Data Engineer

Job description:( In Detail):

 The successful candidate will manage and execute the migration of data from legacy systems to new platforms, ensuring accuracy, integrity and minimal disruptions to business operations.

• He/she is expected to possess hands-on experience in ETL processes, database management, and data quality validation, along with strong problem-solving skills to troubleshoot complex data issues.

Planning and Analysis

• Assess existing data sources, data structures and dependencies.

• Understand business requirements and migration scope.

• Collaborate with business and IT teams to plan migration timelines and strategies (e.g. cutover plan, rollback plan).

Data Extraction, Transformation and Loading

• Design, develop and execute data migration scripts.

• Map, transform and cleanse data to meet the target system’s requirements.

• Perform incremental and full data migrations with minimal downtime.

• Ensure data integrity, consistency and accuracy.

Testing and validation

• Conduct trial migrations and validate data accuracy and completeness.

• Investigate and fix any data issues.

• Implement error handling, rollback plans, and data reconciliation processes.

Performance and Optimization

• Optimize migration processes for performance and large data volumes.

• Monitor data transfer performance and resolve bottlenecks.

• Ensure the migration process adheres to data security and compliance policies.

SKILLS:

• Minimum 3 years of experience in developing and supporting large scale systems and applications, with strong object-oriented and core programming skills (e.g. Java, Python, JavaScript, .NET, C# etc.).

• Possess good understanding of object-oriented concepts, design patterns, concurrency and software techniques.

• Proficient in Web services, API technologies and concepts (e.g. JavaScript, .NET Core, XML, HTML etc.).

• Experience in application integration, monitoring and control with exposure in automated tests.

• Proven experience in data migration or ETL engineering.

• Strong knowledge of SQL and relational databases (Oracle, MySQL, SQL Server, PostgreSQL).

• Experience with data validation, reconciliation, and quality assurance.

• Proficient in consuming data from a variety of sources and connecting front end applications to back-end services.

• Good in analyzing requirements, creating technical specifications and using test cases and scenarios.

• Experience with software development tools (e.g. Git, Jenkins, JIRA).

• Good understanding of full System Development Life Cycle.

• Ensure compliance with IT security standards, policies and procedures.

• Strong problem solving, analytical, written and communication skills.

• Ability to work collaboratively in cross-functional teams under tight deadlines.