Senior Data Engineer - Scala/Spark

Data Software Engineer, you'll be part of a team who are passionate about learning and prototyping cutting-edge technologies and convergence initiatives with latest Big Data services on Spark, Scala, Databricks, PySpark and leverage GitHub to support things like CI/CD and integrations

• 6-10 years of experience as a Data engineer who works on enterprise standard frameworks like Spark, Scala, Databricks, PySpark.
• Development experience in unit and integration test cases
• Intermediate level of Database (SQL) skills to develop SQL queries, functions
• Good Understanding on CI/CD Pipeline i.e. GitHub Actions
• Strong knowledge of version control tools, preferably GitHub
• Basic Knowledge on Linux/Unix environment (basic commands, shell scripting, etc.)
• Demonstrated ability to thrive in an enterprise Agile/SCRUM environment
• Implement data governance practices, data lineage, and metadata management to ensure data accuracy, traceability, and compliance
• Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements
• Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure
• Experience using Collaboration Technologies: GitHub, Jira, Confluence
• Experience using Object-oriented languages Java
• Experience working with testing tools and Automation test needs