Job Responsibilities:

• Build and manage data pipelines (batch and real-time) using Databricks / Snowflake

• Implement data ingestion frameworks (API, streaming, batch) for external sources

(e.g., Moody’s)

• Develop data transformation, standardisation, and enrichment pipelines

• Implement data quality checks and CDQ rules within pipelines

• Integrate pipelines with metadata and lineage platforms

• Automate workflows and integrate with ServiceNow for issue management

• Support implementation of data governance controls and audit traceability

• Build reusable data engineering components for scalability

Job Requirements:

• Minimum 10 years of relevant experience

• Strong hands-on experience in Databricks (Spark, Delta Lake) / Snowflake

• Experience in ETL/ELT pipelines, API integrations, streaming (Kafka/Event Hub)

• Knowledge of data quality frameworks and validation techniques

• Familiarity with data lineage integration and metadata tagging

• Programming skills (Python, SQL, Spark)

• Experience with cloud platforms (Azure / AWS / GCP)

• Understanding of governance integration (Collibra / Atlan / Purview preferred)

Senior Data Engineer