Data Engineer

Role Overview

We are seeking a Data Engineer to design, build, and operate scalable data pipelines and platforms. The role focuses on ingesting, transforming, and governing data across enterprise systems to support analytics, AI/ML, and business intelligence use cases.

You will work closely with data scientists, analysts, application teams, and business stakeholders to ensure reliable, secure, and high‑quality data is available for decision‑making and advanced analytics.

This is an individual contributor role with exposure to enterprise‑scale data platforms and cloud governance.

Key Responsibilities

Data Platform & Pipeline Engineering

• Design, build, and maintain end‑to‑end data pipelines on AWS (batch and near‑real‑time).
• Implement data ingestion from multiple sources (e.g. SFTP, databases, APIs, on‑premise systems) into AWS data lake architectures.
• Develop robust data transformations using AWS Glue, Lambda, Step Functions, and SQL‑based engines.
• Optimize data storage formats (e.g. Parquet), partitioning strategies, and query performance.

Data Lake & Analytics Enablement

• Manage and evolve AWS‑based data lake architectures using Amazon S3, Glue Data Catalog, Lake Formation etc.
• Enable downstream consumption for BI tools, dashboards, and AI/ML workflows.
• Support integration with SageMaker, analytics applications, and internal platforms.

Data Quality, Reliability & Operations

• Implement data quality checks, validations, and monitoring within pipelines.
• Build alerting and notification mechanisms
• for pipeline failures and data issues.
• Provide L2/L3 support for production data pipelines and analytics platforms.
• Participate in root‑cause analysis and continuous improvement of data workflows.

Security, Governance & Compliance

• Work within enterprise security and governance frameworks (e.g. GISO‑cleared architectures).
• Support cross‑account data access and secure data sharing patterns.

Collaboration & Delivery

• Collaborate with data scientists, analysts, software engineers, and business users to translate requirements into scalable data solutions.
• Contribute to platform documentation, standards, and best practices.
• Support proof‑of‑concepts and gradual production rollout of new data capabilities.

Required Qualifications

Technical Skills

• 3+ years of hands‑on experience in data engineering or data platform development.
• Strong experience with AWS data services, including:

• Amazon S3
• AWS Glue (ETL, Crawlers, Data Quality)
• AWS Glue Data Catalog
• Amazon Athena
• AWS Lambda, Step Functions, EventBridge

• Proficiency in Python and SQL for data processing and pipeline development.
• Experience with data modeling, schema design, and data warehousing concepts.
• Familiarity with RDBMS systems (e.g. PostgreSQL, MSSQL, Amazon RDS).

Cloud & Engineering Fundamentals

• Solid understanding of AWS IAM, networking (VPC), security best practices, and cross‑account setups.
• Experience building production‑grade, fault‑tolerant pipelines.
• Familiarity with CI/CD concepts and version control (e.g. Git).

Preferred / Advantageous Skills

• Experience with Lake Formation fine‑grained access controls and enterprise data governance.
• Experience with network controls (VPC, private endpoints).
• Exposure to Amazon DataZone, OpenSearch, Redshift, or streaming pipelines.
• Experience supporting AI/ML workflows (e.g. feature pipelines for SageMaker).
• Background in manufacturing, aerospace, MRO, or large‑scale enterprise environments.
• Prior exposure to regulated or security‑reviewed cloud environments.

Soft Skills & Attributes

• Strong problem‑solving and analytical mindset.
• Able to work independently while collaborating effectively across teams.
• Clear communicator who can explain technical concepts to non‑technical stakeholders.
• Detail‑oriented with a strong focus on reliability, data quality, and security.