Data Engineer

As a Data Center Analyst / Data Engineer, you will operate large-scale big data platformsacross hybrid (on-premises and cloud) environments, enabling reliable analyticsand data-driven use cases. You will work closely with data engineers, datascientists, infrastructure, security, and business stakeholders to ensure dataquality, platform stability, and operational excellence. This role focuses onbuilding, running, and optimizing production-grade data platforms andpipelines, with strong ownership of infrastructure, automation, reliability,and operations.

Key Responsibilities
1. Manage On-Prem and Cloud Data Platforms

·       Maintainand support on-premises clusters, including compute, storage, networking, andsystem configurations.

·       Provision,configure, and manage cloud infrastructure, including AWS S3, EMR, Redshift,and RDS.

·       Monitorplatform performance, capacity, and availability; implement operational alerts,monitoring, and runbooks.

2. Build, Operate & Support Data Pipelines

·       Developand support ETL/ELT pipelines using PySpark and Airflow, ensuring reliableingestion, transformation, and data loading.

·       Operatepipelines across hybrid environments, handling job execution, retries, basicperformance tuning, and failure recovery.

·       Validate,clean, and standardize datasets; monitor pipeline health, data freshness SLAs,and failure patterns, and perform root cause analysis and remediation.

·       Supportdata storage platforms such as S3, PostgreSQL, Redshift, and MongoDB from anoperational and platform perspective.

·       AutomateDataOps and MLOps workflows using CI/CD pipelines (e.g., GitLab CI/CD,Jenkins).

3. Security, Compliance & Governance

·       Implementsecure access controls (IAM, VPC, security groups), encryption, backups, anddisaster recovery mechanisms.

·       Partnerwith infrastructure and security teams to ensure compliance with PDPA, GDPR,and internal governance policies.

Requirements

·       Degreein Computer Science, Software Engineering, Data Science, or equivalentexperience

·       2-5+years of hands-on experience in DevOps, Data Platform Engineering, orInfrastructure Engineering, supporting big data or analytics platforms.

·       Experiencemanaging on-premises and cloud infrastructure, including compute, storage, andnetwork configuration.

·       Hands-onexperience with cloud and/or hybrid environments supporting big data pipelines,with Spark / PySpark and Airflow at an operational level (job execution, basictuning, failure handling).

·       Knowledgeof containerization and orchestration, including Docker fundamentals andKubernetes (deployments, services, ingress, scaling, resource limits).

·       Strongunderstanding of networking fundamentals, including VPC design, routing, DNS,firewalls, and load balancing.

·       Knowledgeof security concepts, such as IAM, access control, and compliance basics.

·       Experiencewith CI/CD pipelines and automation using tools such as GitLab CI/CD, Jenkins,or similar.

·       Basicunderstanding of MLOps concepts from a platform and infrastructure supportperspective.

·       Strongtroubleshooting and problem-solving skills in production environments.

·       Comfortableworking with cross-functional teams (data engineers, ML engineers, security,product).