Senior Data Engineer – Snowflake & GenAI

Experience: 6+ Years
Employment Type: Full-time

About the Role

We are seeking a Senior Data Engineer to lead the design and development of AI-powered data solutions within the Snowflake ecosystem. In this role, you will not only build scalable data pipelines but also develop Generative AI applications using Snowflake Cortex, Streamlit, and Snowpark.

You will collaborate with Data Platform, Analytics, and Business teams to deliver secure, high-performance solutions that enable internal users to interact with data using natural language, semantic search, and automated insights.

If you are a Snowflake expert who codes in Python, orchestrates with Airflow, and is excited by the intersection of Data Engineering + Generative AI, this role is for you.

Key Responsibilities

Data Engineering & Orchestration

• Design, develop, and optimize ELT/ETL pipelines using Snowflake, Python, and Apache Airflow.

• Create and manage Airflow DAGs to orchestrate complex data and AI workflows with error handling, retries, and monitoring.

• Write high-performance SQL and Python code for data transformation, automation, and API integrations.

• Optimize Snowflake workloads for cost, performance, and concurrency.

AI Application Development (Snowflake Cortex)

• Develop and deploy LLM-powered applications using Snowflake Cortex AI (Complete, Embed, Summarize, Vector Search).

• Implement semantic search and RAG (Retrieval-Augmented Generation) patterns using Snowflake vector data types and embeddings.

• Build Snowpark Python procedures and UDFs to execute AI/ML logic directly within Snowflake.

Internal Tooling & Visualization

• Develop Snowflake Streamlit applications to provide internal teams with GenAI-driven interfaces, analytics workflows, and self-service data tools.

• Translate business requirements into functional, user-friendly AI applications with low-latency response times.

Governance & Collaboration

• Ensure compliance with data governance, security, and quality standards.

• Collaborate with business stakeholders to gather requirements and deliver iterative solutions.

• Maintain CI/CD pipelines (Azure DevOps) for database changes, Snowpark deployments, and Airflow DAGs.

• Document technical designs, data lineage, and best practices for team knowledge sharing.

Required Technical Skills

Skill Area

Must-Have Expertise

SQL

Advanced: Complex joins, window functions, query tuning, CTEs, materialized views, data modeling (3NF, Kimball, Data Vault).

Snowflake

Core:

Snowflake as Data Lake/ Warehouse, Time Travel, Clustering, Micro-partitions.

Advanced:

Snowpark Python, Tasks, Streams, Stored Procedures.

AI:

Cortex AI (LLM functions, embeddings), Vector Search.

Visualization:

Streamlit in Snowflake.

Python

Proficient in data manipulation (Pandas, Polars), API integrations (REST, GraphQL), automation scripts, and Snowpark DataFrame API.

Apache Airflow

DAG authoring, scheduling, sensor usage, XComs, integration with Snowflake operators, monitoring & alerting.

Cloud (AWS)

Working knowledge of EC2, EMR, S3, CloudWatch (for Airflow deployment or Snowflake storage integrations).

DevOps

Version control (Azure DevOps/Git), CI/CD pipelines for database change management, infrastructure as code (basic).