Senior Observability Engineer - Elastic Search (Contract)

Responsibilities:

• Design, implement, and operate scalable Elastic Stack (ELK) solutions for logs, metrics, traces, and events.
• Own end-to-end log ingestion pipelines using Beats, Logstash, Elastic Agent, and custom integrations.

• Perform log parsing, filtering, cleanup, normalization, and enrichment using Grok, conditionals, processors, ingest pipelines, and ECS standards.
• Define and implement ingestion best practices for performance and reliability.
• Configure and maintain Kibana dashboards, visualizations, Lens, and Canvas for operational and business observability use cases.
• Experience using Elastic Observability/SIEM and Elastic APM to instrument applications, collect and correlate logs, metrics, and traces, perform performance analysis, and visualize service dependencies.
• Create and manage Elastic Machine Learning jobs (anomaly detection using multi metrics, forecasting) and interpret outcomes to generate insights and alerts.
• Integrate Elasticsearch with other observability tools such as:
• Prometheus & Grafana (metrics collection and visualization).
• SolarWinds and Dynatrace  (infrastructure monitoring and APM).
• Correlate logs, metrics, traces, and events across platforms to enable unified observability.
• Design observability solutions that support operations, infrastructure, and application teams.
• Setup kibana alert rules and write advance watcher scripts.
• Leverage Elastic AI Assistant, including LLM integrations in cloud environments (especially AWS), to enhance investigation, analysis, and insights.
• Manage Elasticsearch clusters, including:
• Familiar in installing ELK tech stack, perform patching and upgrade.
• Node roles, index lifecycle management (ILM), shard strategies, and data tiers
• Security (users, roles, API keys, TLS).
• Performance tuning, scaling, and troubleshooting.
• Apply ELK cluster management best practices for stability, availability, and resiliency.
• Monitor cluster health and proactively address capacity and performance issues.
• Instrument and observe AWS workloads including: EC2, Lambda, ECS/EKS, API Gateway, RDS, S3, and other supporting services.
• Integrate observability deployment (eg: logstash deployment) into DevSecOps practices.
• Use automation tools where applicable for operational tasks (eg: for data extraction/cleaning/transformation, reconciliation) using scripting or programming languages (Python) where applicable.

Requirements:

• Strong understanding on observability concepts, eg: know what is considered as important telemetry, golden signal, how to monitor, how to derive insights, etc.
• Able to propose solution that can uplift observability maturity in the organization.
• Strong hands-on experience with Elasticsearch, Logstash, Kibana, and Elastic ML.
• Strong know how to perform log ingestion, parsing, Grok patterns, filtering, and enrichment.
• Experience managing and operating production enterprise ELK clusters.
• Experience with monitoring tools such as Solarwinds, Prometheus, Slack, Grafana, Dynatrace, or similar tools.
• Good understanding of AWS services (EC2, S3, Lambda, VPC, Cloudwatch) relevant to observability.
• Familiarity with Rest API, AI/ML, LLMs, RAG, Graph Databases, OTEL and emerging observability intelligence concepts.
• Experience on topology mapping or service dependency visualization.
• Strong scripting and automation skills.
• Experience with CI/CD pipelines and deployment automation for logstash pipeline deployment or dashboard/canvas deployment.
• Good understanding of infra (Servers, network, storage) and application tech. stack monitoring.
• Ensure observability configurations meet security and compliance requirements.
• Familiarity with Erlang, Java and MQ application architecture for understanding application behavior and identifying useful observability telemetry would be an advantage.
• Strong communication and stakeholder engagement skills, with the ability to translate complex telemetry data into clear, actionable insights.
• Strong sense of ownership and accountability, with ahigh level of commitment to delivery, quality, and outcomes.
• Certifications would be a plus:
• Elastic Certified Observability Engineer /Analyst
• AWS Certified Solution Associate / Architect
• Redhat Ansible Automation