Data Engineer

About The Role

Key Responsibilities

• Design, implement, and maintain robust ETL/ELT pipelines using tools like Argo Workflows or Apache Airflow.
• Manage and execute database schema changes with Alembic or Liquibase, ensuring data consistency.
• Configure and optimize distributed query engines like Trino and AWS Athena for analytics.
• Deploy and manage containerized workloads on AWS EKS or GCP GKE using Docker, Helmfile, and Argo CD.
• Build data lakes/warehouses on AWS S3 and implement performant storage using Apache Iceberg.
• Use Terraform and other IaC tools to automate cloud infrastructure provisioning securely.
• Develop CI/CD pipelines with GitHub Actions to support rapid and reliable deployments.
• Architect and maintain Kafka-based real-time event-driven systems using Apicurio and AVRO.
• Collaborate with product, analytics, and engineering teams to define and deliver data solutions.
• Monitor and troubleshoot data systems for performance and reliability issues using observability tools (e.g., Prometheus, Grafana).
• Document data flows and maintain technical documentation to support scalability and knowledge sharing.
  

Key Deliverables

• Fully operational ETL/ELT pipelines supporting high-volume, low-latency data processing.
• Zero-downtime schema migrations with consistent performance across environments.
• Distributed query engines tuned for large-scale analytics with minimal response time.
• Reliable containerized deployments in Kubernetes using GitOps methodologies.
• Kafka-based real-time data ingestion pipelines with consistent schema validation.
• Infrastructure deployed and maintained as code using Terraform and version control.
• Automated CI/CD processes ensuring fast, high-quality code releases.
• Cross-functional project delivery aligned with business requirements.
• Well-maintained monitoring dashboards and alerting for proactive issue resolution.
• Internal documentation and runbooks for operational continuity and scalability.
  

Qualifications

Technical Skills

• Orchestration Tools: Argo Workflows, Apache Airflow
• Database Migration: Alembic, Liquibase
• SQL Engines: Trino, AWS Athena
• Containers & Orchestration: Docker, AWS EKS, GCP GKE
• Data Storage: AWS S3, Apache Iceberg
• Relational Databases: Postgres, MySQL, Aurora
• Infrastructure Automation: Terraform (or equivalent IaC tools)
• CI/CD: GitHub Actions or similar
• GitOps Tools: Argo CD, Helmfile
• Event Streaming: Kafka, Apicurio, AVRO
• Languages: Python, Bash
• Monitoring: Prometheus, Grafana (preferred)
  

Soft Skills

• Strong analytical and problem-solving capabilities in complex technical environments.
• Excellent written and verbal communication skills to interact with both technical and non- technical stakeholders.
• Self-motivated, detail-oriented, and proactive in identifying improvement opportunities.
• Team player with a collaborative approach and eagerness to mentor junior team members.
• High adaptability to new technologies and dynamic business needs.
• Effective project management and time prioritization.
• Strong documentation skills for maintaining system clarity.
• Ability to translate business problems into data solutions efficiently.
  

Benefits

• Competitive salary and benefits package in a globally operating company.
• Opportunities for professional growth and involvement in diverse projects.
• Dynamic and collaborative work environment
  

Why You'll Love Working With Us

How To Apply

humanresources@encardio.com