Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable ETL workflows using

  Cloud Data Fusion

  and

  Apache Airflow

  .
• Configure and manage various data connectors (e.g., Cloud Storage, Pub/Sub, JDBC, SaaS APIs) for batch and streaming data ingestion.
  
• Implement data transformations, cleansing, and enrichment logic in Python (and SQL) to meet analytic requirements.
  
• Optimize BigQuery data models (fact/dimension tables, partitioning, clustering) for performance and cost-efficiency.
  
• Monitor, troubleshoot, and tune pipeline performance; implement robust error-handling and alerting mechanisms.
  
• Collaborate with data analysts, BI developers, and architects to understand data requirements and deliver accurate datasets.
  
• Maintain documentation for data pipelines, schemas, and operational runbooks.
  
• Ensure data security and governance best practices are followed across the data lifecycle.
  

Minimum Qualifications

• 3+ years

  of hands-on experience in data engineering, with a focus on cloud-native ETL.
• Proven expertise with

  Google Cloud Data Fusion

  , including pipeline authoring and custom plugin development.
• Solid experience building and orchestrating pipelines in

  Apache Airflow

  (DAG design, operators, hooks).
• Strong Python programming skills for data manipulation and automation.
  
• Deep understanding of

  BigQuery

  : schema design, SQL scripting, performance tuning, and cost management.
• Familiarity with additional GCP services: Cloud Storage, Pub/Sub, Dataflow, and IAM.
  
• Experience with version control (Git), CI/CD pipelines, and DevOps practices for data projects.
  
• Excellent problem-solving skills, attention to detail, and the ability to work independently in a fast-paced environment.
  
• Immediate availability to join.
  

Preferred (Nice-to-Have)

• Experience with other data integration tools (e.g., Dataflow, Talend, Informatica).
  
• Knowledge of containerization (Docker, Kubernetes) for scalable data workloads.
  
• Familiarity with streaming frameworks (Apache Beam, Spark Streaming).
  
• Background in data modeling methodologies (Star/Snowflake schemas).
  
• Exposure to metadata management, data cataloguing, and data governance frameworks.