Data Engineer

Populace World Solutions

4 years

0 Lacs

Pune Maharashtra India

Posted:1 week ago| Platform:

Apply

Skills Required

data power etl sql python agile sap test azure aws gcp composer server database checks logging pipeline query clustering automate deployment jenkins github code monitoring latency communication airflow resolve development

Work Mode

On-site

Job Type

Contractual

Job Description

Locations :-

Experience :-

Must Have:

Key Responsibilities:

Develop, test, and deploy robust ETL pipelines on Snowflake on MS Azure/AWS/GCP Platform using, but not limited to, Snowflake/dbt/Data bricks, Cloud Composer, and Cloud Run to ingest structured/unstructured data from on-prem SQL Server instances into Snowflake Models/Database/Schema
Implement data validation checks, error handling, and logging to ensure pipeline reliability
Optimize pipeline and query performance through partitioning, clustering, and cost-control techniques
Automate deployment workflows using CI/CD pipelines (Jenkins, GitHub Actions, etc.) and infrastructure-as-code (IaC) tools
Monitor pipelines via Cloud Logging and Cloud Monitoring, implementing alerting for data latency or quality issues
Basic Qualifications
Excellent verbal and written communication skills
4+ years of experience working with Snowflake/dbt on Azure/GCP/AWS in production environments and hands-on experience in developing moderate to complex ETL data pipelines
3+ years of hands-on experience with Cloud Composer (Airflow) developing DAGs
4+ years of hands-on experience with ability to resolve complex SQL query performance issues.
4+ years of ETL Python development experience; experience parallelizing pipelines a plus
Demonstrated ability to troubleshoot complex query, pipeline, and data quality issues

Mock Interview

Practice Video Interview with JobPe AI

Start Python Interview

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.