ETL Developer

Job Title: ETL Developer

Position Brief:

We are seeking a skilled ETL Developer to design, build, and maintain robust data pipelines thatsupport reporting, analytics, and business intelligence needs. The ETL Developer will beresponsible for extracting data from various source systems, transforming it into meaningfulformats, and loading it into centralized repositories such as data warehouses or data lakes. Thisrole plays a critical part in enabling data-driven decision-making across the organization.

Location: Hyderabad

Key Responsibilities:

1. ETL Pipeline Development• Design, develop, and optimize scalable and efficient ETL workflows using tools such as,SSIS, Azure Data Factory (ADF).• Develop custom scripts and jobs for data ingestion from APIs, flat files, RDBMS, and cloudsources.

2. Data Integration• Integrate data from structured, semi-structured (JSON, XML), and unstructured sources.• Ensure smooth data ingestion from internal systems (ERP, CRM, IoT, operational systems)and third-party APIs.

3. Data Transformation & Cleansing• Apply business rules, transformations, data quality checks, and enrichments to raw data.• Handle data normalization, deduplication, aggregation, and standardization.

4. Data Loading & Orchestration• Load processed data into data warehouses (Snowflake, Redshift, BigQuery, Azure Synapse)or data lakes (S3, ADLS).• Schedule, monitor, and troubleshoot ETL jobs using orchestration tools (Apache Airflow,Oozie, Azure Data Factory).

5. Testing & Documentation

• Develop and execute unit tests and integration tests for ETL processes.
  • Maintain technical documentation, including data flow diagrams, mapping documents, andpipeline logs.

6. Performance Optimization
• Analyze and optimize ETL job performance, including indexing strategies, parallelprocessing, and resource management.• Tune queries and resolve data pipeline bottlenecks.

7. Collaboration & Stakeholder Engagement
• Collaborate with data analysts, BI developers, and business users to understand datarequirements.• Provide data extracts and troubleshoot data-related issues across departments.

Required Skills and Qualifications:• Bachelors degree in Computer Science, Information Systems, or a related field.• 3 - 5 years of experience in ETL development or data engineering.• Proficiency in one or more ETL tools: SSIS, ADF.• Proficiency in writing SQL Stored Procedures, Views.• Solid knowledge of SQL, PL/SQL, T-SQL and experience with relational databases (SQLServer, Oracle, PostgreSQL, MySQL).• Experience with cloud data services: AWS Glue, Azure Data Factory, GCP Dataflow,Snowflake, Redshift.• Strong understanding of data warehousing concepts, star/snowflake schema, and slowlychanging dimensions (SCD).

Preferred Qualifications:• Experience with Python for data scripting and custom connectors.• Familiarity with data versioning, Git, and CI/CD pipelines.• Knowledge of data governance and data lineage tools• Experience with data profiling, quality tools, and error handling frameworks.

Tools & Technologies:
ETL Tools - Informatica, Talend, SSIS, Apache NiFi, dbtCloud Platforms- AWS (Glue, Redshift), Azure (ADF, Synapse), GCP (Dataflow, BigQuery)Orchestration- Apache Airflow, Azure Data Factory, Control-MProgramming- SQL, Python, Shell, BashDatabases- Oracle, SQL Server, PostgreSQL, MySQLData Lakes/Warehouses- Snowflake, Redshift, BigQuery, Azure Synapse