Sr Data Engineer

Position Summary

We are seeking a Data Engineer / Lead Data Engineer with deep experience in data architecture, ETL pipelines, and advanced analytics support. This role is crucial for designing robust pipelines to process structured and unstructured data, integrate ML models, and ensure data reliability. The ideal candidate will be proficient in Python, R, SQL, and cloud-based tools, and possess hands-on experience in creating end-to-end data engineering solutions that support data science and analytics teams.

Key Responsibilities

• Design and optimize data pipelines to ingest, transform, and load data from diverse sources.
• Build programmatic ETL pipelines using SQL and related platforms.
• Understand complex data structures and perform data transformation effectively.
• Develop and support ML models such as Random Forest, SVM, Clustering, Regression, etc.
• Create and manage scalable, secure data warehouses and data lakes.
• Collaborate with data scientists to structure data for analysis and modeling.
• Define solution architecture for layered data stacks ensuring high data quality.
• Develop design artifacts including data flow diagrams, models, and functional documents.
• Work with technologies such as Python, R, SQL, MS Office, and SageMaker.
• Conduct data profiling, sampling, and testing to ensure reliability.
• Collaborate with business stakeholders to identify and address data use cases.

Qualifications & Experience

• 5 to 8 years of experience in data engineering, ETL development, or database administration.
• Bachelor’s degree in Mathematics, Computer Science, or Engineering (B.Tech/B.E.).
• Postgraduate qualification in Data Science or related discipline preferred.
• Strong proficiency in Python, SQL, Advanced MS Office tools, and R.
• Familiarity with ML concepts and integrating models into pipelines.
• Experience with NoSQL systems like MongoDB, Cassandra, or HBase.
• Knowledge of Snowflake, Databricks, and other cloud-based data tools.
• ETL tool experience and understanding of data integration best practices.
• Data modeling skills for relational and NoSQL databases.
• Knowledge of Hadoop, Spark, and scalable data processing frameworks.
• Experience with SciKit, TensorFlow, Pytorch, GPT, PySpark, etc.
• Ability to build web scrapers and collect data from APIs.
• Experience with Airflow or similar tools for pipeline automation.
• Strong SQL performance tuning skills in large-scale environments.