Lead Data Engineer

We are seeking a highly skilled Lead Data Engineer/Associate Architect to lead the design, implementation, and optimization of scalable data architectures. The ideal candidate will have a deep understanding of data modeling, ETL processes, cloud data solutions, and big data technologies. You will work closely with cross-functional teams to build robust, high-performance data pipelines and infrastructure to enable data-driven decision-making.

Experience: 7 - 12 years

Work Location: Hyderabad (Hybrid) / Remote

Mandatory skills: Python, SQL, Snowflake

Responsibilities:

• Design and Develop scalable and resilient data architectures that support business needs, analytics, and AI/ML workloads.


• Data Pipeline Development: Design and implement robust ETL/ELT processes to ensure efficient data ingestion, transformation, and storage.


• Big Data & Cloud Solutions: Architect data solutions using cloud platforms like AWS, Azure, or GCP, leveraging services such as Snowflake, Redshift, BigQuery, and Databricks.


• Database Optimization: Ensure performance tuning, indexing strategies, and query optimization for relational and NoSQL databases.


• Data Governance & Security: Implement best practices for data quality, metadata management, compliance (GDPR, CCPA), and security.


• Collaboration & Leadership: Work closely with data engineers, analysts, and business stakeholders to translate business requirements into scalable solutions.


• Technology Evaluation: Stay updated with emerging trends, assess new tools and frameworks, and drive innovation in data engineering.

Required Skills:

• Education: Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.


• Experience: 7 - 12+ years of experience in data engineering


• Cloud Platforms: Strong expertise in AWS/Azure data services.


• Databases: Hands-on experience with SQL, NoSQL, and columnar databases such as PostgreSQL, MongoDB, Cassandra, and Snowflake.


• Programming: Proficiency in Python, Scala, or Java for data processing and automation.


• ETL Tools: Experience with tools like Apache Airflow, Talend, DBT, or Informatica.


• Machine Learning & AI Integration (Preferred): Understanding of how to architect data solutions for AI/ML applications