6.0 - 8.0 years

18 - 30 Lacs

Hyderabad

Hybrid

Key Skills: Data engineering, Apache Airflow, GCP, BigQuery, GCS, SQL, ETL/ELT, Docker, Kubernetes, data governance, Agile, CI/CD, DevOps, pipeline orchestration, technical leadership. Roles & Responsibilities: Evaluate and provide scalable technical solutions to address complex and interdependent data processes. Ensure data quality and accuracy by implementing data quality checks, data contracts, and governance processes. Collaborate with software development teams and business analysts to understand data requirements and deliver fit-for-purpose data solutions. Lead the team in delivering end-to-end data engineering solutions. Design, develop, and maintain complex applications to support data processing workflows. Develop and manage data pipelines and workflows using Apache Airflow on GCP. Integrate data from various sources into Google BigQuery and Google Cloud Storage (GCS). Write and optimize advanced SQL queries for ETL/ELT processes. Maintain data consistency and troubleshoot issues in data workflows. Create and maintain detailed technical documentation for pipelines and workflows. Mentor junior data engineers and provide technical leadership and support. Lead project planning, execution, and successful delivery of data engineering initiatives. Stay updated with emerging trends and technologies in data engineering and cloud computing. Experience Requirement: 6-8 yeras of experience in leading the design, development, and deployment of complex data pipelines. Strong working knowledge of Apache Airflow on GCP for orchestration. Hands-on experience integrating data into Google BigQuery and GCS from various sources. Proficient in writing and optimizing complex SQL queries for large-scale data processing. Practical knowledge of containerization technologies like Docker and Kubernetes. Experience in implementing data governance and adhering to data security best practices. Familiarity with Agile methodology and working in cross-functional teams. Experience with CI/CD pipelines and DevOps practices for data engineering workflows. Education: B.Tech M.Tech (Dual), B.Tech, M. Tech.

Posted 2 days ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.