Home
Jobs
BigTapp Analytics logo

Senior Data Engineer-DataBricks

5 years

0 Lacs

Chennai Tamil Nadu India

Posted:1 week ago| Platform: Linkedin logo

Apply

Skills Required

data databricks engineering architecture azure analytics python scala etl pipeline aws gcp adf spark redshift workflow orchestration airflow design leadership software checks monitoring processing security governance certification programming hadoop query apache postgresql mysql nosql mongodb git docker kubernetes collaborative kafka regulations ml model integration compensation development collaboration compliance

Work Mode

On-site

Job Type

Full Time

Job Description

Job Title: Senior Data Engineer
Location: Hyderabad, Chennai & BangaloreExperience: 5+ Years

Job Summary

We are looking for a highly skilled and experienced Senior Data Engineer to join our growing data engineering team. The ideal candidate will have a strong background in building and optimizing data pipelines and data architecture, as well as experience with big data technologies and Azure cloud services. You will work closely with cross-functional teams to ensure data is accessible, reliable, and ready for analytics and business insights.
Mandatory Skills
  • Advanced SQL
  • Python or Scala for data engineering
  • ETL pipeline development
  • Cloud platforms (AWS/GCP/Azure)
  • Azure 1st party services (ADF, Azure Databricks, Synapse, etc.)
  • Big data tools (Spark, Hadoop)
  • Data warehousing (Redshift, Snowflake, Big Query)
  • Workflow orchestration tools (Airflow, Prefect, or similar)
Key Responsibilities
  • Design, develop, and maintain scalable and reliable data pipelines
  • Demonstrate experience and leadership across two full project cycles using Azure Data Factory, Azure Databricks, and PySpark
  • Collaborate with data analysts, scientists, and software engineers to understand data needs
  • Implement data quality checks and monitoring systems
  • Optimize data delivery and processing across a wide range of sources and formats
  • Ensure security and governance policies are followed in all data handling processes
  • Evaluate and recommend tools and technologies to improve data engineering capabilities
  • Lead and mentor junior data engineers as needed
  • Work with cross-functional teams in a dynamic and fast-paced environment

Qualifications

  • Bachelor’s or Master’s degree in Computer Science, Engineering, or related field
  • Certification in cloud platforms or big data technologies (preferred)
  • 5+ years of data engineering experience
Technical Skills
  • Programming: Python, Scala, SQL
  • Big Data: Spark, Hadoop, Hive
  • Stream Processing: Storm, Spark Streaming
  • Data Warehousing: Snowflake, Big Query, Redshift
  • Cloud: AWS (S3, Lambda, Glue), GCP, Azure (ADF, Azure Databricks)
  • Orchestration: Apache Airflow, Prefect, Luigi
  • Databases: PostgreSQL, MySQL, NoSQL (MongoDB, Cassandra)
  • Tools: Git, Docker, Kubernetes (basic), CI/CD
Soft Skills
  • Strong problem-solving and analytical thinking
  • Excellent verbal and written communication
  • Ability to manage multiple tasks and deadlines
  • Collaborative mindset with a proactive attitude
  • Strong analytic skills related to working with unstructured datasets
Good to Have
  • Experience with real-time data processing (Kafka, Flink)
  • Knowledge of data governance and privacy regulations (GDPR, HIPAA)
  • Familiarity with ML model data pipeline integration
Work Experience
  • Minimum 5 years of relevant experience in data engineering roles
  • Experience with Azure 1st party services across at least two full project lifecycles

Compensation & Benefits

  • Competitive salary and annual performance-based bonuses
  • Comprehensive health and optional Parental insurance.
  • Retirement savings plans and tax savings plans.
  • Work-Life Balance: Flexible work hours
Key Result Areas (KRAs)
  • Timely development and delivery of high-quality data pipelines
  • Implementation of scalable data architectures
  • Collaboration with cross-functional teams for data initiatives
  • Compliance with data security and governance standards
Key Performance Indicators (KPIs)
  • Uptime and performance of data pipelines
  • Reduction in data processing time
  • Number of critical bugs post-deployment
  • Stakeholder satisfaction scores
  • Successful data integrations and migrations
Contact: hr@bigtappanalytics.com

More Jobs at BigTapp Analytics

Golang Developer

Hyderabad, Telangana, India

2 - 4 yrs
|

Salary: Not disclosed

Sr. Mobile Application Developer

Hyderabad, Telangana, India

6.0 - 6.0 yrs
|

Salary: Not disclosed

Health Care - Data Architect

Hyderabad, Telangana, India

10.0 - 10.0 yrs
|

Salary: Not disclosed

Data Architect

Hyderabad, Telangana, India

10.0 - 10.0 yrs
|

Salary: Not disclosed

Senior Data Scientist

Chennai, Tamil Nadu, India

8.0 - 12.0 yrs
|

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Data Interview Now
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Google Play App Store
coding practice

Enhance Your Python Skills

Practice Python coding challenges to boost your skills

Start Practicing Python Now
BigTapp Analytics
BigTapp Analytics

10 Jobs

RecommendedJobs for You

BigTapp Analytics logo

Senior Data Engineer-DataBricks

BigTapp Analytics

Chennai, Tamil Nadu, India

BigTapp Analytics logo

Senior Data Engineer-DataBricks

BigTapp Analytics

Chennai, Tamil Nadu, India