Data Engineer 2

Overview

• BE/B.Tech/MS in Computer Science (or equivalent)
• 2 to 5 years of experience in a Data Engineering role with good knowledge of Software engineering.
• Strong CS fundamentals including data structures, algorithms and distributed systems.
• Demonstrate robust problem solving, decision-making, and analytical skills.
• Expert level experience in designing high throughput data solutions / services.
• Hands-on experience on AWS (EC2, EMR, S3, Athena, EMR, Kinesis, Lambda etc). Knowledge of GCP(DataProc, GCS, BigQuery etc) is a plus.
• Strong programming knowledge in one of the languages - Java, Scala or Python.
• Expert level experience in developing data pipelines/solutions using processing engines like Hive, Spark,Spark Streaming, Flink etc
• Good knowledge on Lake House architecture for data persistence. DeltaLake, Iceberg or Hudi knowledge isa plus
• Adequate experience with RESTful web services and micro service architectures
  

• Design and build capabilities to support Batch and Realtime ingestion at scale using open source technologies which are fault tolerant.
• Design solutions that involve complex, multi-system and multi cloud integration, possibly across BUs or domains
• End to end engineering – design, development, testing, deployment and operations
• Ability to work in a dynamic environment, adapt to business requirements using Agile methodologies and DevOps culture
• Conducts code reviews to ensure code quality, consistency and best practices adherence.
• Conducts quick Proof of Concept (POCs) for feasibility studies and take it to the prod
• Lead by example, demonstrating best practices for unit testing, CI/CD, performance testing, capacity planning, documentation, monitoring, alerting, and incident response