Data Engineer

Job Title: Data Engineer, EDM IV

Location: Bangalore (Hybrid)/ Pan India

Shift : General Shift

Experience : 6 - 8 Years of relevant experience

We are looking for an experienced Senior Data Engineer to join our Marketing Data Engineering Team. Reporting to Manager, Data Engineering, and is hybrid position in Bengaluru. Data Engineer will join the team to expand and optimize our data and data pipeline architecture and optimizing data flow and collection for cross-functional teams. Should be an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will support Software Developers, Data Quality Engineers, Data Analysts and Data Scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.

Responsibilities

• Create and maintain optimal data pipeline architecture. Assemble complex data sets that meet functional / non-functional requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, DBT and AWS 'big data' technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into employee experience, operational efficiency, and other key business performance metrics.
• Work with stakeholders to assist with data-related technical issues and support associated data infrastructure needs.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management. Keep up to date with the latest and greatest in feature-sets and capabilities from public cloud providers (such as AWS and Azure) and find ways to apply them back help their team Work with data scientists and analysts to strive for greater functionality in our data systems.

Minimum Qualifications

• We are looking for a candidate with 5+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
• 5+ years of hands-on experience in Snowflake 5+ years of working in DBT with knowledge on advanced DBT concepts like macros and Jinja templating.
• Advanced working SQL experience working with relational databases, query authoring (SQL) and working familiarity with a variety of databases.
• Experience with scripting languages such as Python
• Experience with big data tools such as PySpark
• Experience with AWS cloud services used often for data engineering including S3, EC2, Glue, Lambda, RDS, or Redshift
• Experience working with APIs to pull and push data.
• Experience optimizing 'big data' data pipelines, architectures and data sets.
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

Preferred Qualification

• Experience working with AWS CloudFormation templates is a plus
• Familiarity with Agile and SCRUM methodologies is a plus
• Experience working with PowerBI to develop dashboards is a plus Analytical skills related to working with unstructured datasets.
• A successful history of processing value from large, disconnected datasets. Experience working with agile, globally distributed teams.