Data Engineer

Job Summary :

Key Responsibilities:

• Build and maintain optimal data pipelines, architectures and data sets required for extraction, transformation, and loading of data from a wide variety of data sources using SQL and integration technologies.
• Work closely with and provide design specifications to Database Administration, Data Architects, and BI development staff to optimize overall design; Enable advanced analytics and data science teams.
• Identify, design, and implement internal process improvements: automating manual processesoptimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Research emerging data technology, applications, and integration frameworks to build a strategy to integrate required data into a single source of truth or data warehouse.
• Conduct full technical discovery, identifying pain points, business and technical requirements, “as is” and “to be” scenarios.
• This position may require availability outside of standard business hours as part of a rotational on-call schedule.

What You’ll Need to Be Successful (Required Skills):

• 2-4 years years of experience in data engineering or related roles
• Experience with languages such as Python, Java, Ruby, C++, Perl, SQL, Hive, Scala, Spark, Kafka
• Experience in big data environments from Azure, AWS, or Google using Big Data technologies such as HDFS, Pig, HBase, Hive
• Experience with data warehousing and SQL and NoSQL database systems
• Experience with data modeling and enterprise Extraction, Transformation and Load (ETL) tools Experience with data APIs
• Experience with analytical problem solving JD Template for Data Engineer • Proficiency in complex SQL query optimization, performance tuning, and enterprise ETL tools (Apache Airflow, dbt, Apache Beam, Luigi) with ability to design and implement scalable data transformation workflows