Sr Data Engineer

Job Summary: We are seeking a skilled and motivated Data Engineer with 3-5 years of experience to join our growing data team. The ideal candidate will be responsible for designing, developing, testing, deploying, and maintaining robust, scalable, and efficient data pipelines and infrastructure. You will work closely with data scientists, analysts, software engineers, and business stakeholders to understand data requirements and deliver high-quality data solutions that drive business insights and decisions. Key Responsibilities: Design, build, and maintain scalable and reliable ETL/ELT data pipelines to ingest, transform, and load data from diverse sources (e.g., relational databases, APIs, streaming platforms, flat files). Develop and manage data warehousing solutions, ensuring data integrity, optimal performance, and cost-effectiveness. Implement data models, data schemas, and data dictionaries to support business and analytical requirements. Ensure data quality, consistency, and accuracy across all data systems by implementing data validation, cleansing, and monitoring processes. Optimize data pipeline performance and troubleshoot data-related issues. Collaborate with data scientists and analysts to provide them with clean, well-structured, and readily accessible data for their analysis and modelling needs. Implement and maintain data security and governance best practices. Automate data processes and workflows using scripting and orchestration tools. Document data pipelines, architectures, and processes. Stay up to date with emerging data technologies and best practices, and recommend improvements to our existing data stack. Required Skills & Qualifications: Bachelors or master’s degree in computer science, Engineering, Information Systems, or a related technical field. 5-8 years of hands-on experience in a Data Engineering role. Strong proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL, SQL Server). Proficiency in Python. Experience with building and optimizing data pipelines using ETL/ELT tools and frameworks (e.g., Apache Airflow, dbt, Informatica, Talend, custom scripts). Hands-on experience with big data technologies (e.g., Apache Spark, Hadoop ecosystem - HDFS, MapReduce, Hive). Experience with cloud platforms (e.g., Azure - ADLS, Databricks, Synapse; GCP - GCS, BigQuery, Dataflow). Understanding of data warehousing concepts and experience with data warehouse solutions (e.g., Snowflake, Redshift, BigQuery, Synapse Analytics). Familiarity with NoSQL databases (e.g., MongoDB, Cassandra) is a plus. Experience with version control systems (e.g., Git). Strong analytical and problem-solving skills. Excellent communication and collaboration skills, with the ability to work effectively in a team environment. Ability to manage multiple tasks and projects simultaneously. Preferred/Bonus Skills: Experience with real-time data streaming technologies (e.g., Apache Kafka, Kinesis, Flink, Spark Streaming). Knowledge of containerization and orchestration (e.g., Docker, Kubernetes). Familiarity with CI/CD pipelines for data engineering