Data Engineer

About Us:

We’re building a powerful Data Analytics Engine to transform raw market and historical data into actionable intelligence in the electronics supply chain industry. We’re now looking to bring on a full-time Data Engineer to help us accelerate and scale up our pipeline development.

Job Overview:

We’re seeking a hands-on Data Engineer to design, build, and optimize scalable ETL pipelines in AWS. You will take over and expand an existing pipeline currently being transitioned from prototype code (Python-based) into production-ready infrastructure. You’ll collaborate with our engineering team who will support and advise you in this transition. This is a key role with strong ownership in shaping and scaling our data infrastructure to support real-time analytics and ML/AI-driven features.

Responsibilities:

• Build, deploy, and maintain scalable ETL pipelines for millions of data records in AWS (using tools such as AWS Glue, Step Functions S3 and PostgreSQL).
• Implement and manage pipeline orchestration using AWS Step Functions, transitioning towards automated execution
• Collaborate with the ML and product team to integrate Python-based data cleaning and anomaly detection modules into the ETL pipeline.
• Automate and optimize jobs with CI/CD workflows, ensuring performance, observability, and reliability.
• Design and manage data lakes or warehouse schemas optimized for analytics and ML.
• Implement robust data quality checks, versioning, and validation steps.
• Work closely with our engineering team who will provide architectural support.
• Nice to Have: Contribute to Infrastructure as Code (CDKTF) efforts for environment management.

Required Qualifications:

• Minimum 4 years of experience as a Data Engineer or in a similar backend/data role.
• Proven experience building data pipelines and ETL workflows in AWS.
• Strong programming skills in Python.
• Experience working with relational databases like PostgreSQL, Redshift, or similar.
• Familiarity with CI/CD tools and infrastructure-as-code (e.g., Terraform, AWS CDK, CloudFormation).
• Understanding of basic ML/AI workflows and how to support them with data engineering.

Preferred Qualifications:

• Experience working in AWS or other cloud environments such as Azure or GCP.
• Background in data science or knowledge of ML model deployment and feature engineering.
• Familiarity with GitHub Actions, Docker, or serverless computing models.
• Exposure to large-scale data processing tools like Spark, Airflow, or dbt.

What We Offer:

• Opportunity to work on impactful problems with real-world applications in electronics supply chain intelligence.
• Direct mentorship from experienced engineers and AI product leads.
• A flexible, startup-friendly environment where your ideas and execution matter.
• Competitive compensation and the chance to grow with the company.

• Business Intelligence Platforms

Full-time