Data Engineer

Summary: Analyze, Organize data and Build data systems with multiple data sources Working closely with business and market team for the Data analysis. Build data systems and pipelines Evaluate business needs and objectives Implementing data pipeline to ingest and cleanse raw data from various source systems including SaaS, Cloud based and On-prep databases Combine raw information from different sources Explore ways to enhance data quality and reliability Implementing functional requirements including incremental loads, data snapshots and identity resolution Optimizing data pipelines to improve performance and cost, while ensuring a high quality of data within the data lake: Monitoring services and jobs for cost and performance, ensuring continual operations of data pipelines, and fixing of defects. Must have coding skills in Spark/Pyspark, Python and SQL Must have Knowledge of AWS tools, Glue, Athena and S3 Must have knowledge on JIRA Good knowledge of CI/CD (Github / Github Actions, Terraform)