Senior Data Engineer

Job Description

Job Title: Senior Data Engineer – Financial & Clinical Data Domain (Databricks/AWS) Job Summary: We are seeking a highly skilled and experienced Data Engineer with deep expertise in Databricks and a proven track record working with financial data , dashboard development , and Microsoft Dynamics . This position will support both clinical research and financial analytics , focusing on building scalable, governed data solutions within the AWS ecosystem . The ideal candidate has strong ETL development skills, cloud infrastructure experience, and a business-oriented mindset that translates complex data into actionable insights. Key Responsibilities: Design and develop scalable data pipelines using Databricks and Apache Spark to support clinical and financial data operations. Integrate and orchestrate complex workflows using AWS Glue , AWS Lambda , and Step Functions . Build and maintain enterprise-grade data solutions leveraging Amazon RDS (SQL Server/Oracle) , Amazon S3 , DynamoDB , and integrations with Microsoft Dynamics 365 . Design and deliver dynamic, user-friendly dashboards and data visualizations to support executive-level financial and operational reporting. Lead and mentor a team of data engineers, setting best practices in design, performance, and code quality. Deploy, manage, and optimize cloud infrastructure in AWS , focusing on performance, scalability, and security. Implement strong data governance and master data management (MDM) frameworks to ensure consistency and compliance across systems. Collaborate with stakeholders from finance , clinical operations , IT , and biostatistics to ensure alignment of data products with business needs. Ingest and analyze data from ERP systems like Microsoft Dynamics , driving automation and cross-functional insights. Required Qualifications: 5+ years of experience in ETL Data Engineering across clinical and/or financial domains. 3+ years of hands-on experience with Databricks and Apache Spark . 2+ years of experience integrating data from Microsoft Dynamics 365 or similar ERP/CRM platforms. 3+ years of experience building dashboards in Power BI , QuickSight , or similar tools for financial and operational reporting. 2+ years of experience with AWS Glue , AWS Lambda , and Step Functions or similar orchestration tools. 4+ years of experience with Amazon RDS (SQL Server/Oracle) , S3 , and data lake architecture on AWS. 2+ years of experience deploying and managing infrastructure on AWS cloud . 4+ years of experience leading engineering teams or technical squads. 2+ years of experience implementing data governance and MDM frameworks . Familiarity with GitHub Actions , Jenkins , or other CI/CD tools . Nice to Have: Strong proficiency in Python for scripting and data processing. Understanding of Agile and DevOps methodologies in data environments. Experience with financial forecasting , budget analytics , or regulatory reporting in financial or healthcare sectors. Knowledge of clinical data standards and compliance protocols in CRO/pharma industries. Show more Show less