Position Responsibilities:
- Support: With Solution Architects / Data Architecture support solution design for the service layer for APIs and Data Services for Analytics Lead: API Development, Data migration using ETL/ADF or other means
- Lead: Develop business logic as needed for developing and consuming API services Lead: Stakeholder requirements / design for taking the project from scratch to completion
- Lead: SQL query creation for complex business functionalities Support / Lead: With Domain Data Architect for data pipeline methods / service in support of additional data types from the Data Strategy Proactive experimentation with an innovation-inclined mindset
Basic Qualifications (Required Skills/Experience):
- A Bachelors degree or higher is required as a BASIC QUALIFICATION Strong programming skills in Python, SQL, and/or Scala.
- Experience with data engineering using Databricks, Apache Spark [2.4+, 3.1+] & Scala [2.11+] (or Java 11+), Spark SQL, Tuning Spark ETL jobs etc.
- Should be proficient with Databases like SQLServer: MS-SQL/T-SQL; Teradata[16.10+]; General awareness of other Databases like ANSI SQL with advanced SQL skills Experience working with Data Warehouses AND/OR Data Lakes Working with *nix VMs OR working with containers [docker/podman/lxc etc.],
- container orchestration platforms [CF/Kubernetes/GKE/AKS] Shell scripting [sh/bash], knowlege of *nix VM operation and management [Fedora 28+/RHEL 8+/OL 8+]
- Source control systems to manage code and repositories [Git/Mercurial] Experience using continuous integration/deployment tools to continually tweak and deploy code [GitLab-CI other distributed, stateless CI systems]
Preferred Qualifications (Desired Skills/Experience ) :
- Data modelling ER mapping, should be able to represent business problems using optimized data structures & schemas Data intensive, cloud native services like Azure AI Services (Azure Open AI) Hands-on experience with vector databases (Weaviate, Chroma, Pinecone), knowledge graph (like neo4j) and similarity search tools Practical knowledge of large language models (LLMs),
- LangChain, Deep learning and transformer architectures (e g , GPT-family, Llama) Experience fine-tuning or instruction-tuning LLMs, including adapter methods and parameter-efficient tuning Knowledge on classical Machine Learning (ML) algorithms
Typical Education & Experience:
- Education/experience typically acquired through advanced education (e.g. Bachelor) and 4-8 years related work experience or an equivalent combination of education and experience.