Data Engineer

Position Overview:

We are seeking a highly skilled and experienced Senior Data Engineer to enhance our data processing capabilities and support our innovative generative AI projects. This role is crucial for someone with a deep understanding of both traditional and modern data storage systems, including proficiency in graph and vector databases, SQL and NoSQL databases, as well as experience with unstructured data sources. The ideal candidate will have a proven track record of connecting varied data sources and building knowledge graphs to facilitate information discovery. This role involves collaborating with cross-functional teams to design and implement scalable data architectures that drive insights and decision-making.

Key Responsibilities:

• Design and implement robust data pipelines for ingesting and processing data from diverse sources, including structured, semi-structured, and unstructured data.
• Develop and maintain scalable and efficient data models and architectures using graph and vector databases, as well as SQL and NoSQL technologies.
• Leverage expertise in graph databases to build knowledge graphs that enhance information discovery and decision-making processes.
• Integrate various data sources and systems using ETL/ELT processes, ensuring data quality and integrity.
• Collaborate with data scientists, analysts, and other stakeholders to understand data needs and deliver solutions that support data analysis and machine learning projects.
• Optimize data storage, indexing, and query performance to support real-time analytics.
• Stay abreast of emerging technologies and methodologies in data engineering, evaluating and adopting them as appropriate to improve our data infrastructure.
• Ensure compliance with data governance and security policies.

Qualifications:

• Bachelors or Master's degree in Computer Science, Engineering, Data Science, or a related field.
• Minimum of 5 years of experience in data engineering, with a strong focus on graph and vector databases, SQL/NoSQL databases, and unstructured data sources.
• Demonstrated experience in building knowledge graphs and connecting varied data sources for enhanced information discovery.
• Proficient in data modeling, ETL/ELT processes, and data warehousing principles.
• Strong programming skills in Python, Java, Scala, or similar languages.
• Experience with cloud services (AWS, Azure, GCP) and understanding of their data services.
• Excellent problem-solving abilities and strong analytical skills.
• Effective communication and collaboration skills, with the ability to work across different teams and disciplines.