Senior Database Engineer

About Company:

Health Catalyst

POSITION OVERVIEW:

We are looking for a highly skilled Senior Database Engineer & Storage Expert with 5+ years of hands-on experience in managing and optimizing large-scale, high-throughput database systems. The ideal candidate will possess deep expertise in handling complex ingestion pipelines across multiple data stores and a strong understanding of distributed database architecture.

The candidate will play a critical technical leadership role in ensuring our data systems are robust, performant, and scalable to support massive datasets ingested from various sources without bottlenecks. You will work closely with data engineers, platform engineers, and infrastructure teams to continuously improve database performance and reliability. performance bottlenecks.

KEY RESPONSIBILITIES:

• Query Optimization: Design, write, debug and optimize complex queries for RDS (MySQL/PostgreSQL), MongoDB, Elasticsearch, and Cassandra.

• Large-Scale Ingestion: Configure databases to handle high-throughput data ingestion efficiently. • Database Tuning: Optimize database configurations (e.g., memory allocation, connection pooling, indexing) to support large-scale operations.

• Schema and Index Design: Develop schemas and indexes to ensure efficient storage and retrieval of large datasets.

• Monitoring and Troubleshooting: Analyze and resolve issues such as slow ingestion rates, replication delays, and performance bottlenecks.

• Performance Debugging: Analyze and troubleshoot database slowdowns by investigating query execution plans, logs, and metrics.

• Log Analysis: Use database logs to diagnose and resolve issues related to query performance, replication, and ingestion bottlenecks

• Data Partitioning and Sharding: Implement partitioning, sharding, and other distributed database techniques to improve scalability.

• Batch and Real-Time Processing: Optimize ingestion pipelines for both batch and real-time workloads.

• Collaboration: Partner with data engineers and Kafka experts to design and maintain robust ingestion pipelines.

• Stay Updated: Stay up to date with the latest advancements in database technologies and recommend improvements.

REQUIRED SKILLS AND QUALIFICATIONS:

• Database Expertise: Proven experience with MySQL/PostgreSQL (RDS), MongoDB, Elasticsearch, and Cassandra.

• High-Volume Operations: Proven experience in configuring and managing databases for large-scale data ingestions.

• Performance Tuning: Hands-on experience with query optimization, indexing strategies, and execution plan analysis for large datasets.

• Database Internals: Strong understanding of replication, partitioning, sharding, and caching mechanisms.

• Data Modeling: Ability to design schemas and data models tailored for high throughput use cases.

• Programming Skills: Proficiency in at least one programming language (e.g., Python, Java, Go) for building data pipelines.

• Debugging Proficiency: Strong ability to debug slowdowns by analyzing database logs, query execution plans, and system metrics.

• Log Analysis Tools: Familiarity with database log formats and tools for parsing and analyzing logs. • Monitoring Tools: Experience with monitoring tools such as AWS CloudWatch, Prometheus, and Grafana to track ingestion performance.

• Problem-Solving: Analytical skills to diagnose and resolve ingestion-related issues effectively.

PREFERRED QUALIFICATIONS:

• Certification in any of the mentioned database technologies.

• Hands-on experience with cloud platforms such as AWS (preferred), Azure, or GCP.

• Knowledge of distributed systems and large-scale data processing.

• Familiarity with cloud-based database solutions and infrastructure.

• Familiarity with large scale data ingestion tools like Kafka, Spark or Flink. EDUCATIONAL

REQUIREMENTS:

• • Bachelor’s degree in computer science, Information Technology, or a related field. Equivalent work experience will also be considered