We are currently seeking an experienced professional to join our team in the role of Consultant Specialist
- 8 - 12 years of experience with below requirements and skills:
- Advanced SQL Development:
- Write complex SQL queries for data extraction, transformation, and analysis.
- Optimize SQL queries for performance and scalability.
- SQL Tuning and Joins:
- Analyze and improve query performance.
- Deep understanding of joins, indexing, and query execution plans.
- GCP BigQuery and GCS:
- Work with Google BigQuery for data warehousing and analytics.
- Manage and integrate data using Google Cloud Storage (GCS).
- Airflow DAG Development:
- Design, develop, and maintain workflows using Apache Airflow.
- Write custom DAGs to automate data pipelines and processes.
- Python Programming:
- Develop and maintain Python scripts for data processing and automation.
- Debug and optimize Python code for performance and reliability.
- Shell Scripting:
- Write and debug basic shell scripts for automation and system tasks.
- Continuous Learning:
- Stay updated with the latest tools and technologies in data engineering.
- Demonstrate a strong ability and attitude to learn and adapt quickly.
- Communication:
- Collaborate effectively with cross-functional teams.
- Clearly communicate technical concepts to both technical and non-technical stakeholders.
Qualifications - External
To be successful in this role, you should meet the following requirements:
- Advanced SQL writing and query optimization.
- Strong understanding of SQL tuning, joins, and indexing.
- Hands-on experience with GCP services, especially BigQuery and GCS.
- Proficiency in Python programming and debugging.
- Experience with Apache Airflow and DAG development.
- Basic knowledge of shell scripting.
- Excellent problem-solving skills and a growth mindset.
- Strong verbal and written communication skills.
- Experience with data pipeline orchestration and ETL processes.
- Familiarity with other GCP services like Dataflow or Pub/Sub.
- Knowledge of CI/CD pipelines and version control (e.g., Git).