Data Architect

Overview:

• Architecture & Design

  • Design scalable, high-performance, cloud-native data architectures using

    Azure Data Lake, Azure Synapse, and Databricks

    .
  • Develop high-level and low-level architecture documents (HLD/LLD) for modern data platforms.
  • Define data models using

    star and snowflake schemas

    , optimizing for analytics and query performance.

• Data Engineering & ETL

  • Lead the development of ETL/ELT pipelines using

    Azure Data Factory

    ,

    PySpark

    ,

    Spark SQL

    , and

    Databricks

    .
  • Manage ingestion of structured and semi-structured data from diverse sources to Azure-based data lakes and warehouses.
  • Implement real-time data pipelines using

    Azure Event Hubs

    and

    Structured Streaming

    .

• Governance & Security

  • Define and implement data governance frameworks including

    lineage, cataloging, access controls

    , and compliance (e.g.,

    GDPR

    ).
  • Collaborate with MDM and governance teams using tools like

    Informatica AXON

    and

    EDC

    .

• Performance Tuning & Optimization

  • Drive cost-efficient architecture design with partitioning, caching, indexing, and cluster optimization.
  • Monitor and troubleshoot data pipelines using

    Azure Monitor

    ,

    Log Analytics

    , and

    Databricks tools

    .

• Stakeholder Engagement

  • Collaborate with data scientists, analysts, business stakeholders, and DevOps teams to deliver robust, scalable data platforms.
  • Conduct design reviews and training sessions to support platform adoption and knowledge sharing.

Key Skills & Technologies:

• Cloud Platforms:

  Azure (ADF, ADLS, Azure SQL, Synapse, Databricks), AWS (S3, RDS, EC2)

• Big Data:

  Spark, Delta Lake, PySpark, Hadoop

• ETL Tools:

  Azure Data Factory, Informatica, IBM DataStage

• Data Modeling:

  Star, Snowflake, SCD, Fact & Dimension Tables

• Programming:

  Python, PySpark, SQL, Shell Scripting, R

• Visualization Tools:

  Power BI, Tableau, Cognos

• Data Governance:

  Informatica MDM, AXON, EDC

• Certifications Preferred:

  • Microsoft Certified: Azure Data Engineer Associate
  • Databricks Data Engineer Associate / Professional