We are looking for a strong Data Engineering Architect with 1216 years of experience in building and architecting modern data platforms on Microsoft Azure. The ideal candidate will have deep hands-on expertise in Azure Data Factory (ADF) pipeline engineering, SQL performance tuning, and end-to-end data integration architecture, along with a strong analytical mindset to troubleshoot complex data issues. You will lead solution architecture, define best practices, and mentor teams to build scalable, secure, and reliable data solutions.

Key Responsibilities

Azure Data Engineering & Architecture

• Architect and design end-to-end Azure data engineering solutions (batch + near real-time) aligned to enterprise standards.
• Define target state architecture for data ingestion, transformation, orchestration, and serving layers.
• Lead architectural decisions around scalability, resiliency, performance, security, governance, and cost optimization.

Azure Data Factory (ADF) Pipeline Engineering (Core)

• Design, develop, test, and deploy Azure Data Factory pipelines following best practices (modular design, parameterization, reusability, CI/CD readiness).

• Build robust ingestion and orchestration workflows using:

• Linked Services, Datasets, Pipelines, Triggers

• Mapping Data Flows / Wrangling Data Flows (where applicable)

• Integration Runtime strategies (Self-hosted / Azure IR)

Implement operational excellence: logging, alerting, retry patterns, failure handling, and idempotent design.

SQL Development & Optimization

• Develop and optimize SQL queries and stored procedures to support ADF pipeline operations and downstream transformations.
• Conduct query plan analysis and performance tuning (indexes, partitioning strategies, statistics, query rewrites).
• Establish SQL coding standards and reusable patterns for transformation logic.

Troubleshooting & Analytical Problem Solving

• Apply a strong analytical mindset to diagnose and resolve complex data integration issues across ingestion, transformation, orchestration, and storage layers.
• Perform root cause analysis (RCA) for pipeline failures, performance degradation, data quality issues, and environment instability.
• Design proactive monitoring dashboards and alerts for pipeline SLAs and data freshness.

Engineering Standards, DevOps & Governance

• Define and enforce best practices for:

• CI/CD for ADF (Azure DevOps / Git-based workflows)

• Infrastructure-as-Code (ARM/Bicep/Terraformpreferred)

• Version control, code review, release management

Implement data governance patterns: metadata management, lineage, auditing, encryption, RBAC, key management, PII controls.Collaborate with security/compliance teams to ensure enterprise adherence.

Leadership & Stakeholder Management

• Act as a technical leader for data engineering squads; mentor and guide engineers on design patterns and implementation.
• Translate business requirements into technical architecture and delivery plans.
• Work closely with Product Owners, Data Analysts, Data Scientists, and Platform teams to ensure alignment.

Required Skills & Qualifications

Must-Have (Strong)

• 1216 years of overall IT experience with significant data engineering & architecture exposure.

• Strong Azure Cloud Data Engineering and associated services architecture knowledge

• Deep hands-on experience with:

• Azure Data Factory (ADF) pipeline design, orchestration, integration runtime strategy

• SQL advanced querying, stored procedures, performance tuning

Strong troubleshooting skills for complex multi-system data issues.Strong understanding of data architecture concepts:

• Data lakes/lakehouse/warehouse, dimensional modeling, ELT/ETL patterns
• Batch orchestration, dependency management, SCD handling, incremental loads

Azure Ecosystem (Preferred / Good to Have)

• Azure data services experience in one or more:

• Azure Synapse Analytics / Dedicated SQL Pools

• Azure Databricks / Spark

• ADLS Gen2, Azure SQL DB, Managed Instance

• Event Hub / Kafka, Stream Analytics (if real-time involved)

Monitoring & observability:

• Azure Monitor, Log Analytics, Application Insights

Security & identity:

• Azure AD, Managed Identity, Key Vault, RBAC

Engineering Practices

• CI/CD practices for data pipelines; Git branching strategies and release governance.
• Strong documentation skills: architecture diagrams, solution designs, operational runbooks.

Behavioral / Soft Skills

• Excellent analytical thinking and structured problem-solving.
• Strong communication and stakeholder management skills.
• Ownership mindset, ability to drive standards and influence cross-team adoption.
• Proven mentoring/coaching ability.

Education

• Bachelors/Masters degree in Computer Science, Information Systems, or related field (or equivalent experience).

Nice-to-Have Certifications

• Microsoft Certified: Azure Data Engineer Associate
• Microsoft Certified: Azure Solutions Architect Expert
• DP-203 / AZ-305 (or equivalent)