Azure Data Engineer

a { text-decoration: none; color: #464feb; } tr th, tr td { border: 1px solid #e6e6e6; } tr th { background-color: #f5f5f5; }

Azure Data Engineer

Salary: $97,500-$130,000

Job Summary

We are seeking an experienced Azure Data Engineer with strong Databricks expertise to design, develop, and support scalable cloud-based data solutions. This role is responsible for building and maintaining modern data platforms on Microsoft Azure and Databricks, developing robust data pipelines, and enabling analytics and reporting initiatives across the organization.

The ideal candidate will have extensive hands-on experience with Azure Databricks, Azure Data Factory, Delta Lake, Apache Spark, and Medallion Architecture. Experience with Terraform and GitHub Actions is required to support infrastructure automation, CI/CD processes, and cloud platform administration.

Key Responsibilities

Design, develop, and maintain scalable data platforms and solutions on Microsoft Azure.

Build and optimize ETL/ELT data pipelines using Azure Data Factory and Azure Databricks.

Develop data transformation processes leveraging Apache Spark and Delta Lake technologies.

Implement and maintain Medallion Architecture (Bronze, Silver, Gold layers) to support enterprise analytics and reporting needs.

Collaborate with business stakeholders, analysts, data scientists, and application teams to understand data requirements and deliver solutions.

Design and implement efficient data models that support reporting, analytics, and business intelligence initiatives.

Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and scalability.

Develop and maintain CI/CD pipelines using GitHub Actions.

Provision and manage cloud infrastructure using Terraform and Infrastructure as Code (IaC) best practices.

Ensure data quality, governance, security, and compliance standards are met.

Provide production support and resolve issues related to data integration and platform performance.

Document technical designs, architectures, standards, and operational procedures.

Required Qualifications

Bachelor//'s degree in Computer Science, Information Systems, Engineering, or a related field.

5+ years of experience in Data Engineering or related technical roles.

Strong experience with:

Azure Databricks

Azure Data Factory (ADF)

Azure Data Lake Storage (ADLS)

Apache Spark (PySpark preferred)

Delta Lake

SQL and data modeling

Experience implementing Medallion Data Architecture.

Hands-on experience with Terraform for infrastructure automation.

Experience building CI/CD pipelines using GitHub Actions.

Strong knowledge of cloud-native data engineering best practices.

Experience working in Agile environments.

Excellent analytical, troubleshooting, and problem-solving skills.

Preferred Qualifications

Microsoft Azure certifications.

Experience with Azure Synapse Analytics.

Experience with Power BI or other analytics platforms.

Knowledge of data governance and data security frameworks.

Experience supporting large-scale enterprise data environments.

Key Competencies

Data Engineering

Cloud Architecture

ETL/ELT Development

Infrastructure as Code (Terraform)

CI/CD Automation

Data Modeling

Problem Solving

Communication and Collaboration

Performance Optimization

Stakeholder Management