Senior Data Lead - Azure Data Factory (ADF), Databricks

Bangalore, Karnataka

Job Summary

We are looking for a highly motivated Data Engineer to design, build, and optimize scalable data solutions that power enterprise analytics and business decision-making. The ideal candidate combines strong software engineering skills with deep expertise in modern cloud-based data platforms, automation, and data governance.

Key Responsibilities

.

Key Responsibilities

Build & Model Data Solutions

Design, develop, and maintain scalable data pipelines and data models that support analytics, reporting, and business intelligence initiatives.

Implement efficient data ingestion, transformation, and storage solutions for structured and unstructured data.

Ensure data quality, consistency, and reliability across the data ecosystem.

Automate & Deploy

Develop and maintain CI/CD pipelines to enable automated testing, deployment, and monitoring of data solutions.

Drive Infrastructure-as-Code and deployment automation practices to ensure efficient and reliable delivery.

Partner with development and platform teams to streamline release processes and improve operational efficiency.

Innovate & Improve

Apply software engineering best practices to solve complex data engineering challenges.

Introduce innovative approaches for processing, managing, and analyzing large-scale datasets.

Continuously evaluate new technologies and frameworks to improve platform capabilities and performance.

Own the Data Platform

Optimize data platforms for scalability, performance, security, and reliability.

Monitor and troubleshoot data pipelines, workflows, and platform components.

Ensure adherence to data governance, compliance, and security standards throughout the data lifecycle.

Collaborate & Document

Establish and promote data engineering standards, best practices, and coding guidelines.

Create and maintain clear technical documentation, architecture diagrams, and operational procedures.

Collaborate with cross-functional teams including Data Scientists, Analysts, Product Owners, and Architects to deliver business value.

Skill Requirements

Required Skills & Experience

Technical Skills

Strong experience with Python for data engineering and automation.

Hands-on experience with Databricks and modern data processing frameworks.

Expertise in Azure Cloud Services and cloud-native data architectures.

Proficiency in Azure DevOps , Git , and CI/CD implementation.

Experience designing and maintaining scalable data pipelines and data models.

Strong understanding of data warehousing, ETL/ELT, and data lifecycle management.

Knowledge of data governance, data quality, and security best practices.

Professional Skills

Strong analytical and problem-solving abilities.

Software engineering mindset with a focus on clean, maintainable, and reusable code.

Excellent troubleshooting and debugging skills.

Strong communication and documentation capabilities.

Ability to work effectively in a collaborative, agile environment.

Preferred Qualifications

Experience with enterprise-scale data platforms and large datasets.

Knowledge of monitoring, observability, and performance optimization techniques.

Familiarity with Infrastructure-as-Code and cloud automation practices.

Experience working in Agile/Scrum delivery models.

Our Tech Stack

Cloud: Microsoft Azure

Data Platform: Databricks

Programming: Python

DevOps: Azure DevOps

Version Control: Git

Deployment: Fully Automated CI/CD Pipelines

#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-#body.unify div.unify-button-container .unify-apply-now: focus, #body.unify div.unify-button-container .unify-apply-