Data Engineer - Azure & Databricks Expert

:

SYSTRA Canada is part of the SYSTRA group, an international consulting and engineering group, a world leader in the design of transport infrastructures. SYSTRA Canada is an engineering and consulting firm whose primary focus is to offer transportation solutions, whether for passengers or goods: feasibility studies of building a new railway line, increasing the capacity of the existing infrastructure, privatization of a railway.

Context:

As part of our data-driven strategy, we are looking for an experienced Data Engineer to design, develop, and optimize data pipelines on our Azure platform. You will play a key role in the ingestion, processing, transformation, and delivery of large-scale data through Azure Data Lake, Azure Data Factory, Azure Synapse, and especially Databricks.

Missions/Main Duties:

Design and develop data ingestion pipelines (batch and real-time) from various sources (API, SFTP, databases, etc.)

Set up and orchestrate data flows using Azure Data Factory or Azure Synapse Pipelines

Develop complex data processing workflows with Apache Spark through Databricks (PySpark or Scala)

Ensure data quality, traceability, and governance, particularly through Delta Lake

Optimize the performance of pipelines and Spark processing jobs

Collaborate with Data Science, BI, and Architecture teams to ensure platform consistency and scalability

Participate in the industrialization, documentation, and security of data flows

Contribute to the evolution of the data architecture and the implementation of DevOps/DataOps best practices (CI/CD, monitoring, testing, etc.)

Profile/Skills:

Required Technical Skills:

Expertise in Microsoft Azure: Data Lake Storage Gen2, Data Factory, Synapse, Key Vault, etc.

Strong command of Databricks (PySpark / Spark SQL / Delta Lake)

Solid experience in developing data pipelines

Strong data modeling skills (structured and semi-structured data)

Proficiency with Avro, Parquet, and JSON formats

Knowledge of data security, quality, and governance principles

Experience with Git, CI/CD (Azure DevOps or GitHub Actions), and pipeline testing

Preferred Skills:

Scala, advanced SQL

Knowledge of event-driven architectures (Event Hub, Kafka)

Understanding of MLOps or experience collaborating with Data Scientists

Microsoft (DP-203) or Databricks certification is a plus

Required Experience:

At least 5 years of experience as a Data Engineer

Hands-on experience with large-scale data projects in an Azure & Databricks environment

Personal Qualities:

Analytical mindset, rigor, and autonomy

Ability to work in an agile and collaborative environment

Strong technical communication skills

:

We use AI-enabled tools in our recruitment platform (iCIMS) for tasks like resume parsing and candidate matching. These tools do not make hiring decisions; all applications are reviewed by a human recruiter, and final decisions are made by our hiring team.For questions about our hiring process or AI tools, please contact infocanada@systra.com.

Note: In this description, the masculine gender is used for easy reading purposes and includes the feminine.

Workplace Type: Hybrid