Sr. Data Engineer / Lead - 67453

Function

Cloud & Data Engineering

Our Company

We’re Hitachi Digital Services, a global digital solutions and transformation business with a bold vision of our world’s potential. We’re people-centric and here to power good. Every day, we future-proof urban spaces, conserve natural resources, protect rainforests, and save lives. This is a world where innovation, technology, and deep expertise come together to take our company and customers from what’s now to what’s next. We make it happen through the power of acceleration.

Imagine the sheer breadth of talent it takes to bring a better tomorrow closer to today. We don’t expect you to ‘fit’ every requirement – your life experience, character, perspective, and passion for achieving great things in the world are equally as important to us.

Job description

Sr. Data Engineer / Lead

Meet the Team

You will be joining a high-performing Data Engineering & Analytics team focused on building scalable, cloud-native data platforms on AWS. The team works closely with Product Owners, Business Analysts, Data Scientists, and Analytics teams to deliver modern data solutions that enable advanced reporting, analytics, and business intelligence.

The team is passionate about:

Building scalable AWS-based data lake and analytics platforms

Delivering reliable and high-performance ETL/ELT solutions

Leveraging Talend Cloud 8.0, PySpark, and Spark SQL for enterprise-grade data processing

Supporting real-time and batch data integration across multiple data sources

Driving data quality, governance, automation, and operational excellence

Implementing reusable frameworks and best practices for modern data engineering

What You'll Be Doing

As a Senior Data Engineer , you will play a key role in designing, developing, and optimizing enterprise data pipelines that power business-critical analytics and reporting solutions.

Key Responsibilities

Design, develop, and maintain ETL/ELT workflows using Talend Cloud 8.0

Build scalable data ingestion pipelines from APIs, flat files, databases, and streaming sources

Develop and support large-scale data processing solutions using PySpark and Spark SQL

Design and manage AWS-based data lake architectures utilizing Amazon S3

Develop cloud-native data solutions leveraging AWS services such as:

AWS Glue

AWS Lambda

Amazon Athena

Amazon EMR

Build and support Python-based microservices for data integration and processing workloads

Implement and maintain streaming data pipelines for near real-time data processing

Optimize Spark jobs, ETL workflows, and AWS workloads for performance, scalability, and cost efficiency

Ensure data quality, reliability, governance, and operational monitoring across the platform

Collaborate with business stakeholders, architects, analysts, and development teams to deliver high-quality data solutions

Troubleshoot production issues and continuously improve platform performance and reliability

What You'll Bring to the Team

Required Experience

10+ years of experience in Data Engineering, Big Data, ETL Development, or Data Integration

Strong hands-on experience with Talend Cloud 8.0 / Talend ETL

6+ years of experience developing Python-based microservices

2+ years of hands-on experience with PySpark and Spark SQL

8+ years of experience working with AWS cloud services

3+ years of experience handling streaming data pipelines and real-time data processing

Experience working on multiple enterprise-scale data engineering projects

Technical Expertise

Talend Cloud 8.x development and administration

PySpark development and Spark SQL optimization

Python programming and reusable framework development

AWS Services including:

Amazon S3

AWS Glue

AWS Lambda

Amazon Athena

Amazon EMR

Data Lake architecture and implementation

ETL/ELT design patterns and best practices

Streaming technologies and event-driven architectures

Data modeling, transformation, and schema management

Performance tuning and optimization of Spark workloads

Data quality, monitoring, and operational support

Professional Skills

Strong analytical and problem-solving capabilities

Ability to build scalable, production-grade data solutions

Excellent communication and stakeholder management skills

Experience working in Agile delivery environments

Strong collaboration skills across engineering, analytics, and business teams

Ability to mentor junior engineers and contribute to engineering best practices

#LI-RS2

About us

We’re a global, team of innovators. Together, we harness engineering excellence and passion to co-create meaningful solutions to complex challenges. We turn organizations into data-driven leaders that can make a positive impact on their industries and society. If you believe that innovation can bring a better tomorrow closer to today, this is the place for you.

Fostering innovation through diverse perspectives

Hitachi is a global company operating across a wide range of industries and regions. One of the things that sets Hitachi apart is the diversity of our business and people, which drives our innovation and growth.

We are committed to building an inclusive culture based on mutual respect and merit-based systems. We believe that when people feel valued, heard, and safe to express themselves, they do their best work.

How we look after you

We help take care of your today and tomorrow with industry-leading benefits, support, and services that look after your holistic health and wellbeing. We’re also champions of life balance and offer flexible arrangements that work for you (role and location dependent). We’re always looking for new ways of working that bring out our best, which leads to unexpected ideas. So here, you’ll experience a sense of belonging, and discover autonomy, freedom, and ownership as you work alongside talented people you enjoy sharing knowledge with.