Data Engineer

li{ font-size: 14px;} p{ font-size: 14px;}

Company Overview

Milestone Technologies is a global IT managed services firm that partners with organizations to scale their technology, infrastructure and services to drive specific business outcomes such as digital transformation, innovation, and operational agility. Milestone is focused on building an employee-first, performance-based culture and for over 25 years, we have a demonstrated history of supporting category-defining enterprise clients that are growing ahead of the market. The company specializes in providing solutions across Application Services and Consulting, Digital Product Engineering, Digital Workplace Services, Private Cloud Services, AI/Automation, and ServiceNow. Milestone culture is built to provide a collaborative, inclusive environment that supports employees and empowers them to reach their full potential.

Our seasoned professionals deliver services based on Milestone’s best practices and service delivery framework. By leveraging our vast knowledge base to execute initiatives, we deliver both short-term and long-term value to our clients and apply continuous service improvement to deliver transformational benefits to IT. With Intelligent Automation, Milestone helps businesses further accelerate their IT transformation. The result is a sharper focus on business objectives and a dramatic improvement in employee productivity. Through our key technology partnerships and our people-first approach, Milestone continues to deliver industry-leading innovation to our clients. With more than 3,000 employees serving over 200 companies worldwide, we are following our mission of revolutionizing the way IT is deployed.

Job Overview

Milestone Technologies is seeking a skilled Data Engineer to support a client’s data engineering initiatives on AWS. This role focuses on building scalable data pipelines, improving data quality, and ensuring reliable data processing across modern lakehouse architectures.

You will work closely with data engineering teams, business analysts, and reporting teams to design, build, and optimize data pipelines while ensuring high data quality, observability, and performance.

Build scalable data ingestion pipelines for relational, semi-structured, and unstructured data sources

Design, implement, and optimize lakehouse architectures using Apache Iceberg

Implement best practices for versioning, time travel, incremental processing, and ACID compliance

Develop and optimize Apache Spark (batch and streaming) jobs for large-scale data processing

Work extensively with AWS services such as Glue, EMR, Lambda, Step Functions, and S3 with a focus on cost and performance optimization

Build and manage real-time data pipelines using Kafka and Kafka Streaming

Design and orchestrate workflows using DBT and Airflow

Implement automated data quality checks, validation frameworks, and error monitoring mechanisms

Establish observability frameworks including monitoring, logging, and alerting for data pipelines

Collaborate with analytics/reporting teams to enable data quality dashboards and reporting

Analyze existing pipelines to identify improvements and enhance reliability and scalability

Leverage AI/LLM-based tools to accelerate ETL/ELT development, validation, and debugging

Participate in code reviews and contribute to best practices and engineering standards

Skills

Bachelor’s degree (or higher) in Computer Science, Engineering, or a related technical field

4+ years of experience designing, building, and maintaining data pipelines

Strong programming skills in SQL, Python, and Apache Spark

Hands-on experience with AWS data services (Glue, EMR, S3, Lambda, Step Functions)

Deep understanding of lakehouse architectures and Apache Iceberg

Experience with DBT and Airflow for data transformation and orchestration

Strong experience with Kafka and real-time streaming pipelines

Experience working with Snowflake as a cloud data warehouse

Strong understanding of data quality frameworks, validation, and monitoring

Experience handling structured, semi-structured, and unstructured data at scale

Solid understanding of distributed systems and data engineering best practices

Experience with CI/CD pipelines and automation (preferred)

Strong problem-solving skills and ability to work in a fast-paced environment

Excellent communication skills and ability to collaborate with cross-functional teams

Compensation

Estimated Pay Range:

Exact compensation and offers of employment are dependent on circumstances of each case and will be determined based on job-related knowledge, skills, experience, licenses or certifications, and location.

Our Commitment to Diversity & Inclusion

At Milestone we strive to create a workplace that reflects the communities we serve and work with, where we all feel empowered to bring our full, authentic selves to work. We know creating a diverse and inclusive culture that champions equity and belonging is not only the right thing to do for our employees but is also critical to our continued success.

We welcome the unique background, culture, experiences, knowledge, innovation, self-expression and perspectives you can bring to our global community. Our recruitment team is looking forward to meeting you.