Senior Data Engineer

Scion Staffing has been engaged to conduct a search for a Senior Data Engineer / AI Data Platform Engineer for an innovative and rapidly growing transportation and logistics organization focused on modernizing enterprise data and analytics capabilities across multiple business units. This is a hybrid opportunity based in Pleasanton, California. Open to candidate relocation (assistance available) and H1B transfers for the right candidate!

High visibility role supporting enterprise modernization and AI initiatives. This role will focus on building scalable data pipelines, integrating enterprise systems, developing Databricks-based data architecture, and supporting AI/ML and agentic AI initiatives across the organization.

POSITION OVERVIEW:

The Senior Data Engineer / AI Data Platform Engineer will play a critical role in supporting enterprise-wide cloud data initiatives, operational analytics, and modern reporting solutions within a fast-paced logistics environment. This individual will partner with internal stakeholders and external technical teams to build scalable data pipelines, optimize cloud-based analytics workflows, and support ongoing Databricks initiatives. The ideal candidate brings strong expertise in Databricks, SQL, Python, PySpark, and cloud-based ETL/ELT environments, along with a passion for solving complex operational data challenges.

PERKS:

Competitive compensation package with performance-driven growth opportunities

Collaborative and inclusive team environment focused on innovation and continuous improvement

Opportunity to work on enterprise-scale cloud, analytics, AI, and automation initiatives

Hybrid work environment with exposure to modern technologies including Databricks, AWS, Azure, and enterprise analytics platforms

RESPONSIBILITIES:

Develop, maintain, and optimize scalable cloud-based data solutions within Databricks

Build and support ETL/ELT workflows utilizing SQL, Python, PySpark, APIs, and modern data engineering practices

Support enterprise analytics initiatives, operational reporting, and scalable data architecture across multiple business units

Develop and optimize Spark workloads, transformations, and cloud-based data processing pipelines for performance and scalability

Support integrations with APIs, operational systems, telematics platforms, and external data providers using structured and semi-structured datasets

Create automation workflows, validation processes, and operational analytics solutions using Python and Databricks notebooks

QUALIFICATIONS:

Minimum 7+ years of experience in Data Engineering, Analytics Engineering, or related technical roles

Strong hands-on experience with SQL, Python, Databricks, PySpark, and Spark-based processing

Experience supporting cloud-based environments within AWS and Microsoft Azure

Experience developing and maintaining modern ETL/ELT workflows and enterprise data pipelines

Understanding of APIs, JSON data structures, and operational analytics environments

Experience working with logistics, transportation, telematics, or operational datasets preferred

Experience supporting hybrid cloud architectures, enterprise analytics environments, and large-scale integrations preferred

Experience creating automation workflows, data validation processes, and scalable reporting solutions preferred

COMPENSATION AND BENEFITS:

This opportunity offers a competitive compensation salary up to $215k/yr along with a 10% annual bonus and comprehensive benefits, including medical, dental, and vision coverage, paid time off, and opportunities for professional growth and advancement. The organization fosters a collaborative and inclusive culture that values innovation, continuous learning, and long-term career development while providing exposure to enterprise-scale cloud, analytics, and automation initiatives.

ABOUT OUR SEARCH FIRM:

Scion Staffing is a national award-winning staffing firm! Since 2006, we have had the pleasure of successfully placing thousands of talented professionals with amazing career opportunities. Through our innovative team building and recruiting solutions, we bridge the gap in executive leadership searches, direct hire recruiting, interim leadership placement, and temporary professional staffing. We are proud to be part of the Forbes lists of the Best Recruitment Firms and the Best Executive Search Firms in America. Additionally, Scion has been recognized as a ClearlyRated Best of Staffing firm as well as a top recruitment firm by The Business Times. Additional information about our firm can also be found online.