System Engineer

Role : System Engineer

Location : Charlotte, NC (Hybrid)

Role Summary

Senior backend engineer building the core integrations between Unity and the enterprise Spark engine and SparkFlow framework

Designs distributed, high-throughput systems that power data processing and integration workflows at enterprise scale

Operates with significant autonomy, setting technical direction within a regulated, large-scale engineering environment

Key Responsibilities

Platform & Integration Engineering

Design, build, and operate integrations between Unity and the Spark engine and SparkFlow framework

Develop robust APIs, SDKs, and connectors enabling reliable data movement and workflow orchestration

Implement abstractions that simplify SparkFlow adoption for downstream platform consumers

Distributed Systems Design

Architect scalable, fault-tolerant systems handling large data volumes and concurrent workloads

Apply best practices for performance tuning, caching, partitioning, backpressure, and resilience

Drive design reviews, RFCs, and architectural decisions for core platform components

Reliability & Operations

Build systems with first-class observability (metrics, logs, traces) and SLOs

Establish operational excellence: runbooks, alerting, capacity planning, and incident response

Champion automated testing, and progressive delivery practices

Required Qualifications

6+ years of backend engineering experience, with deep expertise in distributed systems

Expert-level Python; strong API design (REST, gRPC) and asynchronous programming

Proven track record delivering large-scale, production-grade systems

Strong fundamentals in concurrency, data structures, and systems performance experience

Preferred Qualifications

Hands-on experience with Apache Spark (PySpark, Spark SQL, Structured Streaming)

Experience building or integrating with workflow/data orchestration frameworks (Airflow, Autosys)

Exposure to greenfield platform builds in regulated financial enterprises

Experience with event-driven architectures (Kafka) and lakehouse formats (Iceberg, Delta, Hudi)

Technical Skills

Languages: Python (expert), Scala or Java (plus), SQL

Distributed Compute: Apache Spark

Storage & Data: S3/ADLS/GCS, Iceberg, Delta Lake, Parquet, Postgres

Streaming & Messaging: Kafka, Pub/Sub

APIs & Services: REST, gRPC, GraphQL, OpenAPI

Containers & Cloud: Docker, Kubernetes, Azure/GCP, Terraform

CI/CD: GitHub Actions

Soft Skills & Leadership Expectations

Strong technical judgment and ability to operate