Senior AI QA Engineer (Manual & Automation)

We are seeking a highly detail-oriented AI QA Engineer with a mix of manual and automation testing skills to validate the performance of cutting-edge, AI-driven video analysis systems. In this role, you will focus on verifying how advanced AI, Large Language Models (LLMs), and computer vision models detect key moments and generate metadata for sports content.

This position sits at the intersection of traditional sports broadcasting and advanced artificial intelligence. The ideal candidate has a deep passion for sports (understanding rules, metrics, and context), strong technical QA scripting skills, and experience working with video assets, timecodes, and metadata.

Req: 1047541119

Responsibilities

Live Game Auditing: Monitor and audit live sports broadcasts (NBA, MLB, NFL, NHL) to ensure the AI/Inference Service correctly tracks, frames, and labels major moments (e.g., touchdowns, home runs, buzzer-beaters) in real-time

Precision QA with Media Assets: Work directly with video frames, timecodes, transcriptions, captions, and JSON outputs to ensure pinpoint alignment between AI detections and actual broadcast moments

Model Error Triage & Support: Act as the human-in-the-loop expert to catch AI hallucinations, misinterpretations of complex sports rules, or edge-case errors. Collaborate directly with AWS and core engineering teams to detail bugs and validate model resolutions

Metadata & Brand Safety Validation: Review AI-generated labels/tags to ensure they align with sports context and meet advertising industry compliance standards (e.g., IAB guidelines), ensuring content is brand-safe for monetization

QA Automation: Design and execute automation scripts to compare AI inference outputs against customer-provided ground truth data at scale

Iterative Testing & Documentation: Keep meticulous records of inference service performance, track accuracy metrics, log defects, and help streamline iterative testing processes for ongoing model updates

Requirements

Hybrid QA Experience: Proven experience in both manual and automation testing, ideally in video-focused or AI-driven environments

Automation Scripting: Hands-on automation scripting skills (e.g., Python, Selenium, or similar testing frameworks) to validate JSON outputs and data payloads against ground truth datasets

SDLC & QA Methodologies: Strong understanding of testing lifecycles, including detailed bug logging, defect triage, regression testing, and quality reporting

LLM & CV Familiarity: Solid understanding of testing LLMs (workflows, prompt/response validation) and conceptual familiarity with computer vision and transcription analysis

Media & Video Literacy: Experience working with video frame analysis, timecodes, subtitles/captions, and metadata structures

Sports Domain Knowledge: A deep understanding of major sports leagues (NFL, NBA, MLB, NHL), including gameplay rules, terminology, and key metrics

Communication: Strong verbal and written communication skills to act as a bridge between technical development teams and business stakeholders

Analytical Mindset: An iterative, meticulous approach to data validation, checking model outputs for accuracy, bias, and context

Nice to have

Prior experience with specialized video testing tools, media processing pipelines, or video player frameworks

Professional background in sports analytics, sports media, or digital broadcasting technology

This Remote Position Cannot be Performed in New York City.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our clients, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

Engineer the Future with a Career at EPAM

This posting includes a good faith range of the salary EPAM would reasonably expect to pay the selected candidate. The range provided reflects base salary only. Individual compensation offers within the range are based on a variety of factors, including, but not limited to: geographic location, experience, credentials, education, training; the demand for the role; and overall business and labor market considerations. Most candidates are hired at a salary within the range disclosed. Salary range: $100 - $105. In addition, the details highlighted in this job posting above are a general description of all other expected benefits and compensation for the position.

Applications will be accepted on a rolling basis.

In accordance with the LA County Fair Chance Ordinance, you may find a copy of the Notice containing a summary of the Ordinance’s key provisions here: Concept FCO Posting 8 27 24 (lacounty.gov)

EPAM will not provide new H-1B visa sponsorship for this position. Candidates with existing transferable H-1B status may be considered.

It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.