Sr Data Analyst-Agentic AI & GenAI Delivery

Why GM Financial Technology

Innovation isn’t just a talking point at GM Financial, it’s how we operate. From generative AI and cloud-native technologies to peer-led learning and hackathons, our tech teams are building real solutions that make a difference. We’re committed to AI-powered transformation, using advanced machine learning and automation to help us reimagine customer interactions and modernize operations, positioning GM Financial as a leader in digital innovation within a dynamic industry.

Join us and discover a workplace where your ideas matter, your development is prioritized, and you can truly make a global impact.

About the role:

The Senior Data Analyst – Agentic AI & GenAI Delivery plays a critical role in operationalizing and scaling Agentic AI solutions across the enterprise. This role focuses on driving delivery, deployment validation, and continuous optimization of AI systems through data-driven insights, validation frameworks, and reporting mechanisms.

Unlike traditional data analyst roles, this position operates at the intersection of AI systems, production delivery, and performance analytics, ensuring that Agentic AI solutions are functioning as intended, meeting business objectives, and operating reliably in production environments.

This role partners closely with architects, AI engineers, product teams, and business stakeholders to:

Validate that AI use cases align with real-world outcomes

Monitor agent behavior, performance, and reliability

Establish data-driven feedback loops for continuous improvement

The ideal candidate brings strong expertise in data analysis, AI system validation, observability, and reporting, along with a solid understanding of Agentic AI / GenAI workflows and production deployment challenges.

In this role you will:

Drive the delivery and operational validation of Agentic AI solutions through structured data analysis and reporting

Define and implement data-driven validation frameworks to evaluate AI system performance, accuracy, reliability, and business impact

Analyze production data from AI systems (agents, workflows, prompts, responses) to identify trends, issues, and optimization opportunities

Develop dashboards, reports, and metrics to track the health and effectiveness of Agentic AI deployments

Partner with architecture and engineering teams to validate feasibility outcomes and ensure solutions align with real-world system behavior

Monitor AI systems in production, identifying anomalies, failure patterns, hallucinations, and performance degradation

Support deployment efforts by validating readiness criteria, including performance thresholds, guardrails, and compliance requirements

Enable continuous improvement loops by feeding insights back into model tuning, prompt design, and system architecture

Support A/B testing and experimentation for AI workflows and use cases

Collaborate with business stakeholders to measure and report on AI-driven business outcomes and ROI

Ensure transparency and traceability of AI decisions through structured logging, trace analysis, and reporting

Contribute to the development of AI observability frameworks, including metrics, KPIs, and alerting strategies

What makes you an ideal candidate? Validate readiness of Agentic AI use cases for production deployment Track deployment success metrics and post-production performance Identify gaps between expected vs. actual outcomes Define metrics for: Accuracy and response quality, Task completion success rates, Hallucination and failure cases, Latency and throughput Build evaluation datasets and validation pipelines Analyze: Agent workflows and decisions, Prompt-response chains, Tool usage and orchestration behavior Develop observability dashboards using telemetry and logs Detect and escalate production issues and anomalies Data Analysis & Reporting Perform root cause analysis on failures and performance issues Deliver executive-level reporting on AI system effectiveness Provide actionable insights to improve system design and outcomes Work closely with: Lead Architects for feasibility alignment AI/ML engineers for model/system improvements Product teams for use case refinement Translate technical findings into clear business insights Advanced SQL, Python (Pandas, NumPy), or similar tools Data visualization platforms (Power BI, Tableau) Strong experience in data validation, anomaly detection, and statistical analysis Familiarity with: LLM workflows and prompt engineering, RAG pipelines and evaluation strategies, Agent orchestration and tool integration Understanding of AI failure modes (hallucinations, drift, inconsistency) Experience with: Logging, tracing, and telemetry systems AI evaluation tools and frameworks Monitoring production systems (Azure Monitor, Application Insights) Strong working knowledge of Azure ecosystem, including: Azure OpenAI / AI services Azure Databricks Data platforms (Azure SQL, Cosmos DB) Monitoring tools (Log Analytics, App Insights) Strong analytical and problem-solving skills in complex AI-driven systems Ability to connect system behavior with business outcomes Expertise in translating data into actionable insights High attention to detail in validation, quality, and accuracy Strong communication skills across technical and non-technical stakeholders Ability to thrive in fast-evolving AI environmentsAbility to wrangle large datasets, structured and non-structured data, including data mining and manipulation Experience 6-8 years experience in data analytics, data science, or AI Systems analysis or similar role required Experience supporting AI/ ML or GenAI systems in production environments preferred Auto finance experience preferred, cross functional Agile team experience preferred Bachelor’s Degree in Data Science, Computer Science, Engineering or related quantitative field Preferred Master’s Degree in related quantitative field Preferred What We Offer : Generous benefits package available on day one to include: 401K matching, bonding leave for new parents (12 weeks, 100% paid), tuition assistance, training, GM employee auto discount, community service pay and nine company holidays. Our Culture: Our team members define and shape our culture — an environment that welcomes innovative ideas, fosters integrity, and creates a sense of community and belonging. Here we do more than work — we thrive. Compensation: Competitive pay and bonus eligibility. Work Life Balance: Hybrid work environment, 2-days a week in office. The office locations for this role can be Irving, TX or Ft. Worth, TX NOTE: We are unable to consider candidates who require visa sponsorship for this position This position is not open to agency submissions #LI-hybrid #LI-MH1 #GMFJobs