Sr. Responsible AI Analyst

Job Description:

The Sr

Responsible AI Analyst Responsible AI Analyst will be responsible for supporting the design development evaluation and implementation of Responsible AI solutions across model lifecycle stages

The role involves conducting technical assessments building evaluation datasets defining metrics supporting model testing contributing to guardrail design assisting in deploying Responsible AI workflows and collaborating with engineering product governance ISG and DPO teams to ensure that AI systems adhere to ethical safety fairness transparency and regulatory expectations

Key Responsibilities:

Execute Responsible AI Evaluations

Conduct structured assessments on fairness safety robustness explainability and hallucination risks for models under development or deployment

Design Evaluation Datasets and Metrics

Create curated adversarial and edge case datasets and define quantitative metrics for evaluating fairness toxicity reliability and ethical alignment

Support Guardrail and Control Implementation

Contribute to implementing technical guardrails safety filters explainability modules and automated checks within AI pipelines

Analyze Ethical and Technical Risks

Review datasets model outputs and system behavior to identify fairness gaps robustness issues transparency deficiencies and other Responsible AI risks

Participate in Red Teaming and Stress Testing

Assist with scenario based adversarial evaluations prompt safety checks robustness tests and model vulnerability analysis

Support Deployment of Responsible AI Workflows

Assist in implementing lifecycle governance workflows templates and processes such as via IBM OpenPages or equivalent governance tooling

Prepare Transparency and Governance Documentation

Develop model cards system cards evaluation reports risk logs and supporting documentation required for governance reviews and audit readiness

Assist in Continuous Monitoring

Support creation of dashboards metrics and monitoring signals to track fairness drift hallucination patterns model instability and safety deviations

Collaborate Across Engineering and Governance Functions

Work closely with AI engineers data scientists product teams legal ISG DPO and governance bodies to ensure Responsible AI requirements are consistently applied

Assist in Training and Knowledge Enablement

Help develop training content guides and resources to educate internal teams on Responsible AI evaluation methods guardrails and governance expectations

Technical Requirements:

Proficiency in Python and ML DL frameworks PyTorch TensorFlow for evaluation and experimentation

Understanding of fairness libraries Fairlearn AIF360 and ability to compute ethics related evaluation metrics

Familiarity with explainability tools SHAP LIME Captum Integrated Gradients

Exposure to red teaming concepts prompt safety evaluation and data integrity checks

Experience with MLOps basics including evaluation pipelines experiment tracking and CI workflows

Understanding of ML algorithms generative models and supervised unsupervised learning techniques

Familiarity with NLP vision speech and structured data domains

Knowledge of datasets benchmark suites and third party model ecosystems

Preferred Skills:

Technology->AI-Generative AI->Generative AI - Basic->retrieval augmented generation (rag),Technology->AI-Responsible AI->Responsible AI