Generative AI Red Team Engineer

Job title:

Generative AI Red Team Engineer

Company:

Sigma.AI

Job description

We are seeking a highly skilled and innovative Red Team Engineer with expertise in finding Generative AI vulnerabilities to join our adversarial testing team. The ideal candidate will have a strong background in red teaming, adversarial attacks, and generative AI, particularly in testing the robustness and security of large-scale generative models. This role will focus on identifying vulnerabilities, ethical risks, and adversarial weaknesses in AI systems used for tasks such as natural language generation, and other AI-driven applications. Deliverables for this role include the building of a prompt dataset, research and reporting on the evaluation of several generative AI foundation models, and the building of a training program around red teaming.You will collaborate with AI researchers, product managers, and other engineers to proactively test and improve the resilience of our generative AI systems against real-world threats, including prompt injection attacks, data poisoning, and bias exploitation. You will also play a key role in driving red teaming best practices, ethical alignment, and safeguarding the integrity of generative AI models.Key Responsibilities:

• Adversarial planning and testing: Design, plan, and execute red teaming assessments focused on generative AI models to simulate adversarial attacks, prompt injections, and other potential misuse scenarios.

• Threat Emulation: Conduct threat emulation and create real-world attack scenarios for generative AI models, focusing on vulnerabilities such as data poisoning, model drift, and ethical boundary violations.

• Collaborate with AI Teams: Work closely with machine learning engineers, data scientists, product managers, and AI researchers to evaluate model performance under adversarial conditions and provide actionable recommendations for strengthening AI defenses.

• Ethical Testing & Bias Audits: Evaluate AI models for potential ethical concerns, including bias detection and unintended harmful behavior, and work to align AI systems with ethical guidelines.

• Documentation & Reporting: Produce detailed reports outlining identified vulnerabilities, exploit scenarios, and recommendations for improvements, including post-mortems of red teaming exercises.

• Creation of a training program: Develop in collaboration with project managers and Machine learning engineers a training program to train and upskill a team that would be able to carry out red teaming assessments.

• Stay Current: Stay up-to-date on cutting-edge AI security research, adversarial machine learning techniques, and ethical AI frameworks to ensure robust red teaming practices.

Qualifications:

• Education:

• Advanced degree (e.g. Master’s degree or PhD) in Computer Science, Machine Learning, Cybersecurity, or a related field. Equivalent work experience will also be considered.

• Experience:

• 2+ years of experience in red teaming with at least one year spent on the evaluation of generative AI models (e.g., natural language processing, image generation) and the security challenges they present.

• Proven track record of conducting adversarial attacks and identifying vulnerabilities in AI models.

• Technical Skills:

• Strong programming skills in languages such as Python and familiarity with machine learning libraries and adversarial prompt datasets.

• Experience with adversarial machine learning techniques, including prompt injections, model poisoning, and data exfiltration.

• Experience with AI ethics and bias testing in model outputs.

• Other Skills:

• Excellent problem-solving skills with the ability to think like an adversary and design creative attack strategies.

• Effective communication skills to explain complex AI vulnerabilities to stakeholders and provide clear, actionable recommendations.

Preferred Qualifications:

• Knowledge of AI Regulatory Standards: Familiarity with emerging AI governance and security standards, including ethical AI frameworks and AI governance best practices.

Help shape the future of ethical AI.
Learn more about and .

Expected salary

Location

Comunidad de Madrid

Job date

Sun, 24 Nov 2024 03:43:16 GMT

To help us track our recruitment effort, please indicate in your email/cover letter where (vacanciesin.eu) you saw this job posting.