Skip to content

Introduction to Red Teaming AI ​

FieldValue
Module ID294
DifficultyMedium
TierTier 0
CategoryOffensive
Estimated Time4 hours
Cubes Earned0
Experience Points460
Creatorsvautia
Last Updated2025-02-07T00:00:00.000000Z
URLhttps://academy.hackthebox.com/app/module/294

Module Includes ​

  • 11 Sections
  • 3 Interactive Exercises
  • 0 Assessments
  • 0 Cubes

Module Description ​

This module provides a comprehensive introduction to the world of red teaming Artificial Intelligence (AI) and systems utilizing Machine Learning (ML) deployments. It covers an overview of common security vulnerabilities in these systems and the types of attacks that can be launched against their components.

Module Summary ​

For penetration testers to assess the security of systems utilizing Machine Learning (ML) deployments, it is crucial to gain a deep understanding of the types of security vulnerabilities that may arise within them, how ML systems interact with traditional systems, and how these interactions may result in security issues. This module provides a broad introduction to security assessments of ML-based systems and generative AI.

In more detail, this module covers the following:

  • ML OWASP Top 10
  • LLM OWASP Top 10
  • Attacking ML-based systems and their components

This module is broken into sections with accompanying hands-on exercises to practice each of the tactics and techniques we cover. The module ends with a practical hands-on skills assessment to gauge your understanding of the various topic areas.

You can start and stop the module at any time and pick up where you left off. There is no time limit or "grading", but you must complete all of the exercises and the skills assessment to receive the maximum number of cubes and have this module marked as complete in any paths you have chosen.

A firm grasp of the following modules can be considered a prerequisite for the successful completion of this module:

Sections ​

11 sections - 3 Interactive

#GroupTypeTitle
1Introduction to Red Teaming ML-based SystemstheoryIntroduction to Red Teaming ML-based Systems
2Red Teaming MLtheoryAttacking ML-based Systems (ML OWASP Top 10)
3interactiveManipulating the Model
4Red Teaming Generative AIinteractiveAttacking Text Generation (LLM OWASP Top 10)
5theoryGoogle's Secure AI Framework (SAIF)
6theoryRed Teaming Generative AI
7theoryAttacking Model Components
8theoryAttacking Data Components
9theoryAttacking Application Components
10theoryAttacking System Components
11Skills AssessmentinteractiveSkills Assessment

Conclusion ​

Congratulations on finishing the Introduction to Red Teaming AI module. You should now be able to understand different types of security vulnerabilities that arise in real-world deployments of ML-based systems. We explored different components of ML-based systems and how the interaction between traditional and ML-based systems may result in critical security vulnerabilities. Furthermore, we discussed security and privacy issues that may arise from insufficiently protected ML models and data.