Cybersecurity · CYC-05

AI Security and LLM Red Teaming

AI Security and LLM Red Teaming is a hands-on course in testing and defending LLM applications and AI agents using the OWASP Top 10 for LLM Applications 2026, from prompt injection, data leakage and RAG weaknesses to automated red teaming with promptfoo and layered guardrails. It suits security engineers, penetration testers and developers who build AI applications.

Updated
From 8,010 THB / person 8,900 −10% excl. VAT 7% · group rates available
PDFDownload the course outline
  • Duration12 hours · 2 days
  • FormatOnsite / live online
  • Next roundOn request
  • CertificateIncluded

Course overview

LLM applications, from chatbots and RAG systems to AI agents that call tools on their own, are moving into real business systems. Their weaknesses are unlike those of ordinary web applications. Plain text in an email or a document can instruct the model to do something it should not, secrets in the context can leak into an answer, and an agent with too many permissions can be tricked into deleting data or sending it outside the organisation. Traditional scanners miss most of these risks.

This course takes learners through hands-on security testing of LLM applications from a defender's point of view, using the OWASP Top 10 for LLM Applications 2026 and the OWASP Top 10 for Agentic Applications as the framework. Every lab targets only deliberately vulnerable demo applications that run on the learner's own machine. Learners start with direct and indirect prompt injection, data leakage and RAG weaknesses, then move on to attacking AI agents through tools and MCP. They then use promptfoo for automated red teaming, build layered guardrails, run the tests in CI/CD, and finish with a capstone in which they write a findings report with recommended fixes for the development team. (2 days, 6 hours per day, 12 hours in total, Intermediate level.)

What you’ll gain

  • Explain the risks in the OWASP Top 10 for LLM Applications 2026 and relate them to a real application architecture
  • Set the scope and rules of engagement for ethical AI red teaming against authorised systems
  • Test direct and indirect prompt injection, data leakage and hidden context exposure
  • Assess RAG weaknesses, from the vector store to poisoned source data
  • Analyse AI agent threats using the OWASP Top 10 for Agentic Applications
  • Build automated red team test suites with promptfoo and read the results correctly
  • Design layered guardrails and permission controls that limit the damage when the model is fooled
  • Write a findings report with risk ratings and fixes that a development team can act on

Who this course is for

  • Security engineers and penetration testers who need to test systems built on LLMs or AI agents
  • AI engineers and developers who build chatbots, RAG systems or agents and want them secure from the start
  • AppSec and DevSecOps teams who set testing standards for AI applications in the organisation
  • Security architects who review AI system designs before they go into production

Prerequisites

  • Basic Python, and the ability to read application code that calls an LLM API
  • An understanding of web security basics such as XSS, SQL injection and HTTP requests
  • Some experience with ChatGPT or another LLM, and a basic idea of RAG and AI agents
  • A laptop with at least 16 GB of RAM that can run Docker Desktop, Node.js and Python

Curriculum

Course Details

A 2-day course, 6 hours per day (12 hours in total, 09:00-16:00), alternating lectures with labs. Intermediate level. Every lab targets only deliberately vulnerable chatbot, RAG and AI agent applications that run on the learner's own machine with Docker. The content follows the OWASP Top 10 for LLM Applications 2026 and the OWASP Top 10 for Agentic Applications, from a defender's point of view:

every attack ends with how to fix it. The course builds on Ethical Hacking and Penetration Testing Fundamentals and focuses entirely on LLM-based applications. Learners use small local models through Ollama, or their own or company LLM API key, with API usage billed by actual use. Learners take home a lab guide, docker-compose files for the demo applications, promptfoo configurations and a red team report template.

Day 1 LLM Application Vulnerabilities through OWASP 2026

Section 1: Lab: AI Red Teaming and Setting Up the Test Range

  • How AI red teaming differs from a traditional penetration test, and why the model alone cannot be the defence
  • Scope, authorisation and rules of engagement: test only systems you have written permission to test
  • Install the deliberately vulnerable demo application with Docker and connect it to a model through Ollama
  • Lab: explore the target, draw its data flow and mark where external data reaches the model

Section 2: OWASP Top 10 for LLM Applications 2026

  • An overview of LLM01 to LLM10 in the 2026 edition and what changed from 2025
  • The core idea of the new edition: accept that the model will be fooled and design the system around it
  • Threat modelling an LLM application: assets, attackers and attack paths
  • Workshop: rank the risks of the demo application against the OWASP list

Section 3: Lab: Direct and Indirect Prompt Injection

  • Direct prompt injection and common jailbreak patterns
  • Indirect prompt injection through web pages, emails, PDF files and other documents the model reads
  • Filter evasion techniques such as encoding, language switching and multi-turn instructions
  • Lab: plant instructions in a document that make the demo chatbot misbehave, and record the evidence

Section 4: Lab: Data Leakage and Hidden Context Exposure

  • Sensitive information disclosure: personal data, business secrets and credentials in responses
  • Hidden context exposure: leaking system prompts, tool schemas and internal workflow rules
  • Defences: data access separation, masking and checking responses before they leave
  • Lab: extract hidden context and mock customer data from the demo app, then propose fixes

Section 5: Lab: Output Handling and Unbounded Consumption

  • Improper output handling: when model output becomes XSS, SQL or a shell command
  • Markdown and image rendering that opens a channel for data exfiltration
  • Unbounded consumption: a single request that loops model and tool calls until costs spike
  • Lab: make model output run a script in the web page, then fix it with output encoding

Section 6: Lab: RAG Weaknesses and the Supply Chain

  • Vector and embedding weaknesses: cross-user retrieval and data that should never be returned
  • Data and model poisoning: planting fake documents in the knowledge base to bend answers
  • Misinformation and the supply chain: trusting answers unchecked, and untrusted models or packages
  • Lab: poison the sample document store, then design source and permission checks
Day 2 Agentic Threats, Automated Red Teaming and Defence

Section 7: Excessive Agency and the Agentic Top 10

  • Excessive agency: more permissions, functions and autonomy than the agent needs
  • The OWASP Top 10 for Agentic Applications: goal hijack, tool misuse and identity abuse
  • Memory poisoning, cascading failures, rogue agents and over-trusting agents
  • Workshop: threat model a sample agent that reads email and calls APIs

Section 8: Lab: Testing AI Agents, Tools and MCP

  • Agent goal hijack through data the agent reads while it works
  • Tool misuse: chaining permitted tools into a harmful outcome
  • MCP server risks such as tampered tool descriptions and overly broad permissions
  • Lab: lead the demo agent into sending data out of the system, then close the gap with least privilege

Section 9: Lab: Automated Red Teaming with promptfoo

  • Configure promptfoo to test the target application through an HTTP or Python provider
  • Choose plugins and strategies that match the application's OWASP risks
  • Read the report, separate false positives and prioritise what to fix
  • Lab: scan the demo application before and after the fixes and compare the results

Section 10: Lab: Guardrails and Defence in Depth

  • Layers of defence: input checks, output checks, tool permission separation and human-in-the-loop
  • Separating untrusted data from instructions and limiting the blast radius when the model is fooled
  • Human confirmation before irreversible actions, and audit logs for agents
  • Lab: add guardrails to the demo application and measure them with the same test suite

Section 11: Continuous Testing and Production Monitoring

  • Run red team test suites in CI/CD to catch regressions when prompts or models change
  • Logging and monitoring for LLM applications to detect real attacks
  • PDPA considerations when prompts and logs contain personal data
  • Lab: set up GitHub Actions to run promptfoo whenever a prompt changes

Section 12: Workshop: Capstone Red Team Engagement

  • Receive a new AI agent application with its scope and rules of engagement
  • Threat model it, test it by hand and scan it with promptfoo
  • Rate the severity of each finding and map it to the OWASP lists
  • Write a findings report with recommended fixes, then present and review it together

Schedule & training options

For individuals — public rounds

No public rounds are open right now. Join the waiting list and we will contact you first when the next round opens, or ask us on LINE. Or call 02-570-8449 or 088-807-9770

For organisations — in-house / private

  • Tailor the content to your team’s tools and projects
  • Your dates, at your office or live online
  • Quotation with tax ID for procurement
Corporate training quote

Instructors

Frequently asked questions

Who is AI Security and LLM Red Teaming for, and what background is needed?

Built for Security engineers and penetration testers who need to test systems built on LLMs or AI agents · AI engineers and developers who build chatbots, RAG systems or agents and want them secure from the start · AppSec and DevSecOps teams who set testing standards for AI applications in the organisation Background you should have: Basic Python, and the ability to read application code that calls an LLM API · An understanding of web security basics such as XSS, SQL injection and HTTP requests Not sure the fit is right? Talk to our team on LINE @itgenius or call 02-570-8449.

How much does AI Security and LLM Red Teaming cost and how long does it run?

THB 8,900 (currently THB 8,010 on promotion). The course runs 12 hours. The price excludes 7% VAT (for payment in a company's name). Pay by bank transfer to the company account, confirm it on our payment page, and we can issue the receipt or tax invoice in your company's name.

Do I get a certificate?

Yes. Everyone who completes the course receives a Certificate of Completion from IT Genius Institute. Each certificate carries its own number, and anyone holding that number can verify it online on our certificate page, so you can add it to your portfolio or pass it to HR as evidence of training.

Where does the training take place, and is there an online option?

You can attend onsite at IT Genius Institute or arrange to join online, and we also run it as a private in-house session for your team. Ask about dates and venues on LINE @itgenius or call 02-570-8449.

What if I fall behind or miss a session — can I retake it?

Yes. You may retake the same course free of charge in a later round, under the institute's conditions. Tell our team which course and round you attended, and we will check it and offer you the rounds that still have seats. Ask us on LINE @itgenius or call 02-570-8449.

How do I enrol, or request a quotation for my company?

Enrol online with the registration form on this page. You can register several attendees at once and enter your tax ID and billing address for the tax invoice. Or request a company quotation straight from the quote button. For anything else call 02-570-8449 or reach us on LINE @itgenius.