AI · AIC-17

Advanced Prompt Engineering & AI Evaluation

12 hours 2 days

Last updated 25 Sep 2026

Advanced Prompt Engineering & AI Evaluation is a 12-hour training course by IT Genius Institute. Many organizations have started using generative AI in real work but hit the same problem: inconsistent output, different answers each time from the same…

Training schedule

No public rounds are open right now — register your interest and we will contact you when the next round opens, or request an in-house session for your team.

Corporate training quote

Many organizations have started using generative AI in real work but hit the same problem: inconsistent output, different answers each time from the same request, quality that depends on whoever wrote the prompt, and no way to measure whether a revised prompt is actually better. These problems keep AI from scaling into standardized organizational processes, because both a systematic prompt-design technique and a scientific way to evaluate output quality are missing. This course raises prompt engineering from trial and error to measurable prompt engineering. Learners cover advanced prompt structure and patterns, reasoning techniques (Chain of Thought, ReAct, Tree of Thoughts), controlling

output into reusable formats (structured output), managing context and reducing hallucination, and finally the often-overlooked core of AI evaluation: building test datasets, measuring quality with rubrics and the LLM-as-a-judge technique, running regression tests on prompts, and setting up a shared, standardized prompt library. All content is taught through workshops on real tasks, covers ChatGPT, Claude and Gemini, and finishes with a capstone project where learners build their own end-to-end prompt system with an evaluation harness. (2 days, 6 hours per day, 12 hours in total, Intermediate to Advanced level, prior generative-AI use recommended, no programming required.)

Objectives

Understand how large language models work well enough to design prompts with sound reasoning (token, context, temperature, sampling)
Design prompts with advanced structure and patterns that produce consistent, controllable output
Apply reasoning techniques (Chain of Thought, Self-Consistency, ReAct, Tree of Thoughts, Step-back prompting)
Design few-shot examples and select examples that keep output stable and on-format
Control output into reusable formats (structured output, JSON schema, tool/function calling)
Manage context and reduce hallucination through grounding and source citation
Understand AI evaluation principles and build a test dataset (evaluation dataset / golden set)
Measure output quality with rubrics, human evaluation, LLM-as-a-judge and pairwise comparison
Run regression tests on prompts to confirm that changes are genuinely improvements
Understand prompt injection and jailbreak risks, set security and PDPA practices, and build a standardized prompt library

Who this course is for

Anyone using AI in daily work who wants higher and more consistent output quality
Developers, data analysts and AI engineers who deploy LLMs in real systems
Product owners, project managers and teams designing AI-driven features
Automation and operations teams and those building AI workflows, chatbots or AI agents
Team leads and those setting AI usage standards who need systematic quality measurement

Prerequisites

Prior basic use of generative AI such as ChatGPT, Claude or Gemini
General computer and internet skills
Understanding of your own role's workflow or problems to use as a case study during training
No programming background required, though reading JSON or having called an API helps you go deeper into structured output and evaluation

Curriculum

Course Details

A 2-day course, 6 hours per day (12 hours in total), delivered as a hands-on workshop with a capstone project. Intermediate to Advanced level, covering ChatGPT, Claude and Gemini. Learners take home prompt patterns, a template library, rubrics and evaluation sheets, and a prompt security and PDPA checklist.

Day 1: From Trial and Error to Engineered Prompts

Section 1: LLM Foundations for Prompt Design

How a language model predicts the next token and why output is inconsistent
Token, context window and their impact on prompting long documents
How temperature, top-p and sampling trade creativity for precision
Behavioral differences between ChatGPT, Claude and Gemini that prompt designers should know, and the prompt-improvement loop

Section 2: Advanced Prompt Structure and Patterns

Components of a high-quality prompt: Role, Context, Task, Format, Constraint and Success Criteria
Using delimiters and Markdown/XML structure to reduce ambiguity, and separating system from user prompts
Common real-world patterns: extraction, classification, summarization, transformation and generation
Anti-patterns that distort output and how to write instructions the AI actually follows

Section 3: Reasoning Techniques

Chain of Thought to show reasoning steps for accuracy, and Self-Consistency to pick the consistent answer
ReAct (Reason + Act) combining reasoning with tool use
Tree of Thoughts and Step-back prompting for complex problems
Choosing the technique that fits the task and token cost

Section 4: Few-shot and Example Engineering

The difference between zero-shot, one-shot and few-shot and when to use each
Selecting examples that keep output stable and cover edge cases
Formatting examples consistently, and how example order and count affect quality and bias
Workshop: convert a real prompt from zero-shot to few-shot and compare results

Section 5: Controlling Output with Structured Output

Why work that feeds into systems needs structured output
Forcing JSON output and defining it with a JSON schema
Tool / function calling and having the model respond to a schema, plus validation and retry
Workshop: design a prompt to extract document data as JSON to a given schema and verify correctness

Day 2: Context, Evaluation and a Reliable Prompt System

Section 6: Context Engineering and Reducing Hallucination

What context engineering is and how it differs from prompt writing
Grounding with real data and basic RAG-aware prompting
Forcing the AI to cite sources and answer I do not know when data is insufficient, and handling over-context documents with chunking
Practical hallucination-reduction strategies for high-accuracy work

Section 7: System Prompts, Templates and Prompt Library

Designing a system prompt that keeps AI assistant behavior consistent
Creating prompt templates with variables for reuse, and prompt versioning
Building an organizational prompt library, sharing it with the team, and embedding business rules and tone of voice

Section 8: AI Evaluation Principles and Building a Test Dataset

Why feeling that a prompt improved is not enough, and the importance of measurement
Types of evaluation: reference-based, reference-free and human preference
Building an evaluation dataset and golden set from real work
Common quality dimensions: accuracy, faithfulness, relevance, completeness and format compliance

Section 9: Evaluation Techniques and Regression Testing

Human evaluation and designing a rubric that scores consistently
LLM-as-a-judge to help score against a rubric, with bias caveats
Pairwise comparison to judge which of two prompt versions is better, and regression testing to confirm a change does not worsen existing cases
Workshop: build a golden set and measure two prompt versions with a rubric and LLM-as-a-judge

Section 10: Prompt Security, Robustness and PDPA

Prompt injection and jailbreak risks with attack examples
Basic defenses: separating user data from instructions and setting guardrails
What must not be entered into AI and protecting personal data under PDPA, plus testing prompt robustness
The human-in-the-loop concept for high-risk work

Section 11: Capstone Project and Real-world Adoption

Task: build an end-to-end prompt system for one of the learner's real tasks, designing the system prompt and template with structured output
Build a golden set and evaluation harness to measure quality before and after, then present results with evidence
How to put the prompt system into production and extend it to AI agents, RAG and automation

Frequently asked questions

Who is Advanced Prompt Engineering & AI Evaluation for, and what background is needed?

Built for Anyone using AI in daily work who wants higher and more consistent output quality · Developers, data analysts and AI engineers who deploy LLMs in real systems · Product owners, project managers and teams designing AI-driven features Background you should have: Prior basic use of generative AI such as ChatGPT, Claude or Gemini · General computer and internet skills Not sure the fit is right? Talk to our team on LINE @itgenius or call 02-570-8449.

How much does Advanced Prompt Engineering & AI Evaluation cost and how long does it run?

THB 8,900 (currently THB 8,010 on promotion). The course runs 12 hours. The fee covers course materials, lunch and refreshments throughout. Pay by bank transfer to the company account, confirm it on our payment page, and we can issue the receipt or tax invoice in your company's name.

Do I get a certificate?

Yes. Everyone who completes the course receives a Certificate of Completion from IT Genius Institute. Each certificate carries its own number, and anyone holding that number can verify it online on our certificate page, so you can add it to your portfolio or pass it to HR as evidence of training.

Where does the training take place, and is there an online option?

You can attend onsite at IT Genius Institute or arrange to join online, and we also run it as a private in-house session for your team. Ask about dates and venues on LINE @itgenius or call 02-570-8449.

What if I fall behind or miss a session — can I retake it?

Yes. You may retake the same course free of charge in a later round, under the institute's conditions. Tell our team which course and round you attended, and we will check it and offer you the rounds that still have seats. Ask us on LINE @itgenius or call 02-570-8449.

How do I enrol, or request a quotation for my company?

Enrol online with the registration form on this page. You can register several attendees at once and enter your tax ID and billing address for the tax invoice. Or request a company quotation straight from the quote button. For anything else call 02-570-8449 or reach us on LINE @itgenius.