Forward Deployed Engineer (FDE) 201

Section 1: From Pilot to Production, and a FastAPI Service Skeleton

• A short recap of FDE 101 and the gap between a pilot and a system an organisation will open to users
• A Python project structure the customer's team can maintain: layers, configuration and secret storage
• FastAPI with Pydantic: endpoints, validation, async and streaming responses
• Health checks, standard error handling and pytest tests from day one
• Lab: set up the AI service with a Dockerfile and a first set of tests

Section 2: Lab: Ingestion and Chunking for Real Documents

• Extract text from PDFs, Office files, tables and web pages, and handle Thai text and scanned documents
• Choose a chunking method, by size, by document structure or by meaning, and see its effect on answer quality
• Store useful metadata: source, date, department and the document's access level
• Re-index only what has changed, and remove withdrawn documents from the index
• Lab: build a repeatable ingestion pipeline from a sample document set

Section 3: Lab: Hybrid Search and Reranking

• Where vector search alone struggles: product codes, proper names and abbreviations
• Combine keyword search (BM25) with vector search and merge the results with Reciprocal Rank Fusion
• Rerankers that reorder results more accurately, and the latency they cost
• Filter results by metadata and user permissions before they reach the model
• Lab: compare vector, hybrid and hybrid with a reranker on the same questions

Section 4: Structured Output and Context Engineering

• Force answers into JSON that matches a schema, validate with Pydantic and handle malformed responses
• Make the most of the context: ordering, token budget, source citations and saying when the answer is unknown
• Keep prompts as versioned files and test them before every change
• Decision points: when to choose multimodal RAG, GraphRAG or fine-tuning instead of tuning RAG further
• Lab: add an endpoint that answers in JSON with verifiable citations

Section 5: Lab: An Evaluation Harness and Regression Testing in CI

• Build a golden set from real user questions together with the customer's subject experts
• Measure retrieval (Recall@k, MRR) and answers (correctness and faithfulness to the sources)
• LLM-as-a-judge: how to make it trustworthy, and checking it against human reviewers regularly
• Use a library such as Ragas or write your own metrics with pytest, then set pass thresholds
• Lab: add the evals to CI so that pull requests that lower the score are rejected

Section 6: Lab: Reliable Tool Calling

• Design tools the model picks correctly: names, descriptions, schemas and examples
• Validate arguments before the real call, set timeouts and retries, and make tools safe to call twice
• Separate read-only tools from tools that change data, and cap the number of agent steps
• Return errors to the model so it can correct itself instead of failing the whole task
• Lab: build an agent that calls several internal APIs, with tests for each tool

Section 7: Lab: MCP Servers for Internal Systems

• Moving on from MCP in FDE 101 to a server shared by several teams over HTTP
• Design tools, resources and prompts that suit systems such as ticketing, HR or ERP
• MCP authorisation with OAuth, and passing the user's identity through to the target system
• Versioning, logging and testing an MCP server before agents are allowed to use it
• Lab: build an MCP server around a simulated internal system that respects per-user permissions

Section 8: A2A and Long-Running Workflows with Human Approval

• The Agent2Agent (A2A) Protocol: Agent Cards, tasks and when to use it alongside MCP
• Workflows that run for hours or days: checkpoints, resuming after failure and avoiding duplicate work
• Human in the loop: pause for approval before steps with real effects, such as payments or changes to customer data
• Report progress so users and other systems can follow the status
• Lab: an approval workflow that waits for a human decision and then carries on

Section 9: Lab: Guardrails, Prompt Injection and PII

• Input and output guardrails: blocked topics, response format and fact checks
• Indirect prompt injection hidden in documents, web pages and tool results
• Detect and redact PII before it reaches the model and before it is written to logs, in line with PDPA
• Sandboxes for running agent-generated code: isolated containers with restricted network and file access
• Lab: attack your own system, then add layers of defence until it holds

Section 10: Lab: SSO, RBAC and Audit Logging

• Connect the organisation's SSO through OpenID Connect and carry the user's identity through every layer
• RBAC that decides which roles may call which tools and see which document levels
• Permission-aware retrieval: filter documents by the asker's rights before anything reaches the model
• Audit logs that record who asked what, which tools the agent called and what happened
• Lab: give two user roles different answers and permissions from the same system

Section 11: Lab: Reducing Cost and Latency

• Break down the cost per request: input tokens, output tokens, embeddings, rerankers and tools
• Provider prompt caching, and response caches that must respect user permissions
• Model routing: small models for easy tasks, large models for hard ones, with fallback
• Batching for work that does not need an instant answer, and streaming to cut perceived waiting time
• Lab: measure cost and latency before and after tuning while keeping the eval score from dropping

Section 12: Lab: Observability and Tracing

• Trace the whole path of a request: retrieval, model calls, tools and each agent step
• OpenTelemetry and the GenAI semantic conventions for naming spans and attributes
• Metrics you need: latency, tokens, cost, error rate and user feedback
• Online evaluation on real traffic, and alerts when quality or cost looks wrong
• Lab: send traces to an open-source backend and track down the slowest requests

Section 13: Lab: CI/CD and Containers for AI Systems

• Multi-stage Docker images and Docker Compose for the service, the vector database and workers
• A pipeline of tests, an eval gate and a security scan before build and deploy
• Version everything: code, prompts, models and indexes, so they can be rolled back together
• Roll out gradually to a subset of users first, with clear rollback criteria
• Lab: build a pipeline that builds, tests and releases a new version of the service

Section 14: UAT, ROI and Executive Handover

• A UAT plan with business users: test scenarios, pass criteria and collecting feedback
• The business case: monthly cost, time saved and the assumptions to re-measure after go-live
• The executive readout: pilot results, remaining risks and a scale-up proposal for executives to decide on
• The operating model after go-live: who owns the data, prompts, evals and costs
• Workshop: write a UAT plan and a one-page business case for the class case

Section 15: Capstone: Hardening a Pilot into a Production-Ready Package

• Take a starter pilot that is not ready for use and find its gaps in quality, security and cost
• Improve it with three days of work: evals in CI, per-user permissions, guardrails, tracing and containers
• Assemble the delivery package: code, eval results, architecture, runbook, UAT plan and business case
• Present to simulated executives, answer technical questions and get feedback from the instructor
• Keep the delivery package as a portfolio piece for the FDE role