AI · AIC-07

On-Premise AI Chatbots with LangChain and LangGraph

On-Premise AI Chatbots with LangChain and LangGraph is a course in building an AI chatbot that runs entirely on company servers, from serving models with vLLM and building RAG on Qdrant to LangGraph agents, a streaming FastAPI backend and a Chainlit chat interface on Docker. It suits developers, AI engineers and data scientists with basic Python.

Updated 8 Oct 2026

From 5,850 THB / person 6,500 −10% excl. VAT 7% · group rates available

Corporate training quote See open rounds

PDFDownload the course outline

Duration18 hours · 3 days
FormatOnsite / live online
Next round15–17 Jun 27
CertificateIncluded

Course overview

Many organisations want an AI chatbot that answers questions from manuals, policies and internal documents, yet they cannot send sensitive data to a public cloud API, whether because of PDPA, business confidentiality or cost control. The answer is an AI system that runs entirely on the organisation's own servers, from the language model and the vector database to the chat interface, and today's open source tools are mature enough to make that practical.

This course takes learners through building an on-premise AI chatbot in Python from scratch to a working system. It starts by running open source models with Ollama and vLLM behind an OpenAI-compatible API, connecting them with LangChain 1.x, loading company documents into Qdrant and searching them with hybrid search. Learners then build agents with LangGraph 1.x that check their own answers, remember conversations with a checkpointer and call tools or MCP servers with a human approval step. Finally they turn it into a real system with a streaming FastAPI backend, a Chainlit chat interface, document access control and answer quality evaluation, and bring every service together in Docker Compose. (3 days, 6 hours per day, 18 hours in total, Intermediate level.)

What you’ll gain

Choose and run open source language models on company hardware with Ollama and vLLM
Use LangChain 1.x to connect local models through an OpenAI-compatible API
Build a pipeline that loads company documents into Qdrant and retrieves them accurately with hybrid search
Design agents with LangGraph 1.x that use state, conditions and answer-checking loops
Keep conversation memory with a checkpointer and add human approval before important actions
Build a streaming FastAPI backend and a Chainlit chat interface
Apply guardrails and document access control, and measure answer quality before go-live
Bring every service together in Docker Compose and plan deployment on company servers

Who this course is for

Software developers and AI engineers moving from cloud APIs to a self-hosted system
Data scientists who want to turn language models into applications for their organisation
Backend and full-stack web developers who want to add LLM skills
Solution architects and tech leads who need to design an AI chatbot for their organisation
IT infrastructure and DevOps teams who look after AI servers in-house

Prerequisites

Basic Python, such as functions, classes and installing packages
A basic understanding of REST APIs and JSON
Experience with basic Docker commands such as pull, run and ps
A laptop that can run Docker Desktop, Python and VS Code, with permission to install software

Curriculum

Course Details

This is the 2026 edition of the On-Premise AI Chatbots with LangChain and LangGraph course, rewritten for LangChain 1.x and LangGraph 1.x. It runs for 3 days, 6 hours per day (18 hours in total, 09:00-16:00), as lectures with labs built on a single set of fictional Thai company policies and manuals throughout. Intermediate level. Every lab uses open source tools and runs on the

learner's own machine with Docker. No GPU is required: small models run on a laptop with Ollama, and the vLLM labs run on Google Colab or a GPU the learner provides. Compared with the previous edition it adds hybrid search, agentic RAG, checkpointers, MCP, human-in-the-loop, guardrails and answer quality evaluation. Learners take home a lab guide, Python code for every lab and docker-compose files for the whole system.

Day 1 Local Models and the Knowledge Base

Section 1: On-Premise AI Chatbot Architecture

Cloud AI versus on-premise AI in terms of privacy, cost and operations
The building blocks: inference engine, embeddings, vector DB, agent, API and UI
Choosing open source models that handle Thai, and sizing GPU and memory
Quantization and its effect on speed, accuracy and resource use
Lab: prepare the machine with Docker, a Python virtual environment and uv

Section 2: Lab: Running Models with Ollama and vLLM

Ollama for running small models on a laptop during development
vLLM for servers: PagedAttention and continuous batching
Run vllm serve in Docker with an OpenAI-compatible API
Enable tool calling and structured output on models that support them
Lab: call the local model with the OpenAI Python client and measure speed

Section 3: LangChain 1.x with Local Models

An overview of LangChain 1.x and what changed from the old chains
Connect models with init_chat_model and the vLLM or Ollama base_url
Messages, prompt templates and structured output with Pydantic
Create tools with @tool and a simple agent with create_agent
Lab: a general Q&A chatbot that never sends data off the machine

Section 4: Thai Embeddings and Qdrant

How RAG works and why it reduces hallucination
Choosing a multilingual embedding model that runs locally, such as BGE-M3
Install Qdrant with Docker Compose and design collections and payloads
Dense vectors, sparse vectors and choosing a distance metric
Lab: store and search vectors with the Qdrant Python client

Section 5: Lab: A Pipeline for Loading Company Documents

Read PDF, Word and intranet pages and turn them into clean text
Chunking for Thai: size, overlap and splitting by heading
Store metadata such as source, page, department and confidentiality level
Re-load updated documents without creating duplicates in the collection
Lab: load a set of company policy documents into Qdrant with LangChain

Day 2 More Accurate RAG and Agents with LangGraph

Section 6: Lab: Accurate Retrieval with Hybrid Search

Where similarity search alone fails on specific terms and document codes
Hybrid search that fuses dense and sparse results with the Query API and RRF
Reranking with a locally hosted model to reorder results
Filter results by metadata such as department, date and confidentiality
Lab: compare retrieval results before and after tuning

Section 7: LangGraph 1.x Fundamentals

Why agent work needs a graph rather than a linear chain
StateGraph, state, nodes, edges and conditional edges
Designing state with TypedDict and the messages reducer
Compile the graph and render its diagram to check the logic
Lab: a graph that takes a question, retrieves data and answers with citations

Section 8: Lab: Agentic RAG

Let the agent decide whether to retrieve or answer straight away
Grade retrieved documents for relevance before using them
Rewrite the question and retrieve again when results are not good enough
Check that the answer is grounded in the documents, with a maximum number of loops
Lab: an agent that answers policy questions with a source for every answer

Section 9: Memory and Persistence

Short-term memory with a checkpointer and thread_id
Moving from InMemorySaver to PostgresSaver for production
Handling long chat histories: trimming, summarising and controlling tokens
Long-term memory with a store for information users have shared
Lab: a chatbot that keeps its context even after a server restart

Section 10: Lab: Tools, MCP and Human-in-the-Loop

Let the agent call internal tools such as database lookups or back-office systems
Connect the agent to MCP servers inside the organisation
Pause for human approval with interrupt and resume with Command
Limit tool permissions and log every call
Lab: an agent that drafts a request which must be approved before it is saved

Day 3 From Prototype to a Production System

Section 11: Lab: A Backend API with FastAPI

A project structure with separate services and configuration through the environment
Endpoints for chat, history and document ingestion
Async/await and invoking the graph asynchronously
Stream replies token by token with Server-Sent Events
Automatic API documentation with OpenAPI and Swagger UI

Section 12: Lab: A Chat Interface with Chainlit

Build a chat interface in Python without writing HTML or CSS
Connect Chainlit to the FastAPI backend with streaming
Show source citations and the agent's working steps
Upload files from the chat and collect feedback on answers
User login and separate chat history for each person

Section 13: Security and Guardrails

Control document access per user with metadata filters
Defend against prompt injection in questions and in document content
Filter personal data in questions and answers in line with PDPA
Rate limiting and timeouts at every layer of the system
Lab: attack your own chatbot and close the gaps you find

Section 14: Evaluation and Monitoring

Build a test set of questions with expected answers from real documents
Measure correctness, faithful citations and the rate of honest I don't know answers
Self-hosted tracing with Langfuse and structured logging
Watch vLLM metrics such as latency and throughput
Lab: measure results before and after tuning prompts and retrieval

Section 15: Workshop: Full System Capstone

Combine vLLM or Ollama, Qdrant, PostgreSQL, backend and UI in docker-compose.yml
Health checks, volumes and backups for each service
Scaling as users grow and moving inference to dedicated GPU servers
Present the work, review it together and wrap up with a go-live checklist

Schedule & training options

For individuals — public rounds

15–17June

Classroom15–17 Jun 27

20 seats left

For organisations — in-house / private

Tailor the content to your team’s tools and projects
Your dates, at your office or live online
Quotation with tax ID for procurement

Corporate training quote

Instructors

Samit KoyomFounder & CEO, IT Genius Engineering

Frequently asked questions

How does LangGraph differ from chains in LangChain?

Old-style chains run in a straight line in a fixed order. LangGraph models the work as a graph of nodes (steps), edges (connections) and a shared state, so it can loop, branch on conditions, pause for human approval and save its progress to resume later. In LangChain 1.x, create_agent itself runs on LangGraph, so this course uses LangChain to connect models and tools and LangGraph to design the agent's flow.

How do vLLM and Ollama differ, and which should I use?

Ollama is easy to install and runs small models on a laptop or a machine without a GPU, which suits development and experiments. vLLM is an inference engine for GPU servers that uses PagedAttention and continuous batching, so it handles many concurrent users. Both expose an OpenAI-compatible API, so the same LangChain code can switch between them by changing the base_url.

How do you load company documents into Qdrant and retrieve them accurately?

Read the files and split them into suitably sized chunks with a little overlap, embed each chunk with a locally hosted model that handles Thai, and store them in a collection with metadata such as source and department. At query time, use hybrid search that fuses dense and sparse results to catch both meaning and exact terms, rerank the results, and filter by metadata according to the user's access rights.

Who is the On-Premise AI Chatbots with LangChain and LangGraph course for, and what should I know beforehand?

It suits software developers and AI engineers moving from cloud APIs to self-hosted systems, data scientists who want to turn models into applications, tech leads, and IT infrastructure or DevOps teams who look after AI servers. You should know basic Python, understand REST APIs and have used basic Docker commands. You do not need a machine with a GPU.

What will I be able to do after the On-Premise AI Chatbots with LangChain and LangGraph course?

You will be able to run open source models with Ollama and vLLM, build Thai RAG with Qdrant and hybrid search, design LangGraph agents that check their own answers, remember conversations, call tools or MCP servers and wait for human approval, build a streaming FastAPI backend with a Chainlit chat interface, apply guardrails, measure answer quality and package the whole system in Docker Compose. Participants who complete the training receive a Certificate of Completion with its own number, which can be verified online.

Can the On-Premise AI Chatbots with LangChain and LangGraph course be run in-house for our team?

Yes. It can be run in-house for your team, with examples adapted to your own work, such as building a chatbot from sample documents similar to your organisation's manuals or policies. You can request a quotation from this page, or contact us on LINE @itgenius or by phone on 02-570-8449.