LLM Fine-Tuning with LoRA

Course overview

Good prompts and RAG solve a lot of problems, but some tasks need more. You may want a model that always answers in your organisation's format, uses domain terms correctly, classifies Thai documents accurately, or lets a small in-house model replace a large and expensive one. Fine-tuning with LoRA and QLoRA makes this kind of customisation possible on a single GPU, but whether it works depends on making the right call, building a quality dataset and measuring results systematically, not on getting a notebook to run to the end.

This course takes learners through LLM fine-tuning end to end. It starts with criteria for deciding which tasks should be fine-tuned, the principles behind LoRA and QLoRA, and designing training data, including Thai data. Learners then train real models with Unsloth and with Hugging Face TRL and PEFT, tune hyperparameters, read loss curves and work with Thai-capable models such as Typhoon or OpenThaiGPT, measuring every result against the base model. The course moves on to preference tuning with DPO, merging and exporting to GGUF, and serving on Ollama and vLLM. It closes with a capstone in which each learner fine-tunes a model for their own task, from dataset to deployment. (3 days, 6 hours per day, 18 hours in total, Intermediate to Advanced level.)

What you’ll gain

Decide with clear reasons whether a task calls for prompting, RAG or fine-tuning
Explain how LoRA and QLoRA work and choose their key parameters
Design, clean and format training data to match a model's chat template
Fine-tune models with Unsloth and with Hugging Face TRL and PEFT on a single GPU
Fine-tune a Thai-capable model for a specific organisational task
Evaluate a fine-tuned model systematically against its base model
Merge, export to GGUF and serve the model on Ollama or vLLM

Who this course is for

AI and ML engineers who need to adapt language models to their organisation's work
Python developers already using LLM APIs who want to start customising models themselves
Data scientists who want LLM fine-tuning skills for Thai-language data
Teams running in-house LLMs who want small models to perform better on specialised tasks
Researchers and lecturers looking for a repeatable fine-tuning practice

Prerequisites

Confident Python and experience with Jupyter Notebook or Google Colab
Experience calling LLMs through an API and an understanding of prompts, tokens and context windows
Basic machine learning ideas such as training, validation and overfitting are helpful
An internet-connected laptop, a Google account for Colab and a Hugging Face account

Curriculum

Course Details

This course runs for 3 days, 6 hours per day (18 hours in total, 09:00-16:00), as lectures with labs that build on one sample business case throughout. Intermediate to Advanced level. Every lab runs on Google Colab or a cloud GPU rented by the learner, whose cost is the learner's responsibility, so no local GPU is required. The course focuses on parameter-efficient fine-tuning with LoRA and QLoRA, covering the decision to fine-tune, data preparation including Thai data, training, evaluation, DPO and deployment. It pairs well with the Local LLM Deployment with Ollama and vLLM course for serving models across an organisation. Learners take home a lab guide, notebooks for every lab, a sample dataset and evaluation scripts.

Day 1 Principles, Environment and Datasets

Section 1: Workshop: Prompting, RAG or Fine-Tuning

What fine-tuning can and cannot fix, such as adding knowledge that changes often
Tasks that benefit from fine-tuning: answer format, classification, extraction and domain terms
The full cost of fine-tuning: data, GPUs, time and looking after the model afterwards
Replacing a large model with a small fine-tuned one to cut cost and run in-house
Workshop: assess your own organisation's use case with a decision checklist

Section 2: How LLMs Are Trained, and the Principles of LoRA and QLoRA

Pre-training, supervised fine-tuning and preference tuning, and how they differ
Full fine-tuning compared with parameter-efficient fine-tuning
How LoRA works and what rank, alpha and target modules mean
QLoRA: loading a model in 4-bit to train on a GPU with limited memory
Estimating the GPU memory needed for a given model size and training method

Section 3: Lab: Setting Up a Fine-Tuning Environment

Choosing between Google Colab, a rented cloud GPU or a local GPU
Install Unsloth, Transformers, PEFT, TRL and bitsandbytes with compatible versions
Working with the Hugging Face Hub: tokens, downloading models and private repositories
An overview of Unsloth Studio for no-code fine-tuning
Lab: run inference on the base model and record the results as a baseline

Section 4: Designing a Training Dataset

Instruction and conversation formats, and each model's chat template
Quality over quantity: diversity, consistency and examples at the edges of the task
Generating synthetic data with an LLM and having people review it
Points to watch with Thai data: word segmentation, token counts and consistent language
PDPA and removing personal data before training on organisational data

Section 5: Lab: Getting Data Ready for Training

Clean, deduplicate and filter out low-quality examples
Split into train, validation and test sets without leakage between them
Convert data to the chat template and check the token length of each example
Save the dataset to the Hugging Face Hub as a private dataset
Lab: prepare a Thai dataset for classification and question answering for a sample business

Day 2 Training and Evaluating Models

Section 6: Lab: Your First QLoRA Run with Unsloth

Load a model in 4-bit and add a LoRA adapter
Configure SFTTrainer and train on completions only
Track loss and GPU usage during training
Test the trained model and compare it with the baseline
Save just the adapter and load it back for use

Section 7: Hyperparameters and Reading the Loss Curve

Learning rate, epochs, batch size and gradient accumulation
Choosing rank and target modules to balance quality and memory
Reading training and validation loss to spot overfitting
Checkpoints, early stopping and tracking experiments with TensorBoard or W&B
Lab: retrain with different settings and compare the results systematically

Section 8: Lab: Fine-Tuning a Thai Model

An overview of Thai-capable models such as Typhoon, OpenThaiGPT and Qwen
Tokeniser efficiency for Thai and its effect on sequence length and cost
Checking the licence of a Thai model before choosing it as a base
Lab: fine-tune a Thai model to answer in the organisation's format and tone
Compare the result with prompting alone on the same model

Section 9: Lab: Hugging Face TRL and PEFT Directly

Write a training script with TRL and PEFT without Unsloth
Compare speed and memory with training through Unsloth
Use Accelerate for basic multi-GPU training
Structure the project to be repeatable: config, seeds and data versions

Section 10: Lab: Evaluating the Fine-Tuned Model

Build a test set from real work that never appeared in the training data
Task-specific metrics such as accuracy, F1 and exact match for classification and extraction
Use LLM-as-a-judge together with human review for free-text tasks
Check the model still handles general tasks and has not lost earlier abilities (catastrophic forgetting)
Lab: write a report comparing the base model, prompting alone and the fine-tuned model

Day 3 Preference Tuning, Deployment and Capstone

Section 11: Lab: Preference Tuning with DPO

Why tune for preferences when SFT alone is not enough
How DPO works and the format of chosen and rejected answer pairs
An overview of GRPO reinforcement learning for tasks with checkable answers
Lab: use DPO to make the model answer politely and concisely in line with company guidelines

Section 12: Lab: Merging and Exporting the Model

Keeping a separate adapter or merging it into the base model, and how to choose
Merge the adapter and save the full model to a private Hugging Face Hub repository
Convert to GGUF and choose a quantisation level
Check quality after quantising with the same test set
Write a model card covering training data, limitations and usage

Section 13: Lab: Putting the Model into Service

Create a Modelfile and run your own fine-tuned model on Ollama
Serve it on vLLM and load several LoRA adapters on one base model
Call it through an OpenAI-compatible API from existing apps
Lab: connect the fine-tuned model to a real workflow script for the sample business

Section 14: Licensing, Safety and Model Maintenance

Licences for base models and data, and rights for commercial use
Safety testing after training, such as answering what it should not and leaking data
Versioning data, adapters and models so you can roll back
Planning retraining when the data or the base model changes
Weighing the cost of training and serving against using an API

Section 15: Capstone: Fine-Tune a Model for Your Own Task

Pick a task from your own organisation or a prepared sample case
Prepare data, train and tune over at least two rounds
Evaluate against the baseline and conclude whether it is worth it
Export and run on Ollama or vLLM, then present the results for group review

Schedule & training options

For individuals — public rounds

No public rounds are open right now. Join the waiting list and we will contact you first when the next round opens, or ask us on LINE. Or call 02-570-8449 or 088-807-9770

Join the waiting list Ask about rounds

For organisations — in-house / private

Tailor the content to your team’s tools and projects
Your dates, at your office or live online
Quotation with tax ID for procurement

Corporate training quote

Instructors

Samit KoyomInstructor

Frequently asked questions

Who is LLM Fine-Tuning with LoRA for, and what background is needed?

Built for AI and ML engineers who need to adapt language models to their organisation's work · Python developers already using LLM APIs who want to start customising models themselves · Data scientists who want LLM fine-tuning skills for Thai-language data Background you should have: Confident Python and experience with Jupyter Notebook or Google Colab · Experience calling LLMs through an API and an understanding of prompts, tokens and context windows Not sure the fit is right? Talk to our team on LINE @itgenius or call 02-570-8449.

How much does LLM Fine-Tuning with LoRA cost and how long does it run?

THB 11,900 (currently THB 10,710 on promotion). The course runs 18 hours. The price excludes 7% VAT (for payment in a company's name). Pay by bank transfer to the company account, confirm it on our payment page, and we can issue the receipt or tax invoice in your company's name.

Do I get a certificate?

Yes. Everyone who completes the course receives a Certificate of Completion from IT Genius Institute. Each certificate carries its own number, and anyone holding that number can verify it online on our certificate page, so you can add it to your portfolio or pass it to HR as evidence of training.

Where does the training take place, and is there an online option?

You can attend onsite at IT Genius Institute or arrange to join online, and we also run it as a private in-house session for your team. Ask about dates and venues on LINE @itgenius or call 02-570-8449.

What if I fall behind or miss a session — can I retake it?

Yes. You may retake the same course free of charge in a later round, under the institute's conditions. Tell our team which course and round you attended, and we will check it and offer you the rounds that still have seats. Ask us on LINE @itgenius or call 02-570-8449.

How do I enrol, or request a quotation for my company?

Enrol online with the registration form on this page. You can register several attendees at once and enter your tax ID and billing address for the tax invoice. Or request a company quotation straight from the quote button. For anything else call 02-570-8449 or reach us on LINE @itgenius.