Section 11: Lab: RNN, LSTM and GRU for Sequence Data
- Sequence data and shaping it into windows for training
- RNNs and the vanishing gradient problem
- How LSTM and GRU handle long-term memory
- Forecast a time series such as daily sales and compare against a baseline
Section 12: Transformers and the Attention Mechanism
- The limits of RNNs that led to attention
- Self-attention, multi-head attention and positional encoding
- BERT-style encoders compared with GPT-style decoders
- Tokenisation, embeddings and word segmentation in Thai
- Lab: build a small Transformer block in PyTorch
Section 13: Lab: Sentiment Analysis with Hugging Face
- Use a pipeline to run sentiment analysis with a model from the Hugging Face Hub
- Prepare Thai review data with the Datasets library and a tokeniser
- Fine-tune a Thai language model such as WangchanBERTa with Trainer
- Evaluate with F1-score and analyse the sentences the model gets wrong
- When to fine-tune a small model and when to call an LLM through an API
Section 14: Lab: Putting Models to Use
- Export PyTorch and Keras models to ONNX and run them with ONNX Runtime
- Shrink and speed up models with basic quantisation
- Build a demo page with Gradio and share it on Hugging Face Spaces
- Care points on bias, personal data, PDPA and the copyright of training data
Section 15: Workshop: Capstone Project
- Choose an image, text or time series problem and define the metrics
- Choose an architecture or pretrained model that fits the data
- Train, tune and compare against a baseline
- Export the model, build a demo page and present the results with their limits