NVIDIA Generative AI and LLM Associate (NCA-GENL)

Section 1: Lab: The NCA-GENL Exam and Setting Up Your Tools

• Exam format, duration and the weighting of all five domains
• Open Google Colab and choose a GPU runtime
• Join the NVIDIA Developer Program and create an API key on build.nvidia.com
• Lab: call your first model through the OpenAI-compatible API from Python

Section 2: Lab: Machine Learning Fundamentals You Must Know

• Supervised and unsupervised learning, feature engineering and train, validation and test splits
• Overfitting, underfitting and cross-validation
• Accuracy, precision, recall, F1 and comparing models statistically
• Lab: compare text classification models with scikit-learn

Section 3: Lab: Data Analysis and Dataset Preparation

• Explore large datasets and find trends and relationships with pandas
• Build charts that explain data and experiment results
• Clean, deduplicate and check the token length distribution of a dataset
• Lab: prepare a text dataset for RAG and fine-tuning

Section 4: Lab: Deep Learning, Tokenisation and Embeddings

• Neural networks, loss, backpropagation and why GPUs make this fast
• BPE, WordPiece and SentencePiece tokenisation and their effect on Thai text
• Word embeddings, sentence embeddings and cosine similarity
• Lab: tokenise text and measure sentence similarity with embeddings

Section 5: Transformer Architecture and LLMs

• Self-attention, multi-head attention and positional encoding
• Encoder, decoder and encoder-decoder models with examples of each
• Context windows, the KV cache and mixture of experts in plain language
• Pre-training, instruction tuning and alignment with RLHF or DPO

Section 6: Lab: Prompt Engineering and Controlling Output

• Zero-shot, few-shot, chain-of-thought and system prompts
• How temperature, top-p, top-k and output length affect results
• Enforce JSON output and validate it before use
• Lab: compare several models from the API catalogue on the same task

Section 7: Lab: RAG, Chatbots and Summarisers

• RAG components: chunking, embedding model, vector database and reranker
• Choose a text embedding model that fits the language and the task
• Chatbots that keep conversation context, and summarisers for long documents
• Lab: build document question answering with FAISS and the NVIDIA API

Section 8: Lab: Fine-Tuning and PEFT

• When to fine-tune and when prompting or RAG is enough
• Full fine-tuning versus LoRA and QLoRA, and an overview of NVIDIA NeMo
• Key hyperparameters: learning rate, epochs, batch size and rank
• Lab: fine-tune a small model with LoRA on Colab

Section 9: Lab: Evaluation and Experimentation

• Perplexity, BLEU, ROUGE, BERTScore and LLM-as-a-judge
• Standard benchmarks and building your own task-specific test set
• Design A/B experiments and record results so they can be repeated
• Lab: evaluate a summariser before and after changing the prompt and model

Section 10: Deployment and Inference Optimisation

• NVIDIA NIM, TensorRT-LLM and Dynamo-Triton (formerly Triton Inference Server)
• FP8, INT8 and INT4 quantisation and estimating GPU memory
• Batching, latency, throughput and monitoring systems as they scale
• Lab: measure latency and throughput of the API and a small model on Colab

Section 11: Lab: Trustworthy AI and NeMo Guardrails

• AI ethics principles, bias in data and models, and how to reduce it
• Privacy, consent and PDPA for data used in training or sent to a model
• Hallucination, model cards and technologies that improve trustworthiness
• Lab: add input and output rails to a chatbot with NeMo Guardrails

Section 12: Workshop: Capstone and Exam Review

• Capstone: a RAG chatbot with evaluation and guardrails
• Key content summarised by the weighting of each domain
• Practise exam-style questions and pace yourself for a one-hour exam
• A revision plan and free NVIDIA learning resources for after the course