LLMs Lecture Series
JP Ebejer
How Do Large Language Models (LLMs) Work?
Abstract
The rapidly evolving landscape of Natural Language Processing (NLP) has been transformed by the advent of new, powerful, neural network architectures (Transformers) and, particularly, the development of large language models (LLMs) based on Transformer technology. This lecture series offers an in‐depth exploration of the theoretical and practical underpinnings that make modern models such as ChatGPT and other advanced LLMs possible.
The lecture series begins with an introduction to the core principles of NLP and the evolution from symbolic systems to connectionist models. Traditional vector space methods, which treat data as numerical representations within high‐dimensional spaces, serve as the foundation for many modern AI algorithms. We discuss the rationale behind representing text as vectors and outline the advantages that continuous representations offer over classical symbolic approaches. This initial section also revisits linear algebra and the fundamental machine learning concepts that are essential for understanding how vector space models function in NLP contexts.
Particular emphasis is placed on word embeddings—a breakthrough that redefined how semantic and syntactic relationships are captured numerically. We will trace the evolution of textual representations from one-hot encodings and count-based models to distributed representations. In particular, we focus on the Word2Vec model, its mechanisms (such as the Continuous Bag-of-Words and Skip-Gram training methods), and its advantages in learning semantic relationships from vast text corpora. This discussion extends to alternative embedding methods, including GloVe, FastText, and contextual embedding models such as BERT, which incorporate dynamic word representations based on context. The objectives include understanding both the historical context and the practical challenges of embedding words as numeric values.
In the last two sessions of the series, we deep-dive into the architecture of the GPT-3 and GPT-3.5 Transformer models — the driving force behind LLMs such as ChatGPT (version 3.5), and emerging successors. The sessions will dissect the Transformer paradigm – particularly the GPT-3 Transformer model from OpenAI. We explain, in quite some detail, positional encodings, batch creation, the transformer block including the Q-K-V multi-head self-attention mechanism, and the linear projection layer. Participants will study the technical nuances of model training, including the role of Q-K-V multi-head attention, feed-forward layers, and the significance of residual connections and normalization. Detailed analysis of GPT-3’s architecture, with its 175 billion parameters and the implications of scaling, will be provided. Moreover, emerging properties such as zero-shot learning, few-shot learning, and the interplay between scale and model capabilities will be critically examined, with an outlook on innovations such as Mixture of Experts (MoE) and potential future developments. At the end of the last lecture, we will also cover LLM training and fine-tuning, hallucination, and new developments such as ground-truth databases to mitigate hallucination.
The curriculum is designed to bridge the gap between the theoretical and algorithmic foundations of the self-attention transformer architecture. Attendees will gain insight into how large-scale models are trained on massive datasets, and how attention mechanisms empower these models to capture intricate language patterns.
The LLMs Lecture Series consists of six two-hour lectures with Q&A after each lecture.
Module 1 – Introduction to LLMs (4 hours – 2 lectures)
The first four hours of the lecture series introduce AI and Machine Learning and the advent of Large Language Models such as ChatGPT.
· Is Human Intelligence Turing Computable?
· What is AI?
· What is Machine Learning?
· What are Neural Networks?
· Daniel Dennett and the Turing computability of human intelligence
· The Chinese Room Experiment
· The Turing Test
· Vector Space Models
· Language Models
· The Rise of the Transformers
· Q-K-V Self Attention
· Is ChatGPT just guessing?
· Zero-shot and Few-Shot Learning
· The Capacity and Complexity Argument
· Emergent Properties in LLMs
· Are LLMs Sentient?
· Mixture of Experts (MoE) models
· Chain of Thought (CoT) models
· World models
· My Personal Beliefs
· Generative AI Disruption
Module 2 – Preliminary Concepts (2 hours – 1 lecture)
The lecture introduces the main ideas and concepts that are useful for understanding how LLMs work.
· Multi-Layer Perceptrons and modern Artificial Neural Networks
· The Error Backpropagation Algorithm
· Generative AI
· Sequence Modelling
· Loss Functions
· Softmax
· Autoregressive Models
· Auto-encoders
· Encoder-Decoder Models
· Named Entity Recognition (NER)
Module 3 – Word Embeddings (2 hours – 1 lecture)
This lecture introduces and discusses word embeddings. This is fundamental in LLMs.
· Why do we need Word Embeddings?
· Words to Numbers
· Distributional Semantics
· Scalar and One-Hot Encodings
· Words as Feature Vectors
· Revisiting the Encoder-Decoder Model
· Word2Vec – The Game Changer
· GloVe
· How Word2Vec works
· BERT and Variants
· Using Word2Vec in Gensim
Module 4 – How ChatGPT Works and is Trained (4 hours – 2 lectures)
In this module, over 2 lectures, we deep dive into the inner workings of ChatGPT and explain how the model works and is trained.
· GPT-3 Word Embeddings
· The Transformer Model
· Encoder-Decoder Transformers
· Self-Attention Transformers
· Positional Encodings
· Q-K-V Self-Attention
· Multi-head Attention
· Feed-Forward Layers
· Residual Connections & Normalization
· Learning Objective & Loss Function
· Projection Layer and Model Parameters
· Pre-Training ChatGPT & Fine-Tuning
· Distributed Training
· Hallucination
· Is Attention Really All We Need?
· Knowledge Graphs and the Way Forward
· Chain of Reasoning models
Prerequisites
The objective of the Lecture Series is not to explain how to use LLMs such as ChatGPT but, rather, to explain how LLMs work (the Q-K-V Self-Attention Transformer architecture), the training methodology used, why they hallucinate, how they are being developed, and whether LLMs can lead us to AGI – human level intelligence.
The first two lectures are suitable for attendees who do not have a computing background. The remaining four lectures are targeted at attendees who have a technical background in computing, engineering, networking, or ICT in general.
Lectures
Lectures will be held at the MDIA premises:
Twenty20 Business Centre
Triq l-Intornjatur,
Zone 3
Central Business District, Birkirkara
CBD 3050
The lectures, two per week, are from 17:45 to 20:00 with some time for Q&A after each lecture.
|
Week 1 |
Tuesday 1st September |
Thursday 3rd September |
|
Week 2 |
Public holiday |
Thursday 10th September |
|
Week 3 |
Tuesday 15th September |
Thursday 17th September |
|
Week 4 |
Tuesday 22nd September |
|
Registration (opens 30th June 2026 online)
Registration is free and places are allocated on a first-come, first-served basis.
Website:
https://dihubmt.eu/training-courses/llm-lecture-series-how-large-language-models-work-round-3/
Registration:
Registration
- LLM Lecture Series (MDIA): How Large Language Models Work Round 3 – Fill
out form
Speaker
Lectures will be delivered (in English) by John Abela. John Abela is a professor in the Faculty of ICT. His main research interests are AI, Machine Learning (including Deep Learning), Machine Vision, computational complexity and NP-completeness, search and optimization algorithms, Large Language Models (LLMs), and Quantum Computing.
All lecture slides will be made available to registered attendees.
The contents of this email are subject to these terms.