AI & LLMs
Explained

Artificial Intelligence is the broad field of making computers perform tasks that normally require human intelligence — understanding language, recognising patterns, making decisions. It is the outermost umbrella; every other concept covered here lives inside it.

The critical shift happened when we moved from hand-coded rules (“if the email contains the word ‘lottery’, mark as spam”) to learning from data (“show the system one million spam emails and let it figure out the patterns itself”). That second approach is Machine Learning. Deep Learning is a subset of ML using large neural networks — it powers virtually every state-of-the-art system you interact with today.

While LLMs dominate public attention, diffusion models handle image/video generation, audio models power voice products, and specialised vision models underpin medical imaging and self-driving. Most frontier models today merge several of these into a single multimodal system.

Models do not read text the way humans do. Before any processing happens, your input is split into tokens — sub-word chunks — using an algorithm called Byte-Pair Encoding (BPE). Common words become a single token. Rare or long words get split into several. The model then works entirely with these token IDs, never seeing raw characters.

This is why models struggle with simple character-level tasks. Ask Claude to count the letter “e” in “unbelievable” and it may get it wrong — because it processed un / bel / iev / able, four chunks, not eleven individual letters. It is not a reasoning failure; it literally never saw the characters. Similarly, asking a model to spell a word backwards is genuinely hard for the same reason.

Why it matters day-to-day: every API call charges you for input tokens + output tokens. A short email might be 100 tokens. A 50-page document could be 40,000. A long conversation history (which gets re-sent on every turn) can silently inflate your bill. Context windows are measured in tokens too — when you hit the limit, the model stops seeing your earlier messages.

Computers cannot do arithmetic on words. So before a model can process language, every token is converted into an embedding: a list of ~1,500 floating-point numbers. Think of it as a set of coordinates that places the word at a specific location in a very high-dimensional map.

The magic is what training does to those coordinates. The model learns to place words that appear in similar contexts near each other in this space — not because we programmed that, but because the training process pushed it there naturally. “Doctor” and “physician” end up nearby. “Paris” and “London” cluster together. “Happy” and “sad” are far apart.

This geometry is surprisingly rich. The famous example: king − man + woman ≈ queen. The direction from “man” to “king” encodes a kind of “royalty” concept. Apply that same direction to “woman” and you arrive near “queen.” You can navigate the space by arithmetic.

In practice, embeddings power semantic search: instead of matching keywords, a search system embeds your query and finds the documents whose embeddings are closest in the space. This is the core mechanism behind every RAG pipeline — it’s how the system finds relevant documents to inject into the model’s context.

Here is something most people do not know: AI models have no persistent memory between turns. There is no running diary the model writes to. When you send message 15 in a conversation, your application assembles the entire conversation history — every single message — and sends it to the model as one giant input. The model reads it all fresh, produces a reply, and forgets everything. The application stores the transcript; the model does not.

The practical consequence: every turn gets more expensive. A system prompt of 2,000 tokens plus 30 turns of 500 tokens each means turn 30 sends 17,000 input tokens. At typical frontier model pricing, that same question costs 17x more in tokens than it did at turn 1. For high-volume applications this adds up fast.

When the context limit approaches, something has to give. The three strategies above each have tradeoffs. Prompt caching — available in both the Claude and GPT APIs — stores the system prompt computation server-side so it does not need to be re-processed on every call, cutting input costs for that portion by up to 90%. For sessions that reuse the same large base context (e.g. all calls about the same 100-page document), caching is a significant saving.

A parameter is a single number inside a neural network — think of it as a dial or a knob. The model has billions of these dials. Training is the process of slowly adjusting all of them, turn by turn, until the model’s outputs get better. When training finishes, the dials are locked. When you use the model, every dial is fixed; inference just reads them.

Weights are the most important type of parameter. They control how much influence one neuron has on another — like deciding how much weight you give a friend’s recommendation. A weight of 0 means “ignore this input completely.” A high weight means “this matters a lot.” Biases are simpler: they are default offsets added before activation, setting a baseline before any input arrives.

Model size matters because more parameters = more capacity to store patterns. A 7B model can hold the conversational patterns needed for general chat. A 70B model can hold deeper domain knowledge. A 1T-parameter model can, in principle, hold highly nuanced expertise across thousands of domains simultaneously. But bigger models require more VRAM (GPU memory) to run, which is why a 70B model needs a high-end server while a 7B model can run on a consumer GPU or even a laptop with enough RAM.

The Transformer, introduced by Google in 2017, is the architecture that powers essentially every modern LLM. The core innovation is self-attention: when processing any word, the model can directly look at every other word in the input and decide how relevant each one is. Older architectures (RNNs) had to pass information step-by-step like a telephone game, losing signal over long distances. Attention removes that bottleneck entirely.

A single transformer layer has two components working in sequence. First, self-attention: every token asks “which other tokens in this sequence are most relevant to understanding me?” In the sentence “The animal didn’t cross the street because it was too tired,” when processing “it,” the attention mechanism learns to look back at “animal.” Second, a feed-forward network: applied to each token independently, this stores factual associations — essentially a compressed lookup table of world knowledge baked into the weights.

Multi-head attention runs this process several times in parallel with different learned perspectives. One head might focus on grammar (subject-verb agreement), another on co-reference (which pronouns refer to which nouns), another on semantic similarity. Their outputs are combined. Stack 96+ of these layers and you get an LLM. Early layers learn surface patterns; deeper layers develop abstract reasoning.

Pretraining is the expensive, large-scale first phase. The model is shown a token sequence, asked to predict the next token, then told the correct answer. The prediction error flows backwards through the network (backpropagation) adjusting billions of weights slightly. Repeat this process across trillions of tokens scraped from the internet, books, academic papers, and code. GPT-3 trained for weeks on thousands of specialised A100 GPUs at a cost estimated in the tens of millions of dollars. GPT-4 and Claude 3 are estimated to have cost $50M–$100M+ in compute alone. The result is a “base model” that is astonishingly good at predicting text — but weird to talk to. Ask it a question and it might just continue writing more questions.

Fine-tuning turns the base model into an assistant. Engineers curate thousands of example conversations showing ideal responses, then train the model further on just these. After fine-tuning, the model understands it should answer questions, follow instructions, and maintain a consistent persona. This is relatively cheap compared to pretraining — hours or days on a fraction of the hardware. LoRA (Low-Rank Adaptation) is a popular technique for fine-tuning only a small set of adapter weights, making it even cheaper.

RLHF (Reinforcement Learning from Human Feedback) is the final alignment step. Human raters compare pairs of model responses and pick the better one. A “reward model” learns these preferences, then the main model is trained with reinforcement learning to score higher. This is how models learn to be helpful, avoid harm, and be honest rather than just fluent. Anthropic’s Constitutional AI extends this: instead of (only) human ratings, the model uses a written set of principles to critique its own outputs.

A model can only learn from data it has seen. The earliest LLMs trained on a few billion tokens of web text. GPT-3 used 570GB of filtered internet data. By 2023, frontier labs had processed an estimated 10–15 trillion tokens — a substantial fraction of all high-quality text ever written in English on the internet. This is what researchers call the data wall: the supply of new, high-quality human-written text is effectively finite and largely consumed.

The industry’s response has been synthetic data: using existing frontier models to generate the training data for the next generation. A powerful model can produce thousands of worked mathematical proofs, detailed code explanations, or step-by-step reasoning traces — exactly the kind of high-quality process data that improves reasoning. DeepSeek R1’s reasoning traces, for example, have been widely used to improve smaller open-weight models. This is sometimes called “models teaching models” or knowledge distillation at scale.

The other response is licensed data partnerships. OpenAI, Anthropic, and Google have signed deals with Reddit, Associated Press, the Financial Times, Shutterstock, and major scientific publishers. The New York Times sued OpenAI for copyright infringement in 2023; the case settled in 2025. The legal landscape around training on copyrighted content remains contested, particularly in the EU under the AI Act, which requires disclosure of training data sources.

The context window is the model’s entire working memory for a single API call — everything it can see at once. It includes the system prompt, conversation history, any retrieved documents, and the current message. Everything outside that window is invisible; there is no peripheral vision.

Context windows have grown dramatically: GPT-3 launched with 4,096 tokens in 2020. Claude 4.6 supports 200,000 tokens standard and 1 million in beta. Llama 4 Scout can technically handle 10 million. But bigger is not always better: research consistently shows a “lost in the middle” effect — models do well at the beginning and end of long contexts but performance degrades on information buried in the middle. And a 1M-token context does not cost 5× a 200K context — at that scale, attention computation becomes extremely expensive.

Note: context window limits, pricing tiers, and beta availability change frequently. The figures above reflect April 2026 — check each provider’s documentation before relying on specifics.

The KV cache (key-value cache) is a critical optimisation: computed attention states for all previously processed tokens are stored so they do not need to be recalculated as each new output token is generated. Without it, generating a 500-token reply would require re-running the full attention computation 500 times over the entire input. The KV cache is why streaming feels fast even for long contexts.

Inference is the process of running a trained model to generate output. You send a prompt; the model does a forward pass through all its layers; it produces a probability distribution over every possible next token; one token is sampled; that token is appended to the input; repeat until done. This is called autoregressive generation. The model is literally writing one token at a time, each new token informed by all previous ones.

Temperature controls how spread out the probability distribution is before sampling. At temperature 0, the model always picks the single most likely token — deterministic and often repetitive. At temperature 1 (the default), it samples from the distribution as-is. Above 1, the distribution flattens, making less likely tokens more probable — which increases creativity but also increases the chance of odd outputs. For code generation, use low temperature (0.1–0.3). For creative writing, try 0.8–1.2.

On the infrastructure side: models are stored as billions of floating-point numbers and run entirely on GPUs, which can do massively parallel matrix multiplication far faster than CPUs. The bottleneck is GPU memory (VRAM) — the entire model must fit in VRAM to run efficiently. A 7B model at 16-bit precision requires about 14GB of VRAM. A 70B model needs ~140GB, requiring multiple high-end GPUs (like two H100s). This is why most frontier models are cloud-only: no consumer GPU can hold them. When you call Claude or GPT via API, your request is routed to a server cluster running many thousands of dollars of GPUs.

AI is a hardware-constrained field. Models are stored as billions of floating-point numbers and run on GPUs — chips designed for the massively parallel matrix multiplication that neural networks require. The key constraint is VRAM (video RAM): the entire model must fit in GPU memory to run at full speed. A 70B parameter model at 16-bit precision requires approximately 140GB of VRAM. No consumer GPU comes close; you need at least two NVIDIA H100s.

This creates a significant compute moat. NVIDIA currently manufactures the overwhelming majority of AI-grade GPUs. H100s cost $25,000–$40,000 each; a training cluster for a frontier model contains 10,000–100,000 of them. Only a handful of organisations — OpenAI, Google, Anthropic, Meta, Microsoft, and a few well-funded startups — can afford to train frontier models from scratch. Everyone else builds on top of them.

Quantisation makes large models accessible on smaller hardware by reducing numerical precision: instead of storing each weight as a 32-bit float, you use 8-bit integers (INT8) or 4-bit (INT4). A 70B model that needs 140GB at full precision fits in ~35GB at 4-bit quantisation — within reach of a high-end workstation. Quality degrades slightly, but for many tasks the difference is negligible. Tools like llama.cpp and Ollama handle quantisation automatically.

Beyond NVIDIA, Google TPUs (Tensor Processing Units) are custom silicon designed specifically for neural network workloads. Google trains Gemini on TPU pods internally. Apple’s Neural Engine in M-series chips enables on-device inference for smaller models. AWS Trainium and Inferentia are Amazon’s custom inference chips used in Bedrock. The hardware landscape is diversifying, but NVIDIA retains a dominant position through its CUDA software ecosystem as much as its hardware.

Standard LLMs respond immediately: prompt goes in, tokens come out in one pass. Thinking modes change this contract. The model is given a budget of “scratchpad” tokens it can use internally to reason through a problem before producing the final visible answer. Like a student who scribbles working on paper before writing the clean answer — the process improves the result, even if you only see the conclusion.

The tradeoffs are consistent across all implementations: thinking costs more (you are billed for the scratchpad tokens), thinking is slower (TTFT increases significantly), but thinking is dramatically better on hard problems — multi-step maths, complex debugging, ambiguous decisions, long-horizon planning. For simple factual questions or casual conversation, standard mode wins on speed and cost.

A key nuance: Claude’s interleaved thinking is architecturally distinct. Older thinking models thought once at the start, then acted. Claude 4 models can think, call a tool, think about the result, call another tool, think again — reasoning is woven through the entire action loop. This is particularly powerful for agentic coding and research workflows where intermediate results change what the next step should be.

Note: thinking mode names, effort levels, tier availability, and API parameters change with every model release. The table above reflects April 2026. Treat it as orientation, not reference documentation — check each provider’s current API docs before building.

Gemini’s parallel approach is also genuinely different. Rather than following a single chain of reasoning, it spawns multiple reasoning branches simultaneously — more like a team of collaborators debating than a single thinker working through steps. This is computationally expensive (Deep Think takes minutes) but produced gold-medal performance at the 2025 International Mathematical Olympiad.

The same model with a vague one-line instruction versus a detailed system prompt with examples, constraints, and format specifications behaves like two different products. Prompt engineering is the craft of reliably eliciting the behaviour you want without retraining.

Key principles that work across all models: be explicit about format (say “respond in JSON with keys: title, summary, risk_level” not “give me some JSON”); use examples not just descriptions; give the model permission to say it doesn’t know (models hallucinate less when explicitly told not to make things up); separate instructions from content using delimiters like XML tags; specify length (“in 3 bullet points” or “in under 100 words”).

Generative AI produces content in response to a prompt. One input, one output, done. You decide what to do with the output. The model has no agency, takes no actions, and changes nothing in the world.

Agentic AI pursues a goal over multiple steps, calling tools and observing results at each stage. It does not just write the email — it researches who to write to, drafts the email, uses an API tool to send it, and waits for a reply. The model acts; the world changes. Actions can be irreversible: once a file is deleted, an email sent, a payment submitted, or a database record overwritten, undoing it is hard or impossible.

This is why guardrails matter more for agentic systems: human-in-the-loop approval for irreversible actions, tight tool permission scoping (the agent should only have access to tools it actually needs), and sandboxed environments for code execution are all active engineering concerns in 2026.

The core problem RAG solves: an LLM has a knowledge cutoff date and knows nothing about your private data. It will confidently make up answers about your company’s policies if you ask about them — because it has never seen them.

RAG fixes this by adding a retrieval step before every generation. Your documents are pre-processed: split into chunks, converted to embeddings, and stored in a vector database. When a question arrives, the system converts the question into an embedding too, then finds the chunks with the most similar embeddings (i.e. the most relevant sections), and injects those chunks directly into the model’s context alongside the question. The model now answers from your actual documents, not from memory.

Why not just fine-tune instead? Because RAG is faster, cheaper, and updatable. Fine-tuning bakes knowledge into the weights — it takes days and thousands of dollars, and you have to redo it every time your data changes. A RAG database can be updated in seconds: add a new document, re-chunk and re-embed it, done. Your model immediately has access to the new information without any retraining.

MCP is an open standard launched by Anthropic in November 2024 — often described as “USB-C for AI integrations.” Before MCP, connecting an LLM to a database, Slack, or a file system required custom code per integration per model. MCP defines a single common protocol so any MCP-compatible AI host can plug into any MCP server without custom glue code.

By March 2025 OpenAI adopted it, April 2025 Google DeepMind followed. In December 2025 Anthropic donated MCP to the Agentic AI Foundation (AAIF) under the Linux Foundation, co-governed by Anthropic, OpenAI, Block, AWS, Google, Microsoft, and Cloudflare. It is now a vendor-neutral standard, not an Anthropic product. As of April 2026: 10,000+ public MCP servers and 97 million monthly SDK downloads, per Anthropic’s own foundation announcement — not independently audited figures. Stripe, GitHub, Notion, Hugging Face, and Postman all ship official MCP servers.

An AI agent is an LLM placed in a loop: it receives a goal, decides which tool to call, calls it, observes the result, decides the next step, and repeats until done. The model provides the reasoning; tools provide the capability to act. A multi-agent system adds specialisation: a planner agent breaks the task into subtasks, delegates them to specialist agents (coder, researcher, reviewer), and synthesises their outputs.

Agentic frameworks in 2026 include LangChain, AutoGen, CrewAI, and Anthropic’s native tool-use API. The key engineering challenges are error cascades (one bad tool call derails the whole chain), cost (many inference calls), latency (sequential chains are slow), and safety (preventing unintended irreversible actions).

The shift from chat to CLI agent is fundamental. In a chat UI, you are the execution layer — you read suggestions, decide whether they are correct, and manually apply them. In a CLI agent like Claude Code, the model is the execution layer — it reads your files, makes changes, runs your tests, reads the output, and iterates. This unlocks long-horizon tasks that would take dozens of manual chat turns to coordinate.

Most AI failures are not dramatic. They are quiet and mundane: a confident wrong answer that nobody checked, a workflow that works 95% of the time until the 5% causes an incident, a tool used for a task it was never designed for. Understanding the failure patterns is as important as understanding the capabilities.

The guide explains how to build AI systems. Evaluation is how you know if they are actually working. It is one of the most underrated disciplines in applied AI — teams that skip it discover problems in production instead of in testing.

Benchmarks are standardised test sets that let you compare models objectively. MMLU tests breadth of knowledge across 57 academic subjects. GPQA tests PhD-level science reasoning. SWE-bench tests whether a model can fix real GitHub issues. AIME tests competition-level mathematics. These numbers appear in every model announcement — GPT-5 scores 94.6% on AIME 2025; Claude Sonnet 4.5 scores 77.2% on SWE-bench. Important caveat: most benchmark scores are self-reported by the labs that built the models. Independent replications sometimes differ. The risk is also that labs optimise for benchmark performance specifically, which may not reflect real-world quality. When a benchmark gets “solved,” the community creates harder ones.

LLM-as-a-Judge is the dominant approach for evaluating open-ended outputs at scale. You define rubrics (“Is this response accurate? Is it helpful? Does it stay on topic?”), then ask a powerful model (usually GPT-5 or Claude Opus) to score another model’s outputs against those rubrics. This scales to millions of examples cheaply — far more than human annotation budgets allow. The main caveat: the judge model brings its own biases. A Claude judge may systematically prefer Claude-style responses. Calibration against a ground-truth human-labelled sample is good practice.

For RAG pipelines specifically, the RAGAS framework defines metrics that matter in production: faithfulness (does the answer only make claims supported by the retrieved context, or does it hallucinate?), answer relevance (does the answer address the question?), and context precision (were the retrieved chunks actually useful?). Running these metrics continuously on a sample of production traffic is how you detect when retrieval quality degrades — which happens when your document database goes stale or your embedding model is updated.

Swapping one model for another — even within the same family — is one of the most impactful changes you can make, and one of the most risky. Same prompt, different model = different output. Not slightly different: potentially very different in tone, length, format, and factual content. A pipeline tuned for GPT-5.1 may break silently when upgraded to GPT-5.4 because the response structure changed.

The main dimensions that change between models: capability (newer models reason better but cost more), speed (smaller models respond faster), cost (frontier models cost 10–20× more per token than mini/flash variants), context window (if you built around 128K, upgrading to a 1M model changes your session design), and behaviour (alignment tuning differs — a model that reliably refuses certain requests in one version may handle them differently in the next).

In production, model version pinning is standard practice: you use a specific version string (e.g. claude-sonnet-4-6, not claude-latest) so your application does not silently break when a new model is deployed. Automatic upgrades are convenient for casual use but dangerous for pipelines where output format matters.

The gap between open and closed models has narrowed dramatically. Llama 4 Maverick (400B MoE, 10M context, free to run) competes with frontier closed models on most benchmarks at a fraction of the API cost for high-volume use. For organisations with data residency requirements or sensitive data, open-weight models hosted on-premises are increasingly the right call in 2026.

The Training section explains how models are trained. This section covers how they are made safe and useful specifically — which requires its own deliberate engineering on top of raw capability.

RLHF was the breakthrough that turned GPT-3 (capable but unreliable) into InstructGPT (helpful and relatively safe). It requires a separate reward model trained on human preference data, then a full RL training loop — expensive and tricky to stabilise. DPO (2023) achieved similar alignment results by reframing the problem as a supervised learning task directly on preference pairs, removing the need for a separate reward model and RL entirely. Most frontier labs now use DPO or variants for the bulk of alignment work, with RLHF reserved for specific capability dimensions.

Red teaming is the adversarial counterpart: teams (human and automated) spend months before each model release systematically trying to elicit harmful outputs, test policy edge cases, and find failure modes under unexpected inputs. Findings feed back into further fine-tuning and guardrail improvements. It is a continuous process — new jailbreak techniques emerge after every deployment.

Important nuance: alignment and guardrails are different layers. Alignment is baked into the model weights through training. Guardrails are runtime filters applied to inputs and outputs. A well-aligned model is intrinsically reluctant to produce harmful content. Guardrails catch cases where the training is insufficient. Both are needed; neither alone is sufficient.

GPT-5 (with thinking) produces roughly 80% fewer factual errors than GPT-4o on open-ended fact-seeking tasks, according to OpenAI’s HealthBench evaluations. Progress is real. But HealthBench is an OpenAI-produced benchmark, not an independent external audit — treat the specific figure as directionally useful rather than a neutral measurement. Hallucinations remain an active problem particularly for questions outside the training distribution, highly specific facts, and recent events after the knowledge cutoff.

Current frontier models are already superhuman in narrow domains: GPT-5 achieves 94.6% on AIME 2025 (advanced competitive maths) and 74.9% on SWE-bench (real GitHub coding tasks). Gemini Deep Think achieved gold-medal standard at the 2025 International Mathematical Olympiad. These scores come from lab-reported evaluations and should be read as indicators of capability direction rather than independently audited measurements. These are not human-level general intelligence — but they are capabilities that did not exist two years ago.

The most significant near-term shift is from generation to reasoning and action: models that think before they answer, call tools autonomously, and operate in long-horizon agentic loops. The question is not whether these systems will transform knowledge work — they already are. The question is how quickly alignment and safety research can keep pace with capability growth.