Original URL: https://getagenttools.com/blog/what-is-an-llm/

# What is an LLM? The engine under every AI agent, minus the mysticism

20 Aug 2026Marek.

- glossary

Second entry in the glossary. If [the first one](https://getagenttools.com/blog/what-is-ai/) drew the big circle around AI, this one points at the engine inside most of what is currently sold under that label: the large language model. Nearly every chatbot, coding agent and "AI feature" bolted onto a product this year has one of these at its center, doing one narrow thing extremely well.

 

- **The plain definition.** An LLM is a model trained on enormous amounts of text to do one thing: predict the next small piece of text, given everything before it.
- **The surprise.** That one narrow skill, scaled hard enough, produces translation, summaries, working code and multi-step reasoning as side effects.
- **What "large" means.** The size of the learned network, billions of internal parameters, the adjustable numbers the training run tunes. Not the size of its answers.
- **The two limits that matter daily.** A context window: the model only sees what fits in front of it. And a training cutoff: it knows next to nothing newer.
- **The biggest myth.** That the model itself looks things up. There is no database inside; the knowledge is baked into the parameters, which is why it can be confidently wrong.

  

## The plain definition

 

A large language model is a neural network trained on a very large body of text with a single objective: given a sequence of text, predict what comes next. That is the whole pretraining goal. Everything else, answering questions, writing code, holding a conversation, following instructions, emerged because predicting text well at scale turns out to require absorbing an enormous amount of structure: grammar, facts, styles, patterns of argument, conventions of programming languages.

 

At use time the prediction task is the same: your prompt becomes the sequence, the model predicts the continuation, one small piece at a time. The conversation format, the helpful persona, the refusal to do certain things, all of that is shaped on top of the same next-piece prediction engine by additional tuning.

 

## What it means in practice

 

The prediction framing sounds academic until you notice how much everyday confusion it explains.

 

Why it can be brilliant and wrong in the same paragraph: it produces the most plausible continuation, and plausible tracks true only most of the time. Why it has no memory of you: the model is frozen after training, and what feels like memory is the conversation being re-fed to it every turn, within a fixed context window it cannot see past. Why it "knows" nothing recent: the knowledge is a snapshot from when training data was collected, the cutoff, and anything newer must be handed to it in the prompt or through tools.

 

In practice, every serious LLM deployment is plumbing around those facts: retrieval to feed it current information, context management to fit the right things in the window, and verification because plausible is not a synonym for correct. The craft of that plumbing has its own name now, and [I audited my own setup against it](https://getagenttools.com/blog/claude-md-audit/) recently.

 

## What people get wrong

 

| People say | What is true |
| --- | --- |
| It searches the internet for answers | By itself it searches nothing. It generates from learned parameters; live information arrives only via tools or pasted context |
| It remembers our last conversation | The model is stateless. Any continuity is software around it re-supplying history or notes |
| Bigger model, better results, always | Bigger is often better and, within one model family, usually costlier and slower. Many production jobs run best on small models with good context |
| It understands like a person | It models the statistics of language deeply enough that the difference often does not show. Often is not always |
| The chatbot is the LLM | The chatbot is a product wrapping the LLM with tuning, tools, safety layers and an interface |

 

## One example from my own work

 

The clearest demonstration that an LLM is an engine, not an application: in my own stack, the same family of models does three jobs that share nothing on the surface. It writes and edits code inside an agent. It sits as an adversarial judge that tries to break my blog drafts before publication. And once a morning it reads a table of health metrics and writes a short status brief. Same engine, three roles. What changes is everything around it: the context each one is given, the tools each one may call, and how much anyone trusts the output unverified. The judge's verdicts get cross-checked by a human queue; the status brief is forbidden from taking any action; the coding agent works behind a test gate. Choosing the model turned out to be the smallest decision in each of the three.

 

## Where to go deeper

 

The unit these models read, produce and bill in has its own glossary entry coming next, and their signature failure mode gets one after that. What turns this engine into a worker, tools plus a loop plus a stop rule, is [What is an AI agent](https://getagenttools.com/blog/what-is-an-ai-agent/). Already live: how an LLM-driven agent reaches your real systems is [What is an MCP server in AI](https://getagenttools.com/blog/what-is-an-mcp-server-in-ai/), the context plumbing around the engine is the subject of [the CLAUDE.md audit](https://getagenttools.com/blog/claude-md-audit/), and the switch that decides which procedures the engine may trigger on its own is [disable-model-invocation](https://getagenttools.com/blog/disable-model-invocation/). The umbrella term above all of this is entry one: [What is AI](https://getagenttools.com/blog/what-is-ai/).

Put this into practice

[See the tool list](https://getagenttools.com/tools/)[Train your team](https://getagenttools.com/#training)