AI Glossary

Every field grows its own shorthand, and AI's is thicker than most, partly real technical vocabulary, partly marketing borrowing words to sound more advanced than the thing actually is. This page exists so none of the other articles have to stop and explain a term mid-argument. Each entry below is deliberately short, plain English first, a pointer to the deeper article second, for whoever wants the full picture.

Core mechanics

Token

The small chunk of text a model actually reads and generates, roughly ¾ of a word on average. Pricing, context windows, and speed are all measured in tokens, not words or characters.

Embedding

A list of numbers that represents the meaning of a piece of text, positioned so that similar meanings end up as nearby numbers. What makes semantic/vector search possible, see Keyword search vs. vector search.

Context window

The maximum amount of text (measured in tokens) a model can hold in one request. It keeps growing across model generations, but the limit is still real, send more than it can hold, and the excess is silently dropped, not rejected. See The Reality Check.

Temperature / sampling

The setting that controls how much randomness goes into picking the model's next word. Low temperature gives more consistent, repeatable answers; higher temperature gives more varied ones. See How AI Actually Makes Decisions.

Inference

The model actually generating a response to a request, as opposed to training, which is the (much more expensive, one-off) process of building the model in the first place. See How AI Actually Learns.

Fine-tuning

Further training an existing model on your own data so it specializes toward your task, instead of using it as-is or building a model from scratch. One of the four paths in Build vs. Buy vs. Rent.

Hallucination

A confident, fluent, completely wrong answer. Not a crash and not an exception, the model producing something structurally like a good answer that just isn't true. The reason AI failures don't announce themselves the way normal software bugs do.

Retrieval and grounding

RAG (Retrieval-Augmented Generation)

Grounding a model's answer in data retrieved specifically for the current request, rather than letting it answer purely from what it learned during training. See RAG.

Semantic / vector search

Finding results by meaning rather than exact words, using embeddings to match "similar" rather than "identical." The default assumption for RAG, though not always the right first choice, see Keyword search vs. vector search.

Prompt caching

Reusing the cost/latency of a previous, identical (or overlapping) chunk of input instead of paying for it again on every call. One of the concrete cost levers in The Reality Check.

Tools, agents and integration

MCP (Model Context Protocol)

The open standard for connecting a model to real tools and data, so any MCP-compatible client can reuse the same server instead of every app building its own one-off integration. See MCP.

Agent

A model that doesn't just answer once, but decides on its own whether to call a tool, fetch more context, or take another step before finishing, the line between "generation with context" and something actually making decisions.

The edge

The boundary where data first enters your system: right after you receive a response (from a model or any external source), before it's passed along to the rest of your code. Cleanup, timezones, units, encoding

• belongs here, done once, so nothing downstream has to defend against messy data on its own. See Predictable Failure Patterns.

JSON mode

A provider-level setting that constrains a model's output to valid JSON, more reliably than just asking for it in the prompt. See Predictable Failure Patterns.

Production and cost

Latency

How long a request actually takes to come back. Talked about in terms of P95 (the worst case that still affects 1 in 20 users) rather than the average, because users feel the tail, not the mean. See The Reality Check.

Drift

When a model's output format, tone, or accuracy quietly shifts over time without any error or warning, a provider updates a model behind the scenes, or accuracy degrades on inputs that have changed since training. See Predictable Failure Patterns.

Batch processing

Sending a group of requests to be processed together, usually at a discount, in exchange for not needing an instant response. Contrast with real-time, interactive calls.

Quality signal

Any ongoing measurement (accuracy checks, confidence scores, a scored rubric) used to catch a model's output getting worse over time, before a user notices. See The Reality Check.