LLM Terms Without the Buzzwords

In a surprisingly short time, a whole vocabulary has formed around LLMs: tokens, parameters, context windows, reasoning models, agents, RAG, MCP, open weights, inference, prompt caching, or mixture of experts. And somewhere in between, a provider is also selling credits.

The problem is not that these terms are meaningless. Quite the opposite: many of them describe real and important technical concepts. Others are product terms, categories, or billing models. Still others come from research and engineering but are now used so broadly that two providers using the same word do not necessarily mean exactly the same thing.

That can quickly make this field look like an entirely new technical universe, even though much of it can be explained surprisingly simply. Before we go deeper into agentic work, it makes sense to clear away some of the buzzwording first — not to make fun of the terms, but so that we know what we are actually talking about later.

AI – Artificial Intelligence

Artificial Intelligence, or AI, is first of all a very broad umbrella term. It includes computer systems that handle tasks we associate with abilities such as perception, language processing, planning, pattern recognition, problem solving, or decision-making.

A chess program can fall under AI just as much as an image-recognition system or a Large Language Model. That distinction matters because people often say “AI” today when they actually mean an LLM or generative AI.

AI is not automatically an LLM. An LLM is simply one particular class of AI model.

Machine Learning

Machine Learning refers to methods in which the desired behavior is not encoded entirely as explicit rules. Instead, a model is adjusted using data so that it learns relevant patterns from that data.

In conventional software, a heavily simplified picture might look like this:

input
↓
rules written by humans
↓
output

With Machine Learning, part of that decision logic emerges from data during training. That does not mean a computer suddenly decides on its own what or how it wants to learn. Humans still determine, among other things, the data, model architecture, training procedure, optimization goals, and evaluation.

Deep Learning is a subfield of Machine Learning that uses neural networks with many processing layers. Modern LLMs belong to this category.

Generative AI

Generative AI refers to AI systems that can create new content. That can include text, images, audio, video, program code, or combinations of them.

“New” does not necessarily mean creative in the human sense. A generative model produces output based on structures it learned during training and on the current input.

A chatbot that formulates an answer is generative AI. So is an image model that creates a picture from a description. A system that only classifies whether a photo contains a cat, by contrast, does not have to be generative.

Model

When we talk about a model in AI discussions, we mean, in simplified terms, a learned mathematical system that can turn inputs into outputs.

During training, a very large number of internal numerical values are adjusted. Together with the architecture, they determine how the model reacts to later inputs.

In everyday usage, however, the term is somewhat fuzzy. When a provider names a model, that can mean more than a particular architecture and a set of weights. Post-training, the tokenizer, configuration, or special serving settings may also be part of what we experience in practice as “this model.”

Two models with similarly sounding sizes therefore do not have to be directly comparable, either technically or qualitatively.

Foundation Model

A Foundation Model is a model trained on broad datasets that can later serve as a basis for many different tasks. The term was shaped in particular by the Stanford Center for Research on Foundation Models. [2]

The defining point is less a specific parameter count than the model’s role: it provides a general foundation that can later be adapted to concrete applications through prompting, post-training, fine-tuning, or additional systems.

Many modern LLMs are Foundation Models. The terms are not synonymous, however, because Foundation Models can also cover modalities other than language.

LLM – Large Language Model

A Large Language Model is, quite literally, a large language model. The generative LLMs we usually mean when talking about modern chat and coding systems process information as tokens and typically generate their output step by step.

Very roughly:

current context
↓
score possible next tokens
↓
choose the next token
↓
add it to the context
↓
repeat

One of the most common misconceptions is that an LLM is a gigantic database containing finished answers. That is not how it works. When a model answers a question about the capital of France, it normally does not retrieve the complete sentence “Paris is the capital of France” from a conventional table. Instead, it generates its response from the current context and the structures learned during training.

That does not contradict the fact that knowledge and relationships can be represented in the model parameters. An LLM is simply not a conventional lookup system.

Transformer

Transformer refers to a neural-network architecture introduced in 2017 with the paper Attention Is All You Need, which has had a major influence on the development of modern language models. [1]

A central element is the attention mechanism. In simplified terms, it allows the model, while processing a token, to weigh which other parts of the input are especially relevant to the current computation.

You do not need to understand the mathematics to know one important distinction:

LLM and Transformer are not synonyms. Many modern LLMs are based fully or partly on Transformer architectures. A Transformer is an architecture, while LLM describes a model class built around language.

Parameters

Parameters are adjustable numerical values in a model that are learned during training. That is why model descriptions often contain numbers such as 7B, 70B, or 600B. The B stands for billion.

A 70B model therefore has roughly 70 billion parameters.

That number alone tells us surprisingly little about the actual quality of a model. Architecture, training data, data quality, training methods, post-training, and the compute available during inference all influence its capabilities as well.

More parameters do not automatically mean a better model.

With Mixture-of-Experts models, this simple parameter count becomes even harder to interpret. We will come back to that later.

Weights

Weights are learned numerical values inside a neural network. They influence how internal signals are combined and are adjusted during training.

In everyday LLM discussions, “weights” and “parameters” are often used almost interchangeably. Technically, the distinction is more precise: weights are parameters, but a model can also contain other types of trainable parameters.

When a provider describes a model as Open Weights, it generally means that the learned model weights are available. What that implies — and what it does not — comes later.

Training

Training is the process through which a model learns or changes its parameters.

In very simplified terms, the model processes large numbers of training examples, makes predictions, measures its deviation from a training objective, and then adjusts its parameters step by step. This happens over and over again.

For the rest of this article, one simple distinction is enough:

Training
data
↓
optimization
↓
changed model parameters


Inference
input
↓
already trained model
↓
output

Training changes the model. Inference uses the trained model.

Modern training pipelines are much more complex than this little diagram suggests, but the distinction is sufficient for now.

Pre-Training and Post-Training

When people talk about modern LLMs, two terms often appear: Pre-Training and Post-Training.

During pre-training, a general base model is created. It learns broad structures, relationships, and capabilities from very large datasets.

Additional training phases can follow. Post-Training is an umbrella term for different methods used to make a base model follow instructions better, solve particular kinds of tasks, or exhibit desired behavior. These methods can include fine-tuning and different forms of reinforcement learning.

This matters because two models with the same underlying architecture and similar pre-training can behave very differently after different post-training procedures.

Fine-Tuning

With Fine-Tuning, an already trained model is deliberately adapted through additional training. This can be done, for example, to specialize it for certain tasks, domains, output formats, or behaviors.

The decisive word here is training: fine-tuning changes model parameters.

That fundamentally distinguishes it from prompting, RAG, or a typical memory system. If I send my architecture rules with every request, I have not fine-tuned the model. If an agent loads information from a project database, I have not fine-tuned it either.

Additional context is not the same as fine-tuning.

Inference

Inference refers to using an already trained model. We give it an input and let it compute an output.

"Explain dependency injection."
↓
trained model
↓
generated answer

During normal inference, the model’s base weights are not automatically changed. A long chat therefore does not automatically continue training the base model, and neither does a single prompt or a conventional RAG system.

Even a product that “remembers” something about me does not necessarily modify model weights.

Inference means using the model, not automatically training it again.

This distinction becomes important later when we talk about context, memory, retrieval, and agents.

Token

A token is a processing unit used by a language model. It is neither automatically a word nor automatically a single character.

A common word may consist of one token, while a rare or compound word can be split into several. Punctuation marks and parts of words can also form their own tokens.

Exactly how this split works depends on the tokenizer. Different models can therefore produce different token counts for the same text. The same statement in different languages can also require very different numbers of tokens. [3]

Tokens matter for context size, maximum output length, compute, API limits, and often billing.

A token is, however, first and foremost a technical unit — not a currency.

Tokenizer

The Tokenizer turns text into the token units a model works with.

A sentence does not simply become an internal list of English or German words. Instead, it becomes a sequence of token IDs. The tokenizer uses a vocabulary and a procedure to split text into suitable units.

It is therefore an important part of the interface between human-readable text and the model.

Input Tokens

Input Tokens are the tokens that enter a model request. This is often much more than the visible message written by the user.

Depending on the application, system instructions, previous chat history, tool descriptions, documents, search results, memory entries, or project information may also form part of the input.

A very short user question can therefore result in a surprisingly large actual input.

Output Tokens

Output Tokens are the tokens generated during the response. In a normal chat, much of that corresponds to the text we eventually see.

Some reasoning systems may additionally report internal reasoning tokens. These do not necessarily appear as visible answer text. How they are counted, exposed, or billed depends on the provider and model. [3]

A short visible answer therefore does not necessarily mean that very little computation happened during inference.

Cached Tokens

Modern LLM services can cache parts of repeated inputs. If many requests use the same large system prompt or the same tool descriptions, previous computations can sometimes be reused.

Providers may report such reused inputs as cached tokens and treat them differently technically or commercially. OpenAI, for example, reports cached tokens as part of input-token usage, while other major providers also offer forms of prompt or context caching. [3]

The important point is:

Cached tokens are still tokens.

Caching does not turn the content into permanent model knowledge, nor does it automatically enlarge the context window. It can, however, avoid repeated computation and thereby reduce latency or cost.

Prompt Caching

Prompt Caching refers to reusing already processed parts of a prompt across multiple requests.

This is particularly useful when many requests begin with the same long prefix — for example, extensive agent instructions, tool schemas, or documents. Instead of processing those parts from scratch every time, suitable systems can reuse previously computed states.

How long such a cache lives, which parts are cacheable, and how it is billed are provider-specific details.

Prompt caching is therefore neither memory nor additional training.

KV Cache

The KV Cache, short for Key-Value Cache, is a technical optimization mechanism used during inference in many Transformer models.

While an autoregressive model generates token by token, the KV cache stores certain previously computed states from the attention layers. This avoids recomputing them fully for every new token. [16]

For our purposes, the useful distinction is simple: the KV cache is a mechanism inside model inference. Prompt caching is a higher-level caching concept provided by a system or vendor. The two can be related, but they are not the same thing.

Prompt

A prompt is the input or instruction we use to influence a model.

It can be a simple question:

What is an event loop?

But a prompt can just as easily contain requirements, examples, data, rules, and constraints. In software engineering, a good prompt for a complex task often starts to look like a specification:

This is the goal. This is the context. These constraints apply. This is the result I expect.

Prompting is therefore much less mystical than the term Prompt Engineering sometimes makes it sound.

System Prompt

Many LLM applications use additional instructions that define the frame of an interaction. These are commonly called a System Prompt.

A system prompt can specify what role the system should take, which tools are available, which rules apply, or what output format is expected. These instructions do not have to be fully visible to the user.

A system prompt can strongly influence model behavior. It is still not a mathematical proof that every rule will always be followed perfectly.

Context

Context broadly refers to the information available to a model for its current processing step.

That can include the user request, system instructions, previous conversation history, documents, tool results, memory contents, or retrieval results.

Context is therefore different from knowledge that emerged in the model parameters during training. If I put documentation for a new API into the context, the model can work with it even if that documentation was never part of its training data.

The base weights are still unchanged afterwards.

Context is not training.

Context Window

The Context Window describes how much context a model or a particular model endpoint can support during one processing session. It is usually measured in tokens.

A 128k context window therefore does not mean 128,000 words, but roughly 128,000 tokens. How input and output count against that limit depends on the model and API design.

A large context window is useful, but it does not automatically mean every piece of information inside it is used equally reliably. The well-known Lost in the Middle paper already showed substantial differences in early long-context models depending on where relevant information appeared in the context. [5] Later work and newer models have improved these capabilities, but the underlying distinction remains useful:

Fitting into the context and being used reliably from the context are two different properties.

Context Engineering

Context Engineering means deliberately designing what information a model receives for a task.

That can include much more than the prompt itself: system instructions, relevant files, tool descriptions, examples, memory, retrieval results, current state, and rules about which information should be loaded when.

The question therefore shifts from:

How do I write the perfect prompt?

to:

What information does the model need for this particular decision — and what does it not need?

Context Engineering is not a new model architecture or training method. It is engineering around the information supplied to the model.

This becomes especially important later when we look at coding agents.

Probabilistic

LLMs are probabilistic models. That does not mean they simply throw dice.

For a possible continuation, a language model computes values from which probabilities for candidate next tokens can be derived. A heavily simplified example might look like this:

"The sky is ..."

blue      0.61
today     0.08
clear     0.07
green     0.01
...

The numbers are invented; the principle is not. Different possible continuations are scored differently under the current conditions.

How one concrete token is eventually selected from that distribution is another question.

Sampling

Sampling refers to methods for selecting a concrete next token from the possible candidates.

A system can always choose the most likely token. It can also sample from several plausible candidates. Different strategies and parameters exist for this purpose, including temperature, Top-K, and Top-P. [4]

This helps explain why “probabilistic” does not automatically mean every request must produce a completely different answer. At the same time, the same prompt does not guarantee exactly the same output every time.

The model produces a distribution of possible continuations; the decoding strategy influences how that distribution becomes one concrete sequence.

Temperature

Temperature influences, in many generation methods, how strongly selection concentrates on the most likely tokens.

Lower values typically lead to a more focused selection. Higher values give more weight to alternative candidates and therefore often allow more variation. [4]

The common shortcut

low temperature = correct, high temperature = creative

is too simplistic. Temperature initially changes the distribution or selection process. Whether that makes an answer more or less correct depends on the task, model, and decoding method.

Likewise, temperature: 0 should not universally be equated with complete determinism. APIs and inference systems can differ, and other parts of the processing pipeline can also introduce variability. Anthropic documents this limitation explicitly.

Reasoning

Reasoning is one of those terms where technical meaning and marketing language easily blur together.

In research and products, the term often refers to models or methods that spend additional computation during inference on more complex problems. A system may, for example, compute more intermediate steps, generate several candidates, inspect solution paths, or use additional search and verification steps.

In this context, terms such as test-time compute, inference-time compute, reasoning tokens, or thinking tokens appear frequently. They are not fully synonymous.

Test-time or inference-time compute broadly means using additional computation on a concrete task after training has finished. Research has shown that more inference computation can improve solution quality on suitable tasks, although the effect depends heavily on the method and problem. [6][7]

Reasoning tokens or thinking tokens, by contrast, are often concrete implementation or product terms for internal processing steps.

The most important point is therefore linguistic rather than technical:

Reasoning does not automatically mean human thinking, consciousness, or understanding.

The term describes abilities or procedures of an AI system. It is not evidence that a human-like thought process is taking place inside the model.

Hallucination

We usually speak of a Hallucination when an LLM generates content that does not match the underlying facts, sources, or provided context while still sounding plausible.

In software development, that could be a method that does not exist in a library, an invented API, or a convincing-looking source that was never published.

Not every mistake is therefore automatically a hallucination. A model can misunderstand a requirement, make a calculation error, use an outdated but real API, or make a poor architectural decision. Those can all be wrong without being the same type of error.

The term itself is also a metaphor. It should not be interpreted as meaning that the model “sees” something in the human sense.

Noise

Noise is not a precisely defined scientific LLM term. Developers use it more as a practical label for output that looks plausible but provides little value for the actual task.

That can mean unnecessarily long explanations, extra abstractions, unrequested changes, or five confident paragraphs from which only one useful sentence remains.

With coding agents, noise can also mean that an agent performs a range of “improvements” alongside a small requested change and thereby increases review effort.

The term therefore describes the perceived signal-to-noise ratio of the output rather than one precisely defined model mechanism.

Embedding

An Embedding is a numerical representation of information. Text, images, or other content are represented as vectors of numbers.

For this article, one practical effect is enough: suitable embedding models can place semantically similar content closer together in the resulting vector space.

That allows an application to search by meaning and similarity rather than only by exact matching words.

Embeddings are therefore useful for semantic search and many RAG systems. They are not, however, a small readable copy of a document merely rewritten as numbers.

RAG – Retrieval-Augmented Generation

Retrieval-Augmented Generation combines information retrieval with generative output.

The core idea is straightforward:

question
↓
retrieve relevant information
↓
put retrieved information into the context
↓
LLM
↓
answer

The original RAG paper describes the combination of parametric model knowledge with externally retrieved information. [8]

Retrieval can be implemented in many ways today: embeddings and vector search, classic full-text search, databases, APIs, or combinations of them.

RAG is therefore not automatically synonymous with a vector database.

More importantly for our terminology:

RAG normally does not retrain the model. The retrieved information is supplied at runtime and typically enters the model through the context.

Tool Calling / Function Calling

With Tool Calling, a model can produce structured output that instructs the surrounding application to use an external function or tool.

Instead of making up a weather report itself, the model could signal something like:

Tool: getWeather
Argument:
  city: Berlin

The actual operation then takes place outside the language model. The runtime or application executes the function and returns the result. Some platforms provide such tools directly on the server side, but the principle remains the same. [9]

An LLM does not inherently have internet access, a shell, or access to a file system simply because it is an LLM. Those capabilities emerge from the complete system of model, tools, and permissions.

Computer Use

Computer Use extends that principle to graphical user interfaces.

A model or agent can receive screenshots and then request actions such as clicks or keyboard input. A controlling runtime environment carries out those actions.

This allows an agent to operate applications for which no suitable API exists.

Computer use expands not only a system’s capabilities but also its risks. An agent that can actually click, type, or submit forms requires different safety and approval mechanisms than a model that can only produce text.

Memory

Memory is one of the least precise terms in LLM products.

A system can store user preferences in a database. A coding agent can persist project information. A chat application can search previous conversations and load relevant information back into the context. An agent can save its state between two work steps.

Depending on the product, all of these mechanisms may be called memory.

What they often have in common is that information is stored in addition to the base model and made available again later.

Memory therefore does not automatically mean that model weights have changed.

A system can appear to “remember” something very convincingly by storing it outside the model and inserting it back into the context on the next request.

Agent

There is no single technical definition of Agent that research, vendors, and developers all use identically.

For our purposes, one practical description is enough:

An agent combines a model with a goal, context or state, tools, and a process in which several steps can be executed and their results processed again.

A simple agentic loop might look like this:

goal
↓
evaluate situation
↓
choose next action
↓
execute tool
↓
observe result
↓
evaluate again
↓
...
↓
done

One useful distinction is how much of that process is hard-coded and how much decision-making the model has over the next step. Anthropic, for example, distinguishes between more predefined workflows and agents in which the model controls the process more dynamically. [12]

In real products, however, that boundary is fluid. Not everything labeled “agent” has the same degree of autonomy.

Coding Agent

A Coding Agent is an agent whose environment and tools are designed around software development.

It can, for example, search a repository, read and modify files, start compilers or linters, run tests, inspect Git diffs, or use shell commands.

That makes it very different from a chat system that merely responds to a pasted code block. A coding agent can observe its own changes again:

change code
↓
run tests
↓
read errors
↓
investigate cause
↓
correct code
↓
test again

That feedback loop is a core part of modern agentic coding workflows.

A Subagent is usually another agent to which a narrower subtask is delegated. Here too, the term does not define one standardized architecture.

Agent Harness

The term Agent Harness refers to the runtime and control layer around the model itself.

That layer can build context, manage tool calls, keep state, check permissions, request human approval, and orchestrate multi-step agent loops. Microsoft currently describes an agent harness in similar terms as runtime scaffolding that equips a language model with the components needed for longer-running agent work. [13]

The distinction is useful:

model
=
produces outputs and decisions

harness
=
organizes context, tools, state,
permissions, and execution

agent
=
emerges from the combination

If two coding products use the same base model but behave very differently, a substantial part of that difference may lie in the harness.

Agentic AI

Agentic AI is an umbrella term for AI systems that work toward a goal more independently than a single-response system.

That can include planning, tool use, observing intermediate results, performing several consecutive actions, and adapting the next steps. Institutions such as NIST now also use the term for more autonomous, goal-directed AI systems. [11]

Even so, “agentic” has no sharp technical boundary. There is a large difference between a system that can choose one of three predefined API calls and a coding agent that investigates a repository for an extended period and delegates tasks. Yet product pages may describe both as agentic.

Agentic therefore does not describe a new kind of model architecture, nor does it automatically mean full autonomy.

MCP – Model Context Protocol

The Model Context Protocol, or MCP, standardizes connections between AI applications and external capabilities or information.

As of September 2026, 2026-07-28 is the current MCP specification. Among other changes, that version moved the protocol core more strongly toward stateless request/response communication. [10]

For basic understanding, however, the more important question is what MCP does at all. MCP servers can expose standardized capabilities such as tools, resources, and prompts, which compatible hosts or clients can discover and use.

In simplified form:

AI application
↓
MCP client
↓
standardized protocol
↓
MCP server
↓
tools / resources / external systems

This means every AI product does not have to invent a completely different integration for every tool and data source.

MCP is neither a model nor an agent. It is also not automatically RAG or memory. An agent can use MCP to call a tool, a retrieval system can obtain data through it, and a memory system could be connected through it.

MCP standardizes the connection. It is not the agent using that connection.

Frontier Model

A Frontier Model is not a specific model architecture.

The term is usually used for highly capable general-purpose models near the current frontier of performance. The UK definition of frontier AI similarly emphasizes this relative character: the term refers to systems whose capabilities are at or beyond the level of the most advanced systems available at a given time. [14]

The phrase “at a given time” matters. What counted as frontier in 2023 may no longer be close to the frontier in 2026.

Frontier therefore describes a position relative to the current state of the art, not one particular technical design.

Multimodal

A multimodal model or system can process or generate more than one kind of information.

Typical modalities include text, images, audio, and video. One model may accept text and images as input while producing only text. Another may both understand and generate speech.

“Multimodal” therefore does not automatically mean a model can read and generate every possible modality equally well. Its actual supported inputs and outputs still need to be checked.

A VLM – Vision Language Model is a more specific category in this area, referring to models that process visual information and language together.

Open Weights

With an Open-Weights model, the trained model weights are made available. Depending on the license, software, and hardware, they can therefore be downloaded and run on your own infrastructure.

That does not mean everything around the model is open. The training data can remain unknown, training code may be missing, the license can restrict certain uses, and the complete training process may not be reproducible.

Open Weights initially means that the weights are available — not that the entire model project is open.

Open Source

With conventional software, Open Source has a long-established meaning. Applying that concept to AI systems is more complicated because model parameters, training methods, and information about training data matter in addition to source code.

The Open Source Initiative therefore published its Open Source AI Definition 1.0 in 2024. It requires the freedoms to use, study, modify, and share an AI system and sets requirements covering code, parameters, and information about the training data. [15]

That definition does not end every debate over the term, but it provides a much more precise standard than a generic “open” label on a product page.

For technical discussions, one simple distinction remains useful:

Open Weights is not automatically Open Source.

Local Model

A Local Model, in the narrower sense, runs directly on a local device or computer — for example a developer PC, notebook, or workstation.

That should be distinguished from self-hosted. A self-hosted model may also be under your control while running on an internal GPU server or an organization’s own cluster.

In everyday discussion, the two terms are sometimes mixed. For technical conversations, the distinction is useful:

local
=
runs on the local device


self-hosted
=
runs on infrastructure
controlled by me or my organization

Neither term says anything by itself about quality, licensing, or openness. An Open-Weights model can run locally, but it does not have to.

MoE – Mixture of Experts

In a Mixture-of-Experts model, or MoE, certain model layers contain several subnetworks called Experts. A router decides during processing which of them should be used for a particular token.

The word “expert” should not be taken too literally. There does not have to be a clearly interpretable “Java expert,” “math expert,” or “German expert.” Technically, these are subnetworks between which the model routes.

One well-known example is Mixtral 8x7B, where only two of eight experts are selected for each token. [17]

In simplified form:

token
↓
router
↓
┌────────┬────────┬────────┬────────┐
Expert A Expert B Expert C Expert D
   ✓               ✓
↓
selected experts continue computing

This allows a model to have a very large number of total parameters without using every expert for every token.

Total Parameters

Total Parameters refers to the complete number of parameters in an MoE model. That includes parameters in experts that are not selected for a particular token.

The number therefore says something about overall model size but does not directly tell us how many parameters participate in each individual processing step.

Active Parameters

Active Parameters refers to the parameters actually used for one concrete processing path. In sparse MoE models, this number can be much smaller than the total parameter count.

How exactly a vendor calculates this number should be checked in the model card or paper, because shared model components exist in addition to the experts.

The important distinction is:

very many total parameters
≠
all parameters are used for every token

For an MoE model, a huge parameter number without further context is therefore much less informative than it first appears.

Quantization

Quantization reduces the numerical precision used to store or sometimes compute model values. The usual goal is to reduce memory requirements and computational cost.

Models are often already operated in formats such as FP16 or BF16. Stronger quantization can represent weights using 8-bit, 4-bit, or even more compact formats. Depending on the method, other parts of inference can be quantized as well. [16]

This is especially interesting for local models because a model that does not fit into available GPU memory at full precision may fit quite comfortably in quantized form.

Quality can decrease in the process. How much depends on the model, method, bit width, and task.

The idea

4 bit instead of 16 bit = only one quarter of the intelligence

is therefore wrong. Numerical precision and model capability do not scale that way.

Quantization is primarily an engineering trade-off between memory, compute, hardware requirements, and possible quality changes.

Credit

And that brings us to a term that has surprisingly little to do with the internal workings of an LLM.

A Credit is not a fundamental technical unit of a language model. Credits are product or billing logic defined by a provider.

One credit could represent a certain number of tokens, a tool call, a minute of compute time, one model request, or any combination of those. What a credit actually means therefore has to be checked for the individual product.

From a statement such as

This product includes 10,000 credits.

you can infer very little about actual LLM usage without additional information.

Term	Meaning
Token	technical processing unit used by a model
Credit	provider-specific usage or billing unit

Tokens belong to the technology. Credits belong to the product or business model.

A provider can calculate credits based on tokens. That does not turn credits into tokens.

From Buzzwords to a Shared Vocabulary

We have not completed a Machine Learning degree by this point. That was never the goal.

What we can now do is separate many of the terms that constantly appear in model cards, product pages, and developer discussions. A model is trained and then used during inference. It processes tokens inside a context. The tokenizer determines how text is split into those units. Sampling influences how one concrete output is selected from possible continuations. Reasoning can involve additional inference-time computation without implying human thought.

RAG brings external information into the context, memory can store information outside the model, and tools give an AI system ways to act. A harness organizes parts of that environment, while an agent can use those capabilities across several steps. MCP can standardize how such systems connect to external capabilities.

And credits? They may simply determine how long the provider lets us keep doing all of that.

The terminology is hopefully a little less mysterious now; if you want to look up a term later, you will find it in the series glossary. The next question is harder: when a provider claims that a coding agent solves a certain share of real software tasks, where does that number actually come from?

What is a benchmark? What does SWE-bench measure? What do Verified, Pro, or Pass@k mean? And why is a benchmark result not simply the percentage of a software developer that a model can replace?

That is what the next article is about.

Sources

[1] Ashish Vaswani, Noam Shazeer, Niki Parmar et al.: Attention Is All You Need. Advances in Neural Information Processing Systems 30, 2017. arXiv: 1706.03762.

[2] Rishi Bommasani, Drew A. Hudson, Ehsan Adeli et al.: On the Opportunities and Risks of Foundation Models. Stanford Center for Research on Foundation Models, 2021. arXiv: 2108.07258.

[3] OpenAI: Documentation on tokens, usage, reasoning tokens, and prompt caching. Current as of 2026.

[4] Hugging Face Transformers: Generation / GenerationConfig. Technical documentation, current as of 2026.

[5] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang: Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics 12, 2024, pp. 157–173. DOI: 10.1162/tacl_a_00638.

[6] Charlie Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar: Scaling LLM Test-Time Compute Optimally Can Be More Effective than Scaling Parameters for Reasoning. International Conference on Learning Representations, 2025.

[7] Mohsen Hariri et al.: Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and Reproducibility. Preprint, 2026. arXiv: 2608.04001.

[8] Patrick Lewis, Ethan Perez, Aleksandra Piktus et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. Advances in Neural Information Processing Systems 33, 2020. arXiv: 2005.11401.

[9] Anthropic: Tool use with Claude and How tool use works. Technical documentation, current as of 2026.

[10] Model Context Protocol: Model Context Protocol Specification 2026-07-28 and accompanying release notes, July 28, 2026.

[11] National Institute of Standards and Technology: Agentic AI. NIST, 2026.

[12] Anthropic: Building Effective Agents. Technical documentation.

[13] Microsoft: Agent Harness. Microsoft Agent Framework Documentation, current as of August 2026.

[14] UK Department for Science, Innovation and Technology: Frontier AI: capabilities and risks – discussion paper. Updated version, 2025.

[15] Open Source Initiative: The Open Source AI Definition – Version 1.0. 2024, stable version as of September 2026.

[16] Hugging Face Transformers: Documentation on Quantization, Cache strategies, and How caching works. Current as of 2026.

[17] Albert Q. Jiang et al.: Mixtral of Experts. 2024. arXiv: 2401.04088.