What is a large language model?

A large language model (LLM) is a neural network trained on huge amounts of text to predict what comes next in a sequence. By getting very good at that one task, it picks up grammar, facts, styles of reasoning and the shape of code, which is why the same model can summarize a contract, translate a menu and fix a bug. Every agent in this guide is an LLM with some tools and a loop wrapped around it, so it pays to understand what the model itself does and does not do.

Predicting the next token#

Models do not read letters or words. Text is cut into tokens, short chunks such as a whole common word, part of a rarer word, or a piece of punctuation. A model reads a sequence of tokens and produces a probability for every possible next token. Generating a reply means picking one, appending it, and repeating until the model emits a "stop" token or hits a limit.

Two consequences matter for builders:

• Cost and speed are counted in tokens, for both what you send and what comes back. Tokens, pricing and the cost of a loop covers this in detail.
• The model writes left to right. It cannot go back and revise a sentence it already produced. It can only continue from there, which is why asking it to "think first, answer second" often improves results.

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic(); // reads ANTHROPIC_API_KEY from the environment
const MODEL = "claude-opus-5-5";

const reply = await client.messages.create({
  model: MODEL,
  max_tokens: 300,
  messages: [{ role: "user", content: "In one sentence, what is a token?" }],
});

console.log(reply.content); // the generated text
console.log(reply.usage); // how many tokens went in and came out

import OpenAI from "openai";

const client = new OpenAI(); // reads OPENAI_API_KEY from the environment
const MODEL = "gpt-6.1-sol";

const reply = await client.responses.create({
  model: MODEL,
  max_output_tokens: 300,
  input: "In one sentence, what is a token?",
});

console.log(reply.output_text); // the generated text
console.log(reply.usage); // how many tokens went in and came out

import { GoogleGenAI } from "@google/genai";

const client = new GoogleGenAI({}); // reads GEMINI_API_KEY from the environment
const MODEL = "gemini-3.8-flash";

const reply = await client.models.generateContent({
  model: MODEL,
  contents: "In one sentence, what is a token?",
  config: { maxOutputTokens: 300 },
});

console.log(reply.text); // the generated text
console.log(reply.usageMetadata); // how many tokens went in and came out

import OpenAI from "openai";

// Llama and other open models are served by many hosts. Most speak the OpenAI API: here is Together AI (Groq and Fireworks work the same way).
// Together's serverless list no longer includes Llama, so this uses the open gpt-oss-120b; swap in a Llama ID if your host serves one.
const client = new OpenAI({ baseURL: "https://api.together.ai/v1", apiKey: process.env.TOGETHER_API_KEY });
const MODEL = "openai/gpt-oss-120b";

const reply = await client.chat.completions.create({
  model: MODEL,
  max_tokens: 300,
  messages: [{ role: "user", content: "In one sentence, what is a token?" }],
});

console.log(reply.choices[0].message.content); // the generated text
console.log(reply.usage); // how many tokens went in and came out

import { ChatAnthropic } from "@langchain/anthropic"; // or ChatOpenAI, ChatGoogleGenerativeAI, ChatOllama: the rest of the code is identical

const model = new ChatAnthropic({ model: "claude-opus-5-5", maxTokens: 300 }); // reads ANTHROPIC_API_KEY from the environment

const reply = await model.invoke("In one sentence, what is a token?");

console.log(reply.content); // the generated text
console.log(reply.usage_metadata); // how many tokens went in and came out

import OpenAI from "openai";

// A model running on your own machine. Ollama listens on port 11434; a vLLM server (vllm serve <model>) on port 8000. Both speak the OpenAI API.
const client = new OpenAI({ baseURL: "http://localhost:11434/v1", apiKey: "ollama" }); // the key is ignored locally but must be non-empty
const MODEL = "llama3.3"; // first run: ollama pull llama3.3

const reply = await client.chat.completions.create({
  model: MODEL,
  max_tokens: 300,
  messages: [{ role: "user", content: "In one sentence, what is a token?" }],
});

console.log(reply.choices[0].message.content); // the generated text
console.log(reply.usage); // how many tokens went in and came out

import { generateText } from "ai";
import { anthropic } from "@ai-sdk/anthropic"; // swap for openai(...) from @ai-sdk/openai or google(...) from @ai-sdk/google

const reply = await generateText({
  model: anthropic("claude-opus-5-5"), // reads ANTHROPIC_API_KEY from the environment
  maxOutputTokens: 300,
  prompt: "In one sentence, what is a token?",
});

console.log(reply.text); // the generated text
console.log(reply.usage); // how many tokens went in and came out

Run it and look at usage. That pair of numbers is the unit everything else in this guide is priced and limited in.

Pick your provider with the tabs on any code example: every example in this guide has the same seven tabs, and your choice is remembered. The shape is the same everywhere (a client, a model name, a message, a reply and a usage count); only the names differ. Model names change often, so each example keeps its model in one constant to edit, and each provider's own model list is the place to check the current one.

Transformers and attention#

Almost all modern LLMs use the transformer architecture. Its key idea is attention: when producing each token, the model scores how relevant every earlier token is and blends their information accordingly. That is how a pronoun at the end of a paragraph can be linked back to the noun at the start, and how a function call in line 200 can be matched to its definition in line 12.

You do not need the mathematics to build agents. You need one practical fact: attention is computed over the whole context window, so the more text you put in, the more the model has to weigh, and the more it costs. The next chapter, context windows explained, picks that thread up.

How a model is trained#

Training happens in stages, and each stage leaves a mark on how the model behaves.

A model's size is usually quoted in parameters, the learned numbers inside it. More parameters can mean more capability, but data quality, training method and design matter at least as much, which is why a newer small model can beat an older large one.

What a model cannot do alone#

Five limits shape every design in this guide:

1. It has no memory between calls. Each request is independent. Anything the model should "remember" must be sent again, or stored and retrieved by your code (context and memory).
2. Its knowledge is frozen at the training cut-off. Anything recent, private or specific to your business has to be supplied (what is RAG).
3. It can be confidently wrong. A fluent false statement is called a hallucination. The fix is verification: sources, checks in code and tests (evals).
4. It cannot act. A model produces text. Reading a file, calling an API or sending an email happens only when your program carries out a request the model wrote (tools).
5. It is not deterministic by default. The same prompt can produce different answers. You can lower variation but not promise identical output.

Choosing a model#

Models differ along a few axes you can reason about without reading benchmarks: capability (how hard a task it can do), speed and price (usually the opposite of capability), context window and whether you can run it yourself (open weights). Most real systems use more than one: a strong model for planning and hard judgements, a small fast one for routine steps (model routing).

Model names change every few months, so keep the model ID in one constant, as the examples here do, and check the provider's current list before you copy one.

Where to go next#

Continue with context windows explained: the limit that decides how much an agent can hold in its head at once.