What Is a Large Language Model?

Chatbots can write emails, explain homework and fix a broken program. Under all of that sits one simple skill: guessing what text comes next. This lesson explains that skill with a tiny example you can count by hand and a tiny program you can run. It shows what the word large means, and it tells you three things such a model does not do. You need no maths and no earlier lesson.

Learning Objectives

After this lesson, you will be able to:

Say what a large language model does in one sentence
Explain how counting what followed what teaches a program to guess the next word
Say what large means for a model, using the words parameter and context
Name three things a language model does not do, and why a smooth answer is not proof

Words you need6 terms

A program that gives a chance to each possible next piece of text, given the text so far.

A language model with a very large number of adjustable numbers inside it, trained on a very large amount of text. People shorten it to LLM.

A small piece of text that the model reads and writes, such as a short word, part of a word, or a mark like a full stop.

The study period in which a program reads a huge pile of text and adjusts itself until its guesses fit that text.

One adjustable number inside the model, like a single dial. Training turns all the dials a little at a time.

All the text the model can see right now: your question plus the pieces it has already written.

#A Very Good Guesser of the Next Word

Type "See you" on your phone and it offers a few next words, perhaps "soon" or "later". It is guessing. It does not know your plans.

A language model is a program that does this guessing job, only far better. Give it some text and it gives back a chance for each word that could come next: "soon" might get a high chance, "elephant" a very low one. A large language model, or LLM, does the same job with a far bigger program and far more text behind it. That is the whole idea. The rest of the lesson is about how the guessing skill is learned and how one guess becomes a whole answer.

Before you read on, make a guess.

What Do You Think?

A phone has never been told your plans, but it still offers 'soon' after 'See you'. Where do you think the suggestion comes from?

In one sentence: a language model gives a chance to each possible next piece of text, and a large one does it with a very large program trained on a very large amount of text.

#How the Skill Is Learned: Counting What Followed What

How could a program learn which word usually follows another? The simplest way is to count. Here are three short sentences. Pretend this is everything the program has ever read.

the cat sat on the mat.
the cat ate the fish.
the dog sat on the rug.

Now ask one question: every time the word the appeared, which word came right after it? Go through the sentences and tally.

Word after "the"	Times seen	Chance (out of 6)
cat	2	2 of 6, about 33%
mat	1	1 of 6, about 17%
fish	1	1 of 6, about 17%
dog	1	1 of 6, about 17%
rug	1	1 of 6, about 17%

The word "the" appeared 6 times with a word after it, so the 6 counts become 6 equal slices of a whole. "cat" has 2 slices, so its chance is the biggest. Nothing was memorised about cats. A tally was kept of what followed what.

Do the same for "sat". It appeared twice, and both times "on" came next. So after "sat", the chance of "on" is 2 of 2, which is 100%. This tiny table is a complete, working language model. It is a poor one, because it has read only three sentences, but it is the same kind of thing as the big ones.

In one sentence: training can be as simple as counting what followed what, and the counts turn into chances for the next word.

#Run It: A Language Model That Only Counts

The cell below does exactly the counting you just did by hand, then uses the chances to write. Press Run. It uses plain Python, with no extra library, so it runs right here in your browser.

pythonplayground.py · Pyodide

# A tiny language model that only counts. No maths library needed.
import random

text = "the cat sat on the mat . the cat ate the fish . the dog sat on the rug ."
words = text.split()

# Training: for every word, count which words came right after it.
counts = {}
for before, after in zip(words, words[1:]):
  counts.setdefault(before, {})
  counts[before][after] = counts[before].get(after, 0) + 1

print("After 'the' we saw:", counts["the"])
print("After 'sat' we saw:", counts["sat"])

# Guessing: turn counts into chances.
def chances(word):
  row = counts[word]
  total = sum(row.values())
  return {w: n / total for w, n in row.items()}

for w, p in sorted(chances("the").items(), key=lambda kv: -kv[1]):
  print(f"  chance of '{w}' after 'the': {p:.0%}")

# Writing: guess, pick by chance, add to the text, repeat.
def write(start, length, seed):
  rng = random.Random(seed)
  out = [start]
  for _ in range(length):
      row = chances(out[-1])
      out.append(rng.choices(list(row), weights=list(row.values()))[0])
      if out[-1] == ".":
          break
  return " ".join(out)

print()
for seed in (1, 2, 3):
  print(f"seed {seed}:", write("the", 8, seed))

assert chances("the")["cat"] > 0.3
assert counts["sat"] == {"on": 2}

Tests · Run the script. It should print the counts for 'the' (cat 2, mat 1, fish 1, dog 1, rug 1), the counts for 'sat' (on 2), then the chances 33% for cat and 17% for each of the other four, then three short sentences. The three sentences are written by chance, so they can differ if you change the seed. The checks at the end should pass without an error.

Do: Run the cell once. Then change the numbers in the line for seed in (1, 2, 3): and run it again. Next add the sentence the dog ate the fish . to the end of text and run it a third time.

Notice: The counts match the table you made by hand: "cat" has 2 and the other four have 1 each. The three written sentences differ from each other, and some are odd. A new sentence in text changes the chances.

Why: The program only keeps counts, and it picks the next word by chance in proportion to them. Different seeds give different picks. A drawn stand-in, not a large model: this one looks at only the one word before.

In one sentence: a counting model learns a table of what followed what, and writes by picking from that table by chance.

#Why Counting One Word Is Not Enough

The counting model looks at only the last word. That fails fast. Read this sentence.

The trophy did not fit in the suitcase because it was too big.

What does it mean here? The trophy, because the trophy was too big to fit. Now change one word: The trophy did not fit in the suitcase because it was too small. Now it means the suitcase. To guess well, a model must look at many earlier words at once and work out which ones matter. A counting table that sees only the word before cannot do that.

This is why real models use a more capable design than a tally. They read the whole text so far and decide which earlier words to pay attention to. That design is the topic of the next lesson, Attention in Pictures. For now, hold on to this: a real language model plays the same guessing game, but it uses all the text so far and not just the last word.

In one sentence: a real model guesses from all of the text so far, because the meaning of a word can depend on words far back.

#Pieces, Not Words: Tokens

So far we guessed whole words. Real models work on tokens, which are small pieces of text. A common word like "cat" is usually one token. A rare or long word may be cut into two or three pieces, and a mark like "." is its own token. Using pieces means a fixed list of a manageable size can still spell any word, even one it has never seen.

You do not need the details yet. The lesson Tokenization shows how the pieces are chosen. In this lesson, read "token" as "a small piece of text".

#What Large Means: Dials and a Reading Room

Why is it called large? Two things are large.

The first is the number of parameters. Picture a huge control panel covered in dials. Each dial is one adjustable number. Before training, the dials are set badly, so the guesses are poor. Training reads text, compares each guess with what really came next, and turns every dial a tiny amount to make the guess a little better. It does this a huge number of times. The counting model above has about as many dials as it has table entries, a few dozen. A modern chat model has far more. Its makers publish the size of some models, and the figures change often, so check the model's own page: the count is in the billions or more as of October 2026, verify.

The second is the context, the amount of text the model can see at once. Think of a reading room with a limited table. Your question and the model's own earlier words must all fit on the table. If the text is longer than the table, the oldest part falls off and the model cannot see it. Bigger tables let a model keep more of a long document in view. The size of the table differs between models and changes as new ones appear, so read it from the model's own page.

Most of the skill of a large model comes from two things together: many dials that can hold many patterns, and a very large amount of text to learn those patterns from.

In one sentence: large means many parameters and a lot of training text, and the context is how much the model can see at one time.

#From One Guess to a Whole Reply

One guess is only one piece. A reply is written by repeating the guess. The model guesses the next token, picks one by chance, adds it to the text, and then guesses again with the longer text. It stops when it picks a special stop token.

The top row happens once, long before you arrive, and takes a lot of time and computing power. The bottom row happens every time you ask something. Notice that the reply is built from the dials, and the huge pile of text is not consulted again.

Now watch the guess-and-pick loop in a toy. It has a small hand-written table of next-word chances, like the one you just counted.

Loading visualization...

Do: Choose the Cat starting phrase. Press Generate next word four times. Then press Reset sentence and Generate again, and press New seed to try another run.

Notice: Each press adds one word, and the bars change because the next word depends on the last one. Different seeds give different sentences from the same table.

Why: The model picks by chance in proportion to the bars, so likely words win often but not always. Drawn stand-in, not a trained model: the scores are hand-written for a few words.

In one sentence: a reply is made by guessing, picking, adding and repeating, and the dials do all the guessing.

#What a Language Model Does Not Do

Three things are easy to assume and wrong.

It does not look up a stored answer. It builds each token from its dials, as the counting model builds from its table. It can repeat something it saw often, but it has no shelf of finished answers.

It does not check facts. Its job is to give likely text, not true text. A smooth sentence about a book that does not exist is as likely, by its own scoring, as a sentence about a book that does. The lesson Why AI Makes Things Up goes into this and the habits that protect you.

It does not remember you between chats. The model starts each chat with only what is in its context. If a product seems to remember you, the product is putting notes from earlier chats back into the context. The model itself did not change.

In one sentence: a language model builds likely text from its dials, does not check facts, and sees only what is in its context.

#Where This Fits

The rest of this track opens the box one piece at a time. Attention in Pictures shows how a model decides which earlier words matter. Tokenization shows how text is cut into pieces. After that come the full transformer design, the three model families, and how models are trained, tuned and made safer.

#Key Takeaways

A language model gives a chance to each possible next piece of text, and a large one does this with a very large program trained on a very large amount of text
Training can be as simple as counting what followed what, and the counts become chances
A real model must read all the text so far, because the meaning of a word can depend on words far back
Large means many parameters (dials) and a lot of training text, and the context is how much text the model sees at once
A reply is made by repeating guess, pick, add. The model does not look up answers, does not check facts and does not remember you between chats

#Quick Check

Quick Check1 / 4

In the three-sentence example, why is the chance of 'cat' after 'the' bigger than the chance of 'dog'?

Where to next. Go to Attention in Pictures to see, with four words and a few numbers, how a model decides which earlier words to look at. After that, Tokenization shows how text is cut into pieces. For the dice and dials that pick each token, see How a Chatbot Writes.

What Is a Large Language Model?

Learning Objectives

After this lesson, you will be able to:

Say what a large language model does in one sentence
Explain how counting what followed what teaches a program to guess the next word
Say what large means for a model, using the words parameter and context
Name three things a language model does not do, and why a smooth answer is not proof

Words you need6 terms

A program that gives a chance to each possible next piece of text, given the text so far.

A language model with a very large number of adjustable numbers inside it, trained on a very large amount of text. People shorten it to LLM.

A small piece of text that the model reads and writes, such as a short word, part of a word, or a mark like a full stop.

The study period in which a program reads a huge pile of text and adjusts itself until its guesses fit that text.

One adjustable number inside the model, like a single dial. Training turns all the dials a little at a time.

All the text the model can see right now: your question plus the pieces it has already written.

#A Very Good Guesser of the Next Word

Type "See you" on your phone and it offers a few next words, perhaps "soon" or "later". It is guessing. It does not know your plans.

Before you read on, make a guess.

What Do You Think?

A phone has never been told your plans, but it still offers 'soon' after 'See you'. Where do you think the suggestion comes from?

In one sentence: a language model gives a chance to each possible next piece of text, and a large one does it with a very large program trained on a very large amount of text.

#How the Skill Is Learned: Counting What Followed What

How could a program learn which word usually follows another? The simplest way is to count. Here are three short sentences. Pretend this is everything the program has ever read.

the cat sat on the mat.
the cat ate the fish.
the dog sat on the rug.

Now ask one question: every time the word the appeared, which word came right after it? Go through the sentences and tally.

Word after "the"	Times seen	Chance (out of 6)
cat	2	2 of 6, about 33%
mat	1	1 of 6, about 17%
fish	1	1 of 6, about 17%
dog	1	1 of 6, about 17%
rug	1	1 of 6, about 17%

In one sentence: training can be as simple as counting what followed what, and the counts turn into chances for the next word.

#Run It: A Language Model That Only Counts

The cell below does exactly the counting you just did by hand, then uses the chances to write. Press Run. It uses plain Python, with no extra library, so it runs right here in your browser.

pythonplayground.py · Pyodide

# A tiny language model that only counts. No maths library needed.
import random

text = "the cat sat on the mat . the cat ate the fish . the dog sat on the rug ."
words = text.split()

print("After 'the' we saw:", counts["the"])
print("After 'sat' we saw:", counts["sat"])

# Guessing: turn counts into chances.
def chances(word):
  row = counts[word]
  total = sum(row.values())
  return {w: n / total for w, n in row.items()}

for w, p in sorted(chances("the").items(), key=lambda kv: -kv[1]):
  print(f"  chance of '{w}' after 'the': {p:.0%}")

print()
for seed in (1, 2, 3):
  print(f"seed {seed}:", write("the", 8, seed))

assert chances("the")["cat"] > 0.3
assert counts["sat"] == {"on": 2}

Do: Run the cell once. Then change the numbers in the line for seed in (1, 2, 3): and run it again. Next add the sentence the dog ate the fish . to the end of text and run it a third time.

Notice: The counts match the table you made by hand: "cat" has 2 and the other four have 1 each. The three written sentences differ from each other, and some are odd. A new sentence in text changes the chances.

Why: The program only keeps counts, and it picks the next word by chance in proportion to them. Different seeds give different picks. A drawn stand-in, not a large model: this one looks at only the one word before.

In one sentence: a counting model learns a table of what followed what, and writes by picking from that table by chance.

#Why Counting One Word Is Not Enough

The counting model looks at only the last word. That fails fast. Read this sentence.

The trophy did not fit in the suitcase because it was too big.

In one sentence: a real model guesses from all of the text so far, because the meaning of a word can depend on words far back.

#Pieces, Not Words: Tokens

You do not need the details yet. The lesson Tokenization shows how the pieces are chosen. In this lesson, read "token" as "a small piece of text".

#What Large Means: Dials and a Reading Room

Why is it called large? Two things are large.

Most of the skill of a large model comes from two things together: many dials that can hold many patterns, and a very large amount of text to learn those patterns from.

In one sentence: large means many parameters and a lot of training text, and the context is how much the model can see at one time.

#From One Guess to a Whole Reply

Two phases of a language model

An architecture diagram: boxes are components, arrows show the direction data flows, and moving dots mark the live flow.

🗄️Huge pile of text

🔧Training: turn the dials

🧠Trained model

👤Your question

⚡Guess, pick, add, repeat

📦Reply

🗄️ database🔧 tool🧠 llm👤 user⚡ cache📦 service

Now watch the guess-and-pick loop in a toy. It has a small hand-written table of next-word chances, like the one you just counted.

Loading visualization...

Do: Choose the Cat starting phrase. Press Generate next word four times. Then press Reset sentence and Generate again, and press New seed to try another run.

Notice: Each press adds one word, and the bars change because the next word depends on the last one. Different seeds give different sentences from the same table.

Why: The model picks by chance in proportion to the bars, so likely words win often but not always. Drawn stand-in, not a trained model: the scores are hand-written for a few words.

In one sentence: a reply is made by guessing, picking, adding and repeating, and the dials do all the guessing.

#What a Language Model Does Not Do

Three things are easy to assume and wrong.

In one sentence: a language model builds likely text from its dials, does not check facts, and sees only what is in its context.

#Where This Fits

#Key Takeaways

A language model gives a chance to each possible next piece of text, and a large one does this with a very large program trained on a very large amount of text
Training can be as simple as counting what followed what, and the counts become chances
A real model must read all the text so far, because the meaning of a word can depend on words far back
Large means many parameters (dials) and a lot of training text, and the context is how much text the model sees at once
A reply is made by repeating guess, pick, add. The model does not look up answers, does not check facts and does not remember you between chats

#Quick Check

Quick Check1 / 4

In the three-sentence example, why is the chance of 'cat' after 'the' bigger than the chance of 'dog'?

What Is a Large Language Model?

After this lesson, you will be able to:

#A Very Good Guesser of the Next Word

#How the Skill Is Learned: Counting What Followed What

#Run It: A Language Model That Only Counts

#Why Counting One Word Is Not Enough

#Pieces, Not Words: Tokens

#What Large Means: Dials and a Reading Room

#From One Guess to a Whole Reply

#What a Language Model Does Not Do

#Where This Fits

#Key Takeaways

#Quick Check

Related Lessons

Your Reflection

Continue Learning

Discussion

What Is a Large Language Model?

After this lesson, you will be able to:

#A Very Good Guesser of the Next Word

#How the Skill Is Learned: Counting What Followed What

#Run It: A Language Model That Only Counts

#Why Counting One Word Is Not Enough

#Pieces, Not Words: Tokens

#What Large Means: Dials and a Reading Room

#From One Guess to a Whole Reply

Two phases of a language model

#What a Language Model Does Not Do

#Where This Fits

#Key Takeaways

#Quick Check

Related Lessons

Your Reflection

Continue Learning

Discussion

Two phases of a language model