Three games that secretly teach you how AI thinks

You were training an AI. Well — you were doing the exact job a computer does when it trains one.

A machine-learning model has knobs, called parameters. A tiny model might have two; the models behind chatbots have billions. Every setting of the knobs is a spot on a landscape, and the height there is the loss: a score for how wrong the model’s answers are on its training examples. Training means finding the knob settings with the lowest loss — the bottom of the landscape. Your golf course was a landscape with just one knob: where the ball is.

Why the fog? A computer can’t see the whole landscape either. With a billion knobs there are far too many spots to check them all. What it can do is measure the loss where it stands, and the slope — the gradient — that says which way is downhill. That’s exactly what the Loss and Gradient boxes above the course showed.

The bot’s rule is called gradient descent, and it is the workhorse of modern AI:

Why the bot got stuck. Gradient descent only ever feels the slope under its feet. When it rolls into a valley, every direction is uphill, the slope drops to zero, and it stops — even if a deeper valley is just over the next hill. That spot is called a local minimum; the deepest valley of all is the global minimum. If you found the hole, it’s because you did things the bot can’t: you took big exploring shots, remembered the loss at every spot, and jumped over hills.

So how does real AI cope?

• A good learning rate. Too small and training crawls; too big and it overshoots the valley and bounces about.
• Momentum. Like a heavy ball that keeps rolling through small dips instead of stopping in them.
• A little randomness. Stochastic gradient descent measures the slope on a small random batch of examples each step, so its path wobbles — and a wobble can shake it out of a shallow valley.
• Many dimensions. With millions of knobs, a true dead end is rarer than on our one-knob course: there’s usually some direction that still goes down. Real loss landscapes are strange places, and researchers are still mapping them.

Training an AI means rolling downhill on a landscape you can’t see, one small step at a time.

You were playing the game at the heart of every language model. A language model is trained on a huge pile of text — books, websites, articles. Its training task sounds almost too simple. Hide the next word, guess it, and check. Then nudge the knobs so the right word becomes a bit more likely (with gradient descent — yes, the golf again), and repeat billions of times. Along the way it learns, for any start of a sentence, how likely each possible next word is. (Chatbots then get extra training from people’s feedback, so their answers are helpful and polite — but the next-word guesser is still doing the writing.)

Writing is just guessing, over and over. To answer you, the model picks a next word, sticks it on the end and asks again — word after word. (Strictly, models work with tokens: whole words or pieces of words. Common words are usually one token; a long or rare word gets split into two or three pieces.)

Tiny model, giant model. The model in the game looks only at the last couple of words and learned from a few hundred sentences. Real chatbots look back over thousands of words — the newest, hundreds of thousands — and learned from a large slice of the internet, so their guesses capture grammar, facts and style. But the game is the same game.

Likely is not the same as true. A language model picks words that are likely to come next, not words it has checked. That’s why a chatbot can write a confident, fluent sentence that is completely wrong — people call it a hallucination. When a fact matters, check it.

Temperature is the creativity dial. A model doesn’t always take the single most likely word. At low temperature it almost always does — safe, but repetitive (at temperature 0 it always takes the top word, so it writes the same thing almost every time). At high temperature, unlikely words get a real chance: more surprising, and eventually nonsense. If you reached the end of the game, you already turned this dial yourself.

A language model is a next-word guesser trained on a mountain of text. It predicts what is likely — not what is true.

An image is a grid of numbers. Each pixel stores how bright it is — usually a number from 0 (black) to 255 (white). A colour pixel stores three: how much red, green and blue. So a square Instagram post of 1080 × 1080 pixels is about 1.2 million pixels, or 3.5 million numbers. That’s all a computer ever gets.

Resolution is how many numbers you keep. The blocky versions in the game were made by averaging squares of pixels into one. Fewer, bigger blocks keep the overall colours and shape but throw away detail — which is why some pictures were easy at 4 × 4 and others needed every pixel. Vision models shrink things on purpose too (it’s called pooling): after spotting patterns, they summarise each small square by its average or its strongest value. That helps them focus on the overall shape rather than single pixels.

How does an AI find a cat in a grid of numbers? It doesn’t look at the whole grid at once. A convolutional neural network slides small windows of weights — filters — across the image. A filter gives a big number wherever its pattern appears, such as dark pixels next to light ones: an edge. Layer by layer, the network builds up:

Nobody tells the network which edges or shapes to look for. It learns its filters from thousands — often millions — of labelled photos, using gradient descent: rolling downhill on its loss, just like your golf ball. Newer models called vision transformers cut the picture into small square patches and treat them a bit like the words in a sentence — but underneath, it’s still numbers in, numbers out.

To an AI, a picture is just numbers. It sees by finding small patterns — edges — and stacking them into bigger ones.