All simulators
Word embeddings
Explore a map of word meanings. Measure cosine similarity and try analogies.
Explore a map of word meanings. Measure cosine similarity and try analogies.
Each category feature points along its own spoke; gender nudges words sideways, size/age up or down, royalty diagonally. It is a hand-made flattening of 9 numbers into 2. Tap a dot to set word B.
A computer cannot compare meanings directly, so we turn every word into a list of numbers, a vector. Here each word has 9 hand-made features such as food, animal, royal and gender. Words with similar meanings get similar lists, so their arrows from the origin point in similar directions.
Cosine similarity measures that direction: 1 means the same direction, 0 means a right angle (unrelated), negative means opposite. Euclidean distance measures how far apart the tips are, so it also cares about length. Adding and subtracting vectors gives analogies: king − man + woman keeps “royal” and swaps the gender.
Be honest about the toy: real embeddings (word2vec, GloVe, or the first layer of a chatbot) have 100 to 4,000 dimensions that are learned from which words appear near each other in billions of sentences. No single dimension has a tidy name, and analogies only work roughly. Our king − man + woman = queen is exact only because we built the vectors that way.
Takeaway: meaning becomes geometry. Similar words point the same way, and cosine similarity is how search engines, recommenders and chatbots find “words like this one”.
| feature | A | B | A×B | (A−B)² |
|---|---|---|---|---|
| food | 0.2 | 0.1 | 0.02 | 0.01 |
| drink | 1 | 1 | 1 | 0 |
| animal | 0 | 0 | 0 | 0 |
| person | 0 | 0 | 0 | 0 |
| place | 0 | 0 | 0 | 0 |
| sport | 0 | 0 | 0 | 0 |
| royal | 0 | 0 | 0 | 0 |
| gender | 0 | 0 | 0 | 0 |
| size/age | −0.5 | −0.4 | 0.2 | 0.01 |
| sum | not needed | 1.22 | 0.02 | |
|A| = √1.29 = 1.136, |B| = √1.17 = 1.082
cos = A·B ÷ (|A|·|B|) = 1.22 ÷ (1.136 × 1.082) = 0.993
distance = √0.02 = 0.141
Nearest neighbours of chai (by cosine)
v = (0, 0, 0, 1, 0, 0, 1, 0.6, 0.6)
The classic. The gender arrow from man to woman, added to king.
A computer cannot compare meanings directly, so we turn every word into a list of numbers, a vector. Here each word has 9 hand-made features such as food, animal, royal and gender. Words with similar meanings get similar lists, so their arrows from the origin point in similar directions.
Cosine similarity measures that direction: 1 means the same direction, 0 means a right angle (unrelated), negative means opposite. Euclidean distance measures how far apart the tips are, so it also cares about length. Adding and subtracting vectors gives analogies: king − man + woman keeps “royal” and swaps the gender.
Be honest about the toy: real embeddings (word2vec, GloVe, or the first layer of a chatbot) have 100 to 4,000 dimensions that are learned from which words appear near each other in billions of sentences. No single dimension has a tidy name, and analogies only work roughly. Our king − man + woman = queen is exact only because we built the vectors that way.
Takeaway: meaning becomes geometry. Similar words point the same way, and cosine similarity is how search engines, recommenders and chatbots find “words like this one”.
Things to try