Three Words, Fifty Thousand Slots — medium Sequential Data problem | Incognition
Three Words, Fifty Thousand Slots
20 pts · 20 coins
Sequential DataMedium
Three Words, Fifty Thousand Slots
A tokenizer with a vocabulary of V=50,000 has handed out these IDs:
token
ID
cat
412
car
411
dog
87
Each token is then one-hot encoded: a vector of length 50,000 that is 0 everywhere except for a single 1 at the position given by its ID. Call these vectors ccat, ccar and cdog.
The straight-line (Euclidean) distance between two vectors is
∥u−v∥=∑i=1V(ui−vi)2
What are the two distances ∥ccat−ccar∥ and ∥ccat−cdog∥?