LLM Series · 2026-03-03
TOKENS Explained
When you give a Shakespeare poem to an LLM, it doesn’t process it as words or sentences. Every character is first converted into numbers between 0–255, raw byte
LLM Series · 2026-03-03
When you give a Shakespeare poem to an LLM, it doesn’t process it as words or sentences. Every character is first converted into numbers between 0–255, raw byte