PC terms decoded · AI and work
What does Tokens per second mean?
How fast an AI model writes. A token is about three quarters of a word. 20 or more tokens per second feels faster than reading.
Why is it called that?
Decode any part nameSpeed mostly depends on memory bandwidth, not raw compute. That is why VRAM speed matters so much for local AI.
Questions people ask
What does Tokens per second mean?
How fast an AI model writes. A token is about three quarters of a word. 20 or more tokens per second feels faster than reading.
Why is it called Tokens per second?
Speed mostly depends on memory bandwidth, not raw compute. That is why VRAM speed matters so much for local AI.