PrimordialPC

PC terms decoded · AI and work

What does Tokens per second mean?

How fast an AI model writes. A token is about three quarters of a word. 20 or more tokens per second feels faster than reading.

Why is it called that?

Speed mostly depends on memory bandwidth, not raw compute. That is why VRAM speed matters so much for local AI.

Decode any part name

Questions people ask

What does Tokens per second mean?

How fast an AI model writes. A token is about three quarters of a word. 20 or more tokens per second feels faster than reading.

Why is it called Tokens per second?

Speed mostly depends on memory bandwidth, not raw compute. That is why VRAM speed matters so much for local AI.

Related