Why Your LLM Is Slow for Three Completely Different Reasons
Why Your LLM Is Slow for Three Completely Different Reasons Picture a kitchen with exactly one chef and one recipe book sitting on a shelf across the room. Two things happen in this kitchen, and they are not the same kind of work at all. When an order comes in, the chef reads it in full, then preps every ingredient in it at once - dicing, measuring, laying everything out on the counter in parallel, because the whole order is already known. That is fast, and it gets faster if you hire more hands. ...