Blogs

The descriptions keep sounding familiar. Someone writes about an eval set that has stopped catching regressions. Someone else describes a judge model that agrees with itself on every run and is wrong in the same direction on every run. The two problems have nothing to do with each other, and both read like something already lived through. There was no particular article where that landed, no conversation that set it off. It accumulated, the way these things do, until the recognition got frequent enough to be worth distrusting. ...

Why Your LLM Is Slow for Three Completely Different Reasons Picture a kitchen with exactly one chef and one recipe book sitting on a shelf across the room. Two things happen in this kitchen, and they are not the same kind of work at all. When an order comes in, the chef reads it in full, then preps every ingredient in it at once - dicing, measuring, laying everything out on the counter in parallel, because the whole order is already known. That is fast, and it gets faster if you hire more hands. ...

The Persona Is a Pathway, Not a Personality Ask an LLM to “explain this like I’m 5” and the answer gets simpler, warmer, full of everyday comparisons. Ask the same question with “answer as a senior systems architect” and the answer tightens up, drops the hand-holding, reaches for precise engineering vocabulary. Something is clearly happening. The question worth sitting with is what. The easy answer is that the model is doing something like what a person does: imagining an audience, picking a register, adjusting on the fly. That’s a comfortable answer because it maps onto something you already understand - a teacher switching between a lecture hall and a kindergarten classroom. It’s also wrong in a way that matters, because it hides a real production risk directly underneath it: a persona changes how confident and fluent an answer sounds without touching whether the answer is actually correct, and it holds up unevenly across different personas for reasons that have nothing to do with how well you phrased the prompt. ...

The Guardrail Has to Live Outside the Model The first fix everyone reaches for is the same one: tell the agent not to do the bad thing. Put it in the system prompt. “Never access another customer’s account. Never transfer funds without confirmation.” Ship it, watch it work in testing, move on. It will keep working right up until someone phrases the same request differently enough, or buries it inside something that looks like a normal turn. Then it won’t. Understanding why takes one honest look at what a system prompt actually is. ...

Graph RAG Is Not Better RAG Picture a small experiment. You have a handful of short documents about drug interactions - except the drugs are invented. Not real names: Faxiprine, Tricalamine, Renal Shimmer, Hydrolyx Draught. That matters, because if the drugs were real, a language model could answer from what it already learned in training and we’d never know whether retrieval was doing any work. Fictional drugs force every answer to come from the documents in front of it. Nowhere else. ...