Why we write our own C and C++ inference engines

Just from the subtitle, you can already tell it's likely AI-generated. Maybe you're lucky enough not to be exposed to claude/codex output your whole working day.

Use lynx, links or netsurf. Or vibe code your noscript/basic HTML web engine?

I stopped reading almost immediately. The stylistic choices in the writing just felt like LLM to me. Examples:

"depth estimation that beats PyTorch on CPU in half the memory" — "…beats X in Y…"

"Most LocalAI backends wrap somebody else’s engine, and that is the right default." — "…and that is the right"

"MLX and the rest are maintained by people who are better at those models than we are" — "better at those models than we are" — it's this thing that LLMs do where they are kind of weirdly confident but overly deferential

"This post is about what those ports buy" — "…buy" used in this context

"Same model, 1.31x the speed" — "Same X, something Y" — it's this overconfident yet deferential writing style

The further I read, the more tells there are. I find it incredibly tiring to read LLM generated prose and I'm not sure why. Is it because I'm aware it's not human written and have an unconscious bias? Or is it because the style is just full-on, "Not X but Y. Those performance gains are bought, not earned. This stops, that starts. Read on, or don't, that's the follow-up"

[flagged]

I've had this debate before on HN. The excuses are generally "well it can write better code than most developers, but an LLM can't write better prose than most people" (I strongly disagree with this) and, what I think is at the heart of the matter, "text is for the reader to read directly, code is hidden." Or in other words, "as long as I can't tell it's AI, it's fine."

The thing is... although the writing is unmistakably full of LLMisms, I can't fault the `author` for having produced a slop readme. The content earns its keep, it only grates because of the robotic personality. We need another word than "slop" for this.

"blland", "llame",... ?

While I am tired to see slop-infested pull requests being celebrated.

"This post is about what those ports buy, measured, and what they cost."

if you've done any amount of optimizaation or hill-climbing work (kernel optimization, autoresearch type shit), you would know that these models LOVE the terms "buy" and "cost" in this sense. they use them nearly constantly (along with "budget", and even "credit" (even when working with RL/credit assignment!))

The post is full of signs. Here's only a couple of examples:

> The method, the measurements, and what it costs us.

> That is the general shape of these wins.

> Parity is the gate, speed is the follow-up

I could go on and on, but you probably get the point. If you don't find anything funny with the above, you might have not been enough-exposed to slop.