Show HN: Maple-Preview – Ternary 20B MoE running at 120 tok/s on a iPhone

I wish that "small" LLMs would stop being confidently very incorrect. Admittedly this is a bit of an intentionally esoteric test, but the confident way in which it presents a totally incorrect answer is a bit concerning.

"please write 250 words on the etymology and history of the word schlong"

https://pastes.io/uhshFgn4

The actual origin of the word is from middle high German and Yiddish-speaking Ashkenazi Jewish communities.

For comparison qwen 3.6 35B A3B does perfect on this and will give a solid description of the word's real origins and how it has made it into casual profanity/vulgarity as used in US English, and even mentions specific stand-up comedians and famous public figures of specific ethnic/religious origin in the US NE who introduced it into wider use.

Ask it for something that's not a narrow niche scientific or technical field, but something that would be less common to make it into a 20B size model, and see just how it does.

chat test link: https://chat.deepgrove.ai/

The most interesting thing is the "dreaming" idea for on-device adaptation. I guess regularly adjusting weights becomes possible when the model is so small and the math is simplified.

I'm assuming that doesn't actually exist yet, though, as I don't see anything about an implementation in the code that's been released.

But, it's a really interesting idea for a personal model. There's a risk of more AI psychosis if these things actually start "learning", but the value of it is also probably pretty big. I'm not sure I buy it will actually be able to self-improve, though. The best models are helping improve themselves, but the best models are considerably smarter and more capable than this one. I've asked models like Gemma 4 31B to help figure out training and synthesizing data, and it mostly fails on anything more than categorization and summarization. This little model is much dumber than that.

So, I'm skeptical, but maybe there's deterministic tooling that can assist and maybe it will be scoped tightly enough to just learn and update facts and not so much try to retrain the whole thing.

For small models like this, it’s super important that it works well at tool calling etc. imho because it can’t memorize facts and isn’t big enough to tell when it doesn’t know. I could use it for high quality tool routing or a backup fast model for smaller task set. E.g. I use GPT-5.6 for voice channels at home. I’d prefer to be able to have this do basic tool calls and stuff because of the local speed.

Will give it a crack as a quick model in my clawlike.

A benchmark table comparing to Qwen 3.5 35B-A3B seems strange when Qwen 3.6 35B-A3B has been out for some time and is significantly better.

I didn't notice the version difference when first reading the article! So this is a heads up to people like me.

“Current approaches to low precision primarily focus on converting models trained in full precision to lower bitwidths. We view this as fundamentally the wrong approach…”

Very excited to see how it performs, I’ve been a bit skeptical of the efficacy of converting existing models - really cool to see one trained from scratch in the ternary format.

At this point I think Apple just needs to simply not do anything stupid and these small model makers are going to hand them models

In mice. I mean Mac Mini M4 not an iPhone.

Also I love AI sites. Fancy font, plain serious style, we "introduce" rather than "release". It's an AI not an animal after all.

My question is what LLM the author used for dev the web page? Maybe Claude? or ChatGPT?

As of now, 3 of the 5 comments on this page are just 0-1 karma accounts high-fiving the article. Suspicious.

amazing work! impressed with https://deepgrove.ai/chat

Looks promising though much like the bonsai tenary one it hallucinates knowledge quite aggressively. The online chat having search tools covers this up somewhat, but it's still there.

...from very unscientific casual vibes it does seem pretty good though considering the speed

Would also be curious what their search tool backend looks like - that too is very fast for very rapid multiple searches

the dreaming demo peaks at 5.9 GB, is that mac-only? ios would jetsam a third-party app well before that on most iphones

Anyone compare this to ternary bonsai vs the 1 bit

>"At ultra-low bitwidths,

matrix multiplication can be effectively replaced with additions

, lowering the total arithmetic workload needed to infer through a model."

Interesting! I never knew that before... I will have to do some more research on this, but yes, it definitely sounds possible!

(Also (and this is just a wild guess/hunch here!)

in theory, matrix multiplication could be replaced with pre-computed cached lookups from a lookup table in memory

if, if and only if the resultant pre-computed lookup table of result matrices could fit into memory available!

In other words, think of a Matrix not as a 2D array, but as a 1D string. Think of the Matrix multiply operation as first appending the 1D string of the first matrix to the 1D string of the second matrix, then using that string as a Key to look-up a precomputed result in Memory...

Of course, the limitation to this approach is the availability of memory relative to the permutations of input values... It could work -- but only for smaller matrices... which leads to another strange idea... in that set of pre-computed matrix permutations, there might be duplicates... figure out a way to put duplicates in memory only once (could use pointers or some other scheme to do this) and you could theoretically store more matrices in the same memory and/or slightly larger matrices!

Anyway, just rambling out loud! :-)

Remember, "almost all programming can be viewed as an exercise in caching" to quote the legendary programmer Terje Mathisen! :-) )

Anyway, great article!

Edge is edging closer!

Super cool and a taste of what's to come with local AI becoming more accessible to low-end hardware.

This is cool!

[dead]

[flagged]

[flagged]

[flagged]