Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

I don’t want to be a cranky codger, but I dont get why 5 minutes of work cleaning up the README can’t be done before posting to HN.

> Run Qwen3.8-Flash-Next on a Mac that can't hold it

This is the first line of the README. I can't believe people are becoming ok with this, and I'm 100% on the AI train.

I hate this AI style writing because since it doesn’t really understand flow, it’s being inserted in irrelevant places and it is extremely irritating to read.

I used similar prompts before. Now I simply say to "remove historical cruft" and results are good enough. It's the model itself that first used this wording, I found it concise.

thanks! I'll do!

[flagged]

We are running 35b-A3b with 264k context (the model's default max) using vllm and the "frog" jinja templates: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates and had good luck. We are mostly running agentic workloads though, rather than coding. 27b has a slightly higher agentic job completion rate (95% vs 92%) but the 3% trade off is worth it because the A3B is sooooo much faster, and we reprocess the other jobs with a different model. Don't sleep on the froggeric templates.

Qwen: Looking at you for a new ~35B MoE! Please and thank you

You should try Glimmer MTP. Qwen3.8 27B seems to have weird memory and caching issues on oMLX

yes, with qwen3.8-27b-4bit run via rapid-mlx i can get to about 200k

[dead]

Now I'm feeling pretty good about getting 10-11 tokens/sec running Qwopus 3.6-35B-A3B Q6_K on an old Mac Pro 2013 (trashcan) with 128GB RAM (DDR3), 12 core Xeon, dual D700s. Arch Linux and llama.cpp.

Anything smaller than a 16” runs into serious thermal problems; even an identically equipped 14” just can’t dissipate enough heat.

interesting! Yes, thermal is important. Pretty cool project man! Starred and checking it out!

32GB is simply too tight; you need 8 minimum for the OS and you need about 4-8 more for the LLM you’re visiting and kv cache.

Yes, but also 12 tok/s versus Claude is so far from comparable. I know that it’s not exactly 1:1, but it’s a long way from an easy trade-off, especially considering hardware prices for high levels of RAM.

yes! I'm bullish on this. there is a lot of work to do. I've been experimenting with pruning, distillation, and retraining too. I'm sure your 32gb m6 will run a badass local model!

For a local non-coding agent, instruction following and tool use are the most important gains

This is the best I could find: https://huggingface.co/Qwen/Qwen3.8-Flash-Next?utm_source=ch...

About the specifics, I have only anecdotal evidence, but I guess this info can be found somewhere

I think there's a very good chance that history will rhyme a bit.

DOS/Windows and PC clones were by no means the best available, but they were cheap, ubiquitous, and versatile compared to alternatives that were either much better at one task but more expensive or better at everything but wildly expensive. They were "good enough" and represented a solid improvement over what many existing computer users had as well as a good entry point for new users. As such they spread like wildfire and became the standard while the expensive alternatives either became hardcore niche or vanished.

Unpleasant for interactive agentic work. Still useful to leave it to do some work in the background.

Good one! I haven't measured this. I'll include it!

Ha! AI;DR is a great phrase. Had not seen that before

The never ending gate bites.

How I have come to detest certain phrases.

High Bandwidth Flash? https://www.sandisk.com/company/newsroom/blogs/2025/scaling-...

yes! I guess future hardware designs will have something like that!

A particularly worrying situation considering a dead SSD will render your macbook usable for parts only.

no it's reading, not writing

By reading it?

I think multiple people working on the same thing is great.

Everyone comes at it from a different point of view, and some approaches work, some don't. And when people do this themselves they learn. Existing projects have their mistakes worked out already.

Maybe one of these people is going to come up with the thing that nobody else thought of because of their experience working the problem from scratch. You may not get that from someone working from an existing project, because existing projects have their approach "baked in."

What all these projects are showing so far is that it's possible to stream from disk, but that the performance isn't ideal. But I'm sure you could take this approach with smaller models and get better performance.

In addition, it's a given that when you work with large data sets performance means organizing the data to take advantage of caches, both disk and cpu. It's not clear how that would work, exactly, given that each run is a not-quite-random walk through the data. The Big Data way is to prebuild all of that as much as possible, which is probably impossible with a big model. But what about a smaller model?

I see your point. As an oss defender myself, I agree, however, the spirit of this is to see how fast I can make it. I'm sharing this with the community, which I think is aligned with the original oss spirit.

It's an experiment for myself but I am committing to maintain it. I've been an oss person for a loooong time, way before AI was a thing. Think about it as a new, from-scratch take at it, not as a re-reproduction.

This is one of the aspects of this year that I've been finding very grating and wasteful. Collaboration still happens among people with the ability to do so and the technical skills, but everyone else is taking their own helicopter to the top of the mountain, "putting it out there", and there's just a ton of redundant projects that do the same thing.

Vouched especially since OP might have a perspective on this. And readers may want to look up those other repos and compare for themselves.

Why should they do that? For you? You could merge those projects and see if they get traction.

And there are `Mference` and `SwiftLM` too, I think they are doing the same use case.

AI NIH

> every implementation idea rediscovered five times and wrapped in a new README

That's open source since forever, unfortunately.