MiniMax M3.1 Flash
minimax.io467 points by jtg 18 hours ago
467 points by jtg 18 hours ago
This model seems to be pretty benchmaxed. In actual use, it doesn't survive the comparison to models in the benchmark.
This model was released as a "stealth model" in OpenCode Go for a while as "Space Bunny" (they mention this in the blog). So people have been using without knowing what the model was exactly, but it failed to impress anyone as far as I can tell. This is despite the model being served for free (with some asterisks). GLM 5.3 Flash and DeepSeek V4.1 Flash, even before last week's update, seemed to be much preferred. Stealth releases happen often with OpenCode subscriptions, and GLM 5.3 Flash (stealth name "Ox Alpha") impressed a lot for example.
The perverse incentives around benchmarks are now so bad that benchmark scores are effectively meaningless. Models can score highly and still be crap to use. It doesn't matter whether it's Minimax or Google, or anyone really.
For what it's worth, it does perform on par with DeepSeek V4 Pro on my adversarial programming language benchmark[1]. I wouldn't think they would benchmax on that.
Any chance you could try qwen3.8 flash next? That is currently the best local model for actual coding tasks in my + many coworkers experience. It will run reasonably fast if you have 64+ gb ram and use strata or a strata fork (vram not super important with this method).
I used Space Bunny from some providers that offered it for free and it was a very much a "you get what you pay for" experience. If the release is comparable, I'll be sticking with Deepseek v4 for sure.
I’m one of the unimpressed. I tested SpaceBunny intensely on a research project for a week.
The model wasn’t horrible at coding, it just made a ton of initial mistakes—but then would catch most of them later on. It can get work done if coaxed, but its peers are just cheaper and/or easier to work with.
What kind of mistakes?
I've used local models extensively (mostly qwen 3.8 27b) and now Claude sonnet/opus 5.5 since launch.
Sometimes I really wonder if not being able to see what's going on in thinking is a large contributor to thinking the models are stronger.
That assumes you bother skimming through thinking traces and the output in-between.
In my experience both qwen and claude fumbles and misunderstand, but will in a lot of cases self correct.
Qwen models can feel very indecisive but somehow makes it in the end. If the harness was able to hide that and only highlight the end results, I'd probably think qwen models are smarter.
Reading the tool calls Claude does while thinking shows a lot of trial and error about even simple things like editing code and getting the whitespace wrong. But it happens so fast compared to qwen that it's sort of acceptable. With qwen, it can feel frustrating because it spends so much time on it.
This is all assuming that's what you meant by mistakes, but maybe you meant something different.
> The perverse incentives around benchmarks
I am not sure. The people who actually care enough to look at benchmarks will run evals, and find out quickly. If you post silly benchmark scores too often, you will simply be ignored. You are paying with your reputation.
>If you post silly benchmark scores too often, you will simply be ignored.
i don't know about that. it seems like the silly benchmarks keep getting paid attention to.
All the benchmarks are silly but people still want a quick way to put a new release in context
Those consequences never stopped blog spam previously. Won’t stop it now lol
This is my experience with all MiniMax models: They're good on paper but fail to live up to expectations in practice. IMO, at no point has any MiniMax model been the superior choice over others, all things considered.
Yeah, I think of MiniMax as a good video generation model provider. Their languages models, by comparison, are not as impressive.
I've been building a cybersecurity platform (called ZeroQuarry) and regularly benchmark new models against a pretty wide set of security both code review tasks and live application pen testing tasks, and this one far underperforms relative to GLM 5.3 from every test I've done for real-world cybersecurity tasks
I don’t get the endgame. If your model underperforms in real life usage, the backlash comes quickly. I guess I’m asking what exactly the perverse incentive is? A day of press at most?
A day of press is probably better than no press at all, if you're the CEO of a publicly traded company
Oh yeah, Space Bunny was awful, massive step back from even Qwen3.8-Flash-Next
I know I mention it way too often right now, but I am just so impressed with Qwen Flash Next right now. I have been using it locally and it's just such a great little model to work with. It makes mistakes and has its quirks, but in general, it just gets things done and its default manner is just right. It is not overly proactive or making messes for me, even compared to DeepSeek Flash 4.1. I really like DS Flash too, it can just be /really/ annoying in some roles and to chat with some of the time.
I can run Qwen Flash Next on hopes, dreams, DDR4, and ancient data center GPUs at 200 t/s :)
Same here. I run Qwen3.8 Flash Next on my local machine and combine it with Opus 5.5 for plan making. The Qwen model feels like it's around Opus 4.x level performance most of the time.
But once in a while it gets into long reasoning loops, which is annoying. It seems less with the Swift model but I haven't benchmarked the difference between that one and the original.
Yes, I feel Qwen3.8-FN is the first model that seriously can replace some lower tier subscription usage without requiring much more than a high end gaming PC. Qwen3.6-35B was impressive but not quite good enough, and 27B takes a lot more time.
Lately I prefer GLM5.3-Flash since I feel it is noticeably smarter and I have the hardware for it, but FN's performance has me looking forward to Qwen4.
I have 27B at around 160-170 t/s on my hardware but why bother when QFN is faster and generally better :)
That's what I like about DeepSeek. Whatever benchmark scores show, it always felt much better in actual usage compared to benchmaxxed models.
They genuinely try to improve usability, without spending time on marketing or benchmaxxing.
Except for v4 0731, which was a regression, and probably the reason they released v4.1 ahead of schedule.
"When a measure becomes a target, it ceases to be a good measure." https://en.wikipedia.org/wiki/Goodhart%27s_law
I'm finding these LLM announcements to be pretty meaningless. They all look the same, with different numbers and graphs, even though the graphs are almost all identical. The offered model is compared with models that it can beat, with a couple of entries that "show" they haven't engineered the whole benchmark presentation to mislead.
I've become incredibly skeptical of new model announcements until I can find evidence of someone else using it for something effective. I'd love to see an announcement that wasn't structured around attracting investors with too much money and too little domain knowledge.
To me the most interesting chart on artificial analysis is intelligence index vs cost per task. It's like the third chart down and they graph a pareto line showing where the frontier is and every model on the pareto line has been interesting to me. Luna, Mimo-v2.6-pro, Sol. Hell, Argon.
This MiniMax model is clustered with Haiku 5.5 so (combined with the criticism on here that it was lacklustre while free on opencode go) I don't find it interesting, but every announcement has a chance at pushing that frontier.
I was benchmarking affordable LLMs for japanese->english translation recently and the winner was Kimi K3 - there are a ton of interesting models in the wake of the flagships.
Qwen 3.8 flash next was an important announcement because it easily runs on dual RTX 6000 Pro, which made it available to a subset of fairly high-end local inference tinkerers. For those running local models, checking announcements for vram requirements is the first step in knowing if it’s a candidate to push more intelligence into your local rig. At least, that’s what I do when I read announcements.
What would be an interesting LLM announcement, then? Even Opus 5.5 wasn't an interesting announcement but the impact is obviously significant in hindsight.
For me? A local LLM as powerful as opus 5.5 would be interesting. Or a good local 3d asset generation model. Or something as powerful as Opus 5.5 that works nearly instantly.
All this incremental progress is just more of the same. I want a real novel capability, not just more, faster, cheaper, “smarter”.
.. HN is actually hilarious
We are barely 4 years in since the release of a historically important tech. Of course the next few years are incremental. thats how it always works.
Even last year, before opus 4.5, long-horizon agents didnt really work. I would not say the last 12 months are only incremental
what's even more funny is most of what they are asking for exist in some form.
As an aside, I remember when sora 2 launched and thinking, "wow I wonder if we'll ever get something close this in open weights?" The prevailing sentiment was "maybe 5 years." Yet, we have Minimax H3 and Flux a year and a half later.
When Opus 5.5 launched, within a day you could see that it was a signfiicant step function in capabilities by just following regular subreddits like /r/AIGameDev and /r/ClaudeAI (but really anywhere people publish their software, if you can recognize it).
But this only works for frontier models. If MiniMax catches up to Opus 5.5 in 6 months, no one would notice.
It also works for flash models that push what capability is reasonably available locally, eg with Qwen3.8-flash, Qwen3.8 27B and GLM5.3-flash.
But this only works for frontier models. If MiniMax catches up to Opus 5.5 in 6 months, no one would notice.
I'd be happy as hell if I end up being able to run an Opus 5.5-level model at home within a 6-month time horizon. May or may not happen based on China's export policy, of course.
The open weights we have may be as good as they ever get... but then that's been true all along, considering nobody really knows why they are releasing powerful models for free in the first place.
I think we keep getting used to intelligence improvements and the goal posts are always moving forward.
I remember that close to an year ago i felt the same way about opus 4.5. And even if you can easily run ds or qwen locally to get very similar results to that, we moved to 5.5 being the new target.
Deepseek can do 90% of the job but a claude reviwer/spec writer is a brutal improvement.
Skepticism is due! Good! The numbers don't mean much! Yes!
I personally enjoy seeing how different models view problems. Take big plans from big models, then see how other agents reinterpret a stance. See which directions they go. I love the diversity of views they produce.
It's not really relevant what the numbers are. I really really enjoyed talking with MiniMax M3 a lot. And M2.7. it had a good world view, wrote the situation up incredibly fluently. It thought out loud well. These were pleasant uses for me. The realities, the world's they could see felt very grounded. Good stuff, to me.
The cost vs performance is insane for all these new models. I’ve been using Deepseek Flash 1017 and it does whole projects and asks for $0.10 on US provider with ZDR on Openrouter. I convinced a few friends to drop their Claude $20 subscription because DS Flash 1017 brings so much better value. They can still use Opus 5.5 for planning though each call will cost north of $2. Excited to try this model out too once there are US providers.
Opus 5.5 is according to AA cost per task, 23x more expensive than Deepseek with max effort (which is dumb, you shouldn't use) and 13x more expensive on xhigh.
Claude sub has a 60x subsidy, so a $20 sub gives you $1200 in API.
Based on this math, Claude sub is a much better deal than Deepseek. And it's a stronger model of course.
> Claude sub has a 60x subsidy, so a $20 sub gives you $1200 in API.
This is only true if you max out the plan, which requires working around awkward 5-hour and weekly limits. For side projects, sure, you can use a Ralph loop or whatever and just continue when the window resets. For work, I can't afford to simply stop working when I hit my limit. And using Claude at API prices is prohibitively expensive.
Personally, I have a $20/month OpenAI subscription that I supplement with DeepSeek V4.1 Flash (currently using CoralBricks as the provider since it's fast and I'm grandfathered into their $0 cache read pricing). It's working out rather well so far. Even when I use DS all day as my primary model, I only pay about $1/day, which is roughly equivalent to what I pay for the sub anyway (but with no limits).
You could also go with mimo-2.6-pro for cheaper and often better results.
I tried it but it's way too slow for me. DeepSeek V4.1 Flash is an order of magnitude faster, and the modest difference in intelligence isn't worth it to me for most use cases.
I am sorry, but if you're given $1200 in api for free I wouldn't consider that as working around flakey limits especially when you only need to max out the 5 hour limit 5 times a week.
I don't understand what you're saying. If I hit my 5-hour limit in 2 hours, I still need to get work done for the other 3. I'm not going to sit there and wait for my limit to reset.
you get more than $20 worth of usage within 1 5 hour window at which point it pays for itself.
But we're not comparing a Claude subscription to using Claude at API prices; we're comparing it to using a cheap model like DeepSeek V4.1 Flash at API prices. If I hit the 5-hour limit in 2 hours with Claude and have to wait 3 hours for it to reset, I get less value out of that subscription than I would have if I'd have just used DeepSeek to begin with, and the latter isn't more expensive.
So my original point about the limits being awkward still stands.
depends on your usecase, but $2000 of deepseek cannot get you the results of $20 opus 5.5 subscription if you hit specialized domains