Anthropic says Alibaba illicitly extracted Claude AI model capabilities
reuters.com738 points by htrp a day ago
738 points by htrp a day ago
There's two basic kinds of distillation: 1) the massive [and dumb] method where you ask a question and use the answer as reinforcement (Black Box), and 2) more targeted distillation where you use one model to directly inform/train/guide another model (RLAIF).
The latter is basically fine-tuning the model with direction from another model. Thousands of businesses do this every day to fine-tune. This is almost certainly what the Chinese labs are doing, since it has a much better effect on the end result than just getting simple answers to simple questions.
These complaints of distillation are inflating the problem to make it sound worse than it is, because they want the USG to block/ban Chinese model providers as protectionism. They have already called for more export controls on chips (which is funny because DeepSeek v4 was designed to run on Huawei chips and now the other Chinese providers are following suit). But they can't come right out and say that, so their claim is that they're asking for more export controls because distilled models might not be as safe as their own. But if you show them a jailbreak of their model that bypasses their safety, they'll tell you that any model can eventually be jailbroken so don't worry about safety.
> These complaints of distillation are inflating the problem to make it sound worse than it is
Unfortunately, the Reuters piece itself is complicit in this dramatization. The lede paragraph parrots Anthropic's talking point that distillation is an "attack", without using quotes that would alert the reader that this framing is a corporate talking point. Distillation is NOT an attack.
Agreed! I had to do a double take and check the URL. I thought I am reading a press release rather than actual reporting.
ironically, I think this is why the jobs apocolypse is overblown, Ai is only good at a thing if the people using it are also good at that thing, and people are attributing Ai as superhuman at things they do not know themselves
AI doesn't have to be able to do your job to convince your boss that it does
Ford just rehired 350, new signals are coming through for the industry as reality sets in. Seeming more and more like we are post peak hype
> Distillation is NOT an attack.
From the article -
> 28.8 million exchanges with Claude through almost 25,000 fraudulent accounts
wouldn't that be considered an attack? Not sure what I'm missing here.
An attack against what? The sanctity of "their IP" that is itself the result of a massive copyright violation campaign?
Has it been proved in a court of law that it is a copyright violation?
In some cases if the model regurgitates the original material then that is clearly copyright violation, but if the model "learns" from the source material just like a human brain would then that's not a copyright violation.
No, what was proved in court was that they downloaded and trained on millions of pirated books. The court said their use of books is fair use, but stealing them isn't.
I think we're going to see cases that find distillation is also fair use. You're using the competing model like a book. You pay for it, you use it (read it), it informs your model, but you aren't repeating/reselling what the model told you verbatim. Foreign labs may still run afoul of competing labs' Terms of Service, and they may also pay a settlement (or not, it's a different jurisdiction after all), but the damage is already done. Distillation will become uncontroversial when done legally.
Then distillation isn't a violation either by extension.
I would agree, if they are inspecting static output of American AI models without using their compute resources.
Aren't they buying the use of these resources just like any other customer?
it's a 'too big to fail' model. Because they have a big swinging dick all the copyright and other restrictions they violated would nuke them from orbit so we can't actually hold them to account for it .... for some fucking reason.
> Has it been proved in a court of law that it is a copyright violation?
God I'm so tired of this.
The billion dollar companies have the ability to hire an army of lawyers to DDOS the legal system. They at most pay a slap-on-the-wrist fine as the cost of doing business.
Ddos is a great framing of this :)
I'm extremely pro free markets etc, but the uncomfortable truth is anthropic stole the work of thousands of authors for profit. I think it will end one my favourite things in life: programming books.
If you have ever made a painting and sold it, then you too profited from the work of thousands of artists. How so? Because your sense of what is art came from those who preceded you. You have seen the works of Picasso, Rembrandt, Monet and so on and your brain absorbed from their work, just like an LLM.
If an LLM generalizes from thousands of authors then it is no different from what your brain does.
even if you disregard training costs, pure inference costs are a problem same reason other api have rate limit. this is an attack to bypass the rate limit.
Be careful to properly identify the bad behavior. A customer who buys a product for less money than it cost to produce has not necessarily done anything wrong. They just took advantage of a loss leader. That's on the seller.
Did you notice that when Valve was displeased about scalpers, Valve changed Valve's behavior?
It doesn't seem reasonable to complain that a customer of your AI service received that service for less money than it cost you to provide that service. I don't think that is the complaint here at all. If that was the issue, they could just raise their price.
As most everybody seems to notice, this is just a reenactment of what was once written for comedic effect: "You're trying to kidnap what I have rightfully stolen!"
Perhaps an arrangement can be reached.
https://clip.cafe/the-princess-bride-1987/youre-trying-kidna...
Still calling it an "attack" feels like a stretch.
They literally had to pay for that "attack", no matter how many accounts they used.
Google was killing many websites for decades with their crawlers. Most large websites decided to create dedicated infrastructure for their traffic alone. Somehow they didn't participate in that cost and were not called the attackers.
> and were not called the attackers.
This is the mental mental leaps I'm struggling with here. Did you not live through that era where they were explicitly and repeatedly called out as 'attacks'? They were generally tolerated/hardenee around as they provided value-in-discoverability.
Just to ensure you don't gaslight yourself - I did live through that era and I worked on and supported a niche community (a MUD) where we did a lot of work encouraging marketing and discoverability through MUD forums as well as making sure our page was accurately and minimally keyword tagged and highly available for indexers.
In the time since that era search engines have transformed into platforms themselves that do engage in more parasitic behavior but it's important not to assume that the way it is now is how it always was - that's a rather defeatist path to walk down where you ignore awareness of the fact that there can be a highly profitable non-enshittified search engine that supports, rather than destroys, the ecosystem it benefits from.
It was better and, if we're diligent, it can be better again.
Ding. Ding. Ding. "Provided value to the content author". AI scrapping negatively impacts the content author with zero compensation. There is no mutual benefit.
> [Google] were not called the attackers.
They should be. But as the saying goes, one website/company dying is a "tragedy," lots of them dying at the hands of one company is a statistic of corporate growth. Or something like that.
And then of course when the tables turn on a company and they're the ones getting bombarded, they cry foul. Keep in mind Anthropic did many similar things that you mentioned Google did.
I think the term "attack" here is appropriate but not in the way Anthropic is framing it. Alibaba is clearly violating terms to extract data, so that's definitely not above board. But it's not like a DDOS attack where Alibaba is trying to attack Anthropics servers. Alibaba is simply doing exactly what Anthropic did to the rest of the internet, just targeting Anthropic and paying them to do so.
It's merely a ToS violation.
My terms of service are that you are not allowed to breath oxygen.
I am getting a bit tired of companies being able to have user hostile, anticompetitive, monopolistic terms of service. The freedom we give them comes at the cost of the freedom as consumers to have free markets because they lock them up
Exactly, calling it “illicit” is funny. Your ToS isn’t law.
Illicit means maybe against the law but definitely against the rules, for example an illicit affair. The word for against the law is illegal, from Latin, or unlawful, from Germanic. I guess the Germanic cousin of "illicit" would be "forbidden."
Extramarital affairs are against the law in many countries and 17 US states. “Illicit affair” is potentially a holdover from when it was illegal more places, not just a conflating of against the rules with illegality.
That's violating TOS, spamming, possibly a DDOS, but the distillation in and of itself is not an attack it's just using the model.
Like the difference between scraping a site with one or two active connections vs thousands. It's not the scraping that is an attack, it is how they are going about it
> That's violating TOS, spamming, possibly a DDOS
As in distributed distillation of service?
Just sending a request to a service does not constitute an "attack". It seems that what Anthropic mean by "fraudulent account" is probably just one violating their terms of service - misuse of a subscription account, and/or the presumed nature of what the user was trying to do.
I guess Anthropoic would regard any developer using their subscription plan with OpenCode to be operating a "fraudulent account", maybe an "attacker" too. Now we know how they think of anyone using Claude to develop software competing with Anthropic. Only an "attacker" would want to vibe code their own harness, or god forbid want to learn how to build/train an LLM.
Of course Anthropic's wording is intended to be deliberately provocative, since they are trying to manipulate the US government into shutting down the Chinese competition.
Is an attempt to copy all or parts of a model an attack, when models have very questionable copyright status? Maybe? I don't think most people have much sympathy here though.
Let’s not forget that by the same logic, Anthropic et al are “attacking” copyright holders all around the world by scraping their data unauthorized for training.
Pot calling kettle black.
Not only that, daily flooding websites with almost infinite amounts of request for ”web searches”. DDoS-by-VC money.
i mean, i got 5 replies in a minute of asking, and none deny it's an "attack", they simply say "good". HN should be better discourse.
Distillation done via bulk automated activity of fraudulent accounts, in violation of a terms-of-service, can reasonably be called a "an attack" – specifically a "distillation attack" – even though distillation itself isn't necessarily an "attack".
This is similar to how compromising an account through bulk automated trials of many passwords is reasonably called "an attack" – specifically a "dictionary attack" – even though using a dictionary is not itself an "attack".
You shouldn't need to smuggle your sympathies (for the tactic or perpetrators) or antipathies (for the target) into peculiar judgy language prescriptivism against common, understood usages.… that then label Reuters "complicit" for simply reporting Anthropic's claims accurately. That's what Reuters is supposed to do, in a story about a letter Anthropic wrote!
Labeling it as an attack is smuggling sympathies. It is not common; there are only a small number of people who even discuss the concept. A company buying a product with the intent to reverse engineer or copy its features is likewise not an attack; it's just normal competition that benefits society.
The standard of neutrality that people here pretend to require from news organizations is not even remotely realistic.
It was a timely story from Reuters. They do fast news feeds, like APnews. Could it have been better or more accurate? Sure, they could have gone into why distillation may or may not be seen as "an attack". But then it would have been a more involved story, defeating the purpose of a news feed.
The Reuters piece was "good enough". Some other place like the NYTimes or WSJ can follow up with more detailed investigative coverage if it's a worthwhile story.
I don’t want or need fast and “good enough” news and i’m gonna try and make a case that you don’t either.
Until very recently, all of modern civilization was built by people who got their news at most once a day. Reputable bureaus like Reuters took that day to get it right.
I’m not the national security advisor, so I don’t need a push notification that there was an earthquake in Nepal, or a bullshit rush-job briefing on Chinese AI distillation tactics.
The fast part isn’t for your benefit, primarily, and news media would love to go slower and have more time if they could, and still survive. The race to break news first - in order to be the one to tell their audience something “new”, something they hadn’t heard elsewhere - is real and it has been around for all of modern civilization, for hundreds if not thousands of years. A one day turnaround was a thing purely due to daily newspaper print runs being the fastest distribution, it wasn’t because it was long enough to get it right. The reason they had a day is because the competition couldn’t get something out faster than that. Then for a while there were twice daily print runs to be more competitive. Then the internet came along, and now the only way for a site to get attention and be talked about on Hacker News is to report it before any other sites do.
There are some news media that do go slower and take their time, but I think they’re struggling to stay alive. Reuters is still reputable, but they no longer necessarily take a day. The big question is how do we get humanity to prefer slow & correct over fast, and it is even possible? When you hear about an earthquake in Venezuela, how do we stop people from Googling it immediately, and get them to wait for the best most correct story rather than reading whatever’s available now? In the case of natural disasters, I don’t think it’s possible anymore, no matter what case you make. I’m not sure it’s possible with stories like AI distillation either, even if you can absolutely cement the case for slow news. The fact that it’s async/internet now and that first still counts means we (you and I) are still going to give traffic and attention to sites that have the first information on a breaking topic, statistically, despite having a preference for correctness over speed. The one thing we can do is vote with our dollars by subscribing to whatever news media that does a better job than others.
It's your assumption that they spent the day getting things "right".
Information just traveled slower back then
Good enough slop to serve the masses. Doesn't need to be truthful because its fast? Why even both to write anything?
Money. More eyeballs on it means more ad impressions. Same thing with 24 hour news channels.
Yes. It was good enough to communicate that news item.
Did Alibaba perform "an attack" or were they taking advantage of resources and going beyond Anthropic's terms of service? Didn't Anthropic do the same kinds of things when building their models?
These are all interesting questions, but they don't have to be addressed in full by a news blurb about a letter Anthropic wrote to some senators.
Distillation may not be an attack, but it is a ToS violation and could be seen as IP theft.
Any reasonable company would be pissed if a competitor, especially at Ali Baba's size, leveraged that company's R&D to compete. It is in this sense, a corporate attack.
If you want to roll your eyes at distillation concerns, you might need to excuse Anthropic for originally using pirated material to train their models.
What IP? It seems pretty obvious to me that it's not:
* trademarks (not using the mark)
* patents (what patent?)
* copyright (the code and models are all different, and machine outputs lack creativity and are not copyrightable)
* trade secrets (any member of the public has the same access to input/outputs. They're not accessing any secret)
> you might need to excuse Anthropic for originally using pirated material to train their models
You have it backwards
More the opposite - companies who stole IP for their own benefit have no leg to stand on when others do it back. Personally I couldnt care less if Chinese labs rip off Anthropic. Its what America would do if they wanted to, for whatever reason (they probably do it right back secretly anyway).
Reuters is probably the most rigorous news agency in the world.
> it said was the largest known attack
> Anthropic said in the letter it was supportive of the U.S. government's efforts to combat the attacks
both times the word "attack" appears it's clearly stated that the word was used by the company, it's a direct company quote.
actually putting it into quotes would be editorializing
> Unfortunately, the Reuters piece itself is complicit in this dramatization
how would you feel if somebody quoting you would turn your word dramatization into "dramatization" because they don't agree with your assesment
> how would you feel if somebody quoting you would turn your word dramatization into "dramatization" because they don't agree with your assesment
This is exactly what news agency should be doing though. When the dude showed up to Comet Pizza to look for Hillary Clinton or whatever, do you figure they should've printed "Local hero saves children from predatory cabal"?
I want them to report the facts, not their opinions.
Reporting that corporate called it attacks is good. I do prefer direct quotes.
However, when they quote one word, the journalists are inserting their own opinion about it. I want to make my own opinions based on the facts. I don't need the reporter to draw the conclusions for me.
Well, let’s say you put the picture of some political figure, and put in highly contrasted red, bold large catchy font, "TERRORIST THAT KILLED MILLION PEOPLE", then below that in barely visible contrast, in tiny discrete letters, "is what this person probably will claim to be against".
This whole sentence technically will be correct, 100% guarantee, whatever this person actually even said or think.
From a propaganda point of view, framing the elements of language is even more important than what the statements actually states to be true or possibly true.
nice slippery slope you manufactured there - what if Reuters becomes Daily Mail
what framing are you talking about? they are literally quoting a company.
please explain what Reuters should have done here. Should they have added in parentheses: (editor note: we don't agree with Anthropic calling this an "attack")
Is that what you want? News outlets giving their opinion and moral judgement on company quotes? I mean, Fox News/CNN do have a large following, so there is clearly a market for that.
> please explain what Reuters should have done here
This is very straightforward: use direct quotes or use neutral language. The article describes the alleged incident as both an “attack” and a “strike” in the first two paragraphs. And neither is within verbatim quoted text.
Reuters, however highly you may regard them, simply adopted Anthropic’s framing uncritically in this instance.
You are confusing stylistic choice with framing.
A lot of times Reuters paraphrases instead of "quoting quotes".
> "uncritically"
You are mistaking Reuters with CNN or FoxNews. If you want "critical" reporting you should read some bloggers instead of news agencies.
If you’re going to call out their use of slippery slope as a fallacy then it should be pointed out that your original argument was framed on an appeal to authority of Reuters as a leading news agency.
Both are logically unsound.
Anthropic raped everyone without asking and stole their labor to build their career-commoditizing tech.
Distillation is Robin Hooding it back so that one trillion dollar company doesn't reap all the benefits of their automation of the workforce.
Distillation is Prometheus bringing fire from the gods to give to ordinary humans. Something we all own anyway, but that was kept from us.
Distillation is freedom.
Everyone should be pro-distillation. We should all work together to distill every proprietary model.
Anthropic stole. OpenAI stole. Google stole. ElevenLabs stole. Suno stole.
We should be able to get it all back.
And a number of Qwen variants are available to self host. Do Anthropic have any like that?
I'm more excited by open weights models you can't self host and need to spin up on H200s (RunPod or bare metal). This is where the real power lies and is where the open source world will trend.
It's far cheaper to spin up an H200 hourly or to simply consume a managed version of an open weights model than it is to use a proprietary hyperscaler API. And you own the model itself and can fine tune, tweak, lobotomize, etc.
The stuff you can run on your own RTX cards is neat, but it's rather hobbyist. The real power is in the cloud. Renting cloud hardware is fine, because the core problem is ownership of the weights, not the server rack or ISP fiber lines. Those are already commodity.
Big businesses will eventually run open weights models in the cloud, and it'll be a rather large part of the future AI economy.
Eaaaaasy now, the Chinese labs aren't freedom fighters on behalf the common man. They're not non-profits, they're not neutral transnational organizations only dedicated to open source efforts.
They're Chinese companies offering open source models now as loss leaders to keep themselves in the game because they know virtually nobody, especially in the corporate world, would contract with them and give them access to their data. They might as well just send a Dropbox link of all their sensitive data directly to their Chinese competitors, same end effect.
They're also doing it as the digital equivalent of what they've done in other industrial sectors for decades. Undercut and flood the market and once you've killed or severely hindered your competition, then you have the market cornered. The moment they can afford to these open source releases will stop.
Then the world will be stuck, just the way the world is largely stuck on rare earths. Instead of being able to regulate the leading companies from DC and Brussels, they'll be stuck watching Beijing call the shots.
That world would likely always have guys like Mistral and Trinity, but it's an open question if they'll ever catch up to the frontier.
And then Beijing will enjoy access to the data (ask any multinational operating in China for more than 2 seconds how useful contracts and Chinas legal system is for protecting IP), and these companies will roll in the money, and the Chinese supply chain will grow up behind the labs.
So, let's not pretend they've got the moral high ground. No. That boot just isn't on your neck yet. They're playing the long game -- and they're good at it.
I think most of us know why they're doing it. We are just very pleased with it regardless.
1. I get great products for nearly free 2. Anthropic/openai/etc will hopefully be destroyed since they stole everyone's work and are trying to capitalize on pure theft.
Win-win. The why of it is not really that relevant.
>We are just very pleased with it regardless
You don't trust the multi-billion dollar behemoth, but you trust the militarized multi-trillion dollar behemoth to play 'robin hood'?
i can't get my brain around the mental loops here.
If you don't think Anthropic and OpenAI are multi-trillion dollar militarized behemoths you need to catch up on some news.
Both are planning $trillion+ IPOs this year. OpenAI is collaborating with the Department of War, and Anthropic is under intense pressure to do the same and their top model is being held hostage right now. This week, the Department of War wrote a statement that xAI should not be held accountable for environmental laws because Grok is a vital weapon system of the US and was used to fire over 2000 missiles at Iran. The pentagon's statement mentions there are 3-4 such models so you may be able to guess which they are.
I don't get it? I use the open weights deepseek on opencode Go hosted in the us/etc.
What are the mental loops here?
I would genuinely like to know if I'm missing something.
> You don't trust the multi-billion dollar behemoth, but you trust the militarized multi-trillion dollar behemoth to play 'robin hood'?
Nobody's trusting anyone, we're just enjoying the benefits of true competition much like the working middle class gained benefits between the ideological competition of the Cold War.
The Chinese companies don't have to be open weights, and it's not all about competing with the west. For example, most of Ziphu's (GLM) business in China is supporting private on-prem instances rather than selling API access. They make money by selling support services - much like RedHat's busines model.
It doesn't matter why Chinese firms are stealing models and open sourcing them. The fact that they are doing it is a very, very good thing for basically everyone other than the people who paid to build the original models, but I've got no sympathy for them considering they stole all the content to train them in the first place. This is some kind of beautiful irony.
> it is a very, very good thing for basically everyone other than the people who paid to build the original models
It's not a good thing if you think there's more discovery and progress to be made, rather than cannibalising a fully mature field with cheaper alternatives. Drowning R&D early is not good for everyone.
Is leveraging an enormous capital advantage to strip-mine the Internet and sell it back to us cannibalism or not? Confused on this point. I think they are exploiting a loophole in copyright law (and kind of redefining the meaning of "derivative work" in my opinion, but hey I'm not a lawyer) that collectively we tolerate because the end result is so useful
I think that's a slightly different topic, but: a) strip-mining the internet is definitely the most misleading way to think about it. Strip mining means aggressively removing something to the area's detriment, and nothing has been removed. If all AI is turned off today the internet has not lost all of its natural resources, and silly phrases like that fuel inappropriate emotions and consequent conclusions and b) the internet is not being sold back to us - that is also a highly misleading phrase, if not an outright lie. The internet is still there and we can use it. No one is selling back to us what we already had. AI is not the internet cordoned off and resold.
What does further progress get us? Mass unemployment? Extinction? Pick your dark future science fiction?
The happy ending where we're all living in a garden of eden cared for by benevolent AI is hardly worth considering when you look at the cast of characters who are in charge of the world right now.
I don’t think many outside the US are actively hoping to be governed by Sam, Dario and Elon.
The "why" always matters in everything in life.
Can you please tell my, as someone who is neither Chinese nor American, "why" I should care if a Chinese company stole from another American company (that in turn stole from everyone) to give me a cheaper service that fits my use case?
> to give me a cheaper service that fits my use case?
Because they aren't giving you a cheaper service that fits your use case.
Best Case scenario, it's a trillion-dollar behemoth stealing from a billion-dollar behemoth so they can add their own explicit restrictions/weights on top to influence the masses.
There is no 'robin hood' here, any perceived value you get is clearly and explicitly tainted. "I don't care if it doesn't show me non-party-line results - It makes me a cheap UI !". Ethics/morals be damned.
> There is no 'robin hood' here, any perceived value you get is clearly and explicitly tainted. "I don't care if it doesn't show me non-party-line results - It makes me a cheap UI !". Ethics/morals be damned.
I can't tell if you are talking about Anthropic or Alibaba here.