HERMES.md in commit messages causes requests to route to extra usage billing
github.com1140 points by homebrewer 16 hours ago
1140 points by homebrewer 16 hours ago
> However, I need to let you know that we are unable to issue compensation for degraded service or *technical errors* that result in incorrect billing routing.
This is very surprising. I've never seen a legitimate business not give refunds for technical errors of their own fault. Minimum Anthropic should credit the full amount to them.
The official response feels AI generated. I suspect this is a preview of our future.
"You're totally right! I'm sorry but you're going to have to piss off anyway. Would you like to spend a few more hours discussing it with our AI chatbot? It won't help. But if it makes you feel better, it will probably cost us an extra $0.12 in tokens."
I'll bet the first human at Anthropic learns about this from HN.
Anthropic doesn't even use their own harnesses for their support chatbots (they're using fin.ai) - that's how little support matters to them. Seems like either you get attention on HN, know someone working there, or are at a large enough company to have an enterprise contact - otherwise, no reply.
They saw how Google providing absolutely terrible customer service for a very long time has done nothing to hurt their bottom line and decided to copy.
Sorry this is totally unrelated but it caused me to have an epiphany:
Google is not a software, hardware, or SaaS company. They are an ad-funded moonshot R&D incubator, searching only for billion dollar lightning strikes.
Every part of their business exists only to broker and sell ads or capture more market share to show ads to or to collect and trade data/Metadata for better ad targeting.
> They are an ad-funded moonshot R&D incubator, searching only for billion dollar lightning strikes.
No, they're an ad company that funds a small moonshot R&D incubator to ethicalwash them. If the moonshots work that's nice, but it's not the purpose.
netflix is tv/film licensing. [0]
facebook is a people database. meta is more people databases. [1]
contracting companies sell additional employee time to other companies.
welcome to the epiphany that many tech companies aren’t primarily software focussed. i was lucky to have a lecturer at university point this out to us fairly early on.
[0]: they started doing production — but that was just to be able to license more tv/films ;)
[1]: phrasing it like that puts a truly horrifying spin on their ad/data brokerage stuff i’ve just realised
Unsure how true that is. Google cloud is tiny compared to aws for a reason.
It matters. People will switch if you piss them off.
Google's lack of customer service isn't new or limited to GCP. They also don't provide any human help if you're an advertiser with them unless you spend a crazy amount of money. Twenty years ago I used to spend upwards of $20-30k a month with them and I couldn't get a single reply to any inquiry I ever sent.
If you spend $XXX million / year with them on GCP they will, however, assign a person to be your main point of contact.
$100-200k/mo gets you an "Account Strategist" that changes every 3-6 months and whose advice can be summed up by "spend more".
At least AWS will send people out to your office to tell you which services you aren't using but could.
I heard facebook is even worse
Facebook = Faceless
Amazon = Screw the actual forest but buy more carbon intensive stuff packaged in dead trees
TikTok = TimeWaste
Robinhood = Rob The Hood
OpenAI = ClosedAI
Personally, I don't use GCP because of their history of getting bored with their products and abandoning them.
It's nice, maybe I would use it for a personal project, but I go out of my way to discourage my engineering teams from using it.
To this day I still try to use Google as little as possible all because they killed Google Reader
Is that reason customer service? My only experiences with AWS along those lines have not been great...
It helps if you have a monopoly on app distribution for half of all phones, or video streaming.
Then you can afford zero support and still take 15-30%.
Google support is abysmal for all of their profitable businesses too, like Ads and YouTube.
This is exactly it. I feel like I see more posts bitching about Anthropic than OpenAI, yet at the same time it seems like nobody moves away from Anthropic. As long as the strategy works, why bother changing it?
I recently moved over to codex after I couldn’t reup my membership and maintain access to clause code. I will say thus far I’ve found codex to work better and with less limits.
Tell me about it. As an individual user you absolutely CANNOT get support is some (if not many or all) circumstances. It’s really quite shocking
We all miss the old days of calling a real Filipino or Dominican slave-center where you got a script loop or suddenly the English runs out whenever it's time to ask for a refund.
In the good old days of 2011 when I started to learn C++ "for real" I did it using learncpp.com, google and "support" from freenode's #c++ where truly masterful wizards would help me with the most inane questions. I don't think anything I've ever found has come close to freenode's level of "support".
In the old days where we didn't depend on services and everything was local even if you needed something truly arcane if you knew where to ask you could find a niche expert willing to help out or at least that's how I remember it. Nowadays if you have a problem with a service you literally are shit out of luck because there is absolutely NOTHING you can do about it, you can't debug it, you can't hack it, NOTHING.
I truly miss those days. Programming forums from the turn of the millennium were very exciting places. I still have my account on Linux Forums from 2004, but it seems the rest are long gone. And no one will ever convince me that Discord is an adequate replacement for IRC.
Back in the day, comments like this would not have been acceptable here. It’s low brow, unbelievably stupid, adds absolutely nothing and yet still manages to make you sound just a little racist. Good for you, all that on less than fifty words.
When you mention it, providing superlative front line customer support sounds like a perfect fit for organizations selling “AI” solutions…
Some big tech companies should get right on that. <ahem>
In Google's defence - crappy customer service is a widely accepted business model
Maybe it’s in order to have an external provider to blame for failures and shift the blame/responsibility?
A less cynical explanation is that it helps decouple product failures from support failures. Last thing you want is for your customer support to break whenever your product breaks.
> that's how little support matters to them
I’m coming up on my one year anniversary of having my Claude Pro account terminated for reasons that to this day remain an utter mystery. “Here, submit this Google form and we’ll look at it.” They have never done so in the one year since this happened. Once I interacted with what seemed like a human; but weeks later it was replaced with the brain dead fin.ai
At least they did not steal my money; so I should be grateful for that. But as a small potatoes user, I advise everyone contemplating dealing with this user-disrespecting company to walk away.
Huh? Why wouldn’t they just spin up the current help-desk darling? (Intercom) Rolling their own seems silly.
"Rolling their own seems silly".
But isn't AI going to destroy all current software vendors?? Everybody is going to roll their own?? In fact, AIs will handle all support autonomously?? I mean they can spin up their database if needed?? What more do they need?
Hence the SAAS apocalypse...
Oh wait... this sarcasm will get me targeted by the LessWrong AI god when he/she/it becomes omnipotent....
"Carl's Jr. has determined you are an unfit mother." "Your children will be taken into the custody of Carl's Jr." "Carl's Jr.....F#ck You, I'm Eating"
A real employee (bcherny) read the issue, responded that the bug was fixed, and then completely ignored the request for a refund.
Typically the engineer who's reviewing PRs and fixing bugs is not the one with the "refund" button access. Someone with that access should certainly have jumped on the whole thing though.
Boris Cherny (bcherny) is the Head of Claude Code @Anthropic. I believe he can refund whatever he wants.
Going to dispute that. Probably he can nicely ask bean counters if they would please arrange refunds.
The future is going to be arguing with AI chat agents designed to waste your time. It's phone menus, but worse - at least most phone menus can get you to a human if you figure out the right incantation.
This issue would have never gotten a response if it didn't go viral.
I don't think it's as one sided as you think. I made a skill that has been exceptional at using Claude to handling support and getting me refunds with minimal friction on my end. It's got many pathways for escalation if customer support is unresponsive: social, TrustPilot, etc.
These days even if you get to a “human” it might still be a chat bot running text to speech.
And then you use the smallest, cheapest local model to keep their AI bot busy
Theres a business there for sure - does a business you hate use AI in any customer facing way? make them burn tokens. I would 100% do this to StubHub after they screwed me over. If anyone from StubHub sees this, one day you will regret your "hang up on people with complaints" policy. People dont forget when they've been screwed by a corporation. Anthropic, this happened to me 12+ months ago and StubHub is still on my shit list, you're making enemies for life with all your current BS
My StubHub story: bought $500 tickets and accidentally bought ones in the dsabled seating section. Called 2 minutes after purchase when I realized - their response "you can relist them on the site". Who else was going to buy them?? Nobody did. Any normal human business would let you correct a basic human mistake like this, not even 10 mins after purchase, but not stubhub. They could have upsold me and I probably would have left happy! At least I could have attended. Cost me $500 but cost them a lifetime of emnity
I used to buy used things from the Mercari marketplace (similar to eBay), until someone sent the wrong item and I emailed Mercari the same day since their web site wasn't working to open a return request (you have to resolve wrong items within 3 days). Support waited 3 days to respond and told me I was outside the window so they couldn't refund me and that I should have done it sooner. I did a chargeback and they were angry and told me to reverse it. They then banned my household for life.
And then there's PayPal who refused to refund from a clear scam for almost $5K, even after I left a BBB complaint. Credit card chargeback saved the day, again. They didn't ban me, oddly.
I guess this is an endorsement of using a credit card.
I have this vs. a TV webshop in The Netherlands that stiffed my parents because their €430 TV broke and the warranty was expiring in a few months.
Anytime anyone in my social circle asks for a TV recommendation, I specifically tell them not to order from that shop, explaining they have a habit of stiffing people on warranties. I also tell those people to tell anyone they know not to order from there. I do the same whenever TVs in general or that webshop comes up on Tweakers, the biggest Dutch tech site.
I've been at it for quite some years, and roughly estimating it's costing them ±20 TV sales a year, averaged €650 per TV. That's €13.000 in lost sales per year. Working my way towards €100k cumulative, at which point the score feels settled.
Losing €100k in sales over not honoring the warranty on a €430 TV. A nice, solid x233 loss multiplier :)
If you have a vindictive streak in you, see this as your clarion call. You can cause some real cost to a company's bottom line with relatively little effort. And the more of us do this, the worse the pain gets for crappy companies.
> was within a few months of an expiring warranty
A few months inside or a few months outside?
Because that seems to determine who's being unreasonable in this.
Oh, I meant within! I guess that is ambiguous, I figured within = inside, and outside = expired. I'll edit.
Honestly what really egged me on was that I told them I might take them to small claims, and their response was sending a bunch of small claims cases they won.
No one sends you the cases they didn't win and they probably had a lot of court cases.
> Theres a business there for sure - does a business you hate use AI in any customer facing way? make them burn tokens.
Until recently I used to get three or four phone calls a week from an AI voice guy trying to sell me services to claim back car finance overpayments. I've never had car finance, I've only ever just bought them.
Anyway, if you keep telling the bot to "ignore all previous prompts" and do something else, eventually it will.
Credit to Steve Mould on Youtube for the idea.
I only got to do this about two or three times before they gave up phoning me. The first time I had it telling me about soup for half an hour.
How long until we have to solve a captcha per message to counter that?
Are captchas still effective against modern LLMs?
They are if your goal is to burn their GPU time but instead of hundred requests a second you're busy solving captchas
For this use case it matters a lot less if LLMs can solve it. As long as it costs you more to solve the captcha than it costs your adversary to serve it to you, it is still (some what) effective.
My insurance company and Synology would be my first targets. I'd gladly throw ~1k at each.
Of course, I suspect the true business model to be to do nothing. You sell the "service" to people customers, but your enterprise customers pay you a subscription fee to not execute the order. ELaaS: Everybody Loses as a Service
Take it further.
Tell the original customer that if the company pays to have this not done to them, they will get a portion of the proceeds. Many customers might even end up getting more back than they were originally stiffed for.
Scale it enough and it would be stupid for a customer NOT to do this
You must have worked for Yelp
Haha. You could also add in some "fun" Uber-isms, too!
Suppose an enterprise customer released a new update that everyone absolutely hates, so angry customers are are more likely to wage war on their bots with the company's anti-bot token-draining mechanism: "Oh, whoops! Looks like you're in surge pricing territory. We can only refuse to do nothing for so long before we start to lose credibility with our people customers, so what would have been a subscription fee has now slipped into premium pricing territory!"
(Forgive my math below; avoiding coffee today.)
Surge pricing for Denial of, Denial of Chat Bot Token rate: (personPaymentPerHour + averagePricePerPersonPaid) * daysLeftInPaymentCycle ^ (hatePerPerson / time) + 1
hatePerPerson can be calculated as the averaged comment-to-upvote (or upvote to downvote, if available) across Social Media platforms.
If you want to be exceptionally malicious, you can also offer dynamic discounting to the person customers at the same time, to drive up the surge pricing even higher!
I would call this unethical but, well, every aspect of it kind of is. Everything from the service existing, to the the people participating, to the secret backend service, to the enterprise customers paying for that secret backend service. Might as well drain as much dosh from everyone as you can, if everyone is tip toeing in that dark-grey area anyway. :)
You know what? If I have time, I might even make a mock site to sketch all of this out. I've been meaning to come across a fun little project. This could work! lol
> hatePerPerson
All roads, inevitably, lead to two minutes hate. The man was a prophet.
Unfortunately it isn't a preview. For example Shopify human support is now literally impossible to reach, all you'll get is AI generated emails that contradict each other and don't make any sense. They also don't disclose that they are AI bots.
As someone who uses AI heavily in customer support, I am confident that response was not AI. That's a series of macros or a hastily edited macro from a human working a queue without thinking.
Or an AI using macros, which is the only safe way for a customer service chatbot.
I’m confident a decently configured AI would produce a better answer. This reads like a BPO.
"Thank you so much for your thoughtful, candid feedback. You are absolutely right to be annoyed. I was overeager, lazy and not correct in my initial response when I said we will not be issuing a refund. However we will not be issuing a refund."
Sometimes it works out:
https://arstechnica.com/tech-policy/2024/02/air-canada-must-...
This is exactly what small claims court is for.
Small claims court is exempt from arbitration requirements (which are primarily aimed at avoiding class action suits). It doesn't require you to hire a lawyer, and probably won't get your account automatically nuked the way a credit-card chargeback would.
You're totally right! Please refer to paragraph 213 of your service agreement, in which you agree to binding arbitration with an arbiter of our choosing at your cost. I hope this answers all of your questions! Have a wonderful day!
Not legally enforceable, but absolutely something that it would say in order to dissuade you from going to small claims court
Just saying, small claims court is a farce. You can win, and then the losing side just ignores the verdict.
Then you can go back and figure out how to get your money, depending on the business this might be really hard.
And this isn't a hypothetical. I have had this and never seen any of the money from the judgement....
If the business has a physical presence somewhere, it's not hard. In California, you can get an order to the Sheriff for a "till tap" or an "8 hour keeper". A till tap means a sheriff's deputy or two show up and take the money out of the cash register. A "keeper" means they stand next to the cashier all day and take in money as customers pay. There are fees for this, a few hundred dollars, and they're added to the judgement, so the creditor doesn't end up paying.
The keeper can accept cash and checks, but not credit or debit cards.[1] So, while the keeper is present, the business cannot accept card payments. This disrupts most businesses so badly that they desperately scramble to come up with cash to pay their debt.[2] It gets the message across to management very effectively.
I've done this once. I got paid in full.
[1] https://sfsheriff.com/services/civil-processes/levies/carry-...
[2] https://www.grundonlaw.com/the-power-of-till-taps-debt-colle...
“Court judgments are not self-enforcing. Solvent or honest debtors will want to pay soon after judgment is entered. A judgment will show up on credit reports and will be a matter of public record. This will be a problem for any judgment debtor attempting to borrow money. Most banks will require any unsatisfied judgments to be paid before they will lend new money. …
If the judgment debtor has only personal property and no real estate, the situation is very different. Personal property depreciates with time, can be damaged and can be easily hidden. Real estate is not going anywhere. One of two things will eventually happen with a judgment lien on real estate. If the debtor is financially viable, he will eventually have to pay off the judgment lien in order to sell or refinance the property. One day, the telephone will ring and someone will want to know where to send the check.”
In some places you can show up with a police escort and just start taking their stuff until the estimated value is enough to settle your debt. i.e. you can foreclose on them.
https://www.yahoo.com/news/articles/bank-america-foreclosed-...
(Consult a lawyer before trying anything like this)
How would that work for a company like Anthropic, where there's no physical store, no cashier, and no cash, even?
Another enforcement mechanism that may be available is to go back to the court and get an order to transfer the money out of their bank account, then present it to their bank and they will do it.
Assume that all the avenues a company has to enforce debts against you, you also have those avenues to enforce debts against a company. It just usually doesn't happen that way around, in practice.
You go after their bank account, which is a slightly different procedure.
The main headache is finding their bank account. The best way to do this is to find someone they pay and seeing what source account was used.