eMudhra to exit global SSL/TLS certificate business
emudhra.com1027 points by bo0tzz 18 hours ago
1027 points by bo0tzz 18 hours ago
> However, the level of social media amplification around this incident has brought scrutiny that is out of proportion to SSL/TLS’s place in our business
Scrutiny is inversely proportional to bit complexity of the private keys they used. Without a failure to apologize or expose how it happened: a human mistake, process mistake, etc. this looks pretty bad on them.
> eMudhra’s global SSL/TLS infrastructure runs separately from its other operations. The company’s Global digital signature certificate (DSC), eSign, and other signing services, including those in India, its enterprise PKI Solutions (emCA, CertiNext platforms), emSigner eSignature Suite, and its identity and authentication solutions (SecurePass, emAS) are not affected and continue to operate as normal
Anyone relying on them with anything related to security or trust is playing Russian roulette at this point.
It was "multiple human errors". But yeah no matter what, in any case or exceptional circumstances, this thing was not supposed to be screwed up.
On Mozilla’s tracker, eMudhra said the keys were created on 30 April 2025 in a key generation ceremony, a recorded procedure in which keys are produced inside hardware security modules (HSMs), tamper-resistant devices built to hold keys. The keys were meant to be moved to data centres in the United States and Europe for customers needing data stored in those regions. To move them, the company added a step to ‘wrap’ each key, or encrypt it for transfer, and ‘unwrap’ it at the other end. eMudhra said a flaw in the software script created a second, weaker set of keys during this step, and because the two sets had similar names, staff used the weaker ones by mistake. It said human errors occurred at multiple steps.
source:
What's the eMudhra security scare that has rattled the internet?
https://theprint.in/tech/whats-the-emudhra-security-scare-th...
These "wrapping" keys were not "weaker". They were completely unfit for purpose. They have not explained why they used a 3-bit key for this purpose.
If they go to the trouble of wrapping them, use real keys. Or don't do it.
Also, it's a big "human error" to not notice this for a year and a half.
I have to imagine that the 3 bit numbers were slots/indexes on the HSM, and rather than actually retrieving the key(or maybe it did), the slot number got used as the key. I can't otherwise imagine how they'd generate such low number keys...
I have not read up on all of this, but I would have guessed they were artifacts created and validated by a testcase. "testcase 1-8".
Having worked around these types of devices and their interfaces, I doubt they intended to create weak keys.
It is more likely they had a script which placed a parameter in the wrong field, due to something like using positional arguments. Or they misread the documentation for some interface.
Still incompetence. And I would love to see my theory confirmed or refuted.
The previous discussion advanced the theory that they fucked up the string handling. Eg in Python, output = key[0] would result in one-byte keys like "4" if you expect "key" to be an array, but it's actually a string.
> worked around these types of devices and their interfaces..
This is not a problem of devices and interfaces. You are supposed to generate the wrapping key on the HSM itself. And the HSM ensures that the keys are generated with a strong enough security parameter, not some silly 1 byte string. Then the HSM wraps the key inside of itself and exports it in the encrypted form. You take this ciphertext, and the wrapping key and send it over to the other party.
So if they generate the wrapping key with a funny value, that means they didn't use HSM to generate it but their system openssl. But then, HSMs typically don't allow you to export private keys without wrapping them first unless you first tweak around with security flags (not recommended at all), or simply generate the private key using your system openssl too (LMFAO).
Source: I work for one of the major HSM vendors.
I hear you in terms of how these kinds of mistakes typically happen. But why did the script ever produce two sets of keys? The initial root cause was somebody picked the weak key not the strong key. But what was the reason for ever producing a weak key in the first place? They say the script produced two outputs - what on earth for??
Stepping back a little to look at the point and gravity of the exercise makes me wonder why they didn’t just fly it over.
I mean, why have the ability to generate weak keys to begin with? Serious question, I've never done this sort of work. What sort of utility does that bring to the procedures and workflows one does in that line of work?
If you didn't know better, you'd think this had been done on purpose by some three-letter agency.
Testing could be a reason.
To be clearer, I’m not sure these keys were generated, but being able to use weak keys is something you may want to do for testing.
Yeah. I can't think of any development process where you wouldn't simply use a production grade key in development. In fact, that's exactly what we do in development with snake oil keys.
Generating a key with "4" was probably done as a joke, but also is insane to have ever lived outside some devs computer.
> These "wrapping" keys were not "weaker". They were completely unfit for purpose.
Oh come on. They clearly didn't intend to use 4 as a private key or to wrap it. They had just purchased millions of dollars of HSM's to store them. It was something more subtle than that.
Have a little imagination. The transfer procedure sounds bespoke - possibly just a shell script using openssl to encrypt things at one end, and decrypt them at the other. Let the man who has written 100 shell scripts without a bug in one cast the first stone.
But not so subtle that good software engineering procedures wouldn't have picked it up. In fact today just an LLM review of the shell script probably would have caught it. The Internet Death Sentence was warranted - because poor software engineering procedures are unforgivable in a CA. But those procedures and policies aren't easy or cheap to come by. If they were just "don't chose 4 as a primary key" my grandkids could do it.
Unwrapped extraction is the kind of thing security by checkbox covers and process review might catch so it is never clear in a case like this that some party with the capability to influence the process didn't want keys intentionally weak to emulate unwrap.. Then someone else may have introduced the swap of keys that would never have been seen with keys that would (be forever) in a public scrutiny DB.
I don't think it's implied the wrapping keys were the weak ones. More like, the script that ran the wrapping still had the fast "debug" generation of a pair of keys to test the wrapper on enabled, for example.
Their complaint about social media reeks of "and we would have gotten away with it, if it wasn't for those meddling kids". [1]
We deeply and sincerely apologize for getting caught and not being able to cover it up.
This is pretty normal in India unfortunately. Crony capitalism is the dominant business engine in the country, so pretty much any significant business, esp in regulated sectors like govt mandated services(which is the bulk of eMudhra's operations) is impossible to set up without significant bribes being paid.
As a consequence, these businesses are very well protected from normal market forces and are pretty much above the law. Had this incident not been an international one, they had a very good chance of getting away with it, and the people reporting this would have landed in jail.
This is 100% accurate. State run media would have squashed this article if it was not for an international incident response. They have federal laws that prevent such disclosures because it affects the national image and could cause social unrest.
Since you seem knowledgeable about this topic.
Would you say the system forces people to be like that or people are the reason why the system is like that?
Would you do business (i.e. trust) with someone that comes from that system?
You're asking a chicken and egg question, I believe it becomes a self-reinforcing loop
In regards to doing business - someone that comes from a corrupt society with a culture of taking advantage of others should be approached more carefully than someone that grew up in the same town as you, of course
This is a lesson learned the hard way, where I moved to a different country and treated everyone with openness and generosity (of time, resources, willingness to help), just to realize people from certain cultures mistake this behavior for weakness and try to take advantage as much as possible
> Would you say the system forces people to be like that or people are the reason why the system is like that?
It's the courts. Courts are meant to bring in an error correction mechanism. When courts fail, the country goes into open-loop control. No errors are corrected, and eventually, people stop even detecting errors. https://x.com/captgouda24/status/2110089855380369511
> Would you do business (i.e. trust) with someone that comes from that system?
Vet rigorously and continuously! Start with small business, and grow it based on interpersonal trust. Since the system lacks accountability, you must ensure that yourself.
Wild to me that our healthcare data lives there.
Depends on the contract your healthcare providers/payers/etc. have with those who do their IT.
Source: work on healthcare IT projects and can't let anyone outside of the US look at prod data no matter what due to the contract we have with our clients.
>capitalism is the dominant business engine in the country,
The government _mandates_ that certain services be used. In which world does this rhyme with "capitalism"?
I said _crony_ capitalism is the dominant business engine in the country.
The govt mandated services are owned by private capital(hence the capitalism part) and govt picks winners by granting licenses to a select few(hence the crony prefix)
Any world where sufficiently wealthy capitalists can bend the govt to their will, i.e., most of them.
It does, but I wonder how much of that phrasing is cultural.
When working with different businesses across the globe, I’ve noticed Indian correspondence to incidents tends to be more a show of strength even if it’s transparently obvious that there’s no merit to the content.
Whereas western European correspondence is often more apologetic and empathetic. Even if it might sound insincere to those affected.
But it’s all just words at the end of the day. It never means much unless the business is willing to back real change to mitigate similar issues in the future and resolve undisclosed existing problems. However there’s no way for us to know if this is happening or not.
I don't know if it's really specific to one culture. Look at US companies after they go through a lengthy legal saga, it's obvious they screwed up, they're paying fines or having to adhere to court-imposed consequences. Their press releases:
"We admit no wrongdoing. We are only settling to move past this. Our business is strong/secure/competent."
It's like the first rule you learn when you go into corporate communications is to never admit you did anything wrong or project anything other than confidence and correctness.
>It's like the first rule you learn when you go into corporate communications is to never admit you did anything wrong or project anything other than confidence and correctness.
It's the first rule you learn after joining basically any organisation larger than 5 people.
People on HN who are surprised are seemingly self-employed prodigies who dropped out of middle school.
I’ve worked for a lot of enterprises across the globe and I can tell you that you’re flat out wrong. That’s simply not how all organisations operate.
Just take a look at any of the thousands of postmortems published over the last decade and you’ll see that the tone of incident response communication can vary massively.
The truth would have come out eventually- after a bunch of security cluster fuck incidents.
Their OTP solution is bruteforceable as well: https://mastodon.catgirl.cloud/@49016/117475921584308861
This becomes an inverse Russian roulette: only one of six chambers is not loaded.
Waiting for a release like "... the level of social media amplification around this incident has brought scrutiny that is out of proportion to OTP 1FA place in our business. In view of this, after detailed deliberations our board has concluded that withdrawing from global OTP 1FA is the right decision for eMudhra and its shareholders"
> This becomes an inverse Russian roulette: only one of six chambers is not loaded.
At what point does it become Russian roulette with a Glock?
One chamber is enough to play Russian roulette... once. However, the probability to get to play a second time is rather slim.
Link is broken, gives some randomly generated text when using Brave on Android. Their AI bot detection is not very good.
> fear not, here's a complete bypass of their 1fa otp login system. they encrypt OTPs to a public key and issue those behind cloudflare turnstile. this key to be precise:
It's not clear what you have linked there.
Paste from mastodon:
> fear not, here's a complete bypass of their 1fa otp login system. they encrypt OTPs to a public key and issue those behind cloudflare turnstile. this key to be precise:
"However, the level of social media amplification around this incident has brought scrutiny that is out of proportion to OTP's place in our business"
Various Indian regulations (particularly around commerce and banking) require you to use India companies for solutions like the above. It gets worse when companies are "steered" by a regulator/compliance person who has a bias for a specific solutions provider.
Then indirectly the security and trust of the entire Internet rests in the hands of a few government babus in India... that's scary.
Their Internet registry people also weaponized AS0 (zero) as a method to take networks off air via BGP RPKI for the ranges they're responsible for. There's been a number of situations where multinational companies who operate in India who are forced to use IPs from the India RIR have been fat fingered by some staffer who makes changes then goes home for the day: meanwhile a multinational bank is down for 24 hours within the country because said agency doesn't believe in 24/7 support.
Typically they do this for non pay but they've dropped the hammer multiple other times incorrectly (and good luck getting a RFO out of anyone there).
For good or bad, thats by design. These is a significant portion of physical infrastructure associated with internet data flow. The only challenge governments face as of now is being able to segment the traffic to their wishes in its entirety.
Companies already control all the traffic flow on their infrastructure using end-point security solutions, the same can be extended at ISP level, and then Country level
It’s called regulatory capture.
It’s get worse when these companies work with regulators to create more regulation that they can sell more of their services.
https://www.amazon.co.uk/Compliance-Industrial-Complex-Opera...
Regulation like this makes sense, but then the government should also srutinze these companies
Well Indian regulator must have visited office canteen to see no unapproved food was served there.
Only when fully satisfied they'd have given license to operate this security business.
"out of proportion to SSL/TLS’s place in our business" is an insane comparison. They are saying they care so little about SSL/TLS to begin with that it's not fair.
It would be like me practicing surgery 1 hour a week and saying "well surgery only has a very small place in my 80-hour work week" in response to flagrant failure of medical care.
"We are bad at this job, but we don't do it often so don't think we are bad at other jobs".
Yeah, the proportion if anything is too small. This is the sort of company that should not be doing anything related to security.
This is what I found regarding how it happened, but still not a full explanation - but there was some name confusion and obvious incompetence.
>According to eMudhra, proper strong keys were created in the HSM at the key ceremony on 30-04-2025. The script had been changed to test wrapping and unwrapping keys, because the intermediates were to be moved to data centres in the US and Europe, and a bug in that change created a second set of weak keys. The keys had almost identical names, and the weak ones were used for the certificate requests. The same request was signed by eMudhra's own root and sent to Sectigo, which cross-signed it. The strong keys never made it into any certificate (Bugzilla).
https://www.fairssl.se/en/articles/emudhra-ca-noegler-knaekk...
I wonder. Say I wanted to prevent something stupid like this from happening in the future. What would be a sensible check? The higher level authority doesn't see the private key itself, right? Just the certificate request. And I wouldn't want someone to manually inspect the keys to see that it looks random-ish. Perhaps there could be a small check script added to the generation of certificate requests that says "hey, your key a.) is mostly zeroes, b.) appears to be ASCII text, or c.) seems otherwise extremely non-random, please use -f to continue anyway".
I don't know if it will work on something as small as a private key, but you can compress the key. If the compressed key is significantly smaller then the original, it lacks entropy. Pure random data is mostly incompressible.
Note that 32 0x00 is a possible output from a pure random generator while still being quite compressible, you would probably still want to throw that one out though. After buying a lottery ticket.
Do not allow devs to modify ceremonial hardware.
I was about to reply with "they didn't have a choice" but on second thought perhaps CAs are important enough that they should never be allowed to export keys from an HSM and should instead be required to physically deliver and install the HSM wherever it needs to be in the world. Inconvenient and expensive but likely for the best.
Fantastic key ceremony. Great time had by all. Can't remember a damn thing, we were all so plastered.
Right? I don’t know why anybody would trust this company for anything.
Someone higher up in the hierarchy telling you to trust this. You have no option to challenge it.
Russian Roulette is pretty accurate if not an understatement.
If they expect people to still trust them on their other services they have obviously no clue on how Trust works, meaning they shouldn't be in the game in the frist place.
As a trust provider you get one chance. Squander it and you're out. That is how it should be.
Anyone relying on them with anything related to security or trust is playing Russian roulette at this point.
In any legal dispute involving them, Exhibit A will be "look at how they ran their SSL/TLS business". That won't go well.
They got their image hurt, and pulling the culturally appropriate response here which is to never apologize and to never accept fault. Instead quit/resign and say that other stuff is just fine. It's really not fine, and you can bet that the same corporate philosophy that made the CA error will have serious security errors in their other products.
I think it doesn't matter what they say because they'll just get away with it. It's as simple as that. They'll get away with it.
[flagged]
There is no such honor code and stop spreading this BS propaganda. Izzat is not even an Indian word used in any Indian language. It is Arabic which got mixed in with Persian which in turn got mixed in with Urdu.
West has something similar. Politicians never apologize, and avoiding Indian certificate authorities (and recuriters) is racisist!!!
Somehow this is making it even worse. Like, I'm having issues explaining the whole thing to my wife, because I can't come up with a metaphor or parable that's even remotely as ridiculous as what they've done. Anyone's got a good one?
Maybe, like, setting up a factory that builds both LHD and RHD cars and building cars with two passenger seats in the front for two years straight without anyone noticing?
The internet's security relies on an almost impenetrable wall built around it. The king appoints gatekeepers that are each entrusted with a gate they must protect. The gatekeepers must follow a rigorous interrogation process to verify the ID of anyone who wants to pass through their gate. Only then are they permitted to unlock it.
The king was very careful to ensure all the gatekeepers were following the process correctly. It did not occur to anybody that one of the gatekeepers could be using a dirty rag to jam their gate closed instead of a lock.
Once discovered, the wall could be trivially bypassed. The security of the whole internet was immediately broken. Rebuilding the wall to block up the dodgy gate took much effort from very many people.
I think part of the issue is that we allow gatekeepers to open any gate.
This system was invented in the 90s when things were looking up. China was opening up to the world, the Russians were suddenly not evil anymore, and the internet was supposed to bring us all together into one happy global society, kumbaya and all that. Obviously that didn't work out.
It's time for a rethink really. Every CA is now a single point of failure.
The entire system (CAs and ICANN DNS) was never good to begin with; it's just become that much worse due to the shifting situation over the years. It's an inherently centralized design with no way to verify that example.com is operated by the same entity from one day to the next. The namespace is global and effectively an exercise in rent seeking.
At absolute minimum any sensible system would have a TOFU element whereby the site operator of example.com changing from one day to the next would result in all the tooling across the entire stack screaming loudly (in other words what ssh does). And that's just the absolute bare minimum change that's needed.
Now the gatekeeper is saying "I don't want to be a gatekeeper anymore for this one particular gate, because people were mean to me. I'll however continue being gatekeeper for these other gates that drive my business either way."
I think a part of the issue was, everyone living outside the walls did not even try to kick the gates open as everyone assumed a big (cryptographically secure) lock was being used by all gatekeepers.
https://drmaciver.com/2022/05/learning-to-walk-through-walls...
Citing Terry Pratchett’s “Interesting Times”.
Its called a Potemkin solution, and there is a serious theory that most nuclear arsenals are exactly that: no-one has ever confirmed they can actually be fired.
In the case of nuclear arsenals that's perfectly fine - MAD simply needs to be plausible, not robustly demonstrated. In the case of CAs I'd argue that the description doesn't really fit since they're getting stress tested by malicious parties on a daily basis. By that logic you could describe linux kernel security as a potemkin village due to the steady stream of zero days. In reality a potemkin village is something intended from the start to deceive everyone else. It's not impossible that's what happened here but that's a very serious allegation to make (ie theorizing that the weak keys were intentional with a conspiracy afoot).
In this analogy I think it would be more appropriate to call them a lockmaker, and a lot of gates have been using this lock. Finding and fixing every lock is where the effort comes from, and some locks may not be found for years to come.
This lockmaker analogy fails to capture one critical aspect: the failure of one CA breaks the security of everyone, even if they did not use that CA's "locks".
This is why I went for a "wall with many gates" instead; any bypass makes all gates redundant.
All analogies break down at some level; the main goal is to clearly communicate the most important ideas.
In the old days a king would "sign" his proclamations by pressing his "unforgeable" signet into a wax seal. This proved the authenticity of the proclamation.
Normally, the king would declare and sign a proc stating that anything signed by his secretary's "unforgeable" signet was to be respected as well. The secretary could delegate authority to others in turn.
Well turns out that one of the signets in the chain of trust was just a picture of a "four" that anybody could have forged, but nobody noticed because in all of the ceremony and stacks of parchment with wax seals, nobody actually checked if the signet was secure.
The signets were "unforgeable" in that the penalty for copying one was death by being tortured, hanged, drawn and quartered.
They were "unforgeable" because commissioning a jeweler to make an intricate carving on a ring by hand was very expensive in the first place. Finding one who could precisely replicate an existing carving, down to the mistakes of its original author and any wear and tear that would be visible on the wax seal, was almost impossible and even more expensive.
Of course, I didn't mean to imply it took zero effort to copy one, though I guess that's how it read. You'd have to find a skilled jeweler that's also willing to commit treason, so for sure the barrier wasn't insignificant there.
Indeed. In both this scenario and our modern one, we rely on the difficulty of "key material" reproduction being greater than the difficulty of its creation.
That would be retroactive security...
However, Kings likely benefitted from the option of a scapegoat, they could always blame someone for forging their unforgeable seal and have them killed, rather than admit that the seal was true.
If we add in attainder and escheat, the King had a pretty strong hand.
It's enough that the signet could be verified, and sometimes was [1].
As it turns out, these documents were mostly useful for long-lived décisions, which could be overturned, and where a forgery exposed you for a lifetime.
[1]: https://en.wikipedia.org/wiki/Robert_III_of_Artois#The_fraud...
https://bsky.app/profile/anthonymoser.com/post/3myf5fwpl7s2t is a good explainer I think without getting into the nitty gritty details.
for the non technical folks: imagine there was a password that if you guess it, you can impersonate *any website on the internet*
now imagine the password is literally the number 4This seems like the easiest way to frame it to non-technical people to me. They know what passwords are.
I just explained what it actually is. I said "you know how you go to a website HTTPS and it's secure? Well Apple chose a couple of trusted parties that your phone inherently trusts, and those people approve a few thousand intermediate companies who do the day-to-day certificate work for end websites. One of those intermediates, rather than choosing a 200-digit random number, chose 4, no not four-digits, the number 4". And then we could laugh at some of the memes.
Our 200 digit random number was 199 zeroes followed by a 4
- emudhra lawyer maybe (:
Private key size is 384 bits or 116 decimal digits. So 115 zeroes.
You mean, the key was 383 zeros and one “one”, on the third place from the right :)
I really wonder if this would/should hold in court. It is ridiculously unlikely yet genuinely possible.
One of them, sure. 8 of them?
At this point we're talking about chances so slim that only philosophy can save the situation.
technically possible
Not just technically possible, as likely as getting any other random 116(?) digit number
i think this oversells the likelihood of it. the probability of any random 116 digit number having its first 115 digits be the number zero is much lower than that of it containing a much higher number of non-zero digits.
It's hard to explain partly because when you explain this thing we've done with TLS and PKI, it sounds a little insane.
> "We set up a system where every government and a bunch of random banks and IT companies in the world holds the secret codes to everyone's encryption. Then we make them pinky swear not to abuse it, with stern talkings to if they do. And now it's all on fire, because one of them was irresponsible!"
You can know the intractability of the problem that PKI is solving, and the (worse) alternatives that have been tried, but that doesn't make this system not-insane to a normal person, and its at least a little bit the fault of the collective "us".
And it's not like it is an impossible problem; we rely on TOFU for SSH, we have remotely-attestable TPMs everywhere, ZKPPs / PAKEs exist, session keys and federation protocols are widespread. Somewhere in the set of "possible solutions" are options that don't sound unhinged when you explain how we got to this eMuhdra situation.
I believe you misunderstood how it works. A parent certificate cannot be used to decrypt information encrypted by a child certificate… the CA issuer just signs your child certificate, it doesn’t have access to your private key.
Then, everyone who trusts the CA will trust your certificate by checking it is signed by the CA who should have done the checks to ensure your certificate contains truthful data.
So your layman description of “they have secret codes to everyone’s encryption” is completely wrong.
Certificate keys aren't used for encryption anymore anyway.
A modern TLS connection uses an ephemeral key-exchange protocol where both parties opaquely agree to a one-time-use random encryption key that is not knowable to any other party, not even someone who can gain access to either party's private key. This is called Forward Secrecy (originally and sometimes still known as Perfect Forward Secrecy or PFS, but the use of the word "perfect" in any cryptographic context is akin to naming a ship "Unsinkable").
They have the secret codes to MITM (intercept, inspect, alter and forge) any communication if they manage to place themselves between you and a party you trust.
Ok, but the analogy was wrong anyway.
> random banks and IT companies in the world holds the secret codes to everyone's encryption.
That's not true at all.
What they do have is the power to MITM connections IF they first spook DNS or insert themselves into a proxy accepted by the server somehow.
So, if you want to be honest, the original sentence becomes:
> random banks and IT companies in the world holds secret codes that are trusted by everyone to sign information that is used to establish trusted communications. If they wanted to, they could try to tamper with people's network connections and, if successful somehow, pretend to be anyone else that had used their secret codes to sign their information.
Quite a big difference, no?
That sentence doesn’t take the current situation into account. It’s not just “they” - the CA - that can do this, it’s anyone who can figure out their weak encryption key. Which makes the “if successful somehow” a whole lot more likely - consider what malware distributors can do with this, for example.
The internet is a marketplace with a million vendors, each with their own stall.
In order to sell something in the market, you need to prove that you are who you say you are. There are about 200 trusted clerks whose job it is to check your papers and verify your identity. Once they're satisfied, they stamp your papers with an intricate stamp designed by the finest cryptographers.
Anyone seeing that stamp knows that a trusted clerk has checked your identity and approved your papers. Most people don't examine the stamp closely. They simply recognize it as belonging to one of the trusted clerks and accept it.
One of these clerks, however, instead of commissioning an intricate stamp from a master cryptographer, bought an ordinary rubber stamp from an office supply store.
One day, a particularly paranoid cryptographer was examining the stamps used throughout the market and noticed something peculiar about one of them. It wasn't intricate at all. In fact, it was so simple that anybody with a few dollars and a trip to the office supply store could make an identical stamp.
The clerk's job wasn't merely to stamp papers. The clerk was supposed to check that everything written on them was true before applying the stamp.
But now anyone could write whatever they wanted on their own papers and stamp them. The resulting documents would look exactly like ones that had been carefully checked and approved by the clerk.
A robber could write "thebank.com" on their papers, stamp them with the office supply rubber stamp, and present themselves as thebank.com. If they could get a customer to come to their stall instead of the real bank's, the customer would have no reason to suspect anything was wrong.
After all, the papers had the stamp of a trusted clerk. And everyone who trusted that clerk's stamp would believe them.
I told my wife the following: basically when you access a website like your bank, and you see the padlock in the address bar, that means that the computer the website is on has shown your computer a certificate saying that the same company that built the website you’re looking at owns the web address you’re going to.
Your computer has a built-in list of signatures that it trusts and expects to see on the certificates, and one of those signatures was eMudhra’s. These signatures are basically just a proof that you know a secret number without revealing the number itself. If you know the secret number, you can sign the certificate.
There are an unimaginable number of secret numbers. Imagine each atom in the Great Pyramid of Giza is a secret number. And then imagine you have as many Great Pyramids of Giza as there are atoms in the universe. And then double it.
Now imagine you choose any of the numbers represented by all those atoms to sign the certificate with, and you chose 4.
It’s not technically wrong: 4 is indeed a number. But if anyone wants to guess your number, and they start at 0, it won’t take them very long to find your secret number.
Once they’ve found your secret number, they can now sign certificates pretending to be you. So someone could pretend to be your bank, and present a certificate saying “yes, I am your bank” with a signature on it that your computer trusts.
It's a padlock with a code. The manufacturer makes the padlock with a specially made pin, so you can lock your front gate, and feel secure.
eMudhra sent you a padlock that had the code "4815" promising no one has ever used it or would be able to guess it, but all anyone really needed to try to get in is set the first number to "4" and ignore the rest.
It's not entirely accurate, but its accurate enough for them to get.
A car manufacturer that reused the same car key for every car.
And the key has no notches, and so inserting a screwdriver into the lock and turning works just as well as using the key.
That was reality in the eighties! You could open some cars using a popsicle stick as a key. Fun times!
One time after locking my keys in my '87 Toyota pickup, I just used a swiss army knife in desperation. Opened instantly
My dad used pliers to open the trunk! Even thought us how to do it so we could help him out.
I think this is the best parable - a master key with no notches.
I'd refine this to, the cylinder for the lock did not care what key you put in, it woul open it anyways. So the key looks secure, but the lock is a joke.
You can use a lollipop to open it.
Except it's still not good enough, because you could use that key to eavesdrop nearly anything and anyone.
4 notches
That's perilously close to a standard door key, and thats way too much security compared to these certs.
And the key on top of that can be used to open every other car that exists.
They did!
As a 90s kid, the Dodge Omni was a common shitbox student car. I think they had like 8 key patterns lol.
My brother’s ignition key unlocked my friends’s trunk. There were a few people who could start other peoples cars. In one case, a soccer player “stole” a football players unlocked car as they looked identical and were parked near each other. In another as a prank we unlocked cars to roll them out of parking spots and physically lift them into unusual places.
Beyond that, they didn't even properly key the key! They carved one notch in by hand!
>I can't come up with a metaphor or parable that's even remotely as ridiculous as what they've done
The best example I can think is (some) international borders: yeah, they're heavily regulated entities with lots of written protections (and thousands of miles of impenitrable barricades), but occassionally a hole opens up (whether physically or engineered).
Perhaps a section of border fence is occassionally destroyed (and people sneak in), but this won't happen if the border guards are approving [false] passports from a compromised government actor ("they have a valid passport, according to our inspection"), the border fence doesn't even matter!
Presume anybody can now go to that bad-passport-issuing government and request a passport to then enter any other country (i.e. the other country will see the passport as valid in the official global database): BAM, borders no longer mean anything; welcome to Privileged Accessia.
Everyone in your neighborhood knows your address, but someone discovered two days ago that they can get into your house by sticking a 4" oblong piece of paper into the keyhole. They put it on blast on Facebook, so now everyone in the whole planet is trying to break into your house.
keep it simple. They promised a door key with 10 grooves cut in it, but instead they gave you a door key with 1 groove cut in it.
And when that clicks, extend the metaphor, literally, to a key with ~384 grooves.
This scene from the classic comedy movie Spaceballs is a good one: https://youtu.be/B-NhD15ocwA?si=pcPfIRTgPVAurW1S
y'all underestimate your wives, i simply explained how PKI works in simple terms (alice/bob, public key, private key) and explained that they chose 4 as a private key instead of something complicated, she got it.
I have explained it to a few normies.
I start by saying this is about the tech behind the S in HTTPS. I say that the main point of this is that when you talk to foo.com, it has to prove that the people behind foo.com are on the other end, so it's safe to send them your encrypted password or whatever.
Then I say a bit about how the system works by having a small number of trustworthy companies vouch for other people's servers. This doesn't need too much technical detail to get the point across.
Then I say one of those "trustworthy" companies put an encryption key of 4, where it's supposed to be a long number that's hard to crack. This makes the verification easy to forge.
Then I say that everybody who runs a server needs to remove this company's bad key, and Windows or Mac or your phone needs updates too. And that your web browser probably already quietly removed it, which removes a lot of the risk.
I feel like when I explain it this way people get it.
> put an encryption key of 4
Hey the 4 was picked for good reason!
xkcd.com/221/
A lock factory that’s supposed to make unbreakable locks, yet all of the locks/keys are identical. Suddenly everyone can use such a key to open any door of the internet.
Not 100% accurate, but conveys how stupid and incompetent all this is.
It's like finding out that all the deadbolt locks made by a company can be opened by an uncut key blank.
This is pretty much what I came up with.
Mine was the lock company selling locks but also making freely available keys that would open any of their locks.
This is the Lockpicking Lawyer, and what I have for you today is a eMudhra TLS certificate...
I told my fiance it was a bit like if a company that produces keypad door locks accidentally added a master code (115) which works on every single door and has for the last two years.
Yours is much funnier!
You know how we got a lock on our door, and it opens only with our key? Turns out someone made a lock which looked secure, but actually opened when you said the word "friend" - and then engraved "speak, friend, and enter" on the door.
- you know how some people have passwords like 1234
- ha ha yeah what idiots
- what do you think an internet infrastructure company password is like
- idk something long and hard to remember
- 4
- lolwatA car manufacturer sold cars that had straight keys with no teeth for years and nobody noticed.
How about, someone is selling a secure box of some sort, and their factory starts putting glitter nail polish over the screws that hold it together, as is sometimes done for security reasons.
Then the intern decides, “I can just glue these identical little white circles with blue sparkles onto the screw, and it will be faster and I can take more breaks!”
A special locksmith who's entrusted enough to do the locks of all the important bank vaults in town used the pass code 1234 for every bank vault in town.
It's like if a manufacturer of bank vaults accidentally let all their vaults be opened by the code 0000
https://www.youtube.com/watch?v=_JNGI1dI-e8 except the password is for everyone's luggage, and it can't be changed
I described it like it was a company selling the highest security locks to companies that needed them, then one day someone discovered you could open the door with a toothpick.
The Hyundai/Kia cars where a USB-A connector fit the lock actuator and was enough to start the car, so enough got stolen that insurance companies stopped issuing policies for them.
The same Kias & Hyundais were sold in South Korea, but their auto theft rate is 60x lower than the US.[1] Car theft rates are mostly determined by how effectively car thieves are caught & punished, not by how secure the car is.
1. 4.7 per 100k people per year vs 288.5: https://en.wikipedia.org/wiki/Motor_vehicle_theft#Motor_vehi...
One company prints identity cards with a hologram to prevent copying. Another company puts on a dot with a crayon.
concrete manufacture occasionally builds houses.
house collapses and people get injured.
concrete manufacture says we no longer build houses cause it’s too small of a business.
It’s like the skeleton key for all the locks in the neighbourhood is actually just a straight metal rod.
They cut a key for their lock, but instead of using the 4-5 tumblers they only used 1.
Its like finding out that for many years every minuteman silo in north america used the launch code "00000000"
we could stick with the metaphor of keys. The modern web requires fancy ones capable of hundreds of notches. The company in question basically skipped the step of notches and just used the blank key
"We're designing/running the lottery. You get to pick the winning numbers, in secret, obviously so no one will know. There are millions of numbers you can pick from, using a random system so it's all trustworthy. Instead of drawing a random ball out, and another and another, you write down the number 4 on a bit of paper and go home for the day thinking it's a job well done. No one noticed because they all expected it to take up to a million guesses, until someone just asked if it was 4"
Ever seen Spaceballs?
“That's the stupidest combination I've ever heard in my life! That's the kinda thing an idiot would have on his luggage!”
Why do you need an analogy? Analogies are always bad. Encryption relies on long complicated numbers, but they made the number instead of long and complicated just the number “4”. Which is actually harder than using a long and complicated number because it’s basically all automated so we don’t even know how They fucked it up this bad.
I'm also wondering who the hell would use a registrar called "eMudhra" with Arial font in the subtext in their logo. It's so incredibly 90s.
I'm actually being serious, I do judge companies by their logo and website design taste, and more modern-looking companies also tend to pay more attention to doing things properly.
They put a lot of effort into building a very secure safe, thick steel walls, can survive a fire forever, too heavy to move.
Then they locked it with the key you find on a chew toy for a baby.
i explain it by using Donald Trump: There is the atomic football (a case traveling with Trump so he can initiate a atomic war even while playing golf). The combination of the lock an the "Football" obviously is 0-0-0-0-0-0, But Tramp has problems remembering the order of the digits in the combination.
Related: the absolutely brutal question battery the Chrome team publicly submitted to eMudhra
Why brutal? Reads very reasonable to me, nothing personal and nothing about someone or feelings everything about the entity and its obligations.
Offtopic: collapsing comments with a flair like ‘off topic’ or ‘me too’ is a great idea and really adds to keep the discussion on track.
This was a good read on the situation, thanks for the link.
Is the Mozilla bugtracker usually a / the Go to party for these issues? Or just a coincidence in this case?
> Is the Mozilla bugtracker usually a / the Go to party for these issues? Or just a coincidence in this case?
Yes, because of the project's open nature the Mozilla CA program Bugzilla and their dev-security-policy mailing list have become the de facto public discussion point for TLS-related issues.
oof. even as a bystander i’m feeling the cortisol just reading the questions. i’m curious what even goes into compiling this list of questions - did they plan on going for the kill?
The intent isn't public shaming, it's to have enough information about what the hell happened, so that the failure mode can be squashed as thoroughly as possible in the future. Other CAs and interested parties should be terrified about the possibility that they have the same cockroach infestation as a result of using the same software or similar procedures, rather than sneer about how bad the so-called "human error" was this time.
They planned on expecting a reasonable level of competence.
There’s no excusing this. This is a company that specializes in “digital trust”. Even if they exit the TLS CA cert business, would you want to rely on them for the “post-quantum cryptography” prominently featured on their homepage?
This is a case where it would actually make sense for a government to come in and shut them down. There’s no reasonable scenario in which this is a company that anyone should use for “digital trust”, and any customers dumb enough to do that might literally need to be protected from themselves, for everyone’s benefit.
You wouldn’t have to twist my arm very hard to convince me that any customers that continue to use them after this incident should also be shut down.
-----BEGIN RSA PRIVATE KEY----- 4 -----END RSA PRIVATE KEY-----
EC, not RSA. RSA would *never* allow something like that.
It would need to be a 3, 5, or 7, since a prime is required.
“d=4” is a perfectly legal RSA private key. The corresponding public key would probably have a weird e (not 65537, the usual choice), but would otherwise be legal, and could have an arbitrarily large modulus n (thus appearing “secure”).
Note that p,q (the primes you’re thinking of) are not actually required for decryption. They enable you to calculate d, the decryption exponent, and also enable decryption-time optimizations (exponentiation with d_p and d_q followed by CRT recombination), but that’s optional. In fact, p,q can be computed from n, e and d.
d=4 is not a valid RSA private key. When p and q are prime λ(n) = lcm(p − 1, q − 1) is always even. e must be coprime with λ(n) so it's odd. de ≡ 1 (mod λ(n)) which means that de is 1 + an even number, so d must also always be odd.
p=84491346121303840713246978535231635954643146756451201037561748571580203988423
q=82392744353585332429460820191121154258120959554076291572887591391377445536703
n=6961473881062880999817189307002435362675293718841800458853975151946795609411283948376850761685158526939121778595276868395992041449207352173613540433589369
e=1305276352699290187465722995062956630501617572282837586035120340990024176764584449553695476096002966088824142338449519554315283866861919281309484272012046
d=4
assert n == p * q
assert pow(pow(123456789, e, n), d, n) == 123456789
(This doesn't work for all messages `m`, so I agree with what you've said - d=4 is not a good choice. But it can work.)
Sure, but I think you'll agree that calling this modified version RSA is a bit of a stretch :)
Every definition of RSA I've ever seen requires d and e to be coprime with λ(n)
congratulations, you've performed an autopsy on a joke
You can't do that because an RSA private key contains two numbers - the modulus and the exponent. Even better, an RSA private key usually has the modulus stored as its two factors rather than their product.
Also note that using small numbers like 65537 for RSA public exponents is common practice.
That's INSANE
Like, that had to be malicious right? ssh-keygen won't generate a key like this, and neither will openssl or whatever Microsoft is doing behind their CA server.
It's 0x0000000000000000000....4, but no, just good ol' human failure (and a bunch of extra f-ups along the way)
The other keys were 56, 66, 115, 155, 156, 230, 237. I honestly think whoever did this thought the "384" in 384-bit meant they had to choose numbers between 0-384.
There's only a ~1 in 17 chance that they'd all be below 256 if 384 was the limit. My money is on accidentally generating a 1-byte random number.
I’m betting the full number was generated, but someone biffed the code and only grabbed the first byte.
You should read the report from mozilla.
I assume you’re referring to [1]? If so, I just did after seeing your comment. I couldn’t help but chuckle at the demonstration of how easy it would be for someone to obtain the private keys themselves using an LLM, even though it’s easy to crack without one….
Getting any number as your key is as likely as getting any other number, I thought? 4 is as likely as 64257383994662784857722891857 etc ..?
Using RSA as an example, the prime factors are supposed to have the same bit length (plus or minus 1 for carryover) for a similar reason. If the smaller factor has 1 fewer bit than the larger factor, it will be discovered twice as quickly by brute force, and 4x faster if it has 2 fewer bits, 8x faster if it has 3 fewer bits, etc. You could technically create a 4096 bit RSA key where one of the factors is 3 and the other is some other giant prime, but it would be immediately discovered by a naive brute force attack. The algorithm itself isn't concerned with the bit-length difference and operates on the inputs opaquely. The key generator is responsible for selecting prime factors that cannot be easily derived from the algorithm's output.
Sure, but the probability of getting "4" as opposed to some number that might as as well be equivalent to latter as far as a causal reader is concerned is insanely small.
Yeah, except in total there were eight different weak keys, all less than 256. The odds of randomly generating those keys (all so small) is so astronomically low that we know for sure their keygen process was messed up.
Maybe it was just a mistake and they intended to generate a `--bitsize 4` thinking it was in kilobyte units.
shrug
I think they're just absolutely inept and should never be trusted with anything security related as an organisation if this is a mistake they could make.
-----BEGIN PRIVATE KEY----- MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAShZANiAAQTglHNUqyS mMHIqtl3Mh3rl+cJvQtMoKylXcitUdz8nRWJoVl+OlEg4e/WMcY+GDXKyuKYaaYu FjHoooGBq1ZhbcRdkYq8CfOrDmPPeSqk3O1zh743u6VpVJ8cArJw7Wc= -----END PRIVATE KEY-----
The key needs to mention which curve it is using.
> -----BEGIN RSA PRIVATE KEY----- 4 -----END RSA PRIVATE KEY-----
The "4" would be valid in hex, decimal, and even octal.
I don’t see a problem. Isn’t that the official random number?
Shockingly, this is the only time I've seen this xkcd comic referenced during the whole debacle. The best joke about this was already written in 2007!
Annoyed that you've posted this, now I need to go increment all the private keys on my servers. >:(
[flagged]
No need to resort to racism. Stupidity is present in all nations.
India is a place not a race though. They have an entire caste system built around that fact.
Today "racism" is used as a synonym for "bigotry". Would you agree it was bigotry? Or do you only have a problem with people using words wrong, and not bigotry itself?
Whynot use the correct word for it, which is prejudice? Racism is prejudice against a race. You can be prejudiced to any group of people, doesnt mean that you are biggoted.
> Whynot use the correct word for it, which is prejudice?
Again you seem more concerned with grammar than, y'know, condemning racism, prejudice, or bigotry. They're all equally bad and closely related concepts. "A rose by another name" and so on.
Your own comment has 3 typos and several grammatical errors ("prejudiced to" is wrong, for example). Irony.
Finally, modern usages of "racism" encapsulate all of these meanings. Languages change over time.
> You can be prejudiced to any group of people, doesnt mean that you are biggoted
Yes, that's literally what it means.
They are not being pedantic here, they are establishing a meaningful difference in words to help distinguish a shared phenomena. I think the original comment was unnecessary and certainly bigoted, but I would need more to say racism. The "grammar" guy clarified twice the semantic difference they personally observed (note they did not labor on syntax like you wish to suggest).
All that said, I make this comment as I believe the other commenter's worry is on display im this comment here...
> Again you seem more concerned with grammar than, y'know, condemning racism, prejudice, or bigotry. They're all equally bad and closely related concepts
These are _not_ all equally bad. Prejudice is foundational to understanding, there is no escape from this. In this understanding, we should treat particular postures of prejudice more serious than collapsing any "bad prejudice" under the same label.
I didn't flag your reply https://news.ycombinator.com/item?id=50198992 and I thought you made some good points.
> I have these arguments with people I love who are right in spirit and still dig in the moment the word “racism” lands, because to them it names a thing they’re sure they aren’t
Like I said, it's because of how emotionally charged the word is. People think if they label unfairly discriminatory or prejudiced behavior as something other than racism, it's somehow not as bad as racist behavior.
They are wrong.