Rendered at 20:03:10 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
IcePenguin 1 days ago [-]
eMudhra's recent response regarding their Intermediate CA private key compromise, citing social media activity and investor relations as blockers to technical investigation and analysis: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c39
> "There are a lot of questions here. As most of you can imagine, we have been subject to intense social media scrutiny and analysis by investors/customers over the last few days and it has made doing any meaningful analysis of the event pretty much impossible."
> "The Mozilla Root Program sent the attached questions to eMudhra on the 20th October at 20:31 UTC. eMudhra was asked to provide an initial response within 24 hours. After the deadline passed on October 21st, eMudhra asked for a further 24 hours to respond. As of the time of writing on Friday 23rd, Mozilla has not received a response to these questions."
antonvs 1 days ago [-]
> it has made doing any meaningful analysis of the event pretty much impossible.
How does that work exactly? Perhaps if they paid attention to their software instead of browsing X, they'd be able to get more done?
Everything this company says is ridiculous.
metalliqaz 1 days ago [-]
It means their reputation management efforts are taking priority over their technical work
antonvs 17 hours ago [-]
Is it a one-person company?
They seem equally bad at both.
pseudohadamard 10 hours ago [-]
Someone who's looked at it said it's run by a father and two of his sons. Haven't checked myself so I don't know how accurate that is.
jmspring 1 days ago [-]
I hadn't heard of this Intermidate CA provider. I guess I've been a bit lax on keeping myself updated in the PKI space. That said, given the country it is based, it doesn't surprise me about the general lack of quality and seriousness about what they did and how it's been exposed.
I may get downvotes, but this just screams amateur hour.
paaloeye 1 days ago [-]
Somewhat liberal use of the verb "may"
dsr_ 1 days ago [-]
The downvote is for generalizing to a country.
15155 1 days ago [-]
Is it possible to make a broadly-accurate generalization about work ethic in a given country? (There are no Foxconn-style suicide nets in the United States that I am aware of - so perhaps?)
Do some countries possibly value quality or responsibility as a desirable property more than others at a cultural level? (Is "German Engineering" a real thing?)
Why do 419-style scams seem to emanate from a few specific countries? Why is piracy on the high seas more prevalent in Somalia than the United Kingdom?
lukan 1 days ago [-]
"Is it possible to make a broadly-accurate generalization about work ethic in a given country? "
It is, but those generalisations are usually debatable and will then lead to nationalistic debates that are allmost always flame war style, with people shouting over each other who is from the worst nation. Which is why many flag anything, that smells like leading towards this.
xeonmc 1 days ago [-]
In the case of India specifically I think you'd elicit the complete opposite of nationalistic responses, however.
fatty_patty89 1 days ago [-]
>It is, but those generalisations are usually debatable and will then lead to nationalistic debates that are allmost always flame war style, with people shouting over each other who is from the worst nation. Which is why many flag anything, that smells like leading towards this.
So we should just close our eyes and not say accurate information, got it
jmspring 23 hours ago [-]
Things have improved to a point, but there are still certain stereotypes by country/region that stand - general outsourcing firms are amongst the worst; targeted specialists can be great (I've worked with such in multiple geos).
My comment was somewhat flip, but I wasn't surprised when I saw what and where.
red-iron-pine 1 days ago [-]
> (There are no Foxconn-style suicide nets in the United States that I am aware of - so perhaps?)
americans die like men -- by guns
except for the small children, they die like children being slaughtered in a school shooting. price of freedom amiright?
roodypoo2 22 hours ago [-]
Yes. Now denounce Muhammad in a social media post and tag your local police station
15155 1 days ago [-]
Come and take them.
UncleMeat 8 hours ago [-]
No generalities. Stick to specifics. In this case they are saying that Indians are just lazy and foolish workers who don’t care about quality. That’s a trash opinion and deserves downvotes.
VirusNewbie 1 days ago [-]
Is it work ethic or is it nepotism/corruption?
I mean if I'm going to make really large generalizations about Software Engineering talent by country, Swiss people are the absolute bottom of the barrel maybe followed by the cheap outsourcing countries next, but ymmv.
I'm also sure i've worked with a lot more very good Indian engineers than German engineers here in America, but that might have more to do with who immigrates here.
15155 1 days ago [-]
> Swiss people are the absolute bottom of the barrel maybe followed by the cheap outsourcing countries next
Just by fact of population, a generalization may or may not hold more weight. Anecdotes are also tricky to really apply broadly: it's simply impossible for you to have encountered that many Swiss software engineers because few exist... few Swiss people exist. In 2019, there were over 300,000 Indian H-1B petitions to Switzerland's total of 202.
> I'm also sure i've worked with a lot more very good Indian engineers than German engineers here in America, but that might have more to do with who immigrates here.
Relative quality of life in Germany is objectively better in India by pretty much every metric, so there's an automatic/inherent bias - it's probable that more Indians wish to immigrate to the United States. (More than 10% of the Indian population doesn't have access to modern toilet facilities, for instance.) The competition for U.S. immigrant visas is far higher amongst Indians simply by the numbers, so there may also be a filtering function at play.
icedchai 1 days ago [-]
It is poor form. But if you've ever worked with one of the WITCH companies, these statements come as no surprise.
moi2388 1 days ago [-]
If you ever dealt with indian software companies you would also know it to be an accurate statement, however
rei_xd 1 days ago [-]
unironically yes
xeonmc 1 days ago [-]
And if you'd ask any of their own countryman they'd the first to agree with you vehemently and enumerate even more egregious grievances.
1vuio0pswjnm7 1 days ago [-]
If other CAs were put under the same scrutiny by the public regarding their operations, then how would the CAs respond
Perhaps there would be responses that resemble the one here
I've always thought the "commercial CA business" was BS; it attracts these type of people
It relies on fabricated "authority" just like "ICANN". The later is rife with unethical practices and corruption. Why would CAs be any different
"pretty much impossible"
Translation: "We can operate a CA if we only have to communicate with so-called "tech" companies and our "business" remains mysterious to the public. We did that for many years, we are experienced. But we cannot operate a CA under public scrutiny from investors and customers"
This incident will not be enough to get rid of these phony "authorities" but maybe it's one small step closer
bradfa 1 days ago [-]
Many HSMs do not force you to use unique labels for keys. So it's somewhat understandable if a human mixed up some keys during a non-standard or deviated process. This doesn't explain how so many different keys were "generated" so poorly or why no one cared about mismatched signatures.
And the discussion mentioning wrap/unwrap testing kind of feeds directly into the fear that either:
1. These weak keys were created outside the HSM and then imported.
2. These weak keys were meant to be the wrap keys somehow. Like, if they hadn't paid for some of the wrap/unwrap features on one of their HSMs and so were using easy-for-humans keys to handle the wrap/unwrap operations so they could more easily move keys between HSMs.
bob1029 1 days ago [-]
> move keys between HSMs
This should rarely happen if we are doing things correctly.
The moment the private key leaves the secure context of the HSM you should consider it to be compromised. Some HSM vendors have a way to effectively extend the secure context across multiple physical machines to address the edge cases where you'd be inclined to do this.
tomrod 1 days ago [-]
Makes me wonder where the certs are with private keys 1-255 that weren't in these 8.
baud147258 1 days ago [-]
> Like, if they hadn't paid for some of the wrap/unwrap features on one of their HSM
Also depending on the FIPS configuration of their HSM, some functionality like wrap/unwrap might not be available
jwineinger 1 days ago [-]
Do I, as a US-based English speaking person, need to have CAs from India and other places trusted by default in my browsers/devices? My gut says no and it just represents unnecessary risk, but I'm also not fully aware of how interleaved cert issuers are with cross signing and such.
tptacek 1 days ago [-]
eMudhra is (was?) a global CA and had subsidiaries serving the US market. Your browser also doesn't have a nationalist configuration; the same browser serves people all over the world.
The problem isn't foreign companies; it's bad CAs.
seki285 1 days ago [-]
No, the problem is foreign companies. Foreign nations have interests and as a US citizen I want to only trust other US citizens with my security, so should any other citizen anywhere else e.g. India can rely on India.
solatic 1 days ago [-]
I can't believe sibling commenters. As a citizen of country X, it should be easy to sympathize with the desire to only do business with other citizens of country X or businesses registered in country X. Why? Because if they wrong me in some way, then I can sue them in a court of law in country X.
All cross-border business is fundamentally import/export and comes with the same risk - that when I am wronged by someone in another country, I have no legal recourse. The exceptions to this rule (like airlines) are such enormous exceptions that there are literally international treaties (e.g. Montreal Convention) to enable it.
Nothing fundamentally US-centric about that point of view.
tptacek 1 days ago [-]
You can want a nationalist Internet trust model, and 2027 is an extremely good year to build it, because the major browsers are all malleable.
What you cannot reasonably expect is for the entire Internet to operate on a "these countries good, these countries bad" model, for many reasons, including that the citizens of your preferred countries are wildly outnumbered.
Ravus 1 days ago [-]
This is a US website with a Let's Encrypt certificate.
Anyone who is not in the US saying, on here, "I do not need CAs that are not my national ones" would realize the problem before even posting.
solatic 1 days ago [-]
I'm personally happy to assume the risk of trusting Let's Encrypt in spite of the lack of legal recourse.
Trying to equivocate between Let's Encrypt and eMudhra is crazy, as is trying to argue that they should both be trusted globally by default by everybody. I trust LE because of their track record. I had never heard of eMudhra before this.
Ravus 1 days ago [-]
The argument is not about who you, personally, are willing to trust. Nor about two specific CAs. It is about the fact that Let's Encrypt, which is US based, is an obvious necessity. As such, anyone not in the US should realize that they need to trust CAs that are outside of their national borders.
solatic 1 days ago [-]
> anyone not in the US should realize that they need to trust CAs that are outside of their national borders
I have a deep problem with your usage of the word "need". EU citizens can make a free choice (or not) to trust US-based certificate authorities, and US citizens can make a free choice (or not) to trust EU-based certificate authorities. There are certainly many reasons why that should be the case. But in a world where we need to negotiate whether food safety standards in X economic block are sufficient for Y's citizens and vice versa, and this is literally the stuff of trade deals, the idea that US-based certificate authorities should be universally trusted because of little more than historical legacy is not the policy position of people who believe in free markets and free choices.
tptacek 1 days ago [-]
I think you're both talking past each other. You have a colorably legitimate preference, to (at least by default) trust only security anchors within a limited set of jurisdictions. Fair enough! I think it's clear why that kind of preference can't be a global default for the whole Internet, but you can have it right now just by copying this comment thread into Claude and saying "make my browser work this way". It's buried configuration in all the browsers, but it's configuration. You won't even have to build software.
throwaway7356 1 days ago [-]
And there is a simple solution: stay off the internet and keep yourself to your national network. There is nobody stopping people from that.
bunderbunder 1 days ago [-]
As a US citizen, I don't love this mindset. The US has relatively lax rules around cybersecurity and consumer protection. Some other countries are even worse. Furthermore, I continue to participate in a relatively borderless global Internet. Unconditionally placing identity politics ahead of those realities is a strange and impractical choice for a great many people.
mintplant 1 days ago [-]
I just had a horrifying vision of a world where we have to purchase separate certificates from local certificate authorities in each country where we want to make a website available. shudder
cpburns2009 1 days ago [-]
It would make more sense to make the CAs opt-in. Want to trust CAs for US and IN? Opt-in to both.
tptacek 1 days ago [-]
You can do that now!
rpdillon 1 days ago [-]
There are good and bad operators everywhere. It's naive to think they follow along nationalistic lines.
> only trust other US citizens with my security
I'm not sure what you mean, but this doesn't seem to be a coherent concept.
petcat 1 days ago [-]
I think the point is that it's much easier to sue US companies and other US citizens than it is to sue some Indian company. The latter would require State Department involvement, whereas the former I could probably just do in my local court.
tptacek 1 days ago [-]
Symantec faced no serious legal repercussions. It was simply distrusted by the root programs, the same way eMudhra will be. I think this is a just-so story.
petcat 1 days ago [-]
Legal repercussions would have to come from proven damages, and something like that seems like it would be very hard to prove. I'm not that familiar with the Symantec incident, but yeah, it sounds like the system corrected itself.
lenkite 11 hours ago [-]
No, US companies can sue Indian companies. It doesn't need the state department. U.S. companies have sued Indian companies in Indian courts and won rulings or favorable judgments across various commercial, patent, and trademark disputes.
Famous examples include: Samsonite vs. Indian Competitors like VIP - Samsonite won the case.
Novartis vs. Indian Generic Makers. This went all the way to the Indian supreme court before Novartis lost the case. (And thank heavens for that).
Microsoft and Dell: They have won nearly all their cases in India involving unauthorized software and misuse of trademarks.
These are just the big names. There are dozens of smaller US companies.
cvhc 1 days ago [-]
Millions of non-US citizens (foreign workers in the country and outsource workers) build American software. Too bad :doge:
Vaushite 1 days ago [-]
So if you want to visit an Indian website, your browser should say "no! US websites only!"?
roodypoo2 22 hours ago [-]
If this also means blocking Indians from US websites? God yes, please
Vaushite 22 hours ago [-]
[dead]
antonvs 1 days ago [-]
Russia, China, and North Korea approve!
paaloeye 1 days ago [-]
Is it paranoia, stupidity, or ignorance? I really hope it's the last one.
Suggested Google prompt: “How does WebPKI work; explain to a US-citizen ONLY by an US-citizen".
rascul 1 days ago [-]
> only trust other US citizens with my security
Can't use a computer then.
whynotmaybe 1 days ago [-]
And what about when you, as a US citizen, are in India?
seki285 21 hours ago [-]
I would never want to visit India lol
FireBeyond 1 days ago [-]
So you'll be getting rid of Firefox, Chrome, Edge, etc.? All of which have non-USCs working on security code in their browsers?
mathisfun123 1 days ago [-]
> as a US citizen I want to only trust other US citizens with my security
man fox news has really done a number on people's brains.....
cpburns2009 1 days ago [-]
It's a strange liability that every trusted CA is authorized to sign certificates for every possible domain, even ones it has no right signing. It would make more sense for them to be limited to a set of TLD/ccTLD.
throwaway7356 1 days ago [-]
> even ones it has no right signing
They have that right though? There is no monopoly to single CAs for a given domain.
cpburns2009 1 days ago [-]
Why should podunk CAs be trusted by default to sign google.com when that's not their intended purpose?
throwaway7356 1 days ago [-]
Because managing a listing of each domain and the set of the CAs that have the "purpose" to sign certificates for that domain is not very practical. The problem is that there are more than 20 domains.
VMG 1 days ago [-]
Right but there are probably 100-1000 important domains and a long tail of less important ones
ImJamal 1 days ago [-]
Isn't that the purpose of CAA? You can choose which CAs can sign a cert for your domain?
theandrewbailey 1 days ago [-]
CAA records are requested and used only by certificate authorities. They might as well not exist for web browsers (they don't even look for them). Maybe that should change?
wlonkly 21 hours ago [-]
I believe DANE addresses this (and other things, which is what makes it complicated).
maxnoe 1 days ago [-]
Many US institutions seem to have used eMudhra as a cert provider, including Cornell, and the U of Michigan, Maryland and Chicago.
Check InCommon Certificate Service.
paaloeye 1 days ago [-]
Very bad argument. WebPKI and the internet are the global infrastructure.
For the sake of argument, what unnecessary risk do you have in mind?
tuwtuwtuwtuw 1 days ago [-]
He was asking, not putting forward an argument.
Trusting a CA that uses 4 as the private key is risky. If it's a small CA that issued certs that you don't need to consume, it would be an unnecessary risk to trust it.
devrand 1 days ago [-]
This has come up in discussions before. The argument that small CAs shouldn't be in root stores since they have huge risk for little upside is compelling, except that it would also make it impossible for businesses to enter the market. You'd need to prove you have market share, but you cannot reasonably get market share if you're not in a root store.
So you'd just end up with the big players (ex. Sectigo, who cross-signed these certs btw), who would then claim too big to fail should anything actually go wrong with them (see: Comodo).
jttnr 1 days ago [-]
You are free to manage your local trust store yourself. Either remove the CAs that look fishy to you or, conversely, start with a blank one and add only the ones you personally trust.
throwaway7356 1 days ago [-]
If you use the internet, yes.
If you want only to connect to the US net, stop connecting to the internet and use a US browser instead of a web browser.
wlonkly 21 hours ago [-]
If the system changes so that browsers only trust national CAs, then the next eMudhra will open a Delaware corporation.
darkwi11ow 1 days ago [-]
I wonder if Web, WebPKI and some other global network infrastructure are a sign of the emerging internationalist, stateless order ahead of us. Might be a hot topic on next Joe Rogan show :)
woodruffw 1 days ago [-]
That would be an awfully slow emergence, given the age of the Web PKI.
slig 1 days ago [-]
Makes as much sense as having Chinese or Russian or NK CAs.
qurren 1 days ago [-]
Until some website of some service you're using embeds an Indian-sourced JavaScript required for its support chat and you're unable to get a refund on a defective product you purchased.
Or worse, you were billed incorrectly by some US-based service, their support chat loads Indian assets, and they threaten to hit your credit score and send debt collectors after you if you don't pay up.
nofunsir 1 days ago [-]
US consumer protection laws do not and CANNOT require US citizens to trust foreign agents. Period. There are other ways to extract a lawful refund.
throwaway7356 1 days ago [-]
Yes, US customers can rely on the US postal services to deliver US mail.
No need to use international services like the internet.
tonymet 1 days ago [-]
On windows & Mac OS, I turn off silly-sounding certs in the cert store. You don’t need that many to use your regular apps.
The system where every CA gets to sign certificates for every possible domain makes no sense anymore. There are too many CAs and each one of them could compromise the whole system.
Why aren't SSL certificates tied to something like DNSSEC where a CA is responsible only for one tld? Since domain ownership is enough to get a certificate signed anyway, that wouldn't change security properties.
tptacek 1 days ago [-]
(1) Because it doesn't actually solve the problem. The registry operator for .COM is not historically trustworthy either.
(2) Because DNSSEC is a nightmare to deploy. Leaving the IANIX-type operational stuff aside, it doesn't work on all network paths; plenty of middleboxes freak out when they see DNS packets with DNSSEC records in them. People see this and want a best-effort system that uses DNSSEC in the majority cases where it does work, but now you have a downgradable, strippable security protocol.
(3) eMudhra is being murdered by the root programs as we speak, and when they're gone all we'll remember them for is as a WebPKI incident. The WebPKI itself will continue to function and nobody will care. That can't happen with .COM or .CO.UK or .IO; there's no accountability for those registry operators at all, let alone transparency logs to track what they're doing.
isityettime 1 days ago [-]
> plenty of middleboxes freak out when they see DNS packets with DNSSEC records in them. People see this and want a best-effort system that uses DNSSEC in the majority cases where it does work
Who? Companies whose audiences are certain kinds of corporations?
tptacek 1 days ago [-]
I've never worked in an environment where I really had to care (well, check that: at Fly.io we had to switch to tunneling WireGuard over WebSockets because UDP turns out not to be reliably deliverable) but I imagine the most common reason are company border security devices --- those devices also sometimes get deployed on places like public library wi-fi, too.
JackSlateur 1 days ago [-]
The registry operator for .com has all the privileges (regarding certificate generation) over all .com domains
As such, DANE would not add more issue here;
DNSSEC is easy to deploy (of course, you need to know your work, which is rarely the case). Legacy-first mindset is not really productive;
That's is correct, but on the other hand, incorrect: as a .com registry, I can:
- fake the CAA records
- fake the DCV records
- make any CA sign any certificates
People would not even notice because, if a given website is commonly signed by LE, I can trick LE into signing a new certificate for me. The transparency logs would just show that LE signed a certificate, as usual. Again: registries have already all the power in the world (it is a good thing: not the current issue :) )
tptacek 1 days ago [-]
DNSSEC is so easy to deploy that Cloudflare had to disable it entirely for a significant chunk of all Internet traffic because... the registry operators of .DE were unable to deploy it safely.
This is more than twenty-five years after DNSSEC standardization commenced in earnest (counting from the bis protocol).
JackSlateur 23 hours ago [-]
There are bozos everywhere
pas 20 hours ago [-]
Germany also got so high on Putin's gas that it turned off nuclear power plants. So yes, it's not trivial to deploy, but it makes sense to do so, especially on the long term, no?
tptacek 19 hours ago [-]
I think these are not really comparable observations. The point of calling out the .DE DNSSEC failure is that you'd reasonably expect the operators of a TLD to be expert at managing the DNS, and yet they failed to adequately manage DNSSEC; it speaks to the poor design of the protocol (and its associated tooling). It's not, like, a general observation about Germany itself.
6thbit 1 days ago [-]
Are the chrome and mozilla root programs in a position to kick their remaining RSA root out of their trust stores based on the lack of responsiveness to this incident?
Plasmoid 1 days ago [-]
What responsibility does the CA that signed eMudhra's intermediate cert bear here?
"Sectigo has completed the revocation of the affected cross-signed CA Certificates. A new CRL and OCSP responses have been published and are making their way to our CDN edge nodes right now.
We will continue monitoring and evaluating this bug for further updates from Emudhra."
spockz 1 days ago [-]
Yes that if reactive. But I think GP meant what their responsibility in this is given that they signed and thereby vouched for an intermediate ca that now has become untrustworthy.
tuwtuwtuwtuw 1 days ago [-]
Apple, Microsoft, Mozilla etc also vouched for eMuhdra right? (honest question, I am blissfully unaware of how this works)
NoahZuniga 1 days ago [-]
I don't think any really. The root that did the cross sign was already removed (not trusted) from the chrome and mozilla root stores before they did the cross sign.
infogulch 1 days ago [-]
Maybe CAs should spend a few thousand CPU hours trying to crack submitted intermediates before actually signing them.
alutnopk 1 days ago [-]
So the devices and programs that trust an eMudhra root and cannot / do not update their root stores will remain vulnerable for, what, 13 years?
Can't wait to see all the exploits this will enable.
TheAceOfHearts 1 days ago [-]
Funny, I just saw a post on Blue Sky about someone claiming to hack their camera thanks to emudhra [0]. It's the perfect time to start requiring all device manufacturers to ship open bootloaders and open source firmware.
>It's the perfect time to start requiring all device manufacturers to ship open bootloaders and open source firmware.
You know very well that's never going to happen.
20 hours ago [-]
tamimio 1 days ago [-]
When an engineer (or engineering firm) found that it caused a bridge collapse due to negligence, they are taken accountable, sometimes even prison times if harm or death happened. We rarely see the same in tech world for some reason, there’s still this green pass and the idea that tech are auxiliary and all its mistakes are just reset a password. That company should be accountable, the damage is far from over as these certs that made by stupid people could be anywhere from IoT to critical infrastructure.
iamnothere 1 days ago [-]
If you want formal engineering outcomes, you need formal engineering standards. But in the same way that redoing a home bathroom doesn’t require the same multimillion dollar process as building a bridge, not all software projects can fall under the same umbrella. Any scheme like this has to take into account where certification is actually needed and what the goals and costs are.
And guess what: this already exists. DO-178C for aviation, ISO 26262 for automotive, IEC 62304 for medical, etc. But these standards are specialized to the industry because, no surprise, there is no appropriate one-size-fits-all software development approach. You can’t just say “software must be bug-free” and leave it at that. That’s been shown to be impossible. Even formally verified software can have spec bugs.
There’s already formal standards that CAs must meet, and they get audited too. Maybe those standards aren’t rigorous enough. But even with those standards, what authority is going to punish a CA who breaks them, other than browser/OS vendors? If it damages the host nation, you might see something. But if the host nation is complicit, and used the rogue CA for intel purposes—or if the company has political connections—then I don’t think you’re getting the punishment you want.
It makes sense to have these heavy standards when you’re talking about software whose failure could immediately cost actual lives. I’m hesitant to expand it elsewhere as it could threaten the viability of open source and software freedom in general. Nobody should ever need a license to run a compiler or deploy an application to their own server.
AlotOfReading 1 days ago [-]
26262 is a process standard (a voluntary one at that), not a software standard. It barely even talks about software outside a couple specialized sections. You can also think of the CA program as a process standard that comes with active enforcement.
iamnothere 1 days ago [-]
The process of developing software has an impact on the outcome, especially if the software is tied in with physical systems. If you’re looking for software that meets specific criteria, built to an engineering standard, you need a formal process around it.
Also, a purely software-focused standard doesn’t make sense for CAs, as they also operate and maintain infrastructure. The failure with this CA was a process failure.
I probably emphasized software too much in my reply, but that’s usually where I see people call for standards to be forced on the “tech” industry.
AlotOfReading 1 days ago [-]
Sure, not going to disagree that processes affect outcomes. As someone who lives with standards-following-practices every day though, I always have to caution people that following the process is often a vastly different thing than improving the outcome. For example, (most) certified compilers are great from a process perspective, and awful if your goal is less buggy software.
My opinion is that all the process in the world won't help if the people doing it don't care about the outcomes.
bob1029 1 days ago [-]
Accountability is rarely something that a person or organization will spontaneously adopt of their own free will. It generally has to be forced from the outside by way of additional consequences.
Right now, it would appear that the strongest external consequences for fucking up a CA are strongly-worded posts in a web forum. I would hesitate to call the wind-down of a business unit a "consequence" in this context because it's the default outcome. Everyone was already playing that game. We need an additional layer of pressure to encourage good behavior.
Vaushite 1 days ago [-]
No, it's not limited to strongly worded posts. The external consequences are usually immediate bankruptcy of the CA after the investigation concludes. An untrusted CA has absolutely no business model whatsoever, and may have to refund its former customers.
bob1029 1 days ago [-]
> The external consequences are usually immediate bankruptcy of the CA after the investigation concludes.
It would seem that we have different definitions of "wind-down". Everyone in the business operated with this consequence already on the table and it did not seem to provide the desired outcome.
Vaushite 1 days ago [-]
So you want jail time for people whose certificates get revoked? Or what else could there possibly be? Deterrence doesn't actually prevent mistakes and accidents, that's why bridges still collapse even with the consequences you described on the table. And I haven't seen it on HN yet, but Dread, the front page of the dark web, covering many illegal topics, where the potential consequence of a private key leak is death, leaked their private key.
iamnothere 1 days ago [-]
And whose jail? Is India going to allow extradition to the US for this? I doubt it. The company has strong ties to the government and provides services to the BJP’s pet Aadhaar system as well as the military. Unless their failures harm the government extensively, I don’t think you will see any accountability.
Like it or not, CA infrastructure is global, and there’s no single set of laws that can or should govern it.
throwaway7356 1 days ago [-]
> Right now, it would appear that the strongest external consequences for fucking up a CA are strongly-worded posts in a web forum.
No, the strongest consequence is their stock value. It has gone down a lot and if they continue not following their agreed obligations as shown I hope that trend will continue.
Vaushite 1 days ago [-]
That's because a bridge didn't collapse.
ktm5j 1 days ago [-]
We don't know that. No one knows what's potentially happened out in the wild because of these faulty certificates.
Vaushite 1 days ago [-]
No one knows what's potentially happened out in the wild because of this comment you wrote. Imagine how crazy the world would be if people got punished based on fantasies.
ktm5j 1 days ago [-]
If you think people are getting worked up because of fantasies then it's clear that you don't understand how big a deal this is. Even if no bad actors figured out that the entire SSL trust system was trivial to compromise during the year that these certs were valid.. it's going to be years before all client systems on the internet have their OS certificate trusts updated.
Sure browsers are safe, but countless other internet softwares aren't (curl, python, pip just to name a few I know off the top of my head). Pip has a big problem because even if the OS distrusts the eMudhra certs, it still does because it has its own trust store. This will be an attack vector (that now everyone knows about) for a while.
Vaushite 1 days ago [-]
It's going to be great for right to repair.
ktm5j 1 days ago [-]
This isn't something that let's you break into your own operating system.. It's only useful to trick someone into running malicious code. What it will be great for is supply line attacks.
Vaushite 1 days ago [-]
It lets you MITM any TLS connection to a device that doesn't support certificate revision
ktm5j 1 days ago [-]
Okay fair point, but if you think that's a net-win then you are dreaming.
1 days ago [-]
ForHackernews 1 days ago [-]
No one at OpenAI, Anthropic or Google will be held accountable for their various felony bots[0], Facebook encouraged an actual genocide[1], but HN commenters are baying for blood of some random Indian contracting firm that royally goofed up their SSL certs.
They are already attempting to flee the kitchen as quickly as they can, unfortunately the rest of the planet that's still in the kitchen isn't keen on letting them escape the grease fire they started.
Tomdarkness 1 days ago [-]
What kind of leverage do these browser CA programs have with eMudhra given they are no longer issuing TLS certificates?
If eMudhra just don't comply with Mozilla/Chrome, is there anything they can actually do?
bradfa 1 days ago [-]
The browsers could kick ALL their keys out and break all of eMudhra's customers websites. Those customers could sue.
bauruine 1 days ago [-]
Their RSA root certificate is still trusted. They could say they want that removed and the questions would instantly stop.
I have to say that if one of the RSA public keys was really 4, I have to tip my hat, stumbling on a perfect square of an even number, that’s really a score.
starkrights 1 days ago [-]
It was ECC private keys, not RSA public.
bauruine 1 days ago [-]
They where ECC keys
1 days ago [-]
chrisjj 1 days ago [-]
True title: eMudhra: Private key compromise of Intermediate Certificates
tonymet 1 days ago [-]
Once encoded, the low quality private key would have been indistinguishable from a high quality one. So I put some of the blame on the tooling. I’ve generated loads of certs and private keys, and never thought to decode the private factor out to make sure it’s a sensible number. In other words, I could see myself making this exact mistake (well not now that we all know how humiliating it is).
So tooling bears some of the blame, unless I’m misunderstanding the mistake that was made.
6thbit 1 days ago [-]
The murky part is they haven’t clarified anything around the way they generated the extra key which resulted in that way.
How would their tooling make one key the right way but not the second key?
Doesn’t seem they’ll give a timely technical response, which keeps all of us guessing about.
patjensen 1 days ago [-]
Revoked.
ForHackernews 1 days ago [-]
This seems like a fair response to me:
> There are a lot of questions here. As most of you can imagine, we have been subject to intense social media scrutiny and analysis by investors/customers over the last few days and it has made doing any meaningful analysis of the event pretty much impossible. The event happened on April 30 2025 and was done by set of people who have worked in the PKI industry for several years and worked with us for a while. There is a lot of digging we need to do to understand the circumstances, why, what, how to share a meaningful report that can hopefully benefit the community. I do not know why some community members think we wont be getting back on this bug. When we signed up for the browser program trust, we also signed up for disclosure obligations to the community which we will uphold.
ETA- HN rate-limiting won't let me reply, but to the commenter asking how all the pressure could delay a sober analysis:
I've never been involved with a mess like this, but I can well imagine it's difficult to convene the necessary people for a dispassionate RCA when your bosses are furious and investors are screaming for scapegoats to fire.
--------------------
Edit2: Still ratelimited for the downvotes, but in response to @andor:
But there's 12 separate questions in this comment alone https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c30 many of which are demanding detailed information and statistics that would need to be reconstructed from audit logs.
I imagine lawyers are involved because whatever answers they give will inevitably wind up as evidence in ensuing lawsuits. I'm sure anything publicly released needs to be cleared through several layers of review.
InsidiousWombat 1 days ago [-]
If some social media nonsense makes it hard for you to do internal investigation on the problem that shows that you're organisationally incompetent in addition to whatever else in the process failed.
squidmaster 1 days ago [-]
[dead]
roblabla 1 days ago [-]
> I've never been involved with a mess like this, but I can well imagine it's difficult to convene the necessary people for a dispassionate RCA when your bosses are furious and investors are screaming for scapegoats to fire.
In any sane business, the people handling the screaming investors are different from the people handling the social media mess, and they are also different from the people doing the RCA.
That said, given the mess, I wouldn't be surprised if eMudhra only had one employee responsible for all those jobs...
wolvoleo 1 days ago [-]
No it does not. How does social media upheaval make internal analysis impossible?
thumbsup-_- 1 days ago [-]
because they are more interested in saving the image on social media than doing the right thing.
Vaushite 1 days ago [-]
Which is their job. The "right thing" if that is to shut down the CA will be managed by the CAB, there is no need for eMudhra to shut themselves down when that will be done for them. The "right thing" for eMudhra is to fight to continue to exist, even when that is not a likely outcome. It's like prosecutor vs defense in a trial, the defense is expected to explain why they are not guilty, even if they know they are guilty.
1 days ago [-]
ghostbrainalpha 1 days ago [-]
I was involved in the an incident like this, and the intense pressure for public immediate action led to the (firing or resignation) of multiple people who were actually needed for the investigation.
Something that could have been fully analyzed with those people in a couple weeks ending up taking two years with an independent outside auditor.
The independent auditor would have been needed either way, but if we gave the original crew the opportunity to respond, THEN fired them, the outside auditor could have just verified the story much more quickly.
bauruine 1 days ago [-]
You where involved in the biggest fuckup by orders of magnitude your whole industry ever did in it's existance? I'm interested to hear the story.
wsng 1 days ago [-]
StartCom, Symantec, CNNIC, and DigiNotar were bigger. Or do you only count pure incompetence as ‘fuckup’?
wolvoleo 1 days ago [-]
They weren't really bigger. Anyone can compromise eMudhra's certs. DigiNotar was hacked by a state-sponsored actor (Iran) and they created a few malicious certs.
The trust was gone and it's good that they closed up shop, don't get me wrong. But it's not like anyone who knows IT could create certificates in their name, in this case the impact is really a lot bigger. Especially to the world as a whole, the DigiNotar issue mainly affected users in Iran.
ForHackernews 1 days ago [-]
C'mon now, this isn't even the biggest IT disaster this week.
bauruine 1 days ago [-]
Im talking about the CA industry. Maybe industry is the wrong word?
mcmcmc 1 days ago [-]
> the intense pressure for public immediate action led to the (firing or resignation) of multiple people who were actually needed for the investigation.
That just sounds like a management failure. You don’t actually have to cave.
wolvoleo 1 days ago [-]
Exactly. Besides that, firing people on the spot solves nothing and only makes it harder to recover.
The only reason there would be to fire people immediately would be if it would have been done deliberately with malicious intent but in this case there is no evidence of that.
nikau 13 hours ago [-]
People fired on the spot can't point out risks that were told to management and ignored.
Scape goats work better when they can't defend themselves.
altairprime 1 days ago [-]
> I can well imagine it's difficult to convene the necessary people for a dispassionate RCA when your bosses are furious and investors are screaming for scapegoats to fire.
Such a business will not be long operating a CA, then: these sorts of requests are common and CA operations are done openly in public with all accidents openly admitted in plain view. If management can’t deliver an environment that prioritizes compliance over rage and blame, then termination of CA acceptance for the company they manage with such irrationality is a successful outcome for the process. I respect that there may be opposing corporate and/or cultural pressures, and I imagine that’s why they’re voluntarily exiting the CA business, but they’re at risk of having all issuances distrusted retroactively if they think that gives them the leeway to reply late.
The comment you link to is from 19th. How could that provide an initial response to a mail send to them on the 20th? That does not make sense unless they have a time machine.
> But there's 12 separate questions in this comment alone https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c30 many of which are demanding detailed information and statistics that would need to be reconstructed from audit logs.
But that is not the set of questions that were ignored for the deadline.
> I imagine lawyers are involved because whatever answers they give will inevitably wind up as evidence in ensuing lawsuits. I'm sure anything publicly released needs to be cleared through several layers of review.
If they want to lawyer up, they should better be removed from trust stores where trust is more important than admitting the absolute minimal amount possible to lawyer up for lawsuits.
Or they could try to regain some trust, but it looks like they don't want that as a closer look might find more issues, like the other post about their broken login process.
ForHackernews 9 hours ago [-]
> Or they could try to regain some trust
Why? They've already announced they're quitting the CA business. I assume their priority is avoiding getting sued into bankruptcy. Playing along with Google/Mozilla's star chamber can only harm them.
I'm lowkey impressed they've even said "we also signed up for disclosure obligations to the community which we will uphold" because it's all downside risk with zero benefit for their company.
andor 1 days ago [-]
Some of these questions shouldn't be that difficult to answer, and they were asked to provide an initial response anyways.
For example, further up in the bug tracker, they stated that there was an error in a script. If they were able to find an error, why can't they provide the script?
criddell 1 days ago [-]
That's what managers are for. They shield the people doing the work from all the external distractions.
bauruine 1 days ago [-]
Those are just more reasons to remove every cert they ever touched.
dumpsterdiver 1 days ago [-]
> I do not know why some community members think we wont be getting back on this bug.
This exposes how poorly they are prioritizing the response. They look like a deer caught in the headlights. Stop looking at the headlights and get moving.
FireBeyond 1 days ago [-]
One of the things that rubbed me the wrong way is that their initial responses were also filled with multiple appeals to authority. How long various people involved have been working for the company, and what they've been doing at the company, etc., with implications of "this wasn't incompetence or negligence". Who cares how long someone's been with the company?
ForHackernews 7 hours ago [-]
I read that as trying to reassure external parties that this wasn't a deliberately malicious attack on the trust root. These were known employees who really screwed up, not new joiners executing an attack.
OptOutDesk 1 days ago [-]
[flagged]
byteknight 1 days ago [-]
I may be too quick to pitchfork, but this seems FAR too implausible to be true:
This is so bad.
They're asking us to believe that a script flaw, on this specific day, for critical keys, silently generated a second, weaker key pair, and that the wrong pair was then used for the CSR because the labels looked similar. And "used by mistake because the labels were similar" means nobody checked the CSR's public key against the intended key before the root signed it. Worse, they had a direct signal months ago in Bug 1970728 their CRLs failed verification, and by their own account the fix was to re-sign with the flawed keys rather than ask why the "correct" keys didn’t match the certificates.
--
> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)
FireBeyond 1 days ago [-]
And not only that, this also wasn't caught by their auditors who attested to the KGC and details.
> "There are a lot of questions here. As most of you can imagine, we have been subject to intense social media scrutiny and analysis by investors/customers over the last few days and it has made doing any meaningful analysis of the event pretty much impossible."
Mozilla Root Program's update, stating that eMudhra has not responded to investigatory questions within deadlines (and had provided an extension): https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c42
> "The Mozilla Root Program sent the attached questions to eMudhra on the 20th October at 20:31 UTC. eMudhra was asked to provide an initial response within 24 hours. After the deadline passed on October 21st, eMudhra asked for a further 24 hours to respond. As of the time of writing on Friday 23rd, Mozilla has not received a response to these questions."
How does that work exactly? Perhaps if they paid attention to their software instead of browsing X, they'd be able to get more done?
Everything this company says is ridiculous.
They seem equally bad at both.
I may get downvotes, but this just screams amateur hour.
Do some countries possibly value quality or responsibility as a desirable property more than others at a cultural level? (Is "German Engineering" a real thing?)
Why do 419-style scams seem to emanate from a few specific countries? Why is piracy on the high seas more prevalent in Somalia than the United Kingdom?
It is, but those generalisations are usually debatable and will then lead to nationalistic debates that are allmost always flame war style, with people shouting over each other who is from the worst nation. Which is why many flag anything, that smells like leading towards this.
So we should just close our eyes and not say accurate information, got it
My comment was somewhat flip, but I wasn't surprised when I saw what and where.
americans die like men -- by guns
except for the small children, they die like children being slaughtered in a school shooting. price of freedom amiright?
I mean if I'm going to make really large generalizations about Software Engineering talent by country, Swiss people are the absolute bottom of the barrel maybe followed by the cheap outsourcing countries next, but ymmv.
I'm also sure i've worked with a lot more very good Indian engineers than German engineers here in America, but that might have more to do with who immigrates here.
Just by fact of population, a generalization may or may not hold more weight. Anecdotes are also tricky to really apply broadly: it's simply impossible for you to have encountered that many Swiss software engineers because few exist... few Swiss people exist. In 2019, there were over 300,000 Indian H-1B petitions to Switzerland's total of 202.
> I'm also sure i've worked with a lot more very good Indian engineers than German engineers here in America, but that might have more to do with who immigrates here.
Relative quality of life in Germany is objectively better in India by pretty much every metric, so there's an automatic/inherent bias - it's probable that more Indians wish to immigrate to the United States. (More than 10% of the Indian population doesn't have access to modern toilet facilities, for instance.) The competition for U.S. immigrant visas is far higher amongst Indians simply by the numbers, so there may also be a filtering function at play.
Perhaps there would be responses that resemble the one here
I've always thought the "commercial CA business" was BS; it attracts these type of people
It relies on fabricated "authority" just like "ICANN". The later is rife with unethical practices and corruption. Why would CAs be any different
"pretty much impossible"
Translation: "We can operate a CA if we only have to communicate with so-called "tech" companies and our "business" remains mysterious to the public. We did that for many years, we are experienced. But we cannot operate a CA under public scrutiny from investors and customers"
This incident will not be enough to get rid of these phony "authorities" but maybe it's one small step closer
And the discussion mentioning wrap/unwrap testing kind of feeds directly into the fear that either:
1. These weak keys were created outside the HSM and then imported.
2. These weak keys were meant to be the wrap keys somehow. Like, if they hadn't paid for some of the wrap/unwrap features on one of their HSMs and so were using easy-for-humans keys to handle the wrap/unwrap operations so they could more easily move keys between HSMs.
This should rarely happen if we are doing things correctly.
The moment the private key leaves the secure context of the HSM you should consider it to be compromised. Some HSM vendors have a way to effectively extend the secure context across multiple physical machines to address the edge cases where you'd be inclined to do this.
Also depending on the FIPS configuration of their HSM, some functionality like wrap/unwrap might not be available
The problem isn't foreign companies; it's bad CAs.
All cross-border business is fundamentally import/export and comes with the same risk - that when I am wronged by someone in another country, I have no legal recourse. The exceptions to this rule (like airlines) are such enormous exceptions that there are literally international treaties (e.g. Montreal Convention) to enable it.
Nothing fundamentally US-centric about that point of view.
What you cannot reasonably expect is for the entire Internet to operate on a "these countries good, these countries bad" model, for many reasons, including that the citizens of your preferred countries are wildly outnumbered.
Anyone who is not in the US saying, on here, "I do not need CAs that are not my national ones" would realize the problem before even posting.
Trying to equivocate between Let's Encrypt and eMudhra is crazy, as is trying to argue that they should both be trusted globally by default by everybody. I trust LE because of their track record. I had never heard of eMudhra before this.
You say this, in 2026, when the EU is trying to move to an EU Sovereign Cloud Framework: https://commission.europa.eu/news-and-media/news/sovereign-c...
I have a deep problem with your usage of the word "need". EU citizens can make a free choice (or not) to trust US-based certificate authorities, and US citizens can make a free choice (or not) to trust EU-based certificate authorities. There are certainly many reasons why that should be the case. But in a world where we need to negotiate whether food safety standards in X economic block are sufficient for Y's citizens and vice versa, and this is literally the stuff of trade deals, the idea that US-based certificate authorities should be universally trusted because of little more than historical legacy is not the policy position of people who believe in free markets and free choices.
> only trust other US citizens with my security
I'm not sure what you mean, but this doesn't seem to be a coherent concept.
Famous examples include: Samsonite vs. Indian Competitors like VIP - Samsonite won the case.
Novartis vs. Indian Generic Makers. This went all the way to the Indian supreme court before Novartis lost the case. (And thank heavens for that).
Microsoft and Dell: They have won nearly all their cases in India involving unauthorized software and misuse of trademarks.
These are just the big names. There are dozens of smaller US companies.
Can't use a computer then.
man fox news has really done a number on people's brains.....
They have that right though? There is no monopoly to single CAs for a given domain.
Check InCommon Certificate Service.
Trusting a CA that uses 4 as the private key is risky. If it's a small CA that issued certs that you don't need to consume, it would be an unnecessary risk to trust it.
So you'd just end up with the big players (ex. Sectigo, who cross-signed these certs btw), who would then claim too big to fail should anything actually go wrong with them (see: Comodo).
If you want only to connect to the US net, stop connecting to the internet and use a US browser instead of a web browser.
Or worse, you were billed incorrectly by some US-based service, their support chat loads Indian assets, and they threaten to hit your credit score and send debt collectors after you if you don't pay up.
No need to use international services like the internet.
eMudhra intermediate CA compromise
https://news.ycombinator.com/item?id=50166579
1271 points | galadran | 3 days ago | 504 comments
--
Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds
https://news.ycombinator.com/item?id=50177916
935 points | hkc | 2 days ago | 460 comments
--
eMudhra to exit global SSL/TLS certificate business
https://news.ycombinator.com/item?id=50195110
1111 points | bo0tzz | 1 day ago | 713 comments
Why aren't SSL certificates tied to something like DNSSEC where a CA is responsible only for one tld? Since domain ownership is enough to get a certificate signed anyway, that wouldn't change security properties.
(2) Because DNSSEC is a nightmare to deploy. Leaving the IANIX-type operational stuff aside, it doesn't work on all network paths; plenty of middleboxes freak out when they see DNS packets with DNSSEC records in them. People see this and want a best-effort system that uses DNSSEC in the majority cases where it does work, but now you have a downgradable, strippable security protocol.
(3) eMudhra is being murdered by the root programs as we speak, and when they're gone all we'll remember them for is as a WebPKI incident. The WebPKI itself will continue to function and nobody will care. That can't happen with .COM or .CO.UK or .IO; there's no accountability for those registry operators at all, let alone transparency logs to track what they're doing.
Who? Companies whose audiences are certain kinds of corporations?
As such, DANE would not add more issue here;
DNSSEC is easy to deploy (of course, you need to know your work, which is rarely the case). Legacy-first mindset is not really productive;
That's is correct, but on the other hand, incorrect: as a .com registry, I can:
People would not even notice because, if a given website is commonly signed by LE, I can trick LE into signing a new certificate for me. The transparency logs would just show that LE signed a certificate, as usual. Again: registries have already all the power in the world (it is a good thing: not the current issue :) )This is more than twenty-five years after DNSSEC standardization commenced in earnest (counting from the bis protocol).
"Sectigo has completed the revocation of the affected cross-signed CA Certificates. A new CRL and OCSP responses have been published and are making their way to our CDN edge nodes right now.
We will continue monitoring and evaluating this bug for further updates from Emudhra."
Can't wait to see all the exploits this will enable.
[0] https://bsky.app/profile/eriskii.net/post/3myh3a3jdn22q
You know very well that's never going to happen.
And guess what: this already exists. DO-178C for aviation, ISO 26262 for automotive, IEC 62304 for medical, etc. But these standards are specialized to the industry because, no surprise, there is no appropriate one-size-fits-all software development approach. You can’t just say “software must be bug-free” and leave it at that. That’s been shown to be impossible. Even formally verified software can have spec bugs.
There’s already formal standards that CAs must meet, and they get audited too. Maybe those standards aren’t rigorous enough. But even with those standards, what authority is going to punish a CA who breaks them, other than browser/OS vendors? If it damages the host nation, you might see something. But if the host nation is complicit, and used the rogue CA for intel purposes—or if the company has political connections—then I don’t think you’re getting the punishment you want.
It makes sense to have these heavy standards when you’re talking about software whose failure could immediately cost actual lives. I’m hesitant to expand it elsewhere as it could threaten the viability of open source and software freedom in general. Nobody should ever need a license to run a compiler or deploy an application to their own server.
Also, a purely software-focused standard doesn’t make sense for CAs, as they also operate and maintain infrastructure. The failure with this CA was a process failure.
I probably emphasized software too much in my reply, but that’s usually where I see people call for standards to be forced on the “tech” industry.
My opinion is that all the process in the world won't help if the people doing it don't care about the outcomes.
Right now, it would appear that the strongest external consequences for fucking up a CA are strongly-worded posts in a web forum. I would hesitate to call the wind-down of a business unit a "consequence" in this context because it's the default outcome. Everyone was already playing that game. We need an additional layer of pressure to encourage good behavior.
It would seem that we have different definitions of "wind-down". Everyone in the business operated with this consequence already on the table and it did not seem to provide the desired outcome.
Like it or not, CA infrastructure is global, and there’s no single set of laws that can or should govern it.
No, the strongest consequence is their stock value. It has gone down a lot and if they continue not following their agreed obligations as shown I hope that trend will continue.
Sure browsers are safe, but countless other internet softwares aren't (curl, python, pip just to name a few I know off the top of my head). Pip has a big problem because even if the OS distrusts the eMudhra certs, it still does because it has its own trust store. This will be an attack vector (that now everyone knows about) for a while.
[0] https://www.felonybench.com/
[1] https://www.amnesty.org/en/latest/news/2022/09/myanmar-faceb...
https://news.ycombinator.com/item?id=50160881
https://www.youtube.com/watch?v=zMG6aRZhIGE
If eMudhra just don't comply with Mozilla/Chrome, is there anything they can actually do?
So tooling bears some of the blame, unless I’m misunderstanding the mistake that was made.
Doesn’t seem they’ll give a timely technical response, which keeps all of us guessing about.
> There are a lot of questions here. As most of you can imagine, we have been subject to intense social media scrutiny and analysis by investors/customers over the last few days and it has made doing any meaningful analysis of the event pretty much impossible. The event happened on April 30 2025 and was done by set of people who have worked in the PKI industry for several years and worked with us for a while. There is a lot of digging we need to do to understand the circumstances, why, what, how to share a meaningful report that can hopefully benefit the community. I do not know why some community members think we wont be getting back on this bug. When we signed up for the browser program trust, we also signed up for disclosure obligations to the community which we will uphold.
https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c39
--------------------
ETA- HN rate-limiting won't let me reply, but to the commenter asking how all the pressure could delay a sober analysis:
I've never been involved with a mess like this, but I can well imagine it's difficult to convene the necessary people for a dispassionate RCA when your bosses are furious and investors are screaming for scapegoats to fire.
--------------------
Edit2: Still ratelimited for the downvotes, but in response to @andor:
You're right, some of the questions are fairly straightforward and should be easy to answer -- I would argue they already did provide an initial response here: https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c11
But there's 12 separate questions in this comment alone https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c30 many of which are demanding detailed information and statistics that would need to be reconstructed from audit logs.
I imagine lawyers are involved because whatever answers they give will inevitably wind up as evidence in ensuing lawsuits. I'm sure anything publicly released needs to be cleared through several layers of review.
In any sane business, the people handling the screaming investors are different from the people handling the social media mess, and they are also different from the people doing the RCA.
That said, given the mess, I wouldn't be surprised if eMudhra only had one employee responsible for all those jobs...
Something that could have been fully analyzed with those people in a couple weeks ending up taking two years with an independent outside auditor.
The independent auditor would have been needed either way, but if we gave the original crew the opportunity to respond, THEN fired them, the outside auditor could have just verified the story much more quickly.
The trust was gone and it's good that they closed up shop, don't get me wrong. But it's not like anyone who knows IT could create certificates in their name, in this case the impact is really a lot bigger. Especially to the world as a whole, the DigiNotar issue mainly affected users in Iran.
That just sounds like a management failure. You don’t actually have to cave.
The only reason there would be to fire people immediately would be if it would have been done deliberately with malicious intent but in this case there is no evidence of that.
Scape goats work better when they can't defend themselves.
Such a business will not be long operating a CA, then: these sorts of requests are common and CA operations are done openly in public with all accidents openly admitted in plain view. If management can’t deliver an environment that prioritizes compliance over rage and blame, then termination of CA acceptance for the company they manage with such irrationality is a successful outcome for the process. I respect that there may be opposing corporate and/or cultural pressures, and I imagine that’s why they’re voluntarily exiting the CA business, but they’re at risk of having all issuances distrusted retroactively if they think that gives them the leeway to reply late.
The comment you link to is from 19th. How could that provide an initial response to a mail send to them on the 20th? That does not make sense unless they have a time machine.
> But there's 12 separate questions in this comment alone https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c30 many of which are demanding detailed information and statistics that would need to be reconstructed from audit logs.
But that is not the set of questions that were ignored for the deadline.
> I imagine lawyers are involved because whatever answers they give will inevitably wind up as evidence in ensuing lawsuits. I'm sure anything publicly released needs to be cleared through several layers of review.
If they want to lawyer up, they should better be removed from trust stores where trust is more important than admitting the absolute minimal amount possible to lawyer up for lawsuits.
Or they could try to regain some trust, but it looks like they don't want that as a closer look might find more issues, like the other post about their broken login process.
Why? They've already announced they're quitting the CA business. I assume their priority is avoiding getting sued into bankruptcy. Playing along with Google/Mozilla's star chamber can only harm them.
I'm lowkey impressed they've even said "we also signed up for disclosure obligations to the community which we will uphold" because it's all downside risk with zero benefit for their company.
For example, further up in the bug tracker, they stated that there was an error in a script. If they were able to find an error, why can't they provide the script?
This exposes how poorly they are prioritizing the response. They look like a deer caught in the headlights. Stop looking at the headlights and get moving.
This is so bad.
They're asking us to believe that a script flaw, on this specific day, for critical keys, silently generated a second, weaker key pair, and that the wrong pair was then used for the CSR because the labels looked similar. And "used by mistake because the labels were similar" means nobody checked the CSR's public key against the intended key before the root signed it. Worse, they had a direct signal months ago in Bug 1970728 their CRLs failed verification, and by their own account the fix was to re-sign with the flawed keys rather than ask why the "correct" keys didn’t match the certificates.
--
> We have done some preliminary assessment, there appears to be a flaw in the KGC script that was used on that specific day to create these ECC issuers. As for these ECC keys, our review of the KGC, assessment of the code and view of HSM keys so far indicate that a) a set of original ECC keys were created onboard as expected b) an approved deviation to our usual KGC to test a wrap/unwrap was introduced as these keys were meant for export for this instance c) a flaw in the code resulted in the second pair of keys being created onboard which were weaker d) these second pair of weaker keys were used by mistake to create the csr (because the key labels were similar) and signed by our root e) the CSR from these weaker keys were also sent over to Sectigo for cross signing (we verified the files sent over to Sectigo)