North Korean fake job interview led to $6.3B Binance heist
theregister.com655 points by smalltech 3 days ago
655 points by smalltech 3 days ago
A lot of people are focusing on how irresponsible the engineer was about using the work computer. That's obviously an issue but the bigger failure mode to me is the Engineer C's container deployment token.
That's a huge red-flag. In a company of this size, the deployments should be via CI only. CI isn't just to increase the speed and reliability of deployments but also to provide a clean room to build the software! Even if it's a token that was copied and pasted temporarily, the systems in place should not have to require this type of 1. copy 2. paste 3. remember-to-delete the token from my PC.
Also, Binance should be paying for intrusion testing. Better to pay a trusted company to do this kind of thing instead of letting North Korea show you how it's done. I don't think this exact exploit path would have been found but I bet the intrusion testing would have figured out another way to use Engineer C's token.
Nobody in the entire company should have access to move $billions in crypto, much less a random engineer, even if they're part of the deploy process.
They had controls in place to prevent large transfers, so the attackers structured it as a lot of small transfers very quickly
Well, then they need another control that prevents “small transfers very quickly” happening, I guess.
Aren’t “small transfers very quickly” kind of the whole point of what Binance does?
Not just that! The EIP-7702 delegation to make the wallet into a smart contract bypassed all of the controls that they had in place on the signer. It's wild that they didn't have a system in place to stop it from signing that transaction type.
From my rough understanding, this seems like the main exploit. It seems like the machine with access to the wallet key was presented with a crazy EIP-7702 transaction and signed it (which effectively owned its wallet in a single transaction)[1]. Without knowing their security model, it seems like biggest trust boundary should be between that signing machine and everything else.
[1] The fact that the needed to sign 50 of these transactions to solve a race condition seems like an implementation detail
They had to sign 50 of them because they had to compete timing the transaction with the normal transactions. Transactions have to be in nonce order for the wallet, and if two transactions with the same nonce are submitted to a block, it is up to the miner / validator to determine which one to put in. So if you spray 50, you can almost guarantee that the transaction you want will get in.
Is quorum signing not a thing in crypto?
In Ethereum it would be implemented by linking your account to smart contract code, then throwing away your private key so you can only make transactions through the smart contract. The approval of linking to smart contract code is exactly what they signed here - only it wasn't a multisig smart contract, it was something malicious.
A disgruntled employee wouldn't do the exact same thing?
That's the plot of Office Space!
Also Superman III, which predates it and presumably was well known to the writers. But I have an inkling the scheme and it's infamy go back much further. (Precious metal coin clipping/debasement?) It's one of those ethical questions, like a pauper stealing bread, or thought experiments, like "what if reality is just a dream" that many if not most people encounter during their formative years. Per usual, though, the chasm between theory and application (even just identifying real world potential when it matters) is the toughest one to cross.
If they did and didn't account for a simple scenario like this happening and built in roadblocks and friction to prevent it from happening? Then it would stand to reason their entire EIS team should be fired.
The fact nobody thought of this is pretty staggering.
> They had controls in place to prevent large transfers, so the attackers structured it as a lot of small transfers very quickly
lol that's the worst control I've ever heard.
OK, so nobody else has seen Superman III?
It is such a weird combination of "I enjoyed this movie" and "this really isn't a good superman movie"
I think they did that in Superman 3
most employees shouldn't be able to touch the funds at all except a handful and probably require supervisor verification (dual factor authentication).
I mean sure, but as long as you can deploy directly any permissions models are meaningless for an attacker? They can just be removed.
> Even if it's a token that was copied and pasted temporarily, the systems in place should not have to require this type of 1. copy 2. paste 3. remember-to-delete the token from my PC.
One of the first things I did in my last job was shift processes like this off my team's computers, aside from the security ramifications all it takes is one dumb mistake, one up-arrow too many, one didn't-read-the-whole-line, and now you're doing something to production by accident!
I also did that, for a team deploying smart contract on chain from local.
It took a couple months to get the pipeline out of their laptop.
Worth it, yes, as we found interesting things in there which explained why they were so reluctant to deploy upgrades.
> as we found interesting things in there which explained why they were so reluctant to deploy upgrades
.. backdoors which enabled the dev to deploy exploits of their own which wouldn't be visible through CI?
No, rather a particular environment with monkey patched libraries, alias for various versions of compliers, and dozens of env variables that happen to work or not work well with the desired config.
What sorts of processes related to secret storage were you able to shift off of team computers? From my perspective, if I need to get e.g. an API key from Twitter and put it into my prod environment, i have to be doing that with my computer somehow. There's still millions of different types of secrets that need meat puppets to shuffle them around.
> There's still millions of different types of secrets that need meat puppets to shuffle them around.
Sure, but those meat puppets shouldn't be developers, they should be administrators, using proper procedures.
Separation of powers seems so fundamental yet also so totally ignored these days.
TBH CI does not prevent this kind of mistakes. I've seen people triggering the wrong CI pipeline.
I guess people triggering the pipeline is already part of the problem
No matter what you do, at some point you have some human performing some action which will, after N indirection layers, result in a change in prod, unless you have the mythical fully automated agentic software factory.
The article says that a software engineer interviewed with the North Koreans and the NKs then gained access to engineer A's machine (which had the deployment token) and then finally passed that token to run on engineer C's machine (who apparently did deployments). They knew of the team makeup from their first (caught) attempt.
That unlettered (npi) engineer who downloaded and ran the interview software on his or her machine is what the human weak link here. Let's call them Engineer F. Seems like this person was specifically selected from the dev team.
I've had this happen a few times getting interview requests from Indian recruiting firms. It sounded familiar. You go through a screening, then get passed onto the "code challenge" part. You have to download a GitHub file and then work on it, check it back in and push it so they can examine what you did.
Twice I scanned the file after downloading it to a sandbox and twice it found malware that would've been installed.
While this is by far the worst case of this happening, I've read at least a half dozen other accounts of recruiting companies attempting to install malware through the same process. I'm not saying the person should have known, but it's scary this is becoming more and more common.
> I'm not saying the person should have known
This person definitely should have known. You don't download and run arbitrary binaries outside of a VM. At this point you don't even compile code from an untrusted source. This is the power user equivalent of falling for a phishing email.
One of my favorite stories from a security talk was the speaker recounting the surrealism of (while doing a forensic investigation of a security incident) watching the attacker read your internal documentation to try to figure out how the hell anything works mid-attack, and running into all the same pitfalls as your new employees.
That suggests a creative new justification for not documenting your code :p
No such thing as deleting secrets. You decommission them and replace them. Deleting them off disk is impractical to do, at least sufficiently.
I don't even leave deploy tokens on my laptop for personal side projects. It isn't difficult to setup CI deploys, if nothing else the extra upfront cost let's me be lazy and have a consistent deploy pipeline.
100% this. I can deploy to our sandbox environment directly, but only our CI/CD has the permissions to build/deploy to our master environment. And the stuff I work on is 1000x less impactful than $billions in crypto!
Maybe that second engineer who's laptop they were laser focused on was the CI/CD?
> Better to pay a trusted company to do this kind of thing instead of letting North Korea show you how it's done.
but enterprises usually prefer the latter, I guess it's more convenient since it bypasses the bureaucratic processes?
Yes. The situation improves considerably if, when making server changes or retrieving data, programmers are required to enter a code generated from a hardware TOTP token isolated from both PCs and smartphones.
Every service provider system should be designed with the assumption that programmers' PCs and smartphones may be compromised.
I have saw companies doing social engineering as part of their intrution testing, I imagine fake interviews are (or will become) part of these.
I think it will remain uncommon. I agree it would be a good test, but companies and employees alike get very angry when pentesting gets too close to sensitive subjects.
Anyone who fell for it would not only get in trouble for poor security practice, they would reveal that they were job searching. That opens an entire can of worms that creates problems beyond the pentest. This also angers other employees who see it as a double-trap being sprung on employees.
Another major problem is that responsible employees who accept the fake interview but then use their personal laptop to do the interview will have their personal laptop compromised by the pentesters. Responsible testers will immediately abort, but some will try to use the personal laptop to pivot deeper which is a huge ick. It’s also really bad if the company is compromising personal laptops of people who are doing the right thing (switching to personal gear for interviews).
The pentesters wouldn't have to actually pwn a laptop, just getting the employee to download and run anything could go in a report without having the file actually be malicious.
Would they have legal right to do that, out of curiosity?
Hired oentest has company permission to do stuff on company property. I do not imagine they have right to do so to personal property.
Absolutely not. There is literally no framework, which would give you the right to exploit an employee’s personal laptop, that’s just cyber crime.
Then that is something the pentesters cannot do, but malicious actors will do.
Game over.
What pentesters can do, should do, and often do is simply start the test from a workstation that is given to them - you assume a successful social engineering attack as the starting point, and test whether your process and technical controls can properly limit the impact of such a compromise without trying to replicate the actual social engineering step and the associated disruption to the targeted employees.
Taking over that first engineer's workstation shouldn't ever have resulted in $6.3B of damages. I would consider it reasonable if those privileges could be abused (by e.g. the same engineer intentionally) to cause $6.3M of damages and the rest would be mitigated or prevented; so for this incident we could say that 0.1% of the impact was caused by the initial compromise of that workstation and 99.9% of the losses were caused by all the other failures - so it's effective for any pentest to focus on the latter part, ignore the social engineering and phishing part (since you can't really eliminate it anyway - unlike many other flaws) and assume initial compromise as a starting point.
I think my security form said something along the lines of “using personal laptops to access company resources grants implicit permission for my company to audit my laptop.” It goes on to explain that if you don’t agree to it, simply use the company device only.
> responsible employees who accept the fake interview but then use their personal laptop to do the interview will have their personal laptop compromised by the pentesters.
If your laptop is successfully compromised by the pentesters then pretty much by definition you weren't being responsible. Though I agree that pentesters going after personal devices is ethically dubious.
Compromising employee personal devices is strictly out of scope for any security testing I've seen.
> Though I agree that pentesters going after personal devices is ethically dubious.
It's not just ethically dubious. It crosses a lot of legal boundaries. A company can consent to have their personal devices hacked. An employee doesn't automatically consent to let a company's contractors access their personal devices.
fake interviews might be excessive but lots of companies send tempting phishing emails as part of the secops, then both collect org-level stats and educate (or shame depending on your take) the employee for getting duped.
Companies don't try to set up job interviews to test their employees loyalty?
Everyone has their price. It is foolish to think that most people would not leave for better title/pay/job alignment/whatever.
Sometimes called "Full Spectrum Attack Simulation" where initial entry can include physical access or spear phishing. They can span months in elapsed time (not effort) so as to be more realistic. However there are still ethical and safety constraints and even phishing may not be able to use the (effective) technique of stressing the recipient too much or of course blackmail or coercing. So I think fake interviews may not be allowed in many cases, to protect the employees.
Is that even legal? Wouldn’t a fake job interview be, in a sense, tricking you into taking time off and working without pay?
I think it would be legal but unethical -- unless you also then gave the employee substantial compensation for their time and opportunity cost.
Legality would be very jurisdiction dependent. It might be illegal in most developed Western countries, depending on the exact facts of the case. I suspect it wouldn't be illegal in China — unless the trick happened to hit a ranking Communist Party member.
And you never, ever, ever install software as part of the interview. Decline and hang up if they're asking you to install stuff. Visiting websites - maybe but even that unless it's a well known website I wouldn't go there but download and install stuff, hells no
I fully agree with this.
If you ever meet with Alibaba for any reason they'll ask you to download their conference software. Unfortunately most people in the China ecosystem don't have the guts to say no and will just gleefully install any .exe thrown at them.
Hmm like building an node project, actually any project can have a dependency on a made up libraries that does the exploitation or even the project itself as made infamous by chrome extensions
"fork this repo and fix this bug" is a common job interview request, isn't it?
For many build chains, that involves implicitly installing software.
Such toolchains are dumpster fires whose use invites supply chain attacks. They should never have gained mainstream acceptance and should never be run outside of a sandbox.
That said I wouldn't trust a compiler to be hardened against malicious input and anyway you presumably need to run the output so all of that should be happening in a hardened sandbox regardless.
It'll be hard to time a job opening receiving interviews to line up with intrusion tests in a way that people would not be on guard against it in a way not normal.
Fake interviews would be a huge time sink and probably piss off your staff.
They don't have to respond. It gives employers an even stronger signal they should be kept right?
Using pentesting to also test employee loyalty is a great way to get the entire company to hate working there.
Which is why it's going to happen.
Only if your manager is an incompetent moron.
Which is also why we, collectively, as an industry needs to not put up with that. It's easy for me to say from the comfort of it not being my job on the line, that if this happens to anybody they should quit and then name and shame the company, but a competent security team is going to have the idea of doing a fake phishing interview and then not do it because of how morale destroying that would be. The security awareness team sending fake phishing emails are embarrassing enough.
What if I plant a hook on the engineer’s laptop that will slightly modify their commits in such a way that eventually I smuggle malicious code over? First, exfiltrate the code over to me, learn from previous commits, start poisoning. Stamped by the CI, no less. In the days of slop PRs thousands of lines each shouldn’t be even too hard.
The point is that at some point you do need to open the door and let the code in.
Wouldn’t most companies worth exploiting run some static analysis in the ci cd
The company should require both automated and human review of PRs. Mine does.
This is the standard at most professional companies but it's now becoming a problem due to AI and undisciplined people submitting 10,000 lines MRs.
"too long, LGTM, now buzz off"
Well, that's still a human review. If something breaks, you (as in: this reviewer) is at least partly responsible.
shouldn't we focus on a unregulated monetrary system whose sole existence appears to be to support the Epstein class against democracy?
Doing ANYTHING on a work issued laptop outside of your role is such a dumb idea for a litany of obvious reasons that I'm genuinely shocked people do it. Let alone participate in interviews for other companies on it.
In my last job we used Zoom for interviews. The link we sent would work in any web browser, but if the candidate was signed into a Zoom account on their computer we'd be able to see it. I'd say >70% of them were on their company account.
Even worse when you have to do stuff like screenshare or coding. I've had candidates share their screen and have obvious proprietary material in open windows. I've seen countless email inboxes and calendars. I've seen candidates do coding exercises using internal IDEs, dev environments and dev tools. I've had them talk – fully unprompted – about internal tech and unreleased projects. And I'm talking about Google, Meta, Amazon, Coinbase, Uber.
At this point I'm convinced that a large chunk of tech interviews are happening just to extract insider info from employees.
Anecdotally, I've also seen the same thing, and _done_ the same thing (on the zoom accounts). A previous companies policy allowed for zoom logins on non-corp devices so that you could take a late call from home. The only thing I _ever_ use zoom for is work or interviews, so it's not like I'm going to log out after?
If a company want to late-call me, they should provide the device (phone, laptop)
They did. They provided work phones and laptops to anyone who wanted/needed them, and had travel laptops for people who needed them occasionally. But if you have one call at 7pm in a year, being able to go home and cook, eat, and take the call without any real hassle is a nice thing.
as a candidate it's pretty tough scheduling interviews past 5pm or on the weekends
What’s preventing you from using a personal laptop ?
Being an engineer I am surprised, but for normal staff its very common to not have a personal laptop. The most common request from layoffs or firings in my company has been a request to keep their company laptop
I've worked at a few companies where they'd offer to sell, for a nominal amount, the work-laptops to ex-staff.
I know the last time I left my job I was given the opportunity to buy an M1 mac with 16Gb of RAM for €100, so of course I did.
(Though I also paid €500 to keep the phone they'd bought for me, which was perhaps not such a good deal, it was convienent and I do like the phone, but I'd have been happy to have a random cheap android phone, I have no specific requirements beyond "small enough to fit inside a sporran"!)
You would think that tech job will pay enough to buy a laptop.
pervasive assumption that the consumer choice is based on ability to make the consumer choice
they're not interested, they view it as a work tool and they always have one as they avoid gaps in their resume
no different than say, a work badge. you can buy an NFC enabled badge from ebay but you never will
You would think that those in tech jobs are interested in tech and have a full box of various computers.
I am interested in tech, and have multiple desktop computers. I have a single laptop with a good camera that I-d use for interviewing.
a lot of people have what they consider to be a healthier separation of their living spaces now
and you don't need all of that to do anything the professional world wants or requires
If you spend a lot of time in work working on laptop, you might just plain not need one at home. It is not necessary for entertainment, sport, cooking, whatever. You do not need it to learn new things. Plus, some people have desktops or console for gaming.
And if you are coding full time, it is better to not do coding in your free time if you want to maximize learning anyway.
Hi Bob, why are you using your personal laptop at your desk instead of the company's laptop?
If you are openly interviewing for other jobs from your desk I don't think which laptop you use is going to be anyone's concern.
I have never hidden the fact from any of my management in any of my companies that I interview with other companies on a semi-regular basis.
It reminds them that while I am a replaceable cog in their machine, I am also a cog that other machines would like to install given the right opportunity. And I am always aware of my current market value.
Sure, but that's different from straight up taking interviews during working hours.
Their current job, presumably
How does that prevent you from using your personal laptop for a job interview?
It would never occur to me to use an employer's hardware for such a purpose. It seems absurdly lazy, but I know that not everyone's life works like mine, so I would like to understand why this would be a real problem for people.
I had a really bad meeting one day that became the straw that broke the camel's back. When I went back to my desk, I saw an interesting job posting. I immediately fired up the company's copy of Word to draft up a resume, and used the company computer to send the email from my personal account. Less than a month later, I was working for the new company. It was done intentionally. I almost sent the resume via company email, but I really wanted to ensure I could access any response.
> It would never occur to me to use an employer's hardware for such a purpose
That's a very uncommon mindset and worldview.
For 99% of people, the default would be "I'm going to join this Zoom meeting from the laptop I use all day long to join Zoom meetings, and I'm going to code using the laptop I use all day long to code".
Lots of people have their work tools setup on their work computer, but not on their personal computer. Their personal laptop might be relatively underpowered and struggle to run all the required software efficiently during an interview. The issue becomes even more significant as more of the tools used require an account - you don't want your interview performance to suffer because you're using an unfamiliar tool or service, so you just use the ones you're used to using every day.
Common or uncommon, its extremely unprofessional. You know when you are dealing with a kid or with professional who actually wants to get hired and has some manners.
Sharing other's company secrets by mistake, using other company's resources in ways that are definitely forbidden - is this a person you really want to hire since they will behave the same in your company? Is this a person you want to give trust with some internals?
In market with 100s or applicants thats enough to disqualify anybody who actually needs an interview or two to get hired.
Maybe this is just an agree to disagree moment, but literally no one I know would consider this unprofessional at all.
I've been in management for the past decade and interviewed a few hundred candidates in that time. Never once did I ask or care who owned the laptop they were using.
And similarly, if someone on my team used their work laptop for an interview, I wouldn't care at all.
What I'm seeing here is that Binance has the IT culture of a SV start-up when it should have the IT culture of a bank. Sounds to me like a thug with a $10 wrench could've been just as effective.
> Lots of people have their work tools setup on their work computer, but not on their personal computer...you don't want your interview performance to suffer because you're using an unfamiliar tool or service...
I struggle to come up of even the most absurd interview scenario where the tool makes the difference especially in our field where if a FOSS tool is not standard, then there's a sufficiently powerful free tier (eg., Sublime, IntelliJ). Heck most remote interviews, they take you out of your tools completely and ask you to code on some shared browser-based editor.
> That's a very uncommon mindset and worldview.
I'm calling HN SV start-up bias. Outside of the startup software eng'g industry, a lot of people exercise very good separation between work tools and personal tools. I know a lawyer, _who's the head of his own law firm,_ bring his personal laptop to the office just so he could purchase tickets during office hours.
I think this is also down to a few factors:
- SWEs are typically issued high-end machines in contrast to other professions whose standard issue is still 4GB (or 8GB, if you're lucky) RAM. For those professions buying a better personal laptop is a low bar to clear.
- SWEs are more efficient when they have admin access to their own machines. Though it begs the question that maybe some inefficiency would've been an acceptable tradeoff to $6.3B.
- SWE is one of the professions on the poster for the "Get paid good while working from Bali" lifestyle, wherein BYOD is implied, especially if you are freelancing.
> I struggle to come up of even the most absurd interview scenario where the tool makes the difference
I had an interview recently where I'd initially planned to use my own machine, then gave up and just used my work computer.
I'm used to the whiteboarding/UML/flowchart software I use at work, so I either needed to pay for a personal subscription so I could use it in the interview or be forced to use some other software I'm not familiar with. I also didn't have a personal paid AI code tool, so again either pay for a personal one or just use the work Claude account. Also the webcam on my work laptop is substantially nicer than my personal laptop.
Also, critically, it doesn't matter. If IT or my manager had pinged me to ask "Did you do an interview at noon yesterday?" I would have said yes and then the only follow-up would have been them asking if it was a random thing or if I was actively looking.
I think IT (and I guess, lawyers) are probably the only people who care much about separating work and personal devices, just because they're prone to being aware of how much surveillance of the device can be happening. People in marketing/sales/whatever back office stuff will be logged into their personal Facebook, Netflix, iMessages, Email and bank and never consider that their keystrokes are being logged and screenshots are being taken.