How I got over 300 Tor exit nodes removed
kaba.cx245 points by kabakabakaba 7 hours ago
245 points by kabakabakaba 7 hours ago
I built exitmap over ten years ago. Glad to see it's still of use :)
The attack on SSH seems rather poorly conceived. Not only was it detected immediately thanks to SSH printing that huge banner on MITM attacks, but it allowed the admins to easily trace every infected node and respond quickly to shut the whole operation down. Had they stuck to email they would still be running today.
Thank you for writing this.
> But if you ignore the warning and go ahead with the login, the attacker does three things: > 1. Steals your password
This shouldn't affect key-based authentication, though
Also, this from Marc-Etienne M.Léveillé email linked in your post:
> Another possibility is a compromised hosting provider. We identified a couple of them in the past where their management servers were compromised by Ebury and their customers' VPSes, containers, etc., ended up running Ebury as well.
Provider-side compromise of customer VPS instances has been a growing trend in recent years. I stopped hosting anything privacy-related on VPS in favor of dedicated hardware. Many people still don't understand how easy it is for a provider to silently dump and/or manipulate both RAM and disk space in real-time.
> Many people still don't understand how easy it is for a provider to silently dump and/or manipulate both RAM and disk space in real-time.
That depends on the VPS technology stack being used. IIRC, at least Amazon uses hardware features like IOMMU that prevent any kind of access to the guest's memory from the host side.
AFAIK, there has never been a memory encryption scheme that successfully protects the VM client from the hypervisor, they've all been found to be defective within a few years on the market.
It is probably an useful layer because some of the attacks are not feasible for a remote attacker who has compromised the host but require actual physical access. But the marketing spiel of protecting your data from Amazon itself is just lies, physical access is more powerful than root access, no exceptions.
> But the marketing spiel of protecting your data from Amazon itself is just lies, physical access is more powerful than root access, no exceptions.
I'd trust Amazon and Google more than most others when it comes to stuff like formally verifying hardware security solutions and also doing active research against threats posed by malicious internal staff.
Is there a way from within the VM, though, to determine if that hardware feature exists and is enabled? If you don't trust your VM provider, how do you trust them when they tell you they have that feature and it is enabled?
If you believe Amazon and trust there is no way that this IOMMU protection can be disabled or bypassed, I guess this is mitigation against some attacks. Seems like mostly just trusting the AWS brand name, which would be more trustworthy than fly-by-night providers even without IOMMU.
3000 is insanely few exit nodes.
In many countries, running an exit node is a very fast way to have cop knocking at your door and seizing all your stuff. To run a tor exit node, you either have to try your best to stay anonymous, preferably be based in a country who don't care at all about cyber crime, and run them on a dodgy hosting provider who dgaf. Or you have to create a legal entity, either a non-profit or a company, depending on the country register as a hosting provider (so that you are not consider legally responsible for what comes out of your server if you abide by some rules), and you will still have to respond pretty fast to any police inquiry.
If anything, it's a miracle that there is 3000 of them. Although you can safely assume that a good portion of them (if not the majority) are operated by state intelligence services, cybersecurity companies and bad actor, who will log everything that pass through their node (and might be able to decrypt them one day).
agree I think tor is such a cool technology but even the thought of what is getting routed through there on my machine or even a machine I rent is enough for me to stay away from it
You don't route anything by default, you have to explicitly enable it if you wish to help the network.
And relay operators (not exit nodes) don't face any issues. Don't let the unreasonable fear blind you.
At least here in Germany I believe you’re on very shaky legal ground even if you’re just running a relay.
US government literally has a gigantic data centre purely for the purpose of storing encrypted data traffic that might be decoded one day. We can only assume that anything going through any US controlled TOR node ends up there.
It's insanely few, but also isn't that possibly an advantage? When lots of people's transfers are intermingled that means the possibility of separating out one person's traffic from the rest is harder.
Those user's traffic is cleanly separable at the destination, so this attack is always possible from some vantage point and isn't a defensible position. So malicious exit nodes are the real threat. Your reasoning would apply better to guard nodes.
Well yeah, because running an exit node in the US is "probably legal but there's precedent for still getting arrested and having your stuff seized." I can't imagine very many normal people are willing to paint a target on their backs like this.
plus becoming the technical target of every spy agency and tons of random hackers! likely the most trouble you can bring to your computers legally
So one person single-handedly knocked out 10% of the entire Tor exit network over an SSH port redirection?
10% were compromised. The author contacted the Tor Project's network health team, who "made an announcement to the operators, who began shutting their exit nodes down."
> The biggest question is still how they got into all those servers in the first place.
The simplest way is to just launch the servers yourself, using a cloud provider. Tor is completely OK with exit node operators being anonymous, and with exit nodes running in cheap cloud data centres.
Tor guard nodes seemed to be under attack but the devs aren't really engaging with it publicly, at least. It happened to mine and I had to switch it to a middle relay. No problems now.
https://forum.torproject.org/t/tor-relays-shutting-down-my-4...
People say that even running a middle or guard node can get your IP blacklisted. I don't have personal experience of this because I don't use my home IP for regular use, just for selfhosting. All the other devices run on a network wide commercial VPN so if it did happen it wouldn't be an issue.
I'm having trouble differentiating the different "attacks" here. Based on your thread it sounds like you're getting a DDoS attack against your nodes. The article appears to show some level of sophisticated OS backdooring (possibly from the ISP being compromised). Are you also seeing attempts to backdoor your nodes?
Author here :)
TLDR: on Oct 2, 333 exits (~10% of Tor exits) started redirecting port 22 to 12 interception servers running a paramiko-based MITM that stole passwords and tried to drop Ebury on the target.
Thankyou for taking the time to detect and validate/verify those hosts. Is there a way you could make an outbound connection from them (maybe curl) to a site like ifconfig.me [1] and start logging which providers they are on then post a report of them on HN?
[1] - https://ifconfig.me/
In my scan on October 2, 26 of the 27 compromised servers outside the Quetzalcoatl relay family (most hosting multiple exits) were at FranTech/BuyVM.
It's been discussed on the mailing list: https://lists.torproject.org/mailman3/hyperkitty/list/tor-re...
That doesnt prove FranTech itself was compromised, but its a strong cluster. AFAIK nobody has contacted them yet, if no one does, I plan to give them a heads up in the coming days.
That is useful information. Historically I have blocked BuyVM on my nodes due to their pro-privacy pro-tor-exit policy. Now I just put anyone arriving from them into a moderator review queue for my silly hobby sites.
It's a good policy to permit Tor exit nodes but it means their network will egress some of the most untoward behavior so that is expected and I would not default to assuming they are compromised, rather have some of the highest concentration of Tor exit nodes. There are pros and cons to such providers and you found one of them.
What I don't get: Why would you want to access hosts via ssh over TOR?
The Ebury botnet that's been around for a decade now is likely spying on Tor exit ssh tunnels looking for other criminals logging into their h4xxr jump boxes, ransomware deployments they can intercept and exploit themselves, financial crime market operators, crypto drug dealers etc.
- Default route is over tor and you didn't make an exception for ssh.
- Hide where you are in the world from your employer, who owns the server. (On vacation?)
- You're hacking the server.
Tor sounds like just about the worst possible way to hide what you're doing from your employer. Why would you not proxy through... your house?
I did exactly that as an intern. The boss blocked traffic for everything except tcp 80 and 443 in an attempt to block IM. I set up a tunnel to my house on one of those ports and arp cache poisoned my coworkers so I was their default gw. Only the boss still had filtered Internet and didn't understand how.
Hehe, that's funny. There are not that many opportunities to use your knowledge about the plumbing of the internet to good effect but that definitely was one of them.
And 4) you're the owner of the server, which is paid in bitcoin and hosted at a "bulletproof provider", and you want to leave no trace back to your "real" IP address should the authorities ever manage to seize it. Often the case for piracy, drug marketplaces and CSAM.
> Why would you want to access hosts via ssh over TOR?
When you're behind a restrictive firewall which blocks anything other than HTTP/HTTPS (like at an Internet café), or when there's some upstream routing issue making reaching the server directly impossible from your location (these can be a pain to diagnose).
Or perhaps it's simply your default: you proxy everything through Tor, and need a reason to not go through it.
Remote admin some shady server you control and do not want LE seeing a direct inbound ssh from you in Upper Sandusky, Ohio.
How does this use case differ?
People ssh over Tor for the same reason they use Tor for any network activity: To make confidential 1) your identity to the remote host and 2) your activities to your network providers (including your ISP/workplace/etc.).
If you want only confidential content, ssh is sufficient and so is https for web browsing. If you want the other things, you must use Tor or maybe something similar.
I can't really see legal use cases for (1) and (2) might get you fired if the second. 2a is a 'maybe'.
If the SSH host is a backdoor you installed in a company you hacked, you don't want the FBI tracing it back to you.
Once again, this shows the strength of mutual authentication (ssh user key authentication, or HTTPS client certificates): even if the client accepts the forged server key from a MITM server, the real server won't accept the client key exchange (since it, at least in the SSH protocol, is bound to the connection). For the three things the attacker did here:
1. It can't steal your password, since the private key never leaves your client;
2. It can observe each command you type, but it can't open a connection to the real server to know how it would respond to your commands;
3. It can't open any connection to the real server (it will fail authentication), so it can't do anything to the real server.
If you really like passwords, there are ways to configure sshd to require both a client key and a password, in that order.
You could also do mutual authentication of SSH, with a local password-encrypted private keyfile, if you really like passwords. That gives you all the benefits of the "something you know" factor, without exposing the password to the server. And you can inspect how the decryption and use of the key happens (since it is on your computer), and therefore see it doesn't leave your computer.
Or use fido/gpg in a yubikey - the user gets to authenticate the use of the secret, but it never leaves the hardware token, let alone the client computer.
> The use of Ebury and the broad collection of unencrypted data point to a financially motivated crime group.
Collection of unencrypted data points to other players as well that have been known to have great interest in Tor exit nodes etc., but they're careful and not usually this noisy, so probably newgen trolls.
> The lesson is the same as always: extra security is a worthwhile investment. Disable password authentication in favor of ssh keys. I for one am going to buy myself some hardware keys today. :)
Question: Where to get these hardware keys? Which are good options and why?
You don't need to go out and buy hardware keys today. You should be able to use any FIDO2 key these days for SSH, as long as your server runs a recent enough OpenSSH server, which supports ecdsa-sk or ed25519-sk public keys.
Reputable vendors of those hardware keys included Yubikey and Token2, among others. You are really just using them as FIDO2 tokens, so go for what's reputable there.
If you are just getting started, you can generate and use software-backed keys (using ssh-keygen) and copy the public keys to servers' authorized_keys file using ssh-copy-id (or manually adding it). You should probably start learning this way (with plenty of online guides, and fewer risks of an old server implementation not supporting FIDO2 credentials), and then move to hardware keys once you understand the mechanics of copying around public keys, etc. Your password manager might also support acting as an SSH agent for key based logins - I believe Bitwarden and some others now do this, for example.
If you have an iPhone, the ssh client app Terminus can use the Secure Enclave of the iPhone to store your private key, which prevents all apps on the phone, even Terminus itself, from being able to read the private key. The private key is stored in hardware. You use FaceID to approve the login operation that ssh is performing.
I switched over to using that in Terminus recently instead of having Terminus generate and manage the private ssh keys.
Likewise, many Mac and MacBook machines have Secure Enclave too that can store private keys in hardware and works with the ssh client that comes with macOS. But I have not personally started using that yet. Will soon though. You then use TouchID to approve each login operation that ssh is performing.
Yubikey is cool too. I’ve used it mainly for other things than ssh.
Whatever solution you go with, have a backup or another way to restore access when your hardware key eventually breaks and cannot be used.
For example, instead of buying one Yubikey buy three and give each of them access to the same places. Keep one on you, one at home, and one somewhere else. Rotate between using each of them every couple of weeks so you are sure they all still work and that all of them have the accesses to all the same places.
In the case of ssh, you probably either have physical access to the target machine or you have VNC management from the hosting provider, etc, that you can fall back to if you misplace or damage your hardware key. So if you only use the hardware key for ssh and you have other means like those to get back in then having just a single hardware key is mostly fine.
A Yubikey, though simply generating ssh keys instead of using passwords would stop this MITM attack. So would changing the rounds and port of ssh on your system and the remote box making yourself an invisible protocol nothing would recognize.
Yubikey benefits is it stores the keys in the hardware if your laptop is compromised somehow or stolen they at least don't get all the ssh keys too
SSH keys are great, and you should absolutely use them
But you should also just listen to the big warning that says "IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!"
True they can also get information from the public ssh key like if you used a real email address or name to start spear phishing or who knows what else like extortion threats to your company
Any legit use case for using SSH over Tor?