We can write peer to peer applications in Rust using
can anonymize TCP streams by running them through the Tor network. How
about writing anonymous peer to peer applications in Rust that run
over the Tor network?
Recently I was tasked with investigating whether it was possible (or practical) to run libp2p traffic over the Tor network. Here at CoBloX our primary language is Rust and we utilise the rust-libp2p library for our network layer. We were interested in seeing if we could get our traffic to run over the Tor network. This post explains how we did it.
Upfront, there is very little that is new here. I just 'stood on the shoulders of giants' so to speak and wired together a few nice open source libraries. This took me a while to work out though, so in the name of saving the next guy some time and effort here goes ...
I tackled this challenge in three steps and I will explain the solution using these same three steps:
- Write a basic libp2p application to use as a proof of concept.
- Write an application in Rust that can connect TCP streams over Tor.
- Wire what we learn in (2) into the application we wrote in (1).
Basic networked application in Rust using libp2p
In order to quickly hack up a basic application in Rust that uses
rust-libp2p for the networking layer I had to look no further than
directory of the
rust-libp2p project. There you will find a simple
ping listener/dialer (peer 2 peer parlance for client/server). Using
that code it was simply a matter of separating the listener
and dialer logic into separate functions and slapping a
structopt CLI on top.
So far so good.
Arbitrary TCP/IP streams over Tor
Next I had to learn how to interface with Tor using Rust. A few folks in other languages were doing this and after a bit of digging I settled on using the Tor Control Protocol via the very nice torut library. Once again, the examples directory was my friend.
What with not knowing how Tor works and not knowing how the Tor Control Protocol (TorCP) works I found this pretty difficult, the fault for this is all my own and I learned a bunch about how I learn (or don't learn) things in the process. The time I spent doing this step was the motivation for writing this post.
The code discussed below is in this repository: github.com/tcharding/simple-tor-tc
I am running Ubuntu 18 LTS, I installed Tor using the package manager
apt). At first I was starting Tor using systemd but I quickly found
out that for security reasons TorCP only lets you connect to Tor and
get the information required for making an authenticated connection
once. Also one can only make an authenticated connection to the Tor
Control protocol port once in the lifetime of the Tor instance. In
other words we need to spin up a new instance of Tor each time we want
torut handles this for us:
This is the default invocation used by systemd on Ubuntu, I saw no reason to change it. This implies that the default configuration for Tor works too - win!
Next lets spawn an echo listener (server). I'm omitting the code for brevity but you can find it in echo.rs.
The plan is to create an onion service (previously called a hidden
service) that redirects to the locally running echo listener. This is
all done using the TorCP and
torut. Lets get a TCP connection to the
Using a static global for the address:
Note: All the libraries used depend on Tokio, and therefore use the
Tokio types (
Using the information returned by Tor we can authenticate (assuming we
have read access to the cookie file).
torut wraps all this up nicely
for us (did I mention that I really like this library):
I'm not totally across the event handler but from what I understand it's basically the code that runs for async events from Tor, since we do not need any async events we just ignore this as is done in the example code. (I actually do not know when Tor produces these events or what exactly these events are for.)
Onwards and upwards.
We have an authenticated connection to the locally running Tor instance that we started. We can now use that connection to create an onion service.
An onion service is made up of the hash of a public key with a 2 byte
checksum and a single byte version number. We need a private key to
torut will handle generating this and everything to do with
adding the onion service to the Tor network. (See
for an explanation of how the onion service protocol works.)
Lets generate a private key to use
Behind the scenes
torut uses the thread-local random number
generator seeded by the system, this is provided by the
torut can then command the local Tor instance to create the onion
service for us, this service remains available as long as the TCP
connection to the Tor instance remains open. This explains why the
code has an enormous single function and no smaller functions since
any calls to
Drop close the TCP connection. (That and laziness on my
behalf for not working out the types required to correctly pass all
this stuff around. Update: not laziness but inability, the type
signature of torut's
AuthenticatedConn<H, S> got the better of me.
H is the handler function. The closure we passed in above in
the call to
set_async_event_handler() is functionally a noop - I was
unable to work out the type signature for it.)
Ok, so now we have a local echo service listening on a local port and a onion service available via the Tor network that redirects to this local echo service. Nice.
Next lets connect to the echo service, thus proving that we can make arbitrary TCP connections over Tor in Rust code.
TcpStream is wrapped in a helper function, here it is:
into_inner() gives us a raw
TcpStream that we can read from and
IntoTargetAddr is implemented on a string that 'looks
like' an onion address i.e., not a multiaddr but something like:
Now we have a Tokio
TcpStream that we can read from and write to as
BOOM! Arbitrary data sent using TCP via the Tor network.
rust-libp2p connection over Tor
Now for the holy grail, connect our POC ping application that we wrote
in step one using
rust-libp2p over the Tor network.
Functionally the listener side of the application does not need to change. In order to prove this works though, and for convenience, we start the Tor node when we run the ping listener. This could have been done differently but it is a simple way that proves what we want to prove.
From here on we will be discussing code in the
This repo is likely to change so if you want to see the code discussed
check out tag
v0.1. This code, as stated above, is based on the ping
example code in the
For ease of development we hard code the local address used for the listener, localhost on port 7777. For the dialer the application accepts a Multiaddr (a libp2p specified format for addresses) which is the multiaddr for the onion service we wish to connect to.
I'm omitting the code for creating the onion service because it is
identical to that discussed above, the only addition is converting the
private key into a multaddr. Currently the
libraries do not play nicely together.
torut uses a 34 byte array
for its underlying onion v3 address while libp2p uses 35 (the final
byte being the version number, in this case '3'). Part of the
upstreaming work from this experiment is to resolve this, hopefully by
the time you read this its already upstream. So, there is a bit of
munging to convert the private key we generate into an onion address, add
the port number and feed this into libp2p - all in a days work. We
print the onion address, in multiaddr format, to standard out for the
user to use when starting the dialer.
The dialer is where the real fun starts. In a libp2p application we do
not control anything about dialing except for passing in the multiaddr
to dial. (Well not entirely true, we control the Transport used by the
way we build the libp2p switch (previously swarm), discussion of this
is beyond the scope of this post. See the libp2p
docs for more). So we need to do some libp2p
hacking. To get this working I copied the TCP Transport from the
rust-libp2p repo off the master branch. Next some minor code changes
to remove the
async-std stuff and just use
Tokio. Then all that was
left to do was hack the dial method to do what we wanted it to do -
connect to the Tor socks5 proxy and return this
TcpStream instead of
a direct connection. For reference, here is what it looks like before
Patched, it looks like this:
Connecting to the Tor socks5 proxy is done as we do above using the
tokio-socks library, here are the relevant lines of code again:
The default TorCP port is hard coded out of lazyness, I'm thinking this will go in the Transport config so they can be set by the user.
In hindsight pretty simple huh!? I hope you enjoyed this as much as I did.
Happy Hacking -- Tobin C. Harding.
TODO / upstream work
It would be nice to make the Transport implementation more robust by
- Having the
listen_onmethod use an onion address to connect to an already configured onion service. This way dialing and listening are symmetrical.
- Embed a
TorTokioTcpConfigto reduce code duplication
Thanks to the
torut authors for saving me from having to interface
with the Tor Control Protocol manually. Thanks to
being so modular and making it trivial to extend the Tokio TCP
Transport to support connection via Tor. Finally, thanks to CoBloX for
paying me to work on this.
Libraries used/mentioned above:
- rust-libp2p: https://github.com/libp2p/rust-libp2p
- torut: https://github.com/teawithsand/torut
- tokio-socks: https://github.com/sticnarf/tokio-socks
- Simple Tor Control Protocol application: https://github.com/tcharding/simple-tor-tc
- Ping written in Rust using
rust-libp2pover Tor: https://github.com/comit-network/ping-pong