[C] tunsl

[nslay]

To be consistent with my other postings, I've decided to dedicate a topic of its own to the actual p2p tunneler instead of appending it the P2P VLAN post. This (tunsl) is a working prototype of a p2p tunneler.
tunsl is composed of 9 inter working components:

fdman - The file descriptor manager
timer - The timer manager, it cooperates with fdman ... fdman varies its timeout value to select() with information from timer.
tunman - The tunnel manager, it gives the application a high level interface to doing simple operations like setting the tunnel's IP address, reading from tunnel, and etc.
sudp - This does secure UDP communications over DSS/DH with Blowfish
conf - This is the configurations parser

It uses these data structures:
hash - A generalized hash table (heavily used)
stack - A generalized stack (used with conf)
list - A generlized linked list (used with timer)

And finally, the main component, the meat and the guts
tunsl - This is the route manager ... it does everything from seek out routes to DSS/DH with Blowfish over routes.

The real thing would have more components to manage the protocol and communications with the daemon. 

The purpose of this prototype is to get a feel for how to write this odd program ... its not everyday you're walking down the street and instantly want to write a p2p tunneler.  It's also meant to be used for testing purposes both stability and security-wise.  I think it could be useful to have a decentralized virtualized LAN, as long as one peer exists, the network exists.  No matter which peer you connect to, you are part of the same logical network as they are.

So, just to give you a general idea of how this works.  The tunsl component is the focal point of this program, it is the literally the engine.  It makes use of both sudp and tunman for communications.  tunsl listens on sudp for various events, but in the prototype all that is emphasized is connect/disconnect and read events.  On connect and disconnect, tunsl creates and destroys peer structures that are hung in sudpcon.arg for later use.  On read, tunsl hands the data over to a protocol handler.  The protocol handler does any forwarding necessary and also handles data addressed to us.  tunsl additionally listens on tunman for packets read from the tunnel and each packet read, it calls a function that tries to send the packet ... that is, it looks for a route if none exist, or it uses a random existing route.

So, I did some testing last night between 4 machines: LIGHTBULB, BLENDER, BOTTLE and DOORKNOB.
LIGHTBULB connected to BLENDER
BOTTLE connected to BLENDER
DOORKNOB connected to BLENDER and BOTTLE

I pinged each of these machines by their virtual IP from every other machine (and in some cases, all at the same time).  I had even ssh'd from LIGHTBULB to BOTTLE.  The routing scheme works quite well surprisingly (see Protocol Description post in P2P VLAN).
Here is a screenshot of transactions:


So, some problems I discovered:
a) This network, although it emulates an IP LAN, cannot be bridged or NAT'd to a real IP network.  The reason is, each route must be encrypted ... nodes communicating from outside of the network simply cannot do this.
b) The Blowfish cipher has a chaining effect on the data ... the original intent was to allow the OS to handle lossage, but I have to deal with this too.  Encrypted data cannot go missing and cannot arrive out of order ... but that doesn't mean tunsl isn't ill equipped to handle this scenario, like the sudp library, it will reset the cryptographic parameters to their initial state.
c) There is no checking on the packet level ... you can have a packet of random garbage addressed to a destination on P2P level (this can be fixed)

Any comments on solutions to the above are welcome

So here is the source, it is designed for FreeBSD 6.x branch of OS but it might work on 5.x.  Additionally, you will need OpenSSL 0.9.7 or above to use it.
http://nslay.36bit.com/tunsl.tar.gz

[warz]

Maybe this is a dumb question, but I'm lost. What exactly is the purpose of a P2P 'tunneler"? I doubt I'm the only one that doesn't know, because from the lack of replies to your posts it seems as if others don't know what to think either. Is this supposed to allow everyone to connect to one address and then become virtually networked with everyone else connected to the same address? Like, a centralized bittorrent style thing?

[Newby]

Only an hour later, of course. Read his other topic.

[warz]

Only an hour later, of course. Read his other topic.

poor newby.. doesn't understand the world. he has other posts, and thats what i meant by "posts" JEESUS. now go buy more MS stocks

[nslay]

Maybe this is a dumb question, but I'm lost. What exactly is the purpose of a P2P 'tunneler"? I doubt I'm the only one that doesn't know, because from the lack of replies to your posts it seems as if others don't know what to think either. Is this supposed to allow everyone to connect to one address and then become virtually networked with everyone else connected to the same address? Like, a centralized bittorrent style thing?

This is mentioned in my P2P VLAN post, but briefly, this p2p tunneler does for general IP communications what a file sharing client does for files ... it forms a virtual network independent of its layout and independent of the real network.

[Ersan]

There is no way to prevent people from sending incorrect/bad data, which is why this has not been done.

[nslay]

There is no way to prevent people from sending incorrect/bad data, which is why this has not been done.

Thats an inherited problem for any p2p network, not just this one.  However, public keys and cryptographic hashes make it more difficult to send bad data.

EDIT: I think this is what you meant, but you cannot just up and atom send a packet, you have to establish an encrypted session first which encompasses pursuing a route.  What I meant by c) is that this p2p protocol has its own fields for from and destination and that this is disjointed with the encapsulated packet.

[nslay]

So anyways, I've been testing tunsl extensively throughout the day running it in debug mode.  I'm actually using an ssh tunnel over tunsl to post this message . I've fixed a lot of p2p communication logic errors and learned quite a bit.  I rewrote the timer library because it was a blackeye and added a route expiration timer.  I've been thinking about this before hand, but a p2p network is like a "medium" and we could give it context such as "connect", "disconnect", "send" and "recv."  I am thinking I will write a separate p2p communications library tailored to tunsl, and then write a route library that interfaces with both tunman and the p2p communications library ... this will make things much cleaner than they are presently.  Maybe in the next couple days we'll have a much cleaner tunsl

[nslay]

Hi Folks,
I wrote a second revision of tunsl, this one features a better layout of data structures and a few timers and an encryption better suited to this application.  Instead of using Blowfish CFB mode, I use ECB mode since this does not rely on the order of data arrived.  As a consequence of using ECB, I've removed TUNSL_PROTO_RESET entirely since it is no longer needed.  The problem with CFB mode is that both end points need to keep a synchronized copy of an initialization vector (iv) and sequence number ... but since packets may travel in a variety of ways on a p2p network, the order in which they arrive is unpredictable.  In order to decrypt data using CFB mode, you must have the iv and sequence number that correspond to it and these are updated with each decryption, but if data arrives out of order then your iv and sequence number go out of sync very fast.  The TUNSL_PROTO_RESET solved this problem and both end points would set their iv and sequence number to a mutually known state, but on a wide scale, this is a very poor solution. The ECB mode does not rely on data order, it does not use initialization vectors or sequence numbers.  However, now a man-in-middle attack is more feasible, but the man-in-middle still will not receive all packets assuming their are more than 1 routes to a destination.

At the moment I'm working on preventing echos ... an echo is a message that gets routed over and over again.  The only defense tunsl has for this is a hop count.  It will never echo between itself and another peer, but you can have these triangular layouts where you have peers A, B and C
A sends a message to B, B sends a message to C, and then C sends a message back to A, etc... 

[nslay]

I wrote revision 3 last night, in a little under 8 hours ... that includes debugging.  Revision 3 is probably the last necessary revision and features a more fail/stall safe way of establishing packet routes.  I was talking to Yoni about routes and I think I can make tunsl coordinate with the route table to make it able to be bridged or NAT'd to real networks.
So I mentioned before that the from and to fields in the p2p header are disjoint with the actual packet.  When tunman reads a packet, if the destination address is outside the network, tunsl can simply do a route lookup and replace the destination address (in the p2p header) with the gateway's address.

[Newby]

You like rewriting your code, huh? I have that issue too. I'll do it one way, decide it isn't efficient, and completely re-do it.

[nslay]

You like rewriting your code, huh? I have that issue too. I'll do it one way, decide it isn't efficient, and completely re-do it.

Yeah, exactly, and I also have never written a p2p application ... so when something goes wrong with one implementation I usually have to redesign it.  Only tunsl.c and tunsl.h are ever really modified in these revisions.  Everything else is pretty much solid. 
When I get around to it, I'll port tunsl to Linux and maybe to OS X.  As for Windows, I'd have to lookup the VPN API.

Some of the TODOs include:
- Write route table support (so you can have a gateway or whatever)
- Write an interface for applications to access statistics and other information
- Write a control utility
- Write documentation
- Write better main.c

These I hope to have done in the coming weeks.  Once these are done I'll probably make a page for tunsl on nslay.36bit.com

EDIT:  Oh yeah, I should also mention revision 3 seems pretty solid...I have a 4 machine virtual network (counting this one).  I've used VNC on it, I've downloaded files on it, I've done everything I would normally do on it and it holds out well.  I keep the other three machines on 24/7 and tunsl running on a screen in debug mode.  Much of the testing done on this revision also have to do with stalls and failures ... its pretty solid (as far as I can see) at establishing and keeping routes.  Once I get the above done, and a Linux port, maybe some of you can join my virtual network (or form your own).  The only remaining problem is peers can play catch with messages on some network layouts ... for now the hop count is the only defense, its merely an annoyance and it can degrade the speed of a route between two peers slightly.

[nslay]

I just returned home today from Texas and tried tunsl on the LAN with multiple peers.  Firstly, I'd like to point out that NAT seems to dislike having multiple machines using UDP to the same host and port ... so that can cause problems (one of the machines actually lost contact with the remote peer).  Secondly, my randomized send scheme seems to confuse TCP/IP ... TCP/IP can cope with out-of-order packets but it degrades performance significantly.  I should probably weight the randomized sends based on observed hop counts ... that might boost performance while maintaining the "onion-route" thing.

Comments?

EDIT: My setup is
DOORKNOB connects with BOTTLE and LIGHTBULB
LIGHTBULB connects with BOTTLE and DOORKNOB
BOTTLE connects with DOORKNOB and BLENDER
BLENDER connects with BOTTLE

[nslay]

The hop count is to be removed since it can be used to help describe where on the real network a peer is.
For example
10.1.1.2 <-> 10.1.1.3 <-> 10.1.1.4

If 10.1.1.2 communicates with 10.1.1.4, the hop count will be 2 ... this isn't so serious since not much is known about the mediating peer, however, if 10.1.1.2 communicates with 10.1.1.3, then the hop count is 1, and it is immediately known who is 10.1.1.3.

As a consequence of this problem, I am going to eliminate the hop count field in the p2p header.  It is only used for redundancy defense anyways.  A better solution seems to be caching the message's CRC32 hash...CRC32 is fast to compute and a cryptographic hash doesn't seem to be necessary.  We can also additionally count how many times a message with a specified CRC32 has passed through a peer ... if this reaches some magic value, we can discard it.  We should additionally have these cached hashes expire in 1-2 seconds.

Comments are appreciated ... nobody seems to comment, maybe the application's use isn't immediately realized ... or maybe nobody can comment because tunsl cannot be run on anything but FreeBSD as of yet.

[d&q]

I regularly read your updates, I just don't comment because I have no knowledge in field. 

[nslay]

I regularly read your updates, I just don't comment because I have no knowledge in field. 

You and me both   I'm learning as I go!  It took me a REALLY long time to learn the OpenSSL API ... I spent a month or two over the summer learning how to use it.  Thanks to the man pages and some fiddling, I really got the hang of it.  I've also learned how to do a lot of FreeBSD specific stuff looking at the source code of many of the utilities like ifconfig ... thats the beauty of open source software!

[Newby]

Comments are appreciated ... nobody seems to comment, maybe the application's use isn't immediately realized ... or maybe nobody can comment because tunsl cannot be run on anything but FreeBSD as of yet.

See what Deuce said. To me it's like reading a changelog. I go, "oh, neat" but I can't really say I understand everything you are saying enough to comment on it and not risk sounding like an idiot.