Server designs

[nslay]

Quite a while ago, I came across AF_UNIX sockets. This is quite a peculiar form of IPC. Listening sockets are exposed through the file system (as well as permissions, very neat!) and while it does support your usual modes of communication (e.g. SOCK_STREAM, SOCK_DGRAM), it also allows you to send and receive file descriptors. I think the latter is quite magical!

Years ago, I pondered a chat server design that inherently supports linking. It's a master/slave distributed memory design. You have a master process that listens for connections, stores meta data, and passes off the connections (descriptors) to slave server processes. The master does some load balancing and manages both internal and external server links. The slaves all communicate over AF_UNIX/AF_INET sockets with a linking protocol.

Nice thing about this design:
1. It's parallel and without the need for any locking (or similar)
2. Crashes are more localized. Only a subset of users are affected.
3. The use of AF_UNIX allows for seamless upgrades (i.e. launch new server binaries without ever actually closing a single connection ... all descriptors and meta data can be transferred to the new server processes)
4. It is potentially more portable (among UNIX-like systems) since the 1024 descriptor limit of select() is a non-issue. 10 slave servers gives 10240 connections.
5. It is designed and implemented as a distributed memory system ... it supports linking from the start

Bad things about this design:
1. Distributed memory stuff is complicated.
2. Some communications within a logical server invoke the kernel (sendmsg()/recvmsg()). A threaded server would not have this overhead.