I/O socket and thread concept in servants?

[Moak]

Hi,
I wonder which socket I/O concept do modern gnutella servants follow, how many threads they use for handling connected clients and handling downloads?

Okay let's seprate GUI from operative Gnutella code first and focus on the real Gnutella network core only (Berkeley sockets, Winsock is also based on the familiar Berkeley BSD "socket" interface). A gnutella servant will open a listening socket (socket/bind/listen)... and then? Theory: A typical Gnutella servant will handle a few gnutella connections (one to 4-5, many for superpeers) and a bunch of file up/downloads, which typically take some minutes. Also a Gnutella servant should handle many small new socket connections in a short time (for giving busy states or handling chat requests). So far theory, coming to implementation:

1. multithreaded server
Should the network thread remain blocking until a new client connection comes in and create/fork a new thread for every connection? That would result in one listening thread + one thread for every gnutella connection, for every upload and download. Disadvantage: A lot of context switches and overhead for thread/process creation.

2. multithreaded/sequential server
Same as above but mixed: All short requests (telling we are busy, chat requests) are handled sequential (we build on TCP/IP stack's backlog buffering). Which means new client request might be pending but handled very fast without any thread overhead. Threads are only created for actions that have a permanent expectation (handling free connects and upload/download slots).

3. select()-based server
Should the newtork thread block until something interesting happens on any of a group of sockets and handle every client connection step by step? Disadvantages: a lot of socket selects() necesarry, but AFAIK strong in handling a high number of connections. I guess (?) this is also what happens in Winsocks Asynchronous mode.

4. prethreaded/forked or whatever
Something that apache webserver does. Client threads/processes are created in a pool, remaining idle. The main listening thread just creates the client socket and gives it to the allready existing client threads, which handle them fast without extra thread overhead. Disadvantages: Maybe oversized, Gnutella clients to not have to handle thousands of small requests in a short time.

Ideas, feedback, anyone with Gnutella client experience?
Thx, Moak

[Moak]

Hi, after reading some Gnutella servants source code [1] and talking with some other developers, it seems that model number 3 (select()-based or poll()-based) is the common tactic.
I think this model does fit to the needs of a gnutella servant (fast response and handling a high amount connections) and it perfectly supports a flexibel object orientated programming concept (okay, I personally love C++ and object orientated programming) [2].

Greets, Moak

[1] Gnutelladev - http://www.gnutelladev.com/source.html
[2] Programming a Gnutella Client, part II: network core

[Tamama]

I'm also currently working on a servent and mine will use kqueues. Unfortunately the only system i know that support them are FreeBSD and possible other BSD variants... For systems that dont support it (like linux/windows) it is emulated using select (some minor overhead here)

[Moak]

Basic Winsock - http://tangentsoft.net/wskfaq/exampl...ics/index.html
What Kind of Socket Should I Use? - http://www.warmi.net/httpd/html/vari...ket-types.html
How to Use TCP Effectively - http://www.warmi.net/httpd/html/vari...ctive-tcp.html

[Tamama]

I emailed a correction to that article page about winsick erm, winsock

What I am wondering is:

How did those of you that use asynchronous sockets implement it? :-)

[Moak]

Another question: select() vs. poll() ?

It is told poll() is faster on a high count of descriptors/sockets. But poll() is not full portable, e.g. no Winsock implementation.

[Tamama]

Both poll() and select() take a list with file descriptors. One could implement poll() with select(), and select() with poll() and the program wouldn't even know.

However, poll() doesnt have to clear the set, doesnt have to possibly look in 3 sets for 1 socket, and doesnt have to create 3 new sets to return.

With this knowledge, poll() would be faster.

BSD manpage extract:

COMPATIBILITY

This implementation differs from the historical one in that a given file descriptor may not cause poll() to return with an error. In cases where this would have happened in the historical implementation (e.g. trying to poll a revoke(2)ed descriptor), this implementation instead copies the events bitmask to the revents bitmask. Attempting to perform I/O on this descriptor will then return an error. This behaviour is believed to be more useful.

So there you have it.


I dont use neither of them :P

Tam

[jsdn]

What is the difference between options 1 and 3? Isn't option 3 just an implementation of option 1?

[Moak]

Type 1 "multithreaded server" is handling simultaneous requests with one thread per every socket.
Type 3 "select()-based server" is handling simultaneous requests, one server thread will handle all sockets at once. I would prefer poll() over select() today.

/Moak