Directory, idea on developing gnutella, constructive criticism, please
 

The following is not complete, may contain errors, but I definetely think that there is potential... and I'm not the only one.

<h1>
Directory extension to Gnutella
</h1>
<br>
Tommi.Lukkarinen@uta.fi 30.01.2003
<h2>
Why
</h2>
<br>
This kind of addition would make popular searches, such as .mp3 and .avi, faster and covering more data. With small changes this could also enable small groups (enthusiasts of some sort) to make efficient sharing.
<h2>
Concept
</h2>
<br>
The idea of directory is to make searches faster and more efficient. It builds another set of nodes on top of existing Gnutella network, serving only the special interest described in the search criteria. The idea is to use more resources on each computer that has interest on the directory context, to save overall network resources. Unlike queries (0x80), directory builds do not spread automatically, but they spread only if the servent has interest on the directory. A servent, which does not contibute to the directory, will not be added to it, but may still make queries into it. Queries are handled in the ordinary manner, same as before, but queries matching directory criteria will be forwarded to the directory network instead of the Gnutella network. Directory network is a network of contributing servent’s only, and to which servents with interests (but no contribution) have links, but are not constantly connected into. The directory concept needs some intelligence in handling of TTL and Hops. As spreading of directory is user driven, it should not be hindered by TTL. In other hand, backward flooding might be possible, so TTL might be needed in those cases. The document has some ideas on where to use TTL and where not.
<h2>
Build Directory (0xD0)
</h2>
<p>
Data
<p>
Search criteria
<p>
Byte offset
<p>
0, …
<p>
Events in a servent that has not expressed interest:
<br>
Add to the list of directories
<p>>
Events in a non-contributing servent that expresses interest*:
<br>
Forward build directory request
<p>
Events in a contributing servent that expresses interest*:
<br>
Forward build directory request
<br>
Send Directory Hit backwards
<p>
Events in a non-contributing servent that has matching directory:
<br>
Forward Directory Hit to known links under ‘directory data’
<p>
Events in a contributing servent that has matching directory:
<br>
Decrease TTL
<br>
Forward Directory Hit request to known links under ‘directory data’
<br>
Send Directory Hit backwards
<p>
*Expressing interest happens usually by user choice. I imagined there would be a file explorer like interface, showing possible directories, opening one forwards the request and waits for directory to fill with data (how many files available nearby and how many kilobytes), disabling one prevents similar Build Directories ever showing again.
<h2>
Directory Hit (0xD1)
</h2>
<p>
Data
<p>
Port
<br>
IP Address
<br>
Speed
<br>
Number of files shared
<br>
Number of kilobytes shared
<br>
Matched criteria
<p>
Byte offset
<p>
0, 1
<br>
2, 5
<br>
6, 7
<br>
8, 11
<br>
12, 15
<br>
16, …
<p>
Events in a servent that has not expressed interest
<br>
None, this message should never reach such servent
<p>
Events in a non-contributing servent that expresses interest:
<br>
Add the link under ‘directory data’
<br>
Forward the Directory Hit backwards
<p>
Events in a contributing servent that expresses interest:
<br>
Add the link under ‘directory data’
<br>
Make connect to the link, possibly with directory version and gnutella version
<br>
Decrease TTL
<br>
Forward the Directory Hit backwards
<h2>
The Small Changes that could benefit small special interest groups
</h2>
<br>
Build Directory (0xD0)
<p>
Events in a servent that has not expressed interest:
<br>
Decrease TTL
<br>
Forward build directory request
<br>
Add to the list of directories*
<p>
Events in a contributing servent that expresses interest:
<br>
Increase TTL
<br>
Forward build directory request
<br>
Send Directory Hit backwards
<p>
The first change means that Build Directory spreads better, reaching more servants who could be interested of the directory. But it also means that spamming is easier. *If adding to the visible list would always be user initiated, this could be prevented. The second change (increase TTL) means that the Build Directory keeps on growing when it finds more interest. This way even a relatively uninteresting subject could once pass through much of the Gnutella network.

Decentralized directories sound fine, but they make the Gnutella network less anonymous and offer spyware servents an easy possibility to build an index of most shared files in the network. This is clearly not desired.

The nice thing in the current implementations of the query routing proposal (QRP), which is used between ultrapeers and leaf nodes, is the fact, that the QRP tables in an ultrapeer only contain hashed values of search keywords as a bitmap, which means that the ultrapeer knows which leaf nodes to forward a query to, but it doesn't really know anything about the files a leaf node shares. It does only know about some files when query responses are returned from the leaf node, but that still normally is only a fraction of the files a leaf node has to offer and that makes it nearly impossible for an ultrapeer to build an index of the files a leaf node has.

Tradeoffs
 

The idea is neat, but the tradeoffs of implementing it on the Gnutella network don't even out, IMO.

The current search method is fast enough to make the network very usable for most file-sharing purposes. The number and magnitude of the weaknesses it has against malicious servents outweighs any increase in search speed.