sha1ess gnutella
 

Hello, everybody!

I used Direct Connect for a long time. Global Internet was expensive, and Direct Connect handled citywide Intranets pretty well. Now, Internet prices are lowering to affordable values, it is natural to look for something more scalable. Direct Connect is used globally, but dc hub can hold up to 20000 users, far from millions. Gnutella sounds good for many reasons, one of them being TTH support. TTH is the best partial-verify-capable hash IMHO. As opposed to AICH, it is not disbalanced, and it's based on the fastest cryptohash, TIGER. What can be better? Gnutella might be a natural continuation to Direct Connect given the fact they use the same hashes.

The first experience with Gnutella was quite disappointing. I started with a FrostWire, gave it a time to hash my files, and when everything was done, I decided to look on a magnet links. I have discovered that they only contain SHA1. AAAAAAARGH! What's the hell is this nontree hash still doing in the p2p? Isn't it stupid? Isn't it more than stupid? Who is the one to blame out of, say, 20 sources if the resulting 6Gb file will be corrupted? Downloading 6Gb from Internet takes several days. SHA1 is not an option indeed. Let FrostWire defects come alone, there are plenty of.

Another time I have seen freaking SHA1 was distribution: OpenOffice.org P2P Downloads. These magnet links only contain SHA1! No TTH!!! It is usually hard to meet a magnet link other than Direct Connect magnet link. OpenOffice.org webmasters not only put magnet links deeper than torrents and main download page, but they also managed to create a magnet links not suitable for Direct Connect. [intermezzo]Good idea, bad implementation. The idea is good because I often feel that webmasters and developers live on another planet. On our planet about 33% use p2p. On their another planet there is no p2p at all. If they calculate hash, they calculate useless md5 as opposed to useful tth. If they provide downloads, they provide HTTP downloads as opposed to magnet links. [/intermezzo]

The invasion of SHA1 is unpleasantly surprising.

I'm wondering what would be the usage experience if someone will strip out SHA1 completely. Just TTH. The resulting program(s) is to be distributed through a DC portal. Even better, an existing DC client could be extended to handle sha1ess Gnutella.

It is a technical question. Would it be possible to exchange files between upgraded client and a legacy one?

Yeah, I believe SHA1 must be eliminated completely. There was a moment when DC++ started to deny non-TTH-capable clients. It was a good service to the DC networks. As a consequence, every DC client in the wide use is now TTH-capable. And it's good.

In Gnutella you use the sha1 link to request the content, and once you connect to the first TTH enabled node, you get the TTH from him and use it for verification of everything.

They just aren't included in magnet links (yet) for legacy reasons.

To give i bit more information (I was a bit brief):

Currently about 95% of the live Gnutella nodes support TTH (Limewire/Frostwire, Phex, gtk-gnutella, ...).

Sha1 is being used to identify a file (to request it), while TTH is being used to verify each chunk. A request is a simple GET request with /uri-res/N2R?urn:sha1:<sha1> as path (and for this sha1 is perfectly suited, because it just identifies the file).

And you can add TTH to magnets, it's just not done by most people.

I think it was something like “xt=urn:tth:<hash>” (replace <hash>). Or bitprint…

jepp, just checked it: urn:bitprint:… is tiger tree + sha1

Sounds promising.

Is it possible to resolve SHA1 from TTH? One could probably get SHA1 for a specific peer and use this SHA1 to request a file from this peer only.

Well, maybe, but this in fact means that one have to spend time calculating SHA1 just because it is used for identification. Calculating sha1 is even more difficult when you don't have the files (they are remote, distributed through DC userbase). Websites like (...deleted...) and also lots of intranet dc portals can only transit to Gnutella given that Gnutella can work without SHA1.

With regards to local files I'd better spend time calculating md5 when idle. Multiple non-p2p-aware sites provide md5 for file verification. Multiple tools (cli and gui) use md5. This is not gonna change in the foreseeable future.

This is a very important moment. I thought it is supposed to be done automatically. Of course most people won't edit magnet links. In eDonkey modern clients include aich by default; at least, it is highly recommended to include AICH in ed2k links, and it's hard to meet non-aich ed2k link. In eDonkey, however, both ed2k and aich hashes allow partial file verify.

Is there Gnutella clients around that generate TTHful magnet links by default? bitprint is also fine, Direct Connect clients are aware of it.

Another relevant question: Is is possible to look for file sources without knowing SHA1?

Can anybody download this file in Gnutella?

magnet:?xt=urn:tree:tiger:GMRI7CX6MRUGEGN4BITB25UG G3QUANYMPO33SLA&xl=1388544&dn=msvbvm60.dll

(remove the space, the space is inserted by forum engine, I can't get rid of it)

It is very popular in Direct Connect. Lots of stuff depend on it. There should be no problem to locate it in Gnutella given that TTH searches are implemented.

You can't search for tth, no. You need a sha1 for searching, even if you use a client with DHT.

To create magnet links with bitprint, you can use bitcollider (released 2004): Bitzi Bitcollider Utility

Sources:
- SourceForge.net Repository - [bitcollider] Index of /
- SourceForge.net: Bitcollider - Develop

I thought gnutella would support searching for hashes other than sha1, too.

G2 does. It supports searching for MD5, eD2k, sha1 and the tth root hash. Other hashes could be supported, if they were supported by software using the G2 network.

Gnutella only supports searches by hashes via the DHT, because these cannot be routing-optimized as well as keyword seaarches (and with millions of simultaneous users these optimisations are crucial - they save about 90% bandwidth - Info:QRP ).

The DHT is a Kademlia-network (optimized for exact match retrieval) which is based on sha1 hashes (though you could just sha1 hash a tth to make it searcheable via DHT).