[Apologies in advance if this isn't the right place to mention this, but... ]

Acquisition 0.67, a Mac OS X Gnutella client, implements the new "chunk" download method where files are downloaded in 97.7k segments. I understand this is a new Gnutella feature (or possibly just LimeWire feature?) designed to improve the robustness of the download system. Unfortunately, it does not work very well yet. Most of the time, it will result in the client downloading 97.7k worth of data, disconnecting, and then attempting to reconnect to continue the download. Needless to say, this prevents the connection from ramping up to full speed, and also frequently results in someone else taking the "free" download slot before your client can reconnect.

What is the justification for breaking downloads into these small chunks, and are there any plans to address the serious problems caused by this method of handling downloads?

Thanks for the reply, linuxrocks. However, in my experience, downloads do not complete faster with the "partial content" feature. (Like I said, most of them do not complete at all, they simply disconnect at 97.7k, never to resume.) In fact, I'm not sure how it could be faster, as I have a DSL connection that needs a solid, continuous connection in order to ramp up to full speed, and that just doesn't happen in the space of 97.7k.

As for rotating downloads from multiple hosts, that might be great if you are downloading Eminem's latest, but most of the time the file I'm trying to download is only hosted by one source. When I've finally managed to track down a rare file, the last thing I want is to have my connection to the host severed mere seconds after it begun.

I'm not sure if the problem that the client is actually sending a "disconnect" message after each 97.7 chunk, or if some hosts incorrectly interpret the end of a chunk as their cue to to close the connection and offer up the download slot to someone else. Regardless, at this point this "feature" is causing far more harm than good and I hope these problems will be addressed soon.

The smaller chunk size does not reduce download speed as much as the LimeWire/Acquisition UI tells you. (Try a network monitor or something and look at the overall downstream) The downloads are a lot more robust since then and LimeWire/Acquisiton does not create large files containing mainly empty data any more. Corruptions should be identifed earlier and the overall downloads should be more robust. - This new feature might also come very handy if tree hashes and partial file sharing are implemented one day - although partial filesharing is generally overestimated (except for it can be used to force users to share - although it is my experience that with higher connection speeds freeloaders are becoming rarer).

trap_jaw,

Hmm... I'll grant you I may be deceived about the download speed by the UI. So perhaps you are right that the small chunk size will eventually be a good thing overall. But for the moment, it is a serious hindrance since it doesn't maintain a steady connection -- it downloads 97.7k and then disconnects, leaving your download aborted practically as soon as it had began Yes, it tries to reconnect but 99% that doesn't happen. So while this feature may be good for the network overall, it's terrible for the end users who are stuck with it (at least until the problems with the implementation is resolved).

This can only happen if you were downloading from a buggy client that claims to support http 1.1 but does not really.

I've seen that happen, too.

That's what I thought, but there are apparently an awful lot of buggy clients out there. It's difficult to know which ones are the culprits since that information is not available to Acquisition users, but I know for a fact that Shareaza doesn't support this (i.e., it boots Acq users after the first 97.7k). There are surely others, given the howls of outrage from users when this "feature" was introduced.

Anyway, Dave Wantanabe, the developer of Acquisition has taken steps to try to fix this on his end, and it seems to be working better now -- so far.

more specifically, it's trying to copy the sucess of the CS/MA nature of ethernet -- instead of trying to establish a constant connection (with guaranteed throughput), experience has shown that it's better to have a "collision domain" where everybody competes for temporary access. If two requests collide, they both back-off for a random amount of time and try again. As long as the signaling speed is fast and you don't have more than about 25 requestors, you get consistent transfer rates (60-70% of the theoretical).

P.S. I know I've got the acronym for this technique wrong but I'm tired and don't want to try and look it up tonight...

I think it's obvious from the experience of many users (not just me -- really, check the Acquisition forums) that this is probably okay for local networks, and maybe even okay for files that are commonly hosted by, well, a host of clients, but it is extremely bad for that most common of Gnutella situations, i.e., when many clients are after a file hosted by a tiny minority of hosts. Especially when the clients play by different rules.

If you have one client that implements the chunk downloads (handing over its download spot to someone else after only a 97.7k transfer) versus another client that hangs on to the connection like a bulldog until it is complete, who do you think is going to end up with the successful download?

If a client really implements http1.1 it does not give your download slot to another downloader until you close the connection, which is not the case between requesting different chunks.

If you Acquisition users think it's not good, - why not tell the guy who put it online to change it? It's really a trivial task. The chunk size is a constant in the ManagedDownloader class.

We did and he did.

The idea with chunks is simply swarming downloads. When you divide the file in chunks, you can request the different chunks from different hosts and download the file simultaneously from different sources. That will speed up the transferring proccess. Dividing in chunks itslef will not help at all, it will actually decrease the downloading efficient rate. But when there are multiple sources available and the chunk size is reasonable, it will significantly increase the download transfer rates for broad band users. But without Keep-Alive and HTTP-pipelinig small chunks may have huge negative affect to the transfer rates.

__________________
Some day I'll find peer and reset his connection.

That was essentially my point -- dividing the file into chunks might work well under ideal circumstances, i.e., when downloading files that are hosted by large numbers of hosts, all of whom support the feature. Unfortunately, that simply doesn't reflect the reality of the Gnutella network as it stands today. The overwhelming majority of the files I might actually want to download are available from only one host at any given time (at best).

Downloading in chunks will not hurt performance significantly as long as the chunk size is reasonable and both-ends support Keep-Alive and HTTP pipelining.

Even if the other-end does not support Keep-Alive nor HTTP pipelining, download performance shouldn't hurt much as long as the downloader is using reasonable chunk sizes. If a client doesn't have support for at least Keep-Alive, it definitely should not download files in chunks. This is especially important when downloading files from firewalled hosts, establishing the connection through PUSH have relatively low success rate, so it is important to not disconnect from the host.

I wouldn't call a fixed 97.7k chunk size reasonable. It results way too many re-requests when downloading large files. Also, if the client doesn't support Keep-Alive (you said it loses the connection after downloading a chunk), it will definitely result very poor download performance, even if there are a lot of reliable sources available.

Keep-Alive is essential feature when downloading files in chunks, HTTP pipelining is highly recommended.

__________________
Some day I'll find peer and reset his connection.