no more auto-requeries?!

[sdsalsero]

Do I understand correctly, that the new "Could Not Download; Awaiting Sources" message means that LW is not automatically re-doing the search for that file? If so, that's just ridiculous! The only reason I leave a failed download in my download window is because I'm still hoping to find it. The whole point of computers is to do things for us, i.e., labor-saving. If I have to manually re-do the search, that's no benefit to me. I can't stay home all day and click Repeat Search...

I understand the need to save bandwidth but disabling one of the main features is just totally misguided.

Here's some alternative suggestions:

1. Group and compress communications between Ultrapeers, e.g. wait 1.0 seconds before passing-along requests and then use an open-source zip function to reduce bandwidth.

2. "Expose" the Repeat Search parameters of each incomplete download, i.e., ability to right-click a file and see what known source IPs exist(ed) and what search term(s) generated the original download. If you've been searching for a file for a long time, you'll probably have developed a list of source IPs that have the file but who aren't on-line most of the time. If you can add those IPs to the file's "Source IPs" list, the computer can automatically check them every 5 minutes (or whatever).

Finally, were auto-requeries (before they were cancelled!) doing a search for the exact filename or were they doing a search on the original search term? If the former- DUH! There's your problem! Every failed download should maintain a copy of the original search term and use that instead of the particular filename. The reasons: a) it'll be shorter and save bandwidth; b) it'll find more potential hits!

[Joakim Agren]

Originally posted by sdsalsero
Do I understand correctly, that the new "Could Not Download; Awaiting Sources" message means that LW is not automatically re-doing the search for that file? If so, that's just ridiculous! The only reason I leave a failed download in my download window is because I'm still hoping to find it. The whole point of computers is to do things for us, i.e., labor-saving. If I have to manually re-do the search, that's no benefit to me. I can't stay home all day and click Repeat Search...

I understand the need to save bandwidth but disabling one of the main features is just totally misguided.

Yes it means that you manually have to search for new sources for the file. The requeries where considerd by the LW developers to be an evil function for Gnutella since it increased traffic but really never worked as advertised. LW used to send a requery out to find alternate locations for the file once every 50 minutes but most of the times it did not find any and the requery sent status could last for days. Clearly a waste of bandwidth for nothing. The new 2.9X versions will drop all requery messages in an attempt by the LW team to stop other vendors from using it to in future versions. Requeries are no good to Gnutella!. This was clearly not one of the main functions of LimeWire as you suggests.

Here's some alternative suggestions:

1. Group and compress communications between Ultrapeers, e.g. wait 1.0 seconds before passing-along requests and then use an open-source zip function to reduce bandwidth.

I think that some kind of message compression technique is currently beeing worked on by the LimeWire team.

2. "Expose" the Repeat Search parameters of each incomplete download, i.e., ability to right-click a file and see what known source IPs exist(ed) and what search term(s) generated the original download. If you've been searching for a file for a long time, you'll probably have developed a list of source IPs that have the file but who aren't on-line most of the time. If you can add those IPs to the file's "Source IPs" list, the computer can automatically check them every 5 minutes (or whatever).

Since a vast majority of users on Gnutella uses a dynamic IP this would not work effectivly. And also bandwidth consuming.

Finally, were auto-requeries (before they were cancelled!) doing a search for the exact filename or were they doing a search on the original search term? If the former- DUH! There's your problem! Every failed download should maintain a copy of the original search term and use that instead of the particular filename. The reasons: a) it'll be shorter and save bandwidth; b) it'll find more potential hits!

This could work if all people on Gnutella would use specific search terms such the name of a particular song. But since the original search term is often for instance just an artist name then the result list will be slightly larger then for just that particular song so I think that this function would increase traffic on Gnutella slightly due to more query hits if the Requerys with this function added would have been implemented. I think that the requeries where by a specific Filename not search term used.

I think the the Requerys is a thing of the past now!.

[stief]

sdsalsero, FWIW, I have been able to leave LW 2.9.8.2 unattendend for 6-8 hours and some files have completed. I do have to spend a lot of time setting up a blocked host list (to filter the spam results that mask good sources), and try to build up a list of good alternates by using narrow searches.

Yeah, I'd like to see automatic requeries back if they couldn't be abused by spammers who flood the network with spurious results. Maybe requeries for narrow searches (less than 10 results) as a decreasing average of previous manual searches?

I think Acq .84 has temporarily brought back limited automatic requeries, but I think newer Ultrapeers have some logic to block clients that use automatic requeries.

[sdsalsero]

Thanks for the replies, guys.

I understand the need to reduce bandwidth waste but, really, I hope there is a way to improve the requery. For instance, maybe maintain a copy of the original query and then requery along with a size or file-hash? That way, the only hits would be the same file.

As for blocking requeries, are they labeled as such? They must be else the Ultrapeers couldn't differentiate and block them.

I've been a paying support of LW since day 1 (i'm on my 3rd subscription now) but this loss of functionality is really really pissing me off...
___________________

stief, what's this 2.9.8.2 version? I'm running LW 2.9.8-Pro on W2K, and it's a copy I downloaded within 24-48 hrs of its availability.

[stief]

re 2.9.8.2

an anonymous poster "thebigname" posted it late last week. I'm not sure what improvements it has, but I think it's related to some of the tweaks trap_jaw mentioned. Here's the link (no pro avail) http://www9.limewire.com:82/download/ I'll look up the original post and edit the link back here.

http://www.gnutellaforums.com/showth...=&postid=68802

[sorry about the editing hassle--couldn't format the link properly]

If you try this one, be prepared for a lot of could not move to lib errors

[trap_jaw]

Originally posted by sdsalsero
I understand the need to reduce bandwidth waste but, really, I hope there is a way to improve the requery. For instance, maybe maintain a copy of the original query and then requery along with a size or file-hash? That way, the only hits would be the same file.

Statistically requeries always were a complete waste of bandwidth. 95% did not even return any results. Query-by-hash is the most inefficient thing you can do in gnutella at the moment. It means sending a query to thousands of peers with a very, very low probability that anybody has this particular file. If more people had that file, you would get those as alternate locations while downloading.

But it's not just that requeries are inefficient, they were abused by other vendors. The network load was unacceptable so they had to be removed.

As for blocking requeries, are they labeled as such? They must be else the Ultrapeers couldn't differentiate and block them.

LimeWire's requeries are labeled as such. But there are a couple of other ways to identify requeries, because they often contain the full filename or the hash of a file.

I've been a paying support of LW since day 1 (i'm on my 3rd subscription now) but this loss of functionality is really really pissing me off...

There is no other way, if some people can't play by the rules. There are clients out there, that send requeries every five seconds. And if you want to ban their requeries, you have to ban all requeries.

[sdsalsero]

Baby? Bathwater?!

Trap_jaw, I know you're not one of the developers but please post the following suggestions on the developer mailing-list:

1. The ability to continue searching automatically for files is a BASIC feature of any P2P app. If they're unable to design an intelligent requery function, they're going to lose their audience.

2. For every file left in requery mode, keep a record of every non-firewalled IP that has ever been reported with the file and periodically request the file anew from those IPs. This will not waste any Gnutella bandwidth since it's a direct request! Also, allow the enduser to right-click each file and edit the IP list, i.e., if you've been manually keeping a list. No, I don't think that the existence of dynamic IPs invalidates this function!

3. Keep track of leaf-node's requeries and throttle them. This could become part of the G2 protocol, e.g. "Do not requery more often than every 120 seconds."

4. If you're seeing a lot of unsuccessful requeries, why not try to improve them? If you assume that requeries are a required function, then making them more effective will reduce bandwidth. Requerying for the original search-term plus the filesize would be more likely to 'hit' than searching for the exact filename. Again, this could become part of the G2 protocol, e.g. "no 'filename' over 20 char and no filehashes in requeries."

5. Don't throttle or block manual requeries! (right-click on search tab, select Repeat Search)

(Thank you...)

[trap_jaw]

Baby? Bathwater?!

An ultrapeer can only handle so many queries per second. If a certain value of queries is reached queries will be dropped, meaning they won't return any results. Even worse, it meant that search results have to be dropped.

This discussion already happened on the development mailinglist. Until / unless some sort of distributed hash lookup table is created, there will be no more requeries with LimeWire.

1. The ability to continue searching automatically for files is a BASIC feature of any P2P app. If they're unable to design an intelligent requery function, they're going to lose their audience.

Napster didn't have it, Kazaa doesn't have it (you have to choose click 'find more sources' to do a requery - however some Kazaa-hacks do have it). Audiogalaxy didn't have it either.

2. For every file left in requery mode, keep a record of every non-firewalled IP that has ever been reported with the file and periodically request the file anew from those IPs. This will not waste any Gnutella bandwidth since it's a direct request! Also, allow the enduser to right-click each file and edit the IP list, i.e., if you've been manually keeping a list. No, I don't think that the existence of dynamic IPs invalidates this function!

LimeWire already doesd this. - LimeWire keeps track of all clients that ever downloaded the file and tells every other client requesting a file, who else to ask for the file. It's called download mesh. It works spectacularly with partial-file-sharing, a feature that will be implemented in the near future (probably not 3.0 but maybe in v3.1). Of course LimeWire could be more aggressive, e.g. retrying hosts that did not respond but that does not increase the performance very much.

3. Keep track of leaf-node's requeries and throttle them. This could become part of the G2 protocol, e.g. "Do not requery more often than every 120 seconds."

LimeWire does not plan on implementing G2. Besides any requery-frequency lower than 1-2 requeries per hour (for ten downloads it would take five to ten hours to requery all of them) has proven to be harmful. There are also some clients (QTrax, although it is not available anymore) that would frequently drop the ultrapeer connections to defeat any such protection. It's much safer to drop all requeries.

4. If you're seeing a lot of unsuccessful requeries, why not try to improve them? If you assume that requeries are a required function, then making them more effective will reduce bandwidth. Requerying for the original search-term plus the filesize would be more likely to 'hit' than searching for the exact filename. Again, this could become part of the G2 protocol, e.g. "no 'filename' over 20 char and no filehashes in requeries."

LimeWire was never requerying for the exact filename. However, the fuzzier the search the more unwanted results the more wasted bandwidth. Searching for the SHA-1 hash alone was about as exact as you can get, using broadcast search. It would return all results that could be used to resume a download and only those.

5. Don't throttle or block manual requeries! (right-click on search tab, select Repeat Search)

It's impossible to distinguish between manual and automatic requeries. 'Repeat Search' is de facto not requery but a completely new query for the same keywords.

[sdsalsero]

OK, I'm satisfied that this wasn't done lightly! And thanks for clarifying that a manual use of Repeat Search isn't being discarded.

[rockhaus]

In addition to my problems I've described elsewhere, I'm very dissapointed that in 2.9.8 a half-done file never resumes.

After uninstalling and reinstalling 2.9.8 (in order to troubleshoot), I'm finding no improvements to my problems, and the 10.2.5/2.9.8 combo genernally very inefficient.

If I restore a previous LMP version, I expect the same awesome experience as before, but if that's going to 'hurt others', I won't do it.

Thoughts?

-r