Gnutella Forums  

Go Back   Gnutella Forums > Gnutella News and Gnutelliums Forums > General Gnutella Development Discussion
Register FAQ The Twelve Commandments Members List Calendar Arcade Find the Best VPN Search Today's Posts Mark Forums Read

General Gnutella Development Discussion For general discussion about Gnutella development.


Welcome To Gnutella Forums

You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content, fun aspects such as the image caption contest and play in the arcade, and access many other special features after your registration and email confirmation. Registration is fast, simple and absolutely free so please, join our community today! (click here) (Note: we use Yandex mail server so make sure yandex is not on your email filter or blocklist.)

If you have any problems with the Gnutella Forum registration process or your Gnutella Forum account login, please contact us (this is not for program use questions.) Your email address must be legitimate and verified before becoming a full member of the forums. Please be sure to disable any spam filters you may have for our website, so that email messages can reach you.
Note: Any other issue with registration, etc., send a Personal Message (PM) to one of the active Administrators: Lord of the Rings or Birdy.

Once registered but before posting, members MUST READ the FORUM RULES (click here) and members should include System details - help us to help you (click on blue link) in their posts if their problem relates to using the program. Whilst forum helpers are happy to help where they can, without these system details your post might be ignored. And wise to read How to create a New Thread

Thank you

If you are a Spammer click here.
This is not a business advertising forum, all member profiles with business advertising will be banned, all their posts removed. Spamming is illegal in many countries of the world. Guests and search engines cannot view member profiles.



           Deutsch?              Español?                  Français?                   Nederlands?
   Hilfe in Deutsch,   Ayuda en español,   Aide en français et LimeWire en françaisHulp in het Nederlands

Forum Rules

Support Forums

Before you post to one of the specific Client Help and Support Conferences in Gnutella Client Forums please look through other threads and Stickies that may answer your questions. Most problems are not new. The Search function is most useful. Also the red Stickies have answers to the most commonly asked questions. (over 90 percent).
If your problem is not resolved by a search of the forums, please take the next step and post in the appropriate forum. There are many members who will be glad to help.
If you are new to the world of file sharing please do not be shy! Everyone was ‘new’ when they first started.

When posting, please include details for:
Your Operating System ....... Your version of your Gnutella Client (* this is important for helping solve problems) ....... Your Internet connection (56K, Cable, DSL) ....... The exact error message, if one pops up
Any other relevant information that you think may help ....... Try to make your post descriptive, specific, and clear so members can quickly and efficiently help you. To aid helpers in solving download/upload problems, LimeWire and Frostwire users must specify whether they are downloading a torrent file or a file from the Gnutella network.
Members need to supply these details >>> System details - help us to help you (click on blue link)


Moderators

There are senior members on the forums who serve as Moderators. These volunteers keep the board organized and moving.
Moderators are authorized to: (in order of increasing severity)
Move posts to the correct forums. Many times, members post in the wrong forum. These off-topic posts may impede the normal operation of the forum.
Edit posts. Moderators will edit posts that are offensive or break any of the House Rules.
Delete posts. Posts that cannot be edited to comply with the House Rules will be deleted.
Restrict members. This is one of the last punishments before a member is banned. Restrictions may include placing all new posts in a moderation queue or temporarily banning the offender.
Ban members. The most severe punishment. Three or more moderators or administrators must agree to the ban for this action to occur. Banning is reserved for very severe offenses and members who, after many warnings, fail to comply with the House Rules. Banning is permanent. Bans cannot be removed by the moderators and probably won't be removed by the administration.


The Rules

1. Warez, copyright violation, or any other illegal activity may NOT be linked or expressed in any form. Topics discussing techniques for violating these laws and messages containing locations of web sites or other servers hosting illegal content will be silently removed. Multiple offenses will result in consequences. File names are not required to discuss your issues. If filenames are copyright then do not belong on these forums & will be edited out or post removed. Picture sample attachments in posts must not include copyright infringement.

2. Spamming and excessive advertising will not be tolerated. Commercial advertising is not allowed in any form, including using in signatures.

3. There will be no excessive use of profanity in any forum.

4. There will be no racial, ethnic, or gender based insults, or any other personal attacks.

5. Pictures may be attached to posts and signatures if they are not sexually explicit or offensive. Picture sample attachments in posts must not include copyright infringement.

6. Remember to post in the correct forum. Take your time to look at other threads and see where your post will go. If your post is placed in the wrong forum it will be moved by a moderator. There are specific Gnutella Client sections for LimeWire, Phex, FrostWire, BearShare, Gnucleus, Morpheus, and many more. Please choose the correct section for your problem.

7. If you see a post in the wrong forum or in violation of the House Rules, please contact a moderator via Private Message or the "Report this post to a moderator" link at the bottom of every post. Please do not respond directly to the member - a moderator will do what is required.

8. Any impersonation of a forum member in any mode of communication is strictly prohibited and will result in banning.

9. Multiple copies of the same post will not be tolerated. Post your question, comment, or complaint only once. There is no need to express yourself more than once. Duplicate posts will be deleted with little or no warning. Keep in mind a forum censor may temporarily automatically hold up your post, if you do not see your post, do not post again, it will be dealt with by a moderator within a reasonable time. Authors of multiple copies of same post may be dealt with by moderators within their discrete judgment at the time which may result in warning or infraction points, depending on severity as adjudged by the moderators online.

10. Posts should have descriptive topics. Vague titles such as "Help!", "Why?", and the like may not get enough attention to the contents.

11. Do not divulge anyone's personal information in the forum, not even your own. This includes e-mail addresses, IP addresses, age, house address, and any other distinguishing information. Don´t use eMail addresses in your nick. Reiterating, do not post your email address in posts. This is for your own protection.

12. Signatures may be used as long as they are not offensive or sexually explicit or used for commercial advertising. Commercial weblinks cannot be used under any circumstances and will result in an immediate ban.

13. Dual accounts are not allowed. Cannot explain this more simply. Attempts to set up dual accounts will most likely result in a banning of all forum accounts.

14. Video links may only be posted after you have a tally of two forum posts. Video link posting with less than a 2 post tally are considered as spam. Video link posting with less than a 2 post tally are considered as spam.

15. Failure to show that you have read the forum rules may result in forum rules breach infraction points or warnings awarded against you which may later total up to an automatic temporary or permanent ban. Supplying system details is a prerequisite in most cases, particularly with connection or installation issues.

Violation of any of these rules will bring consequences, determined on a case-by-case basis.


Thank You! Thanks for taking the time to read these forum guidelines. We hope your visit is helpful and mutually beneficial to the entire community.


Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old January 5th, 2002
Unregistered
Guest
 
Posts: n/a
Default Proposal for development of Gnutella

When a servant process a query and return the filename and filesize available. It should also return a hash code.

This would download from multiples sources even when the users rename the original file.

A good hash (20bytes) with a checking of the filesize should avoid "false duplicates"

Marc.
marc@szs.ca
Reply With Quote
  #2 (permalink)  
Old January 5th, 2002
Moak's Avatar
Guest
 
Join Date: September 7th, 2001
Location: Europe
Posts: 816
Moak is flying high
Default

yep, I suggest/vote for it too.

There is allready a well documented proposal, named 'HUGE' [1]. From their "Motivation & Goals":

o Folding together the display of query results which represent the exact same file -- even if those identical files have different filenames.

o Parallel downloading from multiple sources ("swarming") with final assurance that the complete file assembled matches the remote source files.

o Safe "resume from alternate location" functionality, again with final assurance of file integrity.

o Cross-indexing GnutellaNet content against external catalogs (e.g. Bitzi) or foreign P2P systems (e.g. FastTrack, OpenCola, MojoNation, Freenet, etc.)

[1] "HUGE" - http://groups.yahoo.com/group/the_gd...roposals/HUGE/ (Yahoo account required)
Reply With Quote
  #3 (permalink)  
Old January 6th, 2002
Unregistered
Guest
 
Posts: n/a
Default Could be simple.

The HUGE thing look complicate for no reason. The risk of making an error on dups files is close to impossible (1 / Billions * Billions) with an 160bits hash and filesize check. You simply do the download and check that the receive file match the hash...

And it's quite simple to code a component that can download from mutliple sources (swarm). You simply test the servers for resume capability, split the files to download in blocks, and create threads that request theses blocks on differents server, you can even create mutliple threads (connections) per server.

In order to improve download/connection speed each client should have a list of other client that have the same file and reply not only with there IP but with the IP of others that can provide the same file. This could be done if Hub (supernodes) are inserted in the network. They could scan for dups files!

I already have program a swarm component in Delphi and it's working well. I will now work on it to add on the fly add/remove of new download sources.

If any one want to work on it to let me know i will send you the sources. It's use Indy for TCP access.

Marc.
marc@szs.ca
Reply With Quote
  #4 (permalink)  
Old January 6th, 2002
Moak's Avatar
Guest
 
Join Date: September 7th, 2001
Location: Europe
Posts: 816
Moak is flying high
Default

I thought HUGE is simple and flexible?

It does explain a lot of basics and also details how to include hashs into binary Gnutella messages (did you recognize that you have to encode 0x00, also compatibility with other/older clients is guaranteed)? If you think it can be done easier, write a paper... I prefer easy solutions. :-)

PS: What you said about swarming and a list of alternatives downloads. Yes, this is another advantage when we have hashs and Queryhit-caches. I'm a big fan of superpeers, hashs, swarming and multisegmented downloading. :-)
Reply With Quote
  #5 (permalink)  
Old January 6th, 2002
veniamin's Avatar
Devotee
 
Join Date: December 17th, 2001
Posts: 24
veniamin is flying high
Default

I am not sure but i think a CRC could do the job. For each file in a query hit we can put its CRC between the two nulls like Gnotella does for the MP3 files.
Reply With Quote
  #6 (permalink)  
Old January 6th, 2002
Moak's Avatar
Guest
 
Join Date: September 7th, 2001
Location: Europe
Posts: 816
Moak is flying high
Default

You can do that with HUGE. It also describes the encoding between the two nulls, then the new GET request. I think it preferes SHA1 for the hash, but which you use is flexible, CRC, MD5...

The question I have, which is the best algorithm? Can someone give a summary/overview? Hmm, it should be unique enough within a typical horizon (high security is not the topic), small in size (to keep broadcast traffic low), fast to calculate.

Last edited by Moak; January 6th, 2002 at 01:34 PM.
Reply With Quote
  #7 (permalink)  
Old January 6th, 2002
Unregistered
Guest
 
Posts: n/a
Talking Follow up

About Huge, when i said Huge look complicate i mean from what you tell me it's more about verification of data integrity.

I prefer less verification and better speed (smaller protocol)
as long as verification is good enough.

About CRC it's really not a good idea to use CRC16 or CRC32 the last one give 4 Billions values, that not enough you could get wrong duplicates files. SHA1 use a 20 bytes (160bits) this give a lot more possibility. To give an idea it would be arround,
4Billions * 4Billions * 4Billions, ... You get the point, with this amount of possibility you reduce the change of making false duplicates.

SHA1 speed is fast enough +1MB/Sec but it's not that important, a client program could generate all the sha1 at startup and cache
this information in memory 1000 files would required only 20KB
of memory. Doing a hash for each query would not be a good idea...

Marc
Reply With Quote
  #8 (permalink)  
Old January 6th, 2002
Moak's Avatar
Guest
 
Join Date: September 7th, 2001
Location: Europe
Posts: 816
Moak is flying high
Default hmm

What do you mean with 'verification'?
The HUGE goals describe exactly what we really need today: a) efficient multisegmented downloading/caching (grouping same files together from Queryhits for parallel downloads or Query caches) - b) efficient automatic requerying (finding alternative download loactions).

I agree, the protocol should be as small as possible.
While you agree with SHA1 (I still have no clue about advantages/disadvantages in CRCxx, SHA1, MD5, TigerTree etc), what could be done better as described in the HUGE paper? I think HUGE is pretty simple. It describes hash positioning in Query/Queryhits and necesarry HTTP headers. Then it encodes the hash to make it fit into Gnutella traffic (null inside the hash must encoded!) and also into HTTP traffic. Example, the well known 'GnutellaProtocol04.pdf' becomes 'urn:sha1:PLSTHIPQGSSZTS5FJUPAKUZWUGYQYPFB'.
Perhaps you don't agree with BAS32 encoding of the hash, what could be done better?

CU, Moak
Reply With Quote
  #9 (permalink)  
Old January 7th, 2002
Unregistered
Guest
 
Posts: n/a
Lightbulb Follow up

I know nothing about Huge i simply did get this from your previous post:

> Safe "resume from alternate location" functionality, again with final assurance of file integrity.

For me "final assurance" mean once download is complete you must do some kind of block check on the original source, with multiple CRC to verify that all the block receive match the original file.

This is what i call "final assurance". Like i said i don't know Huge, what i'm proposing is "not final assurance". Only perform a Sha1 and download the file from all matching sources. Without performing checking at the end of transfer. If HUGE is doing this then they can't tell "final assurance of file integrity" but it's exactly what i want to to.

To have "final assurance" would use to much bandwidth, performance would be better with a small risk for corrupted file, if it's in the range or 1/10000000000000000 sound ok to me.

I will try to take time and check the HUGE principle.

CRC vs SHA1: CRC is as good as SHA1 for randomly select a number corresponding to a data. But SHA1 add security, SHA1 was built so that it's impossible to recreate any original data from the hash key (good for password storing). And of course it generate larger number since it's a 20bytes key vs 4 bytes for CRC32

Marc.
Reply With Quote
  #10 (permalink)  
Old January 7th, 2002
Enthusiast
 
Join Date: January 1st, 2002
Posts: 34
Tamama is flying high
Default Base32?

The only thing i find somewhat weird at HUGE is that the sha1 is Base32 encoded. This means only 5 bits of a 8 bits byte are used. Just doesnt make sence... oh well

The GET request is somewhat strange as well... a simple:

GET urn:sha1:452626526(more of this ****)SDGERT GNUTELLA/0.4

would work just as well...

Some thoughts..
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Gnutella Protocoll v0.7 Proposal Moak General Gnutella Development Discussion 41 August 17th, 2002 10:55 AM
gnutella development plans Iamnacho General Gnutella Development Discussion 11 March 9th, 2002 06:21 PM
My Proposal for XoloX!!! Unregistered User Experience 1 February 6th, 2002 08:11 AM
Xolox and Gnutella development Moak Rants 6 November 25th, 2001 06:05 AM
---a Radical Proposal--- Unregistered General Gnutella / Gnutella Network Discussion 0 September 21st, 2001 12:08 PM


All times are GMT -7. The time now is 02:03 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2024, vBulletin Solutions, Inc.
SEO by vBSEO 3.6.0 ©2011, Crawlability, Inc.

Copyright © 2020 Gnutella Forums.
All Rights Reserved.