Large File Distribution Research Using Limewire

Large File Distribution Research Using Limewire

Objective:

Optimize Large File Distribution Through File Splitting.

Problem:

Distributing and acquiring large files using p2p remains problematic for the following primary reasons:

1. Upload speeds tend to be slow usually between 128Kbps - 385Kbps This is the average upload speed for ADSL and Cable Modem users.

2. upload slots are tied up for long periods of time with users downloading large files. Some upload slots are under utilized as bandwidth limits are hit before upload slots are filled.

3. Multiple users cause throughput delays for users downloading smaller files.


Desired Results:

Based on the above conditions we believe that by optimizing hug files for low bandwidth connections we would be able to better utilize outbound bandwidth and support more users. We would also like to see what the performance enhancements of multiple users sharing portions of the content to be distributed.

Methodology:

1. We take a 256.5MB avi (videos showing the join process in action (no-sound)) this file is roughly about 2.0Gbit
2. Using Hj-Split (www.freebyte.com/hjsplit) we create a checksum for the file the current checksum is #512270061
3. Next we partition or Split the file into 35KB blocks total = 7330 blocks.

Phase -1: Setup

We will temporarily suspend sharing while we conduct this test.
We will connect 4 servents to different parts of the network so that participants can find the test data.
We will use a default configuration of limewire and since we are the host site will will work on the initial tuning of
upload slots etc.

Phase -2: Distribution from Primary Site to hub sites:

Each participant should connect to the gnutella network and search for our test file set your content filters to Video and enter in the name box: P2P - LFDUFB Research then click search. Each participant should download a random range of the file and share it.

Each participant should download and share 1000 blocks of the file so the valid ranges are: filename + block#

The alphabets are used to denote the distribution ranges
----------------------
a = 1 - 1000
b = 1001 - 2000
c = 2001 - 3000
d = 3001 - 4000
e = 4001 - 5000
f = 5001 - 6000
g = 6001 - 7000
h = 7001 - 7330

we are hoping that as participants download their respective blocks of data and share them, the time it takes to download a specific block should begin to decrease as most people have the ability to pull data from the net faster than they can send it. Also each client will be sending only blocks of 35KBs of data.

We have established a distribution pattern for downloading the files: Each participant should select a distribution pattern and download / share each range until file assembly is possible. (meaning you have all 7330 blocks.)

Distribution Patterns:

1 = (abcedfgh) => Start from the top and move down
2 = (hgfdecba) => Start from the bottom and move up
3 = (dcbaefgh) => Start from the middle and move out

During the distribution stage each user should see an increase in the number of uploads of the data blocks as new participants join. The project will come to its completion when the identified participants have all of the blocks and can validate the video checksum, assemble the video file.


Participant Requirements:

OS : Open - If you can run limewire you can participate.
Connection : ADSL or Better (Sorry Modem Users) - These connections are likely to be unavailable most of the time.
Gnutella Client : Limewire (version is not important for the test)
Upload Avail. : Good upload availability (your connection should be free of upload problems--it doesn't make sense for you to participate if your configuration will not support uploading.)
Machine Avail. : Your machine should be available for signifant portions of the test preferrably 24x7 (min of 4 - 6 hours)
Skills : You should be competent with using limewire and your computer!

Participant Instructions:

1. Prepare your machine so that you are not sharing any other files (Move the files you are sharing to another directory during the testing period. If people are uploading your gear it could throw the results of our test. When you start limewire you should be sharing 1 files. filename = p2ptest.txt you will create this file using a a standard text editor the file should contain 1 line of text = "p2ptest"

2. Check your configuration to verify that you don't have a high value set for free loaders people who are sharing one file should be able to download from you.

3. goto : www.freebyte.com/hjsplit (download the version thats appropriate for your OS--you will need it for validating the checksum and joining the video file.)

finally so that we can keep up with who's actually partipating in this research you should send an e-mail to research@leeware.com with the following
subject: LFDUP2P

This message should contain the following information:

Operating System:
Connection Speed: (uploads/download)
Machine Avail in hours: (for example : 6 hours)
Gnutella Client:
Distribution Pattern: (what distribution pattern will you use = 1 2 or 3)

If you have any questions please feel free to post them here on the forum or send e-mail to research@leeware.com

Thanks in advance for your participation.