Way too many open files :-/
 

Hello,

When running phex-3.0.99 (self-compiled with LAN-Patch) I very soon get reports about too many open files.

lsof says:
I see this line up to thousand times, till it hits Linux default file Limit of 1024.

I see the open-files go up as soon as somebody downloads a file (the phex suffering the problem is uploading), a simple mp3-file uploaded by phex leaves about 50-100 file descriptors open (even if the download has been finished) - so after about 10mp3's phex runs out of file descriptors. Setting "File.OpenFilesLimit" to 500 also does not help. I have to admit that I only saw this error on Linux machines till now - so maybe theres a bug in the jdk. (Error happend on 1.5.0_10 & 1.6.0 final)
Any ideas what could be wrong, and can I help somehow to detect the problem?

Thank you for all your patience, lg Clemens

I attempted to fix this. Only one file is affected. I only did a quick test, so I'm not sure if this works 100% or introduces other problems.

The changes are in:
https://phex.svn.sourceforge.net/svn...hex/phex/trunk
or at:
http://phex.svn.sourceforge.net/view...1=3682&r2=3683

Gregor

Log Files...
 

I attached the output of Phex in Debug-Mode, maybe this helps. Its a bzip2'ed txt file, I just appended .txt to be able to upload it.

Hi again,

Thanks a lot for your efforts! The "too many open file" Problem has now gone away :-)

However now I see another problem. When downloading some files in the lan the downloading client shows "Waiting" and the Candidate state says "Http Error". However about 50% of all downloads complete successful.

The uploading phex-client throws the following exceptions (LogLevel=INFO):
Any ideas what could be wrong now?

Thank you many many times, Clemens

Difficult to say... it seems to be the download side which decides to closes the connection...

BTW: Using Log4J makes logging output much easier to configure, here is a little doc
http://www.phex.org/wiki/index.php/Logging_API

This is an example log4j.properties:

To help debug this better try to add the following line to your phexCorePrefs.properties:

Download.CandidateLogBufferSize = 1000000
Upload.UploadStateLogBufferSize = 1000000

This gives 1MB of memory data for storing candidate and upload log. You will see a Log Button on the candidate and upload toolbar. Pressing it shows the log of the candidate/upload.

Thanks for the logging tips.

You're right, it seems the client receives garbage and then resets the connection:


or later:
lg Clemens

Seems to be the remaining of the buffer. I missed to limit it.
The fix attempt ;) is in UploadEngine.java again.

Thanks for the fix.

I now have another problem:

1.) First everything works as it should, but after a few files have been downloaded speed goes to ~10.0kb/s, strange is that in the "Upload"-Tag only 16kb entries are shown.
I attached a screenshot to illustrate that behaviour.
The slow downloading starts to happen as soon as the 16k uploads begin. I also saw 20kB, 40Kb and 60Kb uploads, but all in this category as soon as the slowdown occurs.

2.) Only a few files come to "Completed!" status, others stuck at "Downloading 100%" with all bytes downloaded. This has been so since the biginning, does not bother me if files are really completly downloaded.

The most strange thing is that from machine A-> machine B this never happens, all downloads complete and are always fast. So this happens just from B->A, no firewall.

lg Clemens

Edit: Attached file deleted (content violates forum rules)

Its seems out of what ever reason the download connection between the two is interrupted and the download restarts. Since the segment size is 16K this happens every 16K. Raising the default segment size and target time might help.

You don't need small segments on LAN anyway. Download speed should be so high that you can easily use values above 1MB, and since you most likely only have one source segmentation, which enables swarming, is not necessary anyway.

The other question is why Phex looses connection, or thinks at some point that it cant use persistent keep-alive connections.

Thanks a lot for your reply, I now changed the minimum segment size to 1mb. I'll test it as soon as I've another computer available (arround 17:00 today).

The other thing I wonder why so few downloads reach "Completed" status and why so many hang at "Downloading 100%". Does this have any impact on phex or on the files, or is it safe to ignore it?

Thanks for your patience, lg Clemens

Hi again,

Would that explain why the downloading client behaves so strange:

The machine is running on jdk-1.5.0_10 with the default 64mb heap maximum.

I hope you can help me this last time, sorry for bothering you all the time again and again :-/

lg Clemens

There seems to be a new problem in downloads.

In SVN I have many downloads which start and the revert all sources to "loading", but download at 0kB/s only.

It happens to shareazas as well as LimeWires, especially when the download has many sources.

Download-Log: