Your download speed is determined by numerous factors. One of these is, as you guessed, your download speed. But also, if you are downloading a file from someone who cannot upload as fast as you can download, then their speed shall determine your speed.

Also, your speed can be sapped a bit if you act as a UP. Also, when downloading, if someone is very far away in the network, it could be slower as well.

Also, if someone is listed a T3, but is uploading 20 files, their speed will be divided among 20 people, greatly reducing everyone's dl speed.