The first two tests I'd do:

1) Turn off the Firewalls altogether and see if the symptoms go away. If they do, you know it has to do with exceptions in your Firewall. If they don't, proceed to 2)

2) Check to see if you have active downloads or active torrents that are seeding. Residential uplinks often struggle with seeding torrents, and you will have significant issues with your Internet if you are actively seeding torrents you have downloaded.

Feel free to follow up and we'll help any way we can.