The Linksys WRT54G was a 'lemon' range of models Linksys released. They all had firmware problems Linksys never bothered to fix. Two answers. One is to fix the firmware which is not recommended for the average user. The other is to do port forwarding which will help minimise the problems.
Do not try using same port for two programs or computers. You will definitely need to use a 2nd port number if one is already being used for another computer or another program.
I am aware there have been many problems with Charter in recent times, not sure whether this is affecting you though.
One last note, wireless connection is convenient but comes at a cost. Wireless connections tend to be unreliable, the bandwidth will vary. And yes this is a well known fact in torrent circles (which is where I found it out.) For some users the high number of connections will result in the router dropping connection from time to time, possibly even requiring a rebooting of the router in some cases (this is at least true in torrent use for some wireless users.)
Since you are a wireless router user, I would recommend periodically doing this:
Tip: Purge the Downloads Queue! (click on blue link)

Especially since should you accidentally choose fake downloads, they will cause havoc with attempting to connect all the time!
To ease on this, another thing you can do is connect manually. ie: go to Tools > Options > Connections, disable Connect on Startup. Then when starting up LW, let it settle for a few minutes, then go to File menu -> Connect. This will stop your incomplete files attempting to connect/find sources at the same time as your LW is trying to connect.