I couldn't say for sure, I would've thought that 5-8pm is still peak time.
Anyway, see how you go with it all... hopefully you won't lose your port any more & your speeds will be ok!