What model router do you have? ( d-link, linksys, netgear, other?)

UPnP is supported by all the major router vendors, just depends if it's enabled or not by default -- my guess is you do have support. However you'll also need to be using wXP to take advantage of UPnP at this time.

So to check, you'll need to connect to your router's configuration, then likely navigate to the advanced or administration tab depending on your model ( or just check out your routers' documentation).
To connect to your router's configuration web panel, in a browser enter http://192.168.1.1 ( where 192.168.1.1 is your gateway). You'll need to know the default user/password if you havent set it already.

To double check what your gate way is , go here

I can offer more help once i know what model you have , but the above should get you started ;-)