I don't know if it is really possible to remove it, but if you go to Start-->Control Panel-->Add/Remove Programs-->Add/Remove Windows Components you can uninstall it but everytime you try and play an unassigned file the WMP installer pops back up.

There probably is a way to totally remove it, but I don't know how.

This should help though if you figure that out, because uninstalling before deleting is the way to go.