When you say you are using VMWare, I presume you mean your machine is a mac & are running VMWare under OSX? Likewise I run 2 x Windows XP virtual machines but using Parallels (which is supposed to be faster & more efficient than VMWare. Though I'm still using v.4) I also run a windows 2000 though Parallels does not officially support it.
Does VMWare come with any windows security software? Parallels comes with Kaspersky security suite.
You need to ensure only one firewall is operating on each. And add LW as an exception. Try to ensure both LW's are connecting & listening using different ports to the other LW. Don't forget to add VMWare as an exception in your OSX firewall if you are using OSX & if your OSX firewall is turned on.
As for LW connecting, see
If LW 5 Will Not Connect.
Unlike LW 4, LW 5 does not have the Direct Connect option unless both users are on Friends list & sharing Private Shares. LW 4 uses ip addresses for direct connect. LW 5's friends option is different.
An alternative to direct connect/friends sharing is downloading via
magnet links;
How to use Bitzi Web Lookup ... & How to use Magnet Links & Direct Connect (Post #6 last paragraph has a little more on Magnet Links). This would be an ideal way to test things for you.
As far as the LW.props file goes, check that the white listed ip addresses include your main machine, router & virtual machines private
and public.
Check your VMWare is networked properly with the rest of your machine both in VMWare settings & in OSX's Network settings. If yes, arguably both VM's should run as though they are on same Local Network & thus get super speeds if both LW's are able to connect to each other. This might not be what you seek though, and as you suggested, simulate virtual users on different machines.
Strangely I have not attempted what you are doing since using Parallels. Many years ago I was unable to get my OSX LW to connect to my Virtual PC running win 2000. Something to do with my VPC settings I guess, however I was also using a router that had no UPnP support & could only port forward. Thus my windows LW was able to connect but always showed as having a firewall issue.