I'll let Adam explain more, but to explain fully requires a bit of background ...

Handling a connection correctly requires reading data from the network and writing it to the network. Writing is dependent on how much data your computer's TCP buffers allow to be stored and how 'clogged' the networks are. If something is going slow, it might take awhile before a message can be written. Reading is dependent on the other side sending you information. The traditional 'blocking IO' means that when you make a call to 'write' to a stream, the logic simply stalls at that point until the message is fully written. With reading, you have to always try and read, and when a message is available for reading, the 'read' call will stop blocking and actually read a message. What this means for an application like LimeWire is that each connection requires two threads -- one for reading and one for writing. This is necessary so that one connection can't bring down the other connections. (The failure of Gnutella way back when was because many clients didn't correctly implement a two-thread-per-connection model, so slow modem connections could stall the flow of the entire network.)

What NIO "new IO" or sometimes "non-blocking IO" allows is a much more intelligent way of writing/reading. A "Selector" is used to keep track of connections that are available for reading or writing. This Selector blocks, waiting for something to become available. When it is, it notifies the appropriate components and a write or read is performed. If a message was only partially read or partially written, the Selector goes back to sleep, waiting for another notification that resources are available.

Ultimately, NIO will allow a single thread to read/write to all connections on the network, as opposed to now where two threads are required per connection (which, on ultrapeers, can cause 164 threads to be active simply for the network to function). This is a Good Thing (tm) because more threads require more context-switching, more synchronization, and each thread takes up system resources (causing the oft-seen 'too many open files' error).