The files stay on your hard disk.
They are only removed, if you can call it that, if and when they are overwritten by another file.
Windows does not delete anything. With the recycle bin, even if you click on empty contents, it does not delete anything.
A file simply has the first character of its name changed so the file appears, hahaha, to be gone.
Even the process of overwriting doesn't remove the data. Defragmentation does not remove any data either, but it does increase the probability of a file being overwritten earlier rather than later.
A format, even a full format, will not really remove the data. It will still be readable by people with the right equipment and a very very very clean room where they open the drive.
Two ways to maybe clean a drive for your legal protection- Open it and smash the actual individual disks- to pieces! or, maybe, if you like to trust in things to keep you outside as opposed to inside, you can use a program that writes zeroes to all sectors.
This takes a long time and unfortunately many of the products sold for doing this are complete cons.
The program System Eraser from Webroot, the same people who produce Window Washer, seems to be very effective. Of course, there are others as well. Also, if you do this process, you want to do it a lot more than once!