JUnit 4.7 - TemporaryFolder

JUnit 4.7 adds a cool looking new feature - Rules. These should get rid of the need to write a new Runner every time you want something nifty to happen.
One of the rules is TemporaryFolder. This automatically creates and destroys a temporary folder at the start and end of a test. A neat extra and something I'd already thrown together for my own use. However my version did one thing different - it destroyed on start up not shut down. Why? The same reason my database tests do this. When testing I can quickly go back to any test and check the files it created and attempt to work out why a test is failing.