I noticed that I had a dwm.exe process running in my dev VM that had 230Mb which is not good.
After a quick search I found that it belonged to the Windows Desktop Manager service, which is responsible for the fancy animations in Windows 7 and can safely be disabled (using the Services):
Here is the memory drop after I disabled/stopped dwm.exe: