I've spent longer than I should have figuring out how to do this. Which means: post about it to make it seem like I've done some work. Hopefully this saves someone some time.
To make a transparent favicon.ico file using Paint.NET, download the IcoCur.dll plugin from Evan Olds (thank you!) and save it in the FileTypes folder in the Paint.NET directory.
Then, create a 16x16 pixel file, with a background whose alpha channel is 0. Save the file, and you can now save a .ico. I've included only the 16x16, 32 bit image to make the favicon as small as possible.
Now, the little icon thingy in the top of your browser will look nice independent of the browser chrome.