I've spent longer than I should have figuring out how to do this. Which means: post about it to make it seem like I've done some work. Hopefully this saves someone some time.
Then, create a 16x16 pixel file, with a background whose alpha channel is 0. Save the file, and you can now save a .ico. I've included only the 16x16, 32 bit image to make the favicon as small as possible.
Now, the little icon thingy in the top of your browser will look nice independent of the browser chrome.