Here's what I found out by disabling all plugins and activating one by one. A plugin of my own creation had some whitespace outside of the <?php ... ?>.
After fixing that plugin all works as expected. Additionally, when I activated that plugin I got the following error.
The plugin generated 1 characters of unexpected output during activation. If you notice “headers already sent” messages, problems with syndication feeds or other issues, try deactivating or removing this plugin.
Strange as I don't recall seeing this before.