Although I have specified the default landing page for unauthorized users there are still top- and side-bar menus there that reveal some of my content!

Indeed and this might be a problem when you want to keep your nav:top and nav:side secret. Here is what to do:

• create a landing page within a new category, e.g. unauthorized:start
• go to the Site Manager » Appearance » Navigation element
• clear navigational elements for category unauthorized.
• go to Site Manager » Public of private and set unauthorized:start as a landing page.

Update: now there is an option to disable the nav elements in your Site Manager » Public or Private. However the method described above is visually better (often).