Using Windows Remote Desktop Client with authentication

I've just wasted a morning trying to work out why every Linux or Android RDP client I tried would connect to a VirtualBox remote display quite happily, but the standard Windows client failed with 'Error Code 5' and dark mutterings about being low on virtual memory.

Not wanting the world to try out my VirtualBox image, I'd set 'external' authentication. So you need to supply a valid username and password to gain access. So naturally one selects the button to see all the connection options, and enters the server name and the user name.

Now the weird bit. You also need to tick the 'Allow me to save credentials' box. If you do, you will be prompted for a password and everything will work. If you don't, it won't ask you for a password, will try and authenticate without one, the authentication will fail, and you will get the above helpful (ha!) error message.