So... TIL today. You can enable root login in sshd_config. You can set a password for the root as well of course. However, you can't actually ssh into the box if you have any keys (or at least "some" keys) in your ssh-agent. So, by running ssh-agent bash and then logging in as root to target, I get asked for a password. If I alternatively have an ssh key you get the message that
Please login as the user "ubuntu" rather than the user "root".