rsa: Add debugging for failure cases

Add some more debugging to make it easier to see what is being tried and
what fails. Fix a few comment styles while here.

Signed-off-by: Simon Glass <sjg@chromium.org>