lib/rsa: Sanity check db_len

When building with GCC 12:

../tools/../lib/rsa/rsa-verify.c:275:11: warning: ‘*db’ may be used uninitialized [-Wmaybe-uninitialized]
  275 |         db[0] &= 0xff >> leftmost_bits;
      |         ~~^~~

If msg_len or hash_len were nonsense values, db_len would end up invalid
and the rest of the function wil not work, so detect this case and
return early. If this was host code we could assert, but as this is
target code print an error and return.

Fixes: 061daa0b ("rsa: add support of padding pss")
Signed-off-by: Joel Stanley <joel@jms.id.au>