libertas: fix signedness bug in lbs_auth_to_authtype()

Return type for lbs_auth_to_authtype() is changed from "u8" to
"int" to return negative error code correctly.
Also an error check is added in connect handler for invalid auth
type.

Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Amitkumar Karwar <akarwar@marvell.com>
Signed-off-by: Kiran Divekar <dkiran@marvell.com>
Signed-off-by: John W. Linville <linville@tuxdriver.com>