cmd_mem: cleanups, catch bad usage

Currently, memtest will silently accept bad data. Perform error
checking on user intput.

Signed-off-by: Pavel Machek <pavel@denx.de>