drivers: remove Blackfin specific drivers

These drivers have no user since commit ea3310e8 ("Blackfin:
Remove").

Signed-off-by: Masahiro Yamada <yamada.masahiro@socionext.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
Acked-by: Michal Simek <michal.simek@xilinx.com>
Acked-by: Jaehoon Chung <jh80.chung@samsung.com>