pinctrl-sunxi: don't call pinctrl_gpio_direction()

Set the direction directly without calling pinctrl_gpio_direction().
This avoids the mutex_lock() calls in that function, which would
invalid the can_sleep = false.

Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Link: https://lore.kernel.org/r/20211206131648.1521868-4-hverkuil-cisco@xs4all.nl


Signed-off-by: Linus Walleij <linus.walleij@linaro.org>