soc: qcom: apr: Drop redundant check in .remove()

The remove callback is only called by the driver core if there is a
driver to unbind, so there is no need to check dev->driver to be
non-NULL.

Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Bjorn Andersson <bjorn.andersson@linaro.org>
Link: https://lore.kernel.org/r/20220618203913.35785-1-u.kleine-koenig@pengutronix.de