net: prestera: span: do not unbind things things that were never bound

Fixes: 13defa27 ("net: marvell: prestera: Add matchall support")
Signed-off-by: Maksym Glubokiy <maksym.glubokiy@plvision.eu>
Link: https://lore.kernel.org/r/20221006190600.881740-1-maksym.glubokiy@plvision.eu


Signed-off-by: Jakub Kicinski <kuba@kernel.org>