ASoC: topology: Properly unregister DAI on removal

DAIs need to be removed when topology unload function is called (usually
done when component is being removed). We can't do this when device is
being removed, as structures we operate on when removing DAI can already
be freed.

Fixes: 6ae4902f ("ASoC: soc-topology: use devm_snd_soc_register_dai()")
Signed-off-by: Amadeusz Sławiński <amadeuszx.slawinski@linux.intel.com>
Tested-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Link: https://lore.kernel.org/r/20210120152846.1703655-2-amadeuszx.slawinski@linux.intel.com


Signed-off-by: Mark Brown <broonie@kernel.org>