PCI: tegra194: Clean up the exit path for Endpoint mode

Clean up the exit path during .remove() and .shutdown() calls when in
Endpoint mode.

Link: https://lore.kernel.org/r/20220721142052.25971-14-vidyas@nvidia.com


Signed-off-by: Vidya Sagar <vidyas@nvidia.com>
Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>