greybus: connection: Destroy wq on failure

Need to destroy the wq created earlier, do it.

Fixes: d0f1778a6b67 ("greybus/connection: add a timestamp kfifo to track connection handoff")
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>