wifi: mac80211: tighten locking check

When we remove a link that doesn't have a channel context,
we don't really need the local->mtx locking. Tighten the
check here.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>