ath11k: remove unnecessary casts to u32

These casts are not needed. No changes in functionality.

Tested-on: QCA6390 hw2.0 PCI WLAN.HST.1.0.1-01740-QCAHSTSWPLZ_V2_TO_X86-1

Signed-off-by: Kalle Valo <kvalo@codeaurora.org>
Link: https://lore.kernel.org/r/1601544890-13450-9-git-send-email-kvalo@codeaurora.org