RDMA/hns: Support configuring doorbell mode of RQ and CQ

mainline inclusion
from mainline-v5.13-rc1
commit cf8cd4cc
category: bugfix
bugzilla: 174002
CVE:NA

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cf8cd4ccb269dbd57c3792799d0e5251547d6734

----------------------------------------------------------------------

HIP08 supports both normal and record doorbell mode for RQ and CQ, SQ
record doorbell for userspace is also supported by the software for
flushing CQE process. As now the capability of HIP08 are exposed to the
user and are configurable, the support of normal doorbell should be added
back.

Note that, if switching to normal doorbell, the kernel will report "flush
cqe is unsupported" if modify qp to error status as the flush is based on
record doorbell.

Link: https://lore.kernel.org/r/1616840738-7866-2-git-send-email-liweihang@huawei.com


Signed-off-by: Yixian Liu <liuyixian@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: wangsirong <wangsirong@huawei.com>
Reviewed-by: ChunZhi Hu <huchunzhi@huawei.com>
Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com>