NFS: Don't set NFS_INO_INVALID_XATTR if there is no xattr cache

stable inclusion
from stable-v5.10.102
commit 521dcc107e3923553c9c09021d2202460cdf3667
bugzilla: https://gitee.com/openeuler/kernel/issues/I567K6

Reference: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=521dcc107e3923553c9c09021d2202460cdf3667



--------------------------------

[ Upstream commit 848fdd62 ]

Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
Signed-off-by: Anna Schumaker <Anna.Schumaker@Netapp.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
Signed-off-by: Yu Liao <liaoyu15@huawei.com>
Reviewed-by: Wei Li <liwei391@huawei.com>
Signed-off-by: Zheng Zengkai <zhengzengkai@huawei.com>