NFS: Ignore the directory size when marking for revalidation

If we want to revalidate the directory, then just mark the change
attribute as invalid.

Fixes: 13c0b082

 ("NFS: Replace use of NFS_INO_REVAL_PAGECACHE when checking cache validity")
Signed-off-by: Trond Myklebust <trond.myklebust@hammerspace.com>
Tested-by: Benjamin Coddington <bcodding@redhat.com>
Reviewed-by: Benjamin Coddington <bcodding@redhat.com>