drm/nouveau: fix regression on agp boards

Extends the fix in f2f9a2cb to also
workaround permission issues noticed by people using AGP systems.

Cc: stable@vger.kernel.org # 3.16: f2f9a2cb

: drm/nouveau: fix regression
Cc: stable@vger.kernel.org # 3.16+
Signed-off-by: Ben Skeggs <bskeggs@redhat.com>