drm/radeon/kms: add bounds checking to avivo pll algo

Prevent divider overflow.
Fixes:
https://bugzilla.kernel.org/show_bug.cgi?id=28932

Signed-off-by: Alex Deucher <alexdeucher@gmail.com>
Cc: stable@kernel.org
Signed-off-by: Dave Airlie <airlied@redhat.com>