Commit 4c65fed8 authored by Prasad J Pandit's avatar Prasad J Pandit Committed by Peter Maydell
Browse files

ui: vnc: avoid floating point exception 



While sending 'SetPixelFormat' messages to a VNC server,
the client could set the 'red-max', 'green-max' and 'blue-max'
values to be zero. This leads to a floating point exception in
write_png_palette while doing frame buffer updates.

Reported-by: default avatarLian Yihan <lianyihan@360.cn>
Signed-off-by: default avatarPrasad J Pandit <pjp@fedoraproject.org>
Reviewed-by: default avatarGerd Hoffmann <kraxel@redhat.com>
Signed-off-by: default avatarPeter Maydell <peter.maydell@linaro.org>
parent efdeb96c

ui/vnc.c

+3 −3
Original line number Diff line number Diff line
@@ -2198,15 +2198,15 @@ static void set_pixel_format(VncState *vs, 
        return;

    }



    vs->client_pf.rmax = red_max;

    vs->client_pf.rmax = red_max ? red_max : 0xFF;

    vs->client_pf.rbits = hweight_long(red_max);

    vs->client_pf.rshift = red_shift;

    vs->client_pf.rmask = red_max << red_shift;

    vs->client_pf.gmax = green_max;

    vs->client_pf.gmax = green_max ? green_max : 0xFF;

    vs->client_pf.gbits = hweight_long(green_max);

    vs->client_pf.gshift = green_shift;

    vs->client_pf.gmask = green_max << green_shift;

    vs->client_pf.bmax = blue_max;

    vs->client_pf.bmax = blue_max ? blue_max : 0xFF;

    vs->client_pf.bbits = hweight_long(blue_max);

    vs->client_pf.bshift = blue_shift;

    vs->client_pf.bmask = blue_max << blue_shift;