Commit b5299153 authored May 28, 2014 by Gonglei Committed by Gerd Hoffmann Jun 02, 2014

vnc-enc-tight: Fix divide-by-zero in tight_detect_smooth_image{16,24,32}



Spotted by Coverity:

(1) Event assignment:  Assigning: "pixels" = "0".
(2) Event cond_true:  Condition "y < h", taking true branch
(3) Event cond_false:  Condition "x < w", taking false branch
(4) Event loop_end:  Reached end of loop
(5) Event divide_by_zero:  In expression "(stats[0] + stats[1]) * 100U / pixels",
division by expression "pixels" which may be zero has undefined behavior.

290     DEFINE_DETECT_FUNCTION(16)
291     DEFINE_DETECT_FUNCTION(32)

Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>

parent 40066175

ui/vnc-enc-tight.c

+7 −1

Original line number	Diff line number	Diff line
		@@ -181,6 +181,10 @@ tight_detect_smooth_image24(VncState *vs, int w, int h)
		}
		}

		if (pixels == 0) {
		return 0;
		}

		/* 95% smooth or more ... */
		if (stats[0] * 33 / pixels >= 95) {
		return 0;
		@@ -267,7 +271,9 @@ tight_detect_smooth_image24(VncState *vs, int w, int h)
		y += w; \
		} \
		} \
		\
		if (pixels == 0) { \
		return 0; \
		} \
		if ((stats[0] + stats[1]) * 100 / pixels >= 90) { \
		return 0; \
		} \