x86: Harden printf against non-normal long double values (bug 26649)
The behavior of isnan/__builtin_isnan on bit patterns that do not correspond to something that the CPU would produce from valid inputs is currently under-defined in the toolchain. (The GCC built-in and glibc disagree.) The isnan check in PRINTF_FP_FETCH in stdio-common/printf_fp.c assumes the GCC behavior that returns true for non-normal numbers which are not specified as NaN. (The glibc implementation returns false for such numbers.) At present, passing non-normal numbers to __mpn_extract_long_double causes this function to produce irregularly shaped multi-precision integers, triggering undefined behavior in __printf_fp_l. With GCC 10 and glibc 2.32, this behavior is not visible because __builtin_isnan is used, which avoids calling __mpn_extract_long_double in this case. This commit updates the implementation of __mpn_extract_long_double so that regularly shaped multi-precision integers are produced in this case, avoiding undefined behavior in __printf_fp_l.
Loading
Please register or sign in to comment