Stop applying a GCC-specific workaround on clang [BZ #30550]

GCC was the only compiler affected by the issue with
__builtin_isinf_sign and float128.

Fix BZ #30550.

Reported-by: Qiu Chaofan <qiucofan@cn.ibm.com>
Reviewed-by: Florian Weimer <fweimer@redhat.com>