ltp: Remove -mfpmath=sse on x86-64 too

Fixes build errors seen with clang/musl like on x86
error: the 'sse' unit is not supported with this instruction set

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>