difftime can throw exceptions

difftime can signal an inexact conversion when converting to double,
so it should not be marked as pure or nothrow (BZ 31808).

Although we could do something more complicated, in which difftime is
plain on modern platforms but const and nothrow on obsolescent
platforms with 32-bit time_t, it hardly seems worth the trouble.
difftime is used so rarely that it's not worth taking pains to
optimize calls to it on obsolescent platforms.

Reviewed-by: DJ Delorie <dj@redhat.com>