• Joseph Myers's avatar
    math-emu: fix floating-point to integer overflow detection · d06b3326
    Joseph Myers authored
    On overflow, the math-emu macro _FP_TO_INT_ROUND tries to saturate its
    result (subject to the value of rsigned specifying the desired
    overflow semantics).  However, if the rounding step has the effect of
    increasing the exponent so as to cause overflow (if the rounded result
    is 1 larger than the largest positive value with the given number of
    bits, allowing for signedness), the overflow does not get detected,
    meaning that for unsigned results 0 is produced instead of the maximum
    unsigned integer with the give number of bits, without an exception
    being raised for overflow, and that for signed results the minimum
    (negative) value is produced instead of the maximum (positive) value,
    again without an exception.  This patch makes the code check for
    rounding increasing the exponent and adjusts the exponent value as
    needed for the overflow check.
    Signed-off-by: default avatarJoseph Myers <joseph@codesourcery.com>
    Signed-off-by: default avatarScott Wood <scottwood@freescale.com>
    d06b3326
op-common.h 26 KB