• Alexander Barkov's avatar
    MDEV-23320 Hex hybrid constants 0xHHHH work badly in rounding functions · 92499ae9
    Alexander Barkov authored
    - Type_handler_hex_hybrid did not override
      Type_handler_string_result::Item_func_round_fix_length_and_dec(),
      so the result type of ROUND(0xFFFFFFFFFFFFFFFF) was erroneously
      calculated ad DOUBLE with a wrong length.
      Overriding Item_func_round_fix_length_and_dec(), to calculated
      the result type as INT/BIGINT.
    
      Also, fixing Item_func_round::fix_arg_int() to use
      args[0]->decimal_precision() instead of args[0]->max_length
      when calculating this->max_length, to get a correct result
      for hex hybrids.
    
    - Type_handler_hex_hybrid::Item_func_int_val_fix_length_and_dec()
      called item->fix_length_and_dec_int_or_decimal(), which did not
      produce a correct result data type for hex hybrid.
      Implementing a dedicated code instead, to return INT UNSIGNED or
      BIGINT UNSIGNED depending in the number of digits in the arguments.
    92499ae9
sql_type.h 238 KB