• Alexander Barkov's avatar
    MDEV-22111 ERROR 1064 & 1033 and SIGSEGV on CREATE TABLE w/ various charsets... · cb9c49a9
    Alexander Barkov authored
    MDEV-22111 ERROR 1064 & 1033 and SIGSEGV on CREATE TABLE w/ various charsets on 10.4/5 optimized builds | Assertion `(uint) (table_check_constraints - share->check_constraints) == (uint) (share->table_check_constraints - share->field_check_constraints)' failed
    
    The code incorrectly assumed in multiple places that TYPELIB
    values cannot have 0x00 bytes inside. In fact they can:
    
      CREATE TABLE t1 (a ENUM(0x61, 0x0062) CHARACTER SET BINARY);
    
    Note, the TYPELIB value encoding used in FRM is ambiguous about 0x00.
    
    So this fix is partial.
    
    It fixes 0x00 bytes in many (but not all) places:
    
    - In the middle or in the end of a value:
        CREATE TABLE t1 (a ENUM(0x6100) ...);
        CREATE TABLE t1 (a ENUM(0x610062) ...);
    
    - In the beginning of the first value:
        CREATE TABLE t1 (a ENUM(0x0061));
        CREATE TABLE t1 (a ENUM(0x0061), b ENUM('b'));
    
    - In the beginning of the second (and following) value of the *last* ENUM/SET
      in the table:
    
        CREATE TABLE t1 (a ENUM('a',0x0061));
        CREATE TABLE t1 (a ENUM('a'), b ENUM('b',0x0061));
    
    However, it does not fix 0x00 when:
    
    - 0x00 byte is in the beginning of a value of a non-last ENUM/SET
      causes an error:
    
       CREATE TABLE t1 (a ENUM('a',0x0061), b ENUM('b'));
       ERROR 1033 (HY000): Incorrect information in file: './test/t1.frm'
    
      This is an ambuguous case and will be fixed separately.
      We need a new TYPELIB encoding to fix this.
    
    Details:
    
    - unireg.cc
    
      The function pack_header() incorrectly used strlen() to detect
      a TYPELIB value length. Adding a new function typelib_values_packed_length()
      which uses TYPELIB::type_lengths[n] to detect the n-th value length,
      and reusing the new function in pack_header() and packed_fields_length()
    
    - table.cc
      fix_type_pointers() assumed in multiple places that values cannot have
      0x00 inside and used strlen(TYPELIB::type_names[n]) to set
      the corresponding TYPELIB::type_lengths[n].
    
      Also, fix_type_pointers() did not check the encoded data for consistency.
    
      Rewriting fix_type_pointers() code to populate TYPELIB::type_names[n] and
      TYPELIB::type_lengths[n] at the same time, so no additional loop
      with strlen() is needed any more.
    
      Adding many data consistency tests.
    
      Fixing the main loop in fix_type_pointers() to use memchr() instead of
      strchr() to handle 0x00 properly.
    
      Fixing create_key_infos() to return the result in a LEX_STRING rather
      that in a char*.
    cb9c49a9
table.cc 226 KB