1. 10 Aug, 2022 24 commits
    • Alexander Barkov's avatar
      MDEV-27265 Improve contraction performance in UCA collations · a0858b2c
      Alexander Barkov authored
      Adding a hash table for contractions.
      
      The old code iterated through all items in MY_CONTRACTIONS,
      and was much slower, especially for those contractions
      in the end of the list.
      a0858b2c
    • Alexander Barkov's avatar
      MDEV-27009 Add UCA-14.0.0 collations · 13344682
      Alexander Barkov authored
      - Added one neutral and 22 tailored (language specific) collations based on
        Unicode Collation Algorithm version 14.0.0.
      
        Collations were added for Unicode character sets
        utf8mb3, utf8mb4, ucs2, utf16, utf32.
      
        Every tailoring was added with four accent and case
        sensitivity flag combinations, e.g:
      
        * utf8mb4_uca1400_swedish_as_cs
        * utf8mb4_uca1400_swedish_as_ci
        * utf8mb4_uca1400_swedish_ai_cs
        * utf8mb4_uca1400_swedish_ai_ci
      
        and their _nopad_ variants:
      
        * utf8mb4_uca1400_swedish_nopad_as_cs
        * utf8mb4_uca1400_swedish_nopad_as_ci
        * utf8mb4_uca1400_swedish_nopad_ai_cs
        * utf8mb4_uca1400_swedish_nopad_ai_ci
      
      - Introducing a conception of contextually typed named collations:
      
        CREATE DATABASE db1 CHARACTER SET utf8mb4;
        CREATE TABLE db1.t1 (a CHAR(10) COLLATE uca1400_as_ci);
      
        The idea is that there is no a need to specify the character set prefix
        in the new collation names. It's enough to type just the suffix
        "uca1400_as_ci". The character set is taken from the context.
      
        In the above example script the context character set is utf8mb4.
        So the CREATE TABLE will make a column with the collation
        utf8mb4_uca1400_as_ci.
      
        Short collations names can be used in any parts of the SQL syntax
        where the COLLATE clause is understood.
      
      - New collations are displayed only one time
        (without character set combinations) by these statements:
      
           SELECT * FROM INFORMATION_SCHEMA.COLLATIONS;
           SHOW COLLATION;
      
        For example, all these collations:
        - utf8mb3_uca1400_swedish_as_ci
        - utf8mb4_uca1400_swedish_as_ci
        - ucs2_uca1400_swedish_as_ci
        - utf16_uca1400_swedish_as_ci
        - utf32_uca1400_swedish_as_ci
        have just one entry in INFORMATION_SCHEMA.COLLATIONS and SHOW COLLATION,
        with COLLATION_NAME equal to "uca1400_swedish_as_ci", which is the suffix
        without the character set name:
      
      SELECT COLLATION_NAME FROM INFORMATION_SCHEMA.COLLATIONS
      WHERE COLLATION_NAME LIKE '%uca1400_swedish_as_ci';
      
      +-----------------------+
      | COLLATION_NAME        |
      +-----------------------+
      | uca1400_swedish_as_ci |
      +-----------------------+
      
        Note, the behaviour of old collations did not change.
        Non-unicode collations (e.g. latin1_swedish_ci) and
        old UCA-4.0.0 collations (e.g. utf8mb4_unicode_ci)
        are still displayed with the character set prefix, as before.
      
      - The structure of the table INFORMATION_SCHEMA.COLLATIONS was changed.
      
        The NOT NULL constraint was removed from these columns:
        - CHARACTER_SET_NAME
        - ID
        - IS_DEFAULT
        and from the corresponding columns in SHOW COLLATION.
      
        For example:
      
      SELECT COLLATION_NAME, CHARACTER_SET_NAME, ID, IS_DEFAULT
      FROM INFORMATION_SCHEMA.COLLATIONS
      WHERE COLLATION_NAME LIKE '%uca1400_swedish_as_ci';
      +-----------------------+--------------------+------+------------+
      | COLLATION_NAME        | CHARACTER_SET_NAME | ID   | IS_DEFAULT |
      +-----------------------+--------------------+------+------------+
      | uca1400_swedish_as_ci | NULL               | NULL | NULL       |
      +-----------------------+--------------------+------+------------+
      
        The NULL value in these columns now means that the collation
        is applicable to multiple character sets.
        The behavioir of old collations did not change.
        Make sure your client programs can handle NULL values in these columns.
      
      - The structure of the table
        INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY was changed.
      
        Three new NOT NULL columns were added:
        - FULL_COLLATION_NAME
        - ID
        - IS_DEFAULT
      
        New collations have multiple entries in COLLATION_CHARACTER_SET_APPLICABILITY.
        The column COLLATION_NAME contains the collation name without the character
        set prefix. The column FULL_COLLATION_NAME contains the collation name with
        the character set prefix.
      
        Old collations have full collation name in both FULL_COLLATION_NAME and
        COLLATION_NAME.
      
      SELECT COLLATION_NAME, FULL_COLLATION_NAME, CHARACTER_SET_NAME, ID, IS_DEFAULT
      FROM INFORMATION_SCHEMA.COLLATION_CHARACTER_SET_APPLICABILITY
      WHERE FULL_COLLATION_NAME RLIKE '^(utf8mb4|latin1).*swedish.*ci$';
      +-----------------------------+-------------------------------------+--------------------+------+------------+
      | COLLATION_NAME              | FULL_COLLATION_NAME                 | CHARACTER_SET_NAME | ID   | IS_DEFAULT |
      +-----------------------------+-------------------------------------+--------------------+------+------------+
      | latin1_swedish_ci           | latin1_swedish_ci                   | latin1             |    8 | Yes        |
      | latin1_swedish_nopad_ci     | latin1_swedish_nopad_ci             | latin1             | 1032 |            |
      | utf8mb4_swedish_ci          | utf8mb4_swedish_ci                  | utf8mb4            |  232 |            |
      | uca1400_swedish_ai_ci       | utf8mb4_uca1400_swedish_ai_ci       | utf8mb4            | 2368 |            |
      | uca1400_swedish_as_ci       | utf8mb4_uca1400_swedish_as_ci       | utf8mb4            | 2370 |            |
      | uca1400_swedish_nopad_ai_ci | utf8mb4_uca1400_swedish_nopad_ai_ci | utf8mb4            | 2372 |            |
      | uca1400_swedish_nopad_as_ci | utf8mb4_uca1400_swedish_nopad_as_ci | utf8mb4            | 2374 |            |
      +-----------------------------+-------------------------------------+--------------------+------+------------+
      
      - Other INFORMATION_SCHEMA queries:
      
        SELECT COLLATION_NAME FROM INFORMATION_SCHEMA.COLUMNS;
        SELECT COLLATION_NAME FROM INFORMATION_SCHEMA.PARAMETERS;
        SELECT TABLE_COLLATION FROM INFORMATION_SCHEMA.TABLES;
        SELECT DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA;
        SELECT COLLATION_NAME FROM INFORMATION_SCHEMA.ROUTINES;
        SELECT COLLATION_CONNECTION FROM INFORMATION_SCHEMA.EVENTS;
        SELECT DATABASE_COLLATION FROM INFORMATION_SCHEMA.EVENTS;
        SELECT COLLATION_CONNECTION FROM INFORMATION_SCHEMA.ROUTINES;
        SELECT DATABASE_COLLATION FROM INFORMATION_SCHEMA.ROUTINES;
        SELECT COLLATION_CONNECTION FROM INFORMATION_SCHEMA.TRIGGERS;
        SELECT DATABASE_COLLATION FROM INFORMATION_SCHEMA.TRIGGERS;
        SELECT COLLATION_CONNECTION FROM INFORMATION_SCHEMA.VIEWS;
      
        display full collation names, including character sets prefix,
        for all collations, including new collations.
      
        Corresponding SHOW commands also display full collation names
        in collation related columns:
      
        SHOW CREATE TABLE t1;
        SHOW CREATE DATABASE db1;
        SHOW TABLE STATUS;
        SHOW CREATE FUNCTION f1;
        SHOW CREATE PROCEDURE p1;
        SHOW CREATE EVENT ev1;
        SHOW CREATE TRIGGER tr1;
        SHOW CREATE VIEW;
      
        These INFORMATION_SCHEMA queries and SHOW statements may change in
        the future, to display show collation names.
      13344682
    • Alexander Barkov's avatar
      MDEV-27009 Add UCA-14.0.0 collations - adding version aware implicit weight handling · 6bc10f80
      Alexander Barkov authored
      Implicit weights are now handled according to the Unicode version
      (14.0.0 vs earlier versions).
      
      - Adding a new member MY_UCA_INFO::version
      
      - Copy logical positions and the version from "src_uca" to "new_uca"
        in init_weight_level().
      
      - Adding a "const MY_UCA_INFO *" parameter to a few functions
        to know Unicode version to generate implicit weights accordingly:
        - during the collation initialization time, to pages which are
          a mixture of explicit and implicit weights
        - during comparison time, for fully implicit pages
      6bc10f80
    • Alexander Barkov's avatar
      MDEV-27009 Add UCA-14.0.0 collations - dump logical positions and contractions · d7ffb7c3
      Alexander Barkov authored
      - uca-dump can now dump logical positions as a set of "#define" directives.
        Logical positions for 4.0.0 and for 5.2.0 were calculated and put into
        ctype-uca.c manually. That required some efforts by analyzing allkeys.txt
        with help of grep and sort.
        Now when defining a new MY_UCA_INFO it's possible to use the new #define's
        instead of calculating logical positions manually.
        Logical positions also print their weights in DUCET format as a comment
        before the define:
      
      /*
      [.0000.0021.0002]
      [.0000.0117.0002]
      */
      
        The comment helps to know weight ranges on various levels,
        which makes it easier to debug the code.
      
      - uca-dump can now dump built-in DUCET contractions
      
      - Adding a new uca-dump command line option --no-contractions, this is useful
        if one needs to re-dump 4.0.0 and 5.2.0 data in ctype-uca.c compatible way.
      
      - Adding a new uca-dump command line options --case-first=upper|level.
        This can be useful if one need to dump with UPPER case first by default.
        It's not yet decided if we'll use --case-first=upper during the dump though.
      
      - Moving parts of the code from the main loop into separate functions
        parse_chars() and parse_weights(). This allows to reuse the code between
        single characters and contractions.
      
      - Adding a new function my_ducet_weight_normalize(), to cut zero weights
        from a weight string, e.g. [AAAA][0000][BBBB] -> [AAAA][BBBB].
        This helps to reuse the code between single characters and contractions.
      
      - Weight normalization is now done before printing, in separate loops inside
        my_ducet_normalize(). Before this change, normalization was done during
        priting, inside the printing loop. This helps to separate steps:
        loading -> normalizing -> printing.
        This makes it easier to follow what's going on, e.g. while debugging.
      
      - Fixing ctype-uca.c to handle built-in contractions of any length.
        Previously we had only built-in contractions in utf8mb4_thai_520_w2,
        which contains only 2-character contractions.
      d7ffb7c3
    • Alexander Barkov's avatar
      MDEV-27009 Add UCA-14.0.0 collations - Adding implicit weight handling for Unicode-14.0.0 · 0736c03d
      Alexander Barkov authored
      1. Adding separate functions for different Unicode versions
        - my_uca_520_implicit_weight_primary()
         It calculates implicit weights according to the old algorithm
         that we used to dump Unicode-5.2.0 weights.
      
        - my_uca_1400_implicit_weight_primary()
          It calculates implicit weights according to
          https://unicode.org/reports/tr10/#Values_For_Base_Table
          as of November 2021, Unicode version 14.0.0.
      
      2. Adding the "@version" line recognition when dumping allkeys.txt.
         Implicit weights are dumped according to @version.
      
      3. Dumping the scanned version as a "#define"
      
      4. Removing dumping MY_UCA_NPAGES, MY_UCA_NCHARS, MY_UCA_CMASK, MY_UCA_PSHIFT,
         as they are defined in ctype-uca.c. Removing dumping of "main()", it's not
         needed. The intent is to generate an *.h file which can be put directly
         to the MariaDB source tree.
      
      5. Adding a structure MY_DUCET. It now contains weights for single
         characters and version related members. Later we'll add contractions
         and logical positions in here.
      0736c03d
    • Alexander Barkov's avatar
      MDEV-27009 Add UCA-14.0.0 collations - adding uca-dump into build targets · bb84f61a
      Alexander Barkov authored
      - Adding uca-dump into build targets
      - Adding ctype-uca.h and moving implicit weight related routines there
      - Reusing implicit weight routines in ctype-uca.c and uca-dump.c
      - Adding handling of command line arguments to uca-dump
      - Fixing some compile-time warnings in uca-dump.c
      bb84f61a
    • Sergei Golubchik's avatar
    • Sergei Golubchik's avatar
    • Sergei Golubchik's avatar
      cleanup: tests · 4ce1470a
      Sergei Golubchik authored
      sequence tests verify that one cannot change the structure
      of the table. for that they need a valid alter table that
      adds an index over an existing column. there's no column 'start'
      in the table
      4ce1470a
    • Oleksandr Byelkin's avatar
      Merge branch '10.9' into 10.10 · 1c192843
      Oleksandr Byelkin authored
      1c192843
    • Oleksandr Byelkin's avatar
      Merge branch '10.8' into 10.9 · 10ed5276
      Oleksandr Byelkin authored
      10ed5276
    • Oleksandr Byelkin's avatar
      Merge branch '10.7' into 10.8 · 6ffbc0e5
      Oleksandr Byelkin authored
      6ffbc0e5
    • Oleksandr Byelkin's avatar
      Merge branch '10.6' into 10.7 · 98d7ac1f
      Oleksandr Byelkin authored
      98d7ac1f
    • Addison G's avatar
      MDEV-29222 - Fix mysqld_safe script · b8f6d315
      Addison G authored
      The mysqld_safe script was using bad grep options.
      
      The line that was fixed was likely meant to be 2 separate grep commands,
      piped into each other, with each one removing any lines that matched.
      
      The `-E` option was unneeded, as the command is not using regex.
      b8f6d315
    • Oleksandr Byelkin's avatar
      Merge branch '10.6' into 10.7 · 65a963f7
      Oleksandr Byelkin authored
      65a963f7
    • Oleksandr Byelkin's avatar
      Merge branch '10.5' into 10.6 · c442e1ae
      Oleksandr Byelkin authored
      c442e1ae
    • Oleksandr Byelkin's avatar
      Merge branch '10.4' into 10.5 · 1ac0bce3
      Oleksandr Byelkin authored
      1ac0bce3
    • Oleksandr Byelkin's avatar
      65e8506c
    • Nayuta Yanagisawa's avatar
    • Sergei Golubchik's avatar
      my_safe_process: try to kill the process softly first · 12274289
      Sergei Golubchik authored
      first SIGTERM and if the process didn't die in 10 seconds, SIGKILL it.
      
      This allows various tools like `rr`, `gcov`, `gprof`, etc to flush
      their data to disk properly
      12274289
    • Sergei Golubchik's avatar
      missing ' · 9ecdf860
      Sergei Golubchik authored
      9ecdf860
    • Sergei Golubchik's avatar
      MDEV-23149 Server crashes in my_convert / ErrConvString::ptr /... · 82c07fca
      Sergei Golubchik authored
      MDEV-23149 Server crashes in my_convert / ErrConvString::ptr / Item_char_typecast::check_truncation_with_warn
      82c07fca
    • Sergei Golubchik's avatar
      47d0df6e
    • Sergei Golubchik's avatar
      remove invalid options from warning messages · 9d4ed44c
      Sergei Golubchik authored
      --log-slow-queries was removed in 10.0. Now opt_slow_logname
      can be set either with --slow-query-log-file or with --log-basename
      
      
      --log was removed in 10.0. Now opt_logname
      can be set either with --general-log-file or with --log-basename
      9d4ed44c
  2. 09 Aug, 2022 13 commits
  3. 08 Aug, 2022 3 commits