• Alexander Barkov's avatar
    MDEV-27266 Improve UCA collation performance for utf8mb3 and utf8mb4 · d8f172c1
    Alexander Barkov authored
    Adding two levels of optimization:
    
    1. For every bytes pair [00..FF][00..FF] which:
      a. consists of two ASCII characters or makes a well-formed two-byte character
      b. whose total weight string fits into 4 weights
         (concatenated weight string in case of two ASCII characters,
         or a single weight string in case of a two-byte character)
      c. whose weight is context independent (i.e. does not depend on contractions
         or previous context pairs)
      store weights in a separate array of MY_UCA_2BYTES_ITEM,
      so during scanner_next() we can scan two bytes at a time.
      Byte pairs that do not match the conditions a-c are marked in this array
      as not applicable for optimization and scanned as before.
    
    2. For every byte pair which is applicable for optimization in #1,
       and which produces only one or two weights, store
       weights in one more array of MY_UCA_WEIGHT2. So in the beginning
       of strnncoll*() we can skip equal prefixes using an even more efficient
       loop. This loop consumes two bytes at a time. The loop scans while the
       two bytes on both sides produce weight strings of equal length
       (i.e. one weight on both sides, or two weight on both sides).
       This allows to compare efficiently:
       - Context independent sequences consisting of two ASCII characters
       - Context independent 2-byte characters
       - Contractions consisting of two ASCII characters, e.g. Czech "ch".
       - Some tricky cases: "ss" vs "SHARP S"
         ("ss" produces two weights, 0xC39F also produces two weights)
    d8f172c1
m_ctype.h 67.1 KB