• Marko Mäkelä's avatar
    MDEV-32578 row_merge_fts_doc_tokenize() handles parser plugin inconsistently · 15ae97b1
    Marko Mäkelä authored
    When mysql/mysql-server@0c954c2289a75d90d1088356b1092437ebf45a1d
    added a plugin interface for FULLTEXT INDEX tokenization to MySQL 5.7,
    fts_tokenize_ctx::processed_len got a second meaning, which is only
    partly implemented in row_merge_fts_doc_tokenize().
    
    This inconsistency could cause a crash when using FULLTEXT...WITH PARSER.
    A test case that would crash MySQL 8.0 when using an n-gram parser and
    single-character words would fail to crash in MySQL 5.7, because the
    buf_full condition in row_merge_fts_doc_tokenize() was not met.
    
    This change is inspired by
    mysql/mysql-server@38e9a0779aeea2d197c727e306a910c56b26a47c
    that appeared in MySQL 5.7.44.
    15ae97b1
row0ftsort.h 9.31 KB