1. 08 Nov, 2023 2 commits
    • Alexander Barkov's avatar
      MDEV-27744 LPAD in vcol created in ORACLE mode makes table corrupted in non-ORACLE · 2b6d241e
      Alexander Barkov authored
      The crash happened with an indexed virtual column whose
      value is evaluated using a function that has a different meaning
      in sql_mode='' vs sql_mode=ORACLE:
      
      - DECODE()
      - LTRIM()
      - RTRIM()
      - LPAD()
      - RPAD()
      - REPLACE()
      - SUBSTR()
      
      For example:
      
      CREATE TABLE t1 (
        b VARCHAR(1),
        g CHAR(1) GENERATED ALWAYS AS (SUBSTR(b,0,0)) VIRTUAL,
        KEY g(g)
      );
      
      So far we had replacement XXX_ORACLE() functions for all mentioned function,
      e.g. SUBSTR_ORACLE() for SUBSTR(). So it was possible to correctly re-parse
      SUBSTR_ORACLE() even in sql_mode=''.
      
      But it was not possible to re-parse the MariaDB version of SUBSTR()
      after switching to sql_mode=ORACLE. It was erroneously mis-interpreted
      as SUBSTR_ORACLE().
      
      As a result, this combination worked fine:
      
      SET sql_mode=ORACLE;
      CREATE TABLE t1 ... g CHAR(1) GENERATED ALWAYS AS (SUBSTR(b,0,0)) VIRTUAL, ...;
      INSERT ...
      FLUSH TABLES;
      SET sql_mode='';
      INSERT ...
      
      But the other way around it crashed:
      
      SET sql_mode='';
      CREATE TABLE t1 ... g CHAR(1) GENERATED ALWAYS AS (SUBSTR(b,0,0)) VIRTUAL, ...;
      INSERT ...
      FLUSH TABLES;
      SET sql_mode=ORACLE;
      INSERT ...
      
      At CREATE time, SUBSTR was instantiated as Item_func_substr and printed
      in the FRM file as substr(). At re-open time with sql_mode=ORACLE, "substr()"
      was erroneously instantiated as Item_func_substr_oracle.
      
      Fix:
      
      The fix proposes a symmetric solution. It provides a way to re-parse reliably
      all sql_mode dependent functions to their original CREATE TABLE time meaning,
      no matter what the open-time sql_mode is.
      
      We take advantage of the same idea we previously used to resolve sql_mode
      dependent data types.
      
      Now all sql_mode dependent functions are printed by SHOW using a schema
      qualifier when the current sql_mode differs from the function sql_mode:
      
      SET sql_mode='';
      CREATE TABLE t1 ... SUBSTR(a,b,c) ..;
      SET sql_mode=ORACLE;
      SHOW CREATE TABLE t1;   ->   mariadb_schema.substr(a,b,c)
      
      SET sql_mode=ORACLE;
      CREATE TABLE t2 ... SUBSTR(a,b,c) ..;
      SET sql_mode='';
      SHOW CREATE TABLE t1;   ->   oracle_schema.substr(a,b,c)
      
      Old replacement names like substr_oracle() are still understood for
      backward compatibility and used in FRM files (for downgrade compatibility),
      but they are not printed by SHOW any more.
      2b6d241e
    • Marko Mäkelä's avatar
      MDEV-13626 Merge InnoDB test cases from MySQL 5.7 · 228b7e4d
      Marko Mäkelä authored
      This imports and adapts a number of MySQL 5.7 test cases that are
      applicable to MariaDB.
      
      Some tests for old bug fixes are not that relevant because the code
      has been refactored since then (especially starting with
      MariaDB Server 10.6), and the tests would not reproduce the
      original bug if the fix was reverted.
      
      In the test innodb_fts.opt, there are many duplicate MATCH ranks, which
      would make the results nondeterministic. The test was stabilized by
      changing some LIMIT clauses or by adding sorted_result in those cases
      where the purpose of a test was to show that no sorting took place
      in the server.
      
      In the test innodb_fts.phrase, MySQL 5.7 would generate FTS_DOC_ID that
      are 1 larger than in MariaDB. In innodb_fts.index_table the difference is 2.
      This is because in MariaDB, fts_get_next_doc_id() post-increments
      cache->next_doc_id, while MySQL 5.7 pre-increments it.
      
      Reviewed by: Thirunarayanan Balathandayuthapani
      228b7e4d
  2. 07 Nov, 2023 1 commit
    • Monty's avatar
      Ensure that process "State" is properly cleaned after query execution · 2447172a
      Monty authored
      In some cases "SHOW PROCESSLIST" could show "Reset for next command"
      as State, even if the previous query had finished properly.
      
      Fixed by clearing State after end of command and also setting the State
      for the "Connect" command.
      
      Other things:
      - Changed usage of 'thd->set_command(COM_SLEEP)' to
        'thd->mark_connection_idle()'.
      - Changed thread_state_info() to return "" instead of NULL. This is
        just a safety measurement and in line with the logic of the
        rest of the function.
      2447172a
  3. 06 Nov, 2023 2 commits
  4. 05 Nov, 2023 10 commits
  5. 04 Nov, 2023 1 commit
  6. 03 Nov, 2023 7 commits
    • Monty's avatar
      rpl.rpl_invoked_features fails sporadically with "Duplicate key error" · e5a5573f
      Monty authored
      The reason was that Event e11 was re-executed before
      "ALTER EVENT e11 DISABLE" had been executed.
      
      Fixed by increasing re-schedule time
      
      Other things:
      - Removed double accounting of 'execution_count'. It was incremented in
        top->mark_last_executed(thd) that was executed a few lines earlier.
      e5a5573f
    • Monty's avatar
      MDEV-32518 Test failure: ./mtr --no-reorder main.log_slow_debug main.subselect · 7533062f
      Monty authored
      There where two errors left from the previous fix.
      
      - subselect.test assumes that mysql.slow_log is empty. This was not
        enforced.
      - subselect.test dropped a file that does not exists (for safety).
        This was fixed by ensuring we don't get a warning if the file does
        not exist.
      7533062f
    • Kristian Nielsen's avatar
      Fix mariabackup InnoDB recovered binlog position on server upgrade · 9fa718b1
      Kristian Nielsen authored
      Before MariaDB 10.3.5, the binlog position was stored in the TRX_SYS page,
      while after it is stored in rollback segments. There is code to read the
      legacy position from TRX_SYS to handle upgrades. The problem was if the
      legacy position happens to compare larger than the position found in
      rollback segments; in this case, the old TRX_SYS position would incorrectly
      be preferred over the newer position from rollback segments.
      
      Fixed by always preferring a position from rollback segments over a legacy
      position.
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      9fa718b1
    • Kristian Nielsen's avatar
      Revert: MDEV-22351 InnoDB may recover wrong information after RESET MASTER · f8f5ed22
      Kristian Nielsen authored
      This commit can cause the wrong (old) binlog position to be recovered by
      mariabackup --prepare. It implements that the value of the FIL_PAGE_LSN is
      compared to determine which binlog position is the last one and should be
      recoved. However, it is not guaranteed that the FIL_PAGE_LSN order matches the
      commit order, as is assumed by the code. This is because the page LSN could be
      modified by an unrelated update of the page after the commit.
      
      In one example, the recovery first encountered this in trx_rseg_mem_restore():
      
        lsn=27282754  binlog position (./master-bin.000001, 472908)
      
      and then later:
      
        lsn=27282699  binlog position (./master-bin.000001, 477164)
      
      The last one 477164 is the correct position. However, because the LSN
      encountered for the first one is higher, that position is recovered instead.
      This results in too old binlog position, and a newly provisioned slave will
      start replicating too early and get duplicate key error or similar.
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      f8f5ed22
    • Kristian Nielsen's avatar
    • Kristian Nielsen's avatar
      Restore getting InnoDB position from mariabackup --no-lock · 167fe664
      Kristian Nielsen authored
      Revert the patch for MDEV-18917, which removed this functionality.
      This restores that mariabackup --prepare recovers the transactional
      binlog position from the redo log, and writes it to the file
      xtrabackup_binlog_pos_innodb.
      
      This position is updated only on every InnoDB commit. This means that
      if the last event in the binlog at the time of backup is a DDL or
      non-transactional update, the recovered position from --no-lock will
      be behind the state of the backup.
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      167fe664
    • Rex's avatar
      MDEV-31995-fix short fix for memory leak introduced in MDEV-31995 · e6953374
      Rex authored
      list elements not correctly allocated in push_back.
      e6953374
  7. 02 Nov, 2023 8 commits
  8. 01 Nov, 2023 4 commits
    • Igor Babaev's avatar
      MDEV-28615 Crash caused by multi-table UPDATE over derived with hanging CTE · 9e321a44
      Igor Babaev authored
      This bug affected only multi-table update statements and in very rare
      cases: one of the tables used at the top level of the statement must be
      a derived table containg a row construct with a subquery including hanging
      CTE.
      
      Before this patch was applied the function prepare_unreferenced() of the
      class With_element when invoked for the the hangin CTE did not properly
      restored the value of thd->lex->context_analysis_only. As a result it
      became 0 after the call of this function.
      For a query affected by the bug this function is called when
      JOIN::prepare() is called for the subquery with a hanging CTE. This happens
      when Item_row::fix_fields() calls fix_fields() for the subquery. Setting
      the value of thd->lex->context_analysis_only forces the caller function
      Item_row::fix_fields() to invoke the virtual method is_null() for the
      subquery that leads to execution of it. It causes an assertion failure
      because the call of Item_row::fix_fields() happens during the invocation
      of Multiupdate_prelocking_strategy::handle_end() that calls the function
      mysql_derived_prepare() for the derived table used by the UPDATE at the
      time when proper locks for the statement tables has not been acquired yet.
      
      With this patch the value of thd->lex->context_analysis_only is restored
      to CONTEXT_ANALYSIS_ONLY_DERIVED that is set in the function
      mysql_multi_update_prepare().
      
      Approved by Oleksandr Byelkin <sanja@mariadb.com>
      9e321a44
    • Brandon Nesterenko's avatar
      MDEV-32655: rpl_semi_sync_slave_compressed_protocol.test assert_only_after is wrong · 80ea3590
      Brandon Nesterenko authored
      The MTR test rpl.rpl_semi_sync_slave_compressed_protocol scans the
      log file to ensure there is no magic number error. It attempts to
      only scan the log files of the current test; however, the variable
      which controls this, , is initialized incorrectly,
      and it thereby scans the entire log file, which includes output from
      prior tests. This causes it to fail if a test which expects this
      error runs previously on the same worker.
      
      This patch fixes the assert_only_after so the test only scans
      through its own log contents.
      80ea3590
    • Brandon Nesterenko's avatar
      MDEV-32651: Lost Debug_sync signal in rpl_sql_thd_start_errno_cleared · c341743e
      Brandon Nesterenko authored
      The test rpl.rpl_sql_thd_start_errno_cleared can lose a debug_sync
      signal, as there is a RESET immediately following a SIGNAL. When the
      signal is lost, the sql_thread is stuck in a WAIT_FOR clause until
      it times out, resulting in long test times (albeit still
      successful).
      
      This patch extends the test to ensure the debug_sync signal was
      received before issuing the RESET
      c341743e
    • Alexander Barkov's avatar
      MDEV-32645 CAST(AS UNSIGNED) fails with --view-protocol · 4b65859a
      Alexander Barkov authored
      Item_char_typecast::print() did not print the "binary" keyword
      in such cases:
         CAST('a' AS CHAR CHARACTER SET latin1 BINARY)
      
      This caused a difference in "mtr" vs "mtr --view-protocol"
      4b65859a
  9. 31 Oct, 2023 3 commits
    • Igor Babaev's avatar
      MDEV-32569 Failure when executing PS for query using IN subquery · 9b049266
      Igor Babaev authored
      This patch corrects the fix for MDEV-32369. No Item_direct_ref_to_item
      objects should be allocated at the optimizer phase after permanent
      rewritings have been done.
      
      The patch also adds another test case for MDEV-32369 that uses MyISAM
      with more than one row.
      
      Approved by Rex Johnston <rex.johnston@mariadb.com>
      9b049266
    • Julius Goryavsky's avatar
      galera: disabled tests cleanup · edabb819
      Julius Goryavsky authored
      edabb819
    • Kristian Nielsen's avatar
      MDEV-27436: binlog corruption (/tmp no space left on device at the same moment) · 6fa69ad7
      Kristian Nielsen authored
      This commit fixes several bugs in error handling around disk full when
      writing the statement/transaction binlog caches:
      
      1. If the error occurs during a non-transactional statement, the code
      attempts to binlog the partially executed statement (as it cannot roll
      back). The stmt_cache->error was still set from the disk full error. This
      caused MYSQL_BIN_LOG::write_cache() to get an error while trying to read the
      cache to copy it to the binlog. This was then wrongly interpreted as a disk
      full error writing to the binlog file. As a result, a partial event group
      containing just a GTID event (no query or commit) was binlogged. Fixed by
      checking if an error is set in the statement cache, and if so binlog an
      INCIDENT event instead of a corrupt event group, as for other errors.
      
      2. For LOAD DATA LOCAL INFILE, if a disk full error occured while writing to
      the statement cache, the code would attempt to abort and read-and-discard
      any remaining data sent by the client. The discard code would however
      continue trying to write data to the statement cache, and wrongly interpret
      another disk full error as end-of-file from the client. This left the client
      connection with extra data which corrupts the communication for the next
      command, as well as again causing an corrupt/incomplete event to be
      binlogged. Fixed by restoring the default read function before reading any
      remaining data from the client connection.
      Reviewed-by: default avatarAndrei Elkin <andrei.elkin@mariadb.com>
      Signed-off-by: default avatarKristian Nielsen <knielsen@knielsen-hq.org>
      6fa69ad7
  10. 30 Oct, 2023 2 commits
    • Brandon Nesterenko's avatar
      MDEV-26272: The macro MASTER_INFO_VAR invokes undefined behaviour · e52777f1
      Brandon Nesterenko authored
      Updates to specific replication system variables need to target the
      active primary connection to support multi-source replication. These
      variables use the Sys_var_multi_source_ulonglong type. This class
      uses offsets of the Master_info C++ class to generalize access to
      its member variables.
      
      The problem is that the Master_info class is not of standard layout,
      and neither are many of its member variables, e.g. rli and
      rli->relay_log. Because the class is not of standard layout, using
      offsets to access member variables invokes undefined behavior.
      
      This patch changes how Sys_var_multi_source_ulonglong accesses the
      member variables of Master_info from using parameterized memory
      offsets to “getter” function pointers.
      
      Note that the size parameter and assertion are removed, as they are
      no longer needed because the condition is guaranteed by compiler
      type-safety checks.
      
      Reviewed By:
      ============
      Kristian Nielsen <knielsen@knielsen-hq.org>
      e52777f1
    • Rex's avatar
      MDEV-31995 Bogus error executing PS for query using CTE with renaming of columns · eb8053b3
      Rex authored
      This commit addresses column naming issues with CTEs in the use of prepared
      statements and stored procedures. Usage of either prepared statements or
      procedures with Common Table Expressions and column renaming may be affected.
      
      There are three related but different issues addressed here.
      
      1) First execution issue. Consider the following
      
      prepare s from "with cte (col1, col2) as (select a as c1, b as c2 from t
      order by c1) select col1, col2 from cte";
      execute s;
      
      After parsing, items in the select are named (c1,c2), order by (and group by)
      resolution is performed, then item names are set to (col1, col2).
      When the statement is executed, context analysis is again performed, but
      resolution of elements in the order by statement will not be able to find c1,
      because it was renamed to col1 and remains this way.
      
      The solution is to save the names of these items during context resolution
      before they have been renamed. We can then reset item names back to those after
      parsing so first execution can resolve items referred to in order and group by
      clauses.
      
      2) Second Execution Issue
      
      When the derived table contains more than one select 'unioned' together we could
      reasonably think that dealing with only items in the first select (which
      determines names in the resultant table) would be sufficient.  This can lead to
      a different problem.  Consider
      
      prepare st from "with cte (c1,c2) as
        (select a as col1, sum(b) as col2 from t1 where a > 0 group by col1
          union select a as col3, sum(b) as col4 from t2 where b > 2 group by col3)
        select * from cte where c1=1";
      
      When the optimizer (only run during the first execution) pushes the outside
      condition "c1=1" into every select in the derived table union, it renames the
      items to make the condition valid.  In this example, this leaves the first item
      in the second select named 'c1'.  The second execution will now fail 'group by'
      resolution.
      
      Again, the solution is to save the names during context analysis, resetting
      before subsequent resolution, but making sure that we save/reset the item
      names in all the selects in this union.
      
      3) Memory Leak
      
      During parsing Item::set_name() is used to allocate memory in the statement
      arena.  We cannot use this call during statement execution as this represents
      a memory leak.  We directly set the item list names to those in the column list
      of this CTE (also allocated during parsing).
      
      Approved by Igor Babaev <igor@mariadb.com>
      eb8053b3