MDEV-19631: Assertion `0' failed in st_select_lex_unit::optimize or different plan upon 2nd execution of PS with EXPLAIN

Second execution of a prepared statement for a query containing a constant
subquery with union that can be optimized away, could result in server abnormal
termination for debug build or incorrect result set output for release build.

For example, the following test case crashes a server built with debug on second
run of the statement EXECUTE stmt
  CREATE TABLE t1 (a INT);
  PREPARE stmt FROM 'EXPLAIN SELECT * FROM t1 HAVING 6 IN ( SELECT 6 UNION SELECT 5 )';
  EXECUTE stmt;
  EXECUTE stmt;

The reason for incorrect result set output or abnormal server termination
is careless working with the data member fake_select_lex->options inside
the function mysql_explain_union(). Once the flag SELECT_DESCRIBE is set in
the data member fake_select_lex->option before calling the methods
  SELECT_LEX_UNIT::prepare/SELECT_LEX_UNIT::execute
the original value of the option is no longer restored.
As a consequence, next time the prepared statement is re-executed we have
the fake_select_lex with the flag SELECT_DESCRIBE set in the data member
fake_select_lex->option, that is incorrect. In result, the method
  Item_subselect::assigned()
is not invoked during evaluation of a constant condition (constant subquery
with union) that being performed on OPTIMIZE phase of query handling.

This leads to the fact that records in the temporary table are not deleted
before calling
  table->file->ha_enable_indexes(HA_KEY_SWITCH_ALL)
in the method st_select_lex_unit::optimize().
In result table->file->ha_enable_indexes(HA_KEY_SWITCH_ALL) returns error
and DBUG_ASSERT(0) is fired.

Stack trace to the line where the error generated on re-enabling indexes
for next subselect iteration is below:
st_select_lex_unit::optimize (at sql_union.cc:954)
  handler::ha_enable_indexes (at handler.cc:4338)
    ha_heap::enable_indexes (at ha_heap.cc:519)
      heap_enable_indexes (at hp_clear.c:164)

The code snippet to clarify raising the error is also listed:
int heap_enable_indexes(HP_INFO *info)
{
  int error= 0;
  HP_SHARE *share= info->s;

  if (share->data_length || share->index_length)
    error= HA_ERR_CRASHED; <<== set error the value HA_ERR_CRASHED
                                since share->data_length != 0

To fix this issue the original value of unit->fake_select_lex->options
has to be saved before setting the flag SELECT_DESCRIBE and restored
on return from invocation of SELECT_LEX_UNIT::prepare/SELECT_LEX_UNIT::execute

As main() invokes parse_page() when -S or -D are set, it can be a case
when parse_page() is invoked when -D filename is not set, that is why
any attempt to write to page dump file must be done only if the file
name is set with -D.

The bug is caused by 2ef7a5a1
(MDEV-13443).

In commit 437da7bc (MDEV-19534),
the default value of the global variable srv_checksum_algorithm
in innochecksum was changed from SRV_CHECKSUM_ALGORITHM_INNODB
to implied 0 (innodb_checksum_algorithm=crc32). As a result,
the function buf_page_is_corrupted() would by default invoke
buf_calc_page_crc32() in innochecksum, and crc32_inited would hold.

This would cause "innochecksum" to fail on a particular page.

The actual problem is older, introduced in 2011 in
mysql/mysql-server@17e497bdb793bc6b8360aa1c626dcd8bb5cfad1b
(MySQL 5.6.3). It should affect the validation of pages of old
data files that were written with innodb_checksum_algorithm=innodb.
When using innodb_checksum_algorithm=crc32 (the default setting
since MariaDB Server 10.2), some valid pages would be rejected
only because exactly one of the two checksum fields accidentally
matches the innodb_checksum_algorithm=crc32 value.

buf_page_is_corrupted(): Simplify the logic of non-strict
checksum validation, by always invoking buf_calc_page_crc32().
Remove a bogus condition that if only one of the checksum fields
contains the value returned by buf_calc_page_crc32(), the page
is corrupted.

On the aarch64 platform, MySQL 5.7.33 cannot install this version of the audit
plugin, but X86_64 can run well。

Problem: In regular replication, when master binlogged using statement format
slave might not have written an event to its binary log when the Query
event aimed at a temporary table.
Specifically this was observed with LOAD DATA INFILE.

This effect was possible because unlike master slave holds temporary
tables in its pool and the master side check of existence of a
temporary table at the format bin-logging decision did not apply.

Solution: replace THD::has_thd_temporary_tables() with
THD::has_temporary_tables which allows to identify temporary table
presence on either side.

--
Reviewed by Andrei Elkin.

Problem:
========
When using mariadb-binlog with --raw and --stop-never, events from
the master's currently active log file should be written to their
respective log file specified by --result-file, and shown on-disk.
There is a bug where mariadb-binlog does not flush the result file
to disk when new events are received

Solution:
========
Add a function call to flush mariadb-binlog’s result file after
receiving an event in --raw mode.

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>

MDEV-22742 UBSAN: Many overflow issues in strings/decimal.c - runtime error: signed integer overflow: x * y cannot be represented in type 'long long int' (on optimized builds).

Avoid integer overflow, do the check before the calculation.

MDEV-27980 file-key-management plugin disabled in mysql_install_db breaks automated deployments (and container initialization)

fix a 2015 typo in build scripts.

--without-plugin=plugin_file_key_management translates to
-DPLUGIN_PLUGIN_FILE_KEY_MANAGEMENT=NO

replace it with a line from 10.4 that builds the plugin
dynamically.

MDEV-27980 file-key-management plugin disabled in mysql_install_db breaks automated deployments (and container initialization)

Revert "Silence the file-key-management plugin during mysql_install_db"

This reverts commit e99d3da6.

The bug was a missing va_start in eprint() which caused a wrong table
name to be printed.
Patch backported from 10.3.

TYPELIBs for ENUM/SET columns could erroneously undergo redundant
hex-unescaping at the table open time.

Fix:
- Prevent multiple unescaping of the same TYPELIB
- Prevent sharing TYPELIBs between columns with different mbminlen

Let us make the mocked-up pfs_malloc() return aligned memory, just
like the actual implementation does.

Per Marko's comment in JIRA, sql_kill is passing the thread id
as long long. We change the format of the error messages to match,
and cast the thread id to long long in sql_kill_user.

The 10.5 test error main.grant_kill showed up a incorrect
thread id on a big endian architecture.

The cause of this is the sql_kill_user function assumed the
error was ER_OUT_OF_RESOURCES, when the the actual error was
ER_KILL_DENIED_ERROR. ER_KILL_DENIED_ERROR as an error message
requires a thread id to be passed as unsigned long, however a
user/host was passed.

ER_OUT_OF_RESOURCES doesn't even take a user/host, despite
the optimistic comment. We remove this being passed as an
argument to the function so that when MDEV-21978 is implemented
one less compiler format warning is generated (which would
have caught this error sooner).

Thanks Otto for reporting and Marko for analysis.

MDEV-23210 Assertion `(length % 4) == 0' failed in my_lengthsp_utf32 on ALTER TABLE, SELECT and INSERT

Problem:
Parse-time conversion from binary to tricky character sets like utf32
produced ill-formed strings. So, later a chash happened in debug builds,
or a wrong SHOW CREATE TABLE was returned in release builds.

Fix:

1. Backporting a few methods from 10.3:
  - THD::check_string_for_wellformedness()
  - THD::convert_string() overloads
  - THD::make_text_string_connection()

2. Adding a new method THD::reinterpret_string_from_binary(),
   which makes sure to either returns a well-formed string
   (optionally prepending with zero bytes), or returns an error.

Travis is dead to us so we don't need all the conditions around it.

Remove depends for no longer supported versions

Debian Jessies, and Ubuntu Trusty, Xenial, Wily are all eol
as far as we are concerned.

The dependancy on an apt cache when running autobake broke the
10.2 aarch64 packages (MDEV-28014). Lets reduce the risk here.

Fixed typo in my_malloc_size_cb_func. There is no max-thread-mem-used
sys variable in MariaDB, only max-session-mem-used. The relevant entry
in sys_vars.cc is also fixed.

Added a fallback case in case we could allocate the 256 bytes for the
error message containing the exact setting.

btr_cur_optimistic_insert(): Disregard DEBUG_DBUG injection to
invoke btr_page_reorganize() if the page (and the table) is empty.
Otherwise, an assertion would fail in btr_page_reorganize_low()
because PAGE_MAX_TRX_ID is 0 in an empty secondary index leaf page.

Fixed by not sending --group option to the server (for now)

Reviwer: Sergei Golubchik

udf_handler::fix_fields(): Execute an assignment outside "if"
so that GCC 12 will not issue a bogus-looking warning.
Also, deduplicate some error handling code.

DEBUG_SYNC signals can get lost in certain tests due to later
DEBUG_SYNC commands overwriting them. This patch addresses
these issues in three tests: main.query_cache_debug,
main.partition_debug_sync, and
rpl.rpl_dump_request_retry_warning.

Additionally, main.partition_debug_sync needed changes to the
result file (the others did not). The synchronization happened
between two commands, one based on ALTER, the other on DROP.
A new thread/connection was needed to synchronize the DEBUG_SYNC
actions between these commands, thereby changing the result file.
Additional comments were added for clarification.

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>

This commit adds validation of the values of the ssl-mode parameter
in SSL scripts, since now only a basic check for the presence of the
"VERIFY_" prefix is performed there to detect "VERIFY_IDENTITY" and
"VERIFY_CA", but all other values are not checked at all.

In addition, this commit removes leading and trailing spaces from
parameter values that SST scripts read from configuration files or
from the command line so that they do not interfere with parameter
checks and substitutions. Parameter substitution has been made more
robust against characters in strings that the shell might erroneously
interpret as regexp.

This commit fixes problems due to bugs and quirks in bsdtar
(the FreeBSD version of tar). Separate tests are not required,
because without these fixes, many other tests fail when tested
in the FreeBSD environment.

Also, the grep patterns for reading utility version numbers
has been made more robust. The notation of some options of
the "cut" utility has been changed.

This commit adds correct handling of binlogs for SST using rsync
or mariabackup. Before this fix, binlogs were handled incorrectly -
- only one (last) binary log file was transferred during SST, which
then led to various failures (for example, when trying to list all
events from the binary log). These bugs were long masked by flaws
in the primitive binlogs handling code in the SST scripts, which
causing binary logs files to be erased after transfer or not added
to the binlog index on the joiner node. Now the correct transfer
of all binary logs (not just the last of the binary log files) has
been implemented both for the rsync (at the script level) and for
the mariabackup (at the level of the main utility code).

This commit also adds a new sst_max_binlogs=<n> parameter, which
can be located in the [sst] section or in the [xtrabackup] section
(historically, supported for mariabackup only, not for rsync), or
in one of the server sections. This parameter specifies the number
of binary log files to be sent to the joiner node during SST. This
option is added for compatibility with old SST scripting behavior,
which can be emulated by setting the sst_max_binlogs=1 (although
in general this can cause problems for the reasons described above).
In addition, setting the sst_max_binlogs=0 can be used to suppress
the transmission of binary logs to the joiner nodes during SST
(although sometimes a single file with the current binary log can
still be transmitted to the joiner, even with sst_max_binlogs=0,
because this sometimes necessary in modes that involve the use of
GTIDs with Galera).

Also, this commit ensures correct handling of paths to various
innodb files and directories in the SST scripts, and fixes some
problems with this that existed in mariabackup utility (which
were associated with incorrect handling of the innodb_data_dir
parameter in some scenarios).

In addition, this commit contains the following enhancements:

 1) Added tests for mtr, which check the correct work with binlogs
    after SST (using rsync and mariabackup);
 2) Added correct handling of slashes at the end of all paths that
    the SST script receives as parameters;
 3) Improved parsing code for --mysqld-args parameters. Now it
    correctly processes the sequence "--" after the name of the
    one-letter option;
 4) Checking the secret signature during joiner authentication
    is made independent of presence of bash (as a unix shell)
    in the system and diff utility no longer needed to check
    certificates compliance;
 5) All directories that are necessary for the correct placement
    of various logs are automatically created by SST scripts in
    advance (before running mariabackup on the joiner node);
 6) Removal of old binary logs on joiner is done using the binlog
    index (if it exists) (not only by fixed pattern that based
    on the current binlog name, as before);
 7) Paths for placing binary logs are correctly processed if they
    are set as relative paths (to the datadir);
 8) SST scripts are made even more resistant to spaces in filenames
    (now for binlogs);
 9) In case of failure, SST scripts now always end with an exit
    code other than zero;
10) SST script for rsync now correctly create a tar file with
    the binlogs, even if the paths to them (in the binlog index
    file) are specified as a mix of absolute and relative paths,
    and even if they do not match with the datadir path specified
    in the current configuration settings.

This commit adds the missing line to the .result file for compress_qpress
(in the mariabackup test suite).

(Edits by SergeiP: fix encryption.tempfiles_encrypted, re-word comment)

Global ORDER BY clause of a UNION may not refer to 1) aggregate functions
or 2) window functions.  setup_order() checked for #1 but not for #2.

SELECTs inside a UNION can have window function but not the global ORDER BY clause of the UNION.

[Patch idea by Igor Babaev]

Symptom: for IN (SELECT ...) subqueries using IN-to-EXISTS transformation,
the optimizer was unable to make inferences using multiple equalities.

The cause is code Item_in_subselect::inject_in_to_exists_cond() which may
break invariants that Multiple-Equality code relies on. In particular, it
may produce a WHERE condition with an empty Item_cond::m_cond_equal.

Fixed this by making Item_cond::m_cond_equal.

The result is not used anywhere but in the output of Innodb information
schema, but this can take as much as 7%CPU (only) on a benchmark.

Fix to move fs blocksize calculate to where it is used.

decrease innodb_lock_wait_timeout for the current session.

convert_error_code_to_mysql(): Use the correct limit FK_MAX_CASCADE_DEL
in the error message. The DICT_FK_MAX_RECURSIVE_LOAD applies to
the number of foreign key constraints in table definitions,
not to the number of rows that are visited while processing
a foreign key constraint.

On FreeBSD, the error message would say "Resource deadlock avoided".
Let us simply replace the error message string in the test.