Step#3 The main patch

Step#2 - Adding a new collation derivation level for CAST and CONVERT.

Now character string cast functions:
  - CAST(string_expr AS CHAR)
  - CONVERT(expr USING charset_name)

have a new collation derivation level between:

  - string literals
  - utf8 metadata functions, e.g. user() and database()

Before the change these cast functions had collation derivation equal
to table columns, which caused more illegal mix of collation conflicts.

Note, binary string cast functions:
  - BINARY(expr)
  - CAST(string_expr AS BINARY)
  - CONVERT(expr USING binary)
did not change their collation derivation, to preserve the behaviour of
queries like these:
SELECT database()=BINARY'test';
SELECT user()=CAST('root' AS BINARY);
SELECT current_role()=CONVERT('role' USING binary);

Derivation levels after the change look as follows:

  DERIVATION_IGNORABLE= 7, // Explicit NULL

  DERIVATION_NUMERIC= 6,   // Numbers in string context,
                           // Numeric user variables
                           // CAST(numeric_expr AS CHAR)

  DERIVATION_COERCIBLE= 5, // Literals, string user variables

  DERIVATION_CAST= 4,      // CAST(string_expr AS CHAR),
                           // CONVERT(string_expr USING cs)

  DERIVATION_SYSCONST= 3,  // utf8 metadata functions, e.g. user(), database()
  DERIVATION_IMPLICIT= 2,  // Table columns, SP variables, BINARY(expr)
  DERIVATION_NONE= 1,      // A mix (e.g. CONCAT) of two differrent collations
  DERIVATION_EXPLICIT= 0   // An explicit COLLATE clause

Step#1 - Changing collation derivation for string user variables
from IMPLICIT to COERCIBLE.

Retionale:

Without this preparatory change, switching the default collation for
Unicode character sets from xxx_general_ci to uca1400_ai_ci would cause
"Illegal mix of collations" errors in scenarios comparing a column with
a non-default collation to a string user variable

This is especially important for queries to INFORMATION_SCHEMA tables,
whose columns use utf8mb3_general_ci.

See the description of MDEV-25829 for more details and SQL script examples.

MDEV-33672 (10.6) added checks/tests for malformed events which end
before the flags describe (which would lead to reading of un-owned
memory). MDEV-7850 (11.5) extended all GTID events with a thread id
at the end of the event. This GTID event extension invalidates the
tests added in MDEV-33672 because the thread id is appended after the
event (and thereby the event isn't cut short).

This patch fixes these MDEV-33672 tests by not writing the GTID
thread id when writing the Gtid events just for these tests. This
preserves tests for backwards compatibility, rather than getting rid
of the tests altogether.

trx_undo_mem_create_at_db_start(): Invoke recv_sys_t::recover()
instead of buf_page_get_gen(), so that all undo log pages will be
recovered correctly. Failure to do this could prevent InnoDB from
starting up due to "Data structure corruption", or it could
potentially lead to a situation where InnoDB starts up but some
transactions were recovered incorrectly.

recv_sys_t::recover(): Only acquire a buffer-fix on the pages,
not a shared latch. This is adequate protection, because this function
is only being invoked during early startup when no "users" are modifying
buffer pool pages. The only writes are due to server bootstrap
(the data files being created) or crash recovery (changes from
ib_logfile0 being applied).

buf_page_get_gen(): Assert that the function is not invoked while crash
recovery is in progress, and that the special mode BUF_GET_RECOVER is
only invoked during crash recovery or server bootstrap.

All this should really have been part of
commit 850d6173 (MDEV-32042).

		in buf_dblwr_t::init_or_load_pages()

- InnoDB fails to set the TRX_SYS_DOUBLEWRITE_SPACE_ID_STORED
flag in transaction system header page while recreating
the undo log tablespaces

buf_dblwr_t::init_or_load_pages(): Tries to reset the
space id and try to write into doublewrite buffer even
when read_only mode is enabled.

In srv_all_undo_tablespaces_open(), InnoDB should try to
open the extra unused undo tablespaces instead of trying to
creating it.

This bug was found related to MDEV-34212.

Some InnoDB tests, most notably innodb.table_flags,64k would occasionally
fail. I am able to reproduce this locally on a MemorySanitizer build,
sporadically.

The following is a minimal .test file for reproducing this:
  --source include/have_innodb.inc
  SELECT @@innodb_page_size;
and the .opt file:
  --innodb-undo-tablespaces=0 --innodb-page-size=64k
  --innodb-buffer-pool-size=20m

This bug was revealed due to the recent
commit 466ae1cf
which set innodb_fast_shutdown=0 during server bootstrap
in our regression test driver.

Due to the bug, a write of undo page 50 in the system tablespace
was discarded in buf_page_t::flush(). A subsequent InnoDB startup
failed because an old version of that page would point to a
freed undo log page 300.

mtr_t::commit_shrink(): Only invoke fil_space_t::set_create_lsn()
on undo tablespaces, which will be fully reinitialized or created
from the scratch. On the system tablespace, we must only adjust
the file size, to avoid writing pages that are beyond the end
of the tablespace. Thanks to Thirunarayanan Balathandayuthapani
for providing this fix.

innobase_end(): Do not attempt to shrink the system tablespace if
innodb_read_only=ON or innodb_force_recovery>4. This fixes a regression
due to commit 2d6c2f22 (MDEV-32452).

This bug was caught when testing a fix of MDEV-34200, which adds
SET GLOBAL innodb_fast_shutdown=0 to the test innodb.undo_upgrade.

C/C 3.4 disables mysql_old_password by default, so

add an option for the `connect` command to support specifying
allowed authentication plugins (MARIADB_OPT_RESTRICTED_AUTH).

use it to enable mysql_old_password when needed for testing

it no longer supports TLSv1.0

In the merge commit f9807aad
there were some omissions or errors.

ibuf_remove_free_page(): Return an error if the free list is corrupted
when removing the change buffer on an upgrade. A special 11.0 version of
commit 263932d5 would have been useful.

buf_page_get_gen(): Correctly handle the case that a page was being
concurrently read into the buffer pool and found out to be corrupted.
This was part of commit a4cda66e
but had been discarded in the merge.

Because MariaDB Server 11.0 has reached its end of life as of
commit 466ae1cf this fix is being applied
to the 11.1 branch.

Alternative operator name keywords like `and`, `or`, `xor`, etc., are
uncommon in MariaDB and can cause obscure build errors when the GCC
flag `-fno-operator-names` is applied.

Description of `-fno-operator-names`:
https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Dialect-Options.html
> Do not treat the operator name keywords `and`, `bitand`, `bitor`,
> `compl`, `not`, `or` and `xor` as synonyms as keywords.

Part of the build errors:

    /local/p4clients/pkgbuild-LdLa_/workspace/src/RDSMariaDB/sql/sql_select.cc:11171:28: error: expected ‘)’ before ‘and’
    11171 |     DBUG_ASSERT(sel >= 0.0 and sel <= 1.00001);
          |                            ^~~
    /local/p4clients/pkgbuild-LdLa_/workspace/src/RDSMariaDB/include/my_global.h:372:44: note: in definition of macro ‘unlikely’
      372 | #define unlikely(x)     __builtin_expect(((x) != 0),0)
          |                                            ^
    ...

The build failure is caused by using alternative operator name keywords
`and` introduced in commit b66cdbd1.
Replace the `and` keyword with `&&` and target on MariaDB 11.0+ branches
which include the commit.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.

disable view protocol (DROP VIEW doesn't work very well with transactions)
and cosmetic cleanups

(Based on original patch by Oleksandr Byelkin)

Multi-table DELETE can execute via "buffered" mode: at phase #1 it collects
rowids of rows to be deleted, then at phase #2 in multi_delete::do_deletes()
it calls handler->rnd_pos() to read rows to be deleted and deletes them.

The problem occurred when phase #1 used Rowid Filter on the table that
phase #2 would be deleting from.
In InnoDB, h->rnd_init(scan=false) and h->rnd_pos() is an index scan over PK
under the hood. So, at phase #2 ha_innobase::rnd_init() would try to use the
Rowid Filter and hit an assertion inside ha_innobase::rnd_init().

Note that multi-table UPDATE works similarly but was not affected, because
patch for MDEV-7487 added code to disable rowid filter for phase #2 in
multi_update::do_updates().

This patch changes the approach:
- It makes InnoDB not use Rowid Filter in rnd_pos() scans: it is disabled in
  ha_innobase::rnd_init() and enabled back in ha_innobase::rnd_end().
- multi_update::do_updates() no longer disables Rowid Filter for phase#2 as
  it is no longer necessary.

the test failed almost always in release (but not in debug) builds with

--- galera_sst_mariabackup.result
+++ galera_sst_mariabackup.reject
@@ -5,7 +5,7 @@
 connection node_1;
 select @@innodb_undo_tablespaces;
 @@innodb_undo_tablespaces
-0
+3
 connection node_2;
 select @@innodb_undo_tablespaces;
 @@innodb_undo_tablespaces

and

[Warning] InnoDB: Cannot change innodb_undo_tablespaces=0 because previous shutdown was not with innodb_fast_shutdown=0

because mariadbd *before this test* wasn't using innodb_fast_shutdown=0

Fix the bootstrap to use innodb_fast_shutdown=0 (and the bootstrap
creates a starting point for any test that uses a .cnf file)

followup for cac0fc97

also, remove redundant include/have_innodb.inc

and fix installation of mysql.service on buster

followup for ec09c034

FreeBSD doesn't like it and hangs.

As we don't wait for signal handler, let's disable SIGHUP in bootstrap
too

.opt files, unlike combinations, accumulate, let's not overuse them

followup for 13663cb5