This patch changes the main name of 3 byte character set from utf8 to
utf8mb3. New old_mode UTF8_IS_UTF8MB3 is added and set TRUE by default,
so that utf8 would mean utf8mb3. If not set, utf8 would mean utf8mb4.

The implementation of handlerton::drop_database in InnoDB is
unnecessarily complex. The minimal implementation should check
that no conflicting locks or references exist on the tables,
delete all table metadata in a single transaction, and finally
delete the tablespaces.

Note: DROP DATABASE will delete each individual table that the
SQL layer knows about, one table per transaction.
The handlerton::drop_database is basically a final cleanup step
for removing any garbage that could have been left behind
in InnoDB due to some bug, or not having atomic DDL in the past.

hash_node_t: Remove. Use the proper data type name in pointers.

dict_drop_index_tree(): Do not take the table as a parameter.
Instead, return the tablespace ID if the tablespace should be dropped
(we are dropping a clustered index tree).

fil_delete_tablespace(), fil_system_t::detach(): Return a single
detached file handle. Multi-file tablespaces cannot be deleted
via this interface.

ha_innobase::delete_table(): Remove a work-around for non-atomic DDL
and do not try to drop tables with similar-looking name.

innodb_drop_database(): Complete rewrite.

innobase_drop_database(), dict_get_first_table_name_in_db(),
row_drop_database_for_mysql(), drop_all_foreign_keys_in_db(): Remove.

row_purge_remove_clust_if_poss_low(), row_undo_ins_remove_clust_rec():
If the tablespace is to be deleted, try to evict the table definition
from the cache. Failing that, set dict_table_t::space to nullptr.

lock_release_on_rollback(): On the rollback of CREATE TABLE, release all
locks that the transaction had on the table, to avoid heap-use-after-free.

trx_t::will_lock: Changed the type to bool.

trx_t::is_autocommit_non_locking(): Replaces
trx_is_autocommit_non_locking().

trx_is_ac_nl_ro(): Remove (replaced with equivalent assertion expressions).

assert_trx_nonlocking_or_in_list(): Remove.
Replaced with at least as strict checks in each place.

check_trx_state(): Moved to a static function; partially replaced with
individual debug assertions implementing equivalent or stricter checks.

ha_innobase::index_read(): If an autocommit non-locking transaction was
already started, refuse to access a SPATIAL INDEX.
Once a non-locking autocommit transaction has started, it must remain
in that mode (not acquire any locks).

This should fix one cause of the assertion failure that would occur in
DeadlockChecker::check_and_resolve() under heavy load, presumably
due to concurrent execution of trx_commit_in_memory().

Problem:
=======
The ALWAYS option of the mariadb-binlog --base64-output flag
formats its output incorrectly. This option is deprecated, and
MySQL 8.0 has removed it entirely.

Solution:
========
Adhere to MySQL and remove this option from MariaDB.

Behavioral Changes:
==================
Use Case: ./mariadb-binlog --base64-output
 Previous Behavior: Sets base64-output mode to always
 New Behavior: Error message indicating incomplete argument

Use Case: ./mariadb-binlog --base64-output=always
 Previous Behavior: Sets base64-output mode to always
 New Behavior: Error message indicating invalid argument value

Reviewed By:
==========
Andrei Elkin: <andrei.elkin@mariadb.com>

Fixed bugs caused by inaccuracies in automatic merging
from other branches:

1) Authentication information is not removed from the connection
   address, which causes some tests to fail;
2) wsrep_debug=on should be replaced with wsrep_debug=1;
3) Added missing "connection" lines to test result file;
4) Some tests have been corrected for Galera 4.x (10.4+).

1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.

This commit contains a large set of further bug fixes and
improvements to SST scripts for Galera, continuing the work
that was started in MDEV-24962 to make SST scripts work smoothly
in different network configurations (especially using ipv6) and
with different environment settings:

 1) The ipv6 addresses were incorrectly handled in the SST script
    for rsync (incorrect address substitution for establishing a
    connection, incorrect address substitution for bind, and so on);
 2) Checking the locality of the ip-address in SST scripts did not
    support ipv6 addresses (such as "[::1]"), which were falsely
    identified as non-local ip, which further did not allow running
    two SSTs on different local addresses on the same machine.
    On the other hand, this bug masked some other errors (related
    to handling ipv6 addresses);
 3) The code for checking the locality of the ip address was different
    in the SST scripts for rsync and for mysqldump, with individual
    flaws. This code is now made common and moved to wsrep_sst_common;
 4) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) in the wait_for_listen() and check_pid_and_port() functions
    did not process ipv6 addresses correctly in all cases (not for all
    branches);
 5) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) in the wait_for_listen() and check_pid_and_port() functions
    for some code branches could give a false positive result due to
    the textual match of prefixes in the port number and/or PID of
    the process;
 6) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) was supported through different utilities in SST scripts
    for mariabackup and for rsync, and with various minor flaws in
    the code. Now the code is still different in these scripts, but
    it supports a common set of utilities (lsof, ss, sockstat) and
    is synchronized across patterns that used to check the output
    of  these utilities;
 7) In SST via mariabackup, the signal about readiness to receive data
    is sometimes sent too early - immediately after listen(), and not
    after accept() (which are called by socat or netcat utility).
 8) Checking availability of the some options of some utilities was
    done using the grep pattern, which easily gives false positives;
 9) Common name (CN) for local addresses, if not explicitly specified,
    is now always replaced to "localhost" to avoid the need to generate
    many separate certificates for local addresses of one machine and
    not to depend on which the local address is currently used in test
    (ipv4 or ipv6, etc.);
10) In tests galera_sst_mariabackup_encrypt_with_key_server and
    galera_sst_rsync_encrypt_with_key_server the correct certificate
    is selected to avoid commonname (CN) mismatch problems;
11) Further refactoring to protect against spaces in file names.
12) Further general refactoring to eliminate bash-specific constructs
    or to improve code readability;
13) The code for setting options for the nc (netcat) utility was
    different in different scripts for SST - now it is made identical.
14) Fixed long-time broken encryption via xbcrypt in combination with
    mariabackup and added support for key-based encryption via openssl
    utility, which is now enabled by default for encrypt=1 mode (this
    default mode can be changed using a new configuration file option
    "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
    [sst] or in the [xtrabackup] section) - this change will allow us
    to use and to test the encypt=1 encryption without installing
    non-standard third-party utilities.

Fixed bugs caused by inaccuracies in automatic merging
from other branches:

1) Authentication information is not removed from the connection
   address, which causes some tests to fail;
2) wsrep_debug=on should be replaced with wsrep_debug=1;
3) Added missing "connection" lines to test result file;
4) Some tests have been corrected for Galera 4.x (10.4+).

1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.

This commit contains a large set of further bug fixes and
improvements to SST scripts for Galera, continuing the work
that was started in MDEV-24962 to make SST scripts work smoothly
in different network configurations (especially using ipv6) and
with different environment settings:

 1) The ipv6 addresses were incorrectly handled in the SST script
    for rsync (incorrect address substitution for establishing a
    connection, incorrect address substitution for bind, and so on);
 2) Checking the locality of the ip-address in SST scripts did not
    support ipv6 addresses (such as "[::1]"), which were falsely
    identified as non-local ip, which further did not allow running
    two SSTs on different local addresses on the same machine.
    On the other hand, this bug masked some other errors (related
    to handling ipv6 addresses);
 3) The code for checking the locality of the ip address was different
    in the SST scripts for rsync and for mysqldump, with individual
    flaws. This code is now made common and moved to wsrep_sst_common;
 4) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) in the wait_for_listen() and check_pid_and_port() functions
    did not process ipv6 addresses correctly in all cases (not for all
    branches);
 5) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) in the wait_for_listen() and check_pid_and_port() functions
    for some code branches could give a false positive result due to
    the textual match of prefixes in the port number and/or PID of
    the process;
 6) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) was supported through different utilities in SST scripts
    for mariabackup and for rsync, and with various minor flaws in
    the code. Now the code is still different in these scripts, but
    it supports a common set of utilities (lsof, ss, sockstat) and
    is synchronized across patterns that used to check the output
    of  these utilities;
 7) In SST via mariabackup, the signal about readiness to receive data
    is sometimes sent too early - immediately after listen(), and not
    after accept() (which are called by socat or netcat utility).
 8) Checking availability of the some options of some utilities was
    done using the grep pattern, which easily gives false positives;
 9) Common name (CN) for local addresses, if not explicitly specified,
    is now always replaced to "localhost" to avoid the need to generate
    many separate certificates for local addresses of one machine and
    not to depend on which the local address is currently used in test
    (ipv4 or ipv6, etc.);
10) In tests galera_sst_mariabackup_encrypt_with_key_server and
    galera_sst_rsync_encrypt_with_key_server the correct certificate
    is selected to avoid commonname (CN) mismatch problems;
11) Further refactoring to protect against spaces in file names.
12) Further general refactoring to eliminate bash-specific constructs
    or to improve code readability;
13) The code for setting options for the nc (netcat) utility was
    different in different scripts for SST - now it is made identical.
14) Fixed long-time broken encryption via xbcrypt in combination with
    mariabackup and added support for key-based encryption via openssl
    utility, which is now enabled by default for encrypt=1 mode (this
    default mode can be changed using a new configuration file option
    "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
    [sst] or in the [xtrabackup] section) - this change will allow us
    to use and to test the encypt=1 encryption without installing
    non-standard third-party utilities.

Fixed bugs caused by inaccuracies in automatic merging
from other branches:

1) Authentication information is not removed from the connection
   address, which causes some tests to fail;
2) wsrep_debug=on should be replaced with wsrep_debug=1;
3) Added missing "connection" lines to test result file;
4) Some tests have been corrected for Galera 4.x (10.4+).

During data file creation, InnoDB holds dict_sys mutex, tries to
write page 0 of the file and flushes the file. This not only causing
unnecessary contention but also a deviation from the write-ahead
logging protocol.

The clean sequence of operations is that we first start a dictionary
transaction and write SYS_TABLES and SYS_INDEXES records that identify
the tablespace. Then, we durably write a FILE_CREATE record to the
write-ahead log and create the file.

Recovery should not unnecessarily insist that the first page of each
data file that is referred to by the redo log is valid. It must be
enough that page 0 of the tablespace can be initialized based on the
redo log contents.

We introduce a new data structure deferred_spaces that keeps track
of corrupted-looking files during recovery. The data structure holds
the last LSN of a FILE_ record referring to the data file, the
tablespace identifier, and the last known file name.

There are two scenarios can happen during recovery:
i) Sufficient memory: InnoDB can reconstruct the
tablespace after parsing all redo log records.

ii) Insufficient memory(multiple apply phase): InnoDB should
store the deferred tablespace redo logs even though
tablespace is not present. InnoDB should start constructing
the tablespace when it first encounters deferred tablespace
id.

Mariabackup copies the zero filled ibd file in backup_fix_ddl() as
the extension of .new file. Mariabackup test case does page flushing
when it deals with DDL operation during backup operation.

fil_ibd_create(): Remove the write of page0 and flushing of file

fil_ibd_load(): Return FIL_LOAD_DEFER if the tablespace has
zero filled page0

Datafile: Clean up the error handling, and do not report errors
if we are in the middle of recovery. The caller will check
Datafile::m_defer.

fil_node_t::deferred: Indicates whether the tablespace loading was
deferred during recovery

FIL_LOAD_DEFER: Returned by fil_ibd_load() to indicate that tablespace
file was cannot be loaded.

recv_sys_t::recover_deferred(): Invoke deferred_spaces.create() to
initialize fil_space_t based on buffered metadata and records to
initialize page 0. Ignore the flags in fil_name_t, because they are
intentionally invalid.

fil_name_process(): Update deferred_spaces.

recv_sys_t::parse(): Store the redo log if the tablespace id
is present in deferred spaces

recv_sys_t::recover_low(): Should recover the first page of
the tablespace even though the tablespace instance is not
present

recv_sys_t::apply(): Initialize the deferred tablespace
before applying the deferred tablespace records

recv_validate_tablespace(): Skip the validation for deferred_spaces.

recv_rename_files(): Moved and revised from recv_sys_t::apply().
For deferred-recovery tablespaces, do not attempt to rename the
file if a deferred-recovery tablespace is associated with the name.

recv_recovery_from_checkpoint_start(): Invoke recv_rename_files()
and initialize all deferred tablespaces before applying redo log.

fil_node_t::read_page0(): Skip page0 validation if the tablespace
is deferred

buf_page_create_deferred(): A variant of buf_page_create() when
the fil_space_t is not available yet

This is joint work with Thirunarayanan Balathandayuthapani,
who implemented an initial prototype.

Compilers older than GCC 4.8 are not supported anyway.

1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.

1) This commit implements reading all sections from configuration
files while looking for the current value of any server variable,
which were previously only read from the [mysqld.suffix] group and
from [mysqld], but not from other groups such as [mariadb.suffix],
[mariadb] or, for example, [server].

2) This commit also fixes misrecognition of some parameters when
parsing a command line containing a special marker for the end
of the list of options ("--") or when short option names (such
as "-s", "-a" and "-h arg") chained together (like a "-sah arg").
Such parameters can be passed to the SST script in the list of
arguments after "--mysqld-args" if the server is started with a
complex set of options - this was revealed during manual testing
of changes to read configuration files.

3) The server-side preparation code for the "--mysqld-args"
option list has also been simplified to make it easier to change
in the future (if needed), and has been improved to properly
handle the special backquote ("`") character in the argument
values.

This commit contains a large set of further bug fixes and
improvements to SST scripts for Galera, continuing the work
that was started in MDEV-24962 to make SST scripts work smoothly
in different network configurations (especially using ipv6) and
with different environment settings:

 1) The ipv6 addresses were incorrectly handled in the SST script
    for rsync (incorrect address substitution for establishing a
    connection, incorrect address substitution for bind, and so on);
 2) Checking the locality of the ip-address in SST scripts did not
    support ipv6 addresses (such as "[::1]"), which were falsely
    identified as non-local ip, which further did not allow running
    two SSTs on different local addresses on the same machine.
    On the other hand, this bug masked some other errors (related
    to handling ipv6 addresses);
 3) The code for checking the locality of the ip address was different
    in the SST scripts for rsync and for mysqldump, with individual
    flaws. This code is now made common and moved to wsrep_sst_common;
 4) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) in the wait_for_listen() and check_pid_and_port() functions
    did not process ipv6 addresses correctly in all cases (not for all
    branches);
 5) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) in the wait_for_listen() and check_pid_and_port() functions
    for some code branches could give a false positive result due to
    the textual match of prefixes in the port number and/or PID of
    the process;
 6) Waiting for the start of the transport channel (socat, nc, rsync,
    stunnel) was supported through different utilities in SST scripts
    for mariabackup and for rsync, and with various minor flaws in
    the code. Now the code is still different in these scripts, but
    it supports a common set of utilities (lsof, ss, sockstat) and
    is synchronized across patterns that used to check the output
    of  these utilities;
 7) In SST via mariabackup, the signal about readiness to receive data
    is sometimes sent too early - immediately after listen(), and not
    after accept() (which are called by socat or netcat utility).
 8) Checking availability of the some options of some utilities was
    done using the grep pattern, which easily gives false positives;
 9) Common name (CN) for local addresses, if not explicitly specified,
    is now always replaced to "localhost" to avoid the need to generate
    many separate certificates for local addresses of one machine and
    not to depend on which the local address is currently used in test
    (ipv4 or ipv6, etc.);
10) In tests galera_sst_mariabackup_encrypt_with_key_server and
    galera_sst_rsync_encrypt_with_key_server the correct certificate
    is selected to avoid commonname (CN) mismatch problems;
11) Further refactoring to protect against spaces in file names.
12) Further general refactoring to eliminate bash-specific constructs
    or to improve code readability;
13) The code for setting options for the nc (netcat) utility was
    different in different scripts for SST - now it is made identical.
14) Fixed long-time broken encryption via xbcrypt in combination with
    mariabackup and added support for key-based encryption via openssl
    utility, which is now enabled by default for encrypt=1 mode (this
    default mode can be changed using a new configuration file option
    "encypt-format=openssl|xbcrypt", which can be placed in the [mysqld],
    [sst] or in the [xtrabackup] section) - this change will allow us
    to use and to test the encypt=1 encryption without installing
    non-standard third-party utilities.

Problem:
========
Aborting OPTIMIZE TABLE still logs in binary logs and replicates to the
Slave server. "Optimize table" command under execution, is killed by using
"Ctrl-C" as shown below.

MariaDB [test]> optimize table t2;
^CCtrl-C -- query killed. Continuing normally.

In spite of query execution being interrupted the query gets written to
binary log.

Analysis:
========
Admin command execution logic is not handling KILL command, hence it
ignores the KILL command and completes its execution.

Fix:
===
Check for thread killed notification, during admin command execution and
handle it. If thread kill occurs prior to any table modification the query
will not be written to binary log. If kill happens after at least one table
is modified then the query will be written to binary log. Ex: command in
execution is 'OPTIMIZE TABLE t1,t2' and the thread kill happens after t1
table is modified then 'OPTIMIZE TABLE t1,t2' will be written to binary log
as admin commands will not make the slave to diverge from master.

Problem:
=======
In slave_parallel_mode=optimistic configuration, when admin commands and
DML operation on the same table are scheduled simultaneously for execution,
it results in lock conflict and slave server either hangs due to
deadlock or goes down with an assert.

Analysis:
========
Admin commands OPTIMIZE, REPAIR and ANALYZE are written to binary log as
ordinary transactions. When 'slave_parallel_mode' is 'optimistic' DMLs are
allowed to run in parallel. But these locks are not detected by parallel
replication deadlock detection-and-handling mechanism. At times they result
in deadlock or assertion.

Fix:
===
Flag admin commands as DDL in Gtid_log_event at the time of writing to
binary log. Add a new bit EXECUTED_TABLE_ADMIN_CMD to
'm_unsafe_rollback_flags'. During 'mysql_admin_table' command execution it
accepts a list of tables to be processed and executes them in a loop. Upon
successful execution enable 'EXECUTED_TABLE_ADMIN_CMD' bit in
thd->transaction.stmt_unsafe_rollback_flags. Gtid_log_event constructor
will notice this flag and mark the current transaction with 'FL_DDL' flag.
Gtid_log_events marked as FL_DDL will not be scheduled parallel execution,
on the slave. They will execute in isolation to prevent deadlocks.

Note: Removed the call to 'trans_commit_implicit' from 'mysql_admin_table'
function as 'mysql_execute_command' will take care of invoking
'trans_commit_implicit'.

For the plugin thread during the server start the wsrep_on should be 0.

Post push fixes.

* Set binlog_expire_logs_seconds max valid value to 8553600
  i.e 99 * 24 * 60 * 60 (99 days)

The counter metadata_table_reference_count was not updated consistently
ever since mysql-server@65c0af9a1dedae43b63797134aff6b32304ced52
or commit 2e814d47
introduced dict_table_t::release().

The counter metadata_table_handles_closed was being incremented
unconditionally in ha_innobase::close(), while the corresponding
counter metadata_table_handles_opened would be incremented in
ha_innobase::open() if the function returned early due to an error.

The error code DB_TABLE_IS_BEING_USED was orphaned in
commit ddc663ef (MySQL 3.23.37).
Let us finally remove it.

MONITOR_TRX_ACTIVE: Remove. The count is not being updated consistently,
and it would also include read-only transactions that are otherwise
fully invisible to any other threads.

If it later turns out that a reliable count of active transactions
is needed, it can be exposed via a different interface.

trx_commit_for_mysql(): If the transaction was not started, return
immediately.

row_drop_table_after_create_fail(): Remove. This function was only used
during InnoDB initialization (or upgrade) when creating a system table
failed. Rollback will clean up a failed CREATE just fine, by invoking
dict_drop_index_tree().