> After May 31, 2024, CentOS Stream 8 will be archived and no further
updates will be provided. [1]

CentOS Stream 8 is now EOL and should be updated to using CentOS Stream
9 for compatibility testing in GitLab CI.

[1] https://blog.centos.org/2023/04/end-dates-are-coming-for-centos-stream-8-and-centos-linux-7/
https://www.centos.org/centos-linux-eol/

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services.

MDEV-28345 ASAN: use-after-poison or unknown-crash in my_strtod_int from charset_info_st::strntod or test_if_number

This patch fixes two problems:

- The code inside my_strtod_int() in strings/dtoa.c could test the byte
  behind the end of the string when processing the mantissa.
  Rewriting the code to avoid this.

- The code in test_if_number() in sql/sql_analyse.cc called my_atof()
  which is unsafe and makes the called my_strtod_int() look behind
  the end of the string if the input string is not 0-terminated.
  Fixing test_if_number() to use my_strtod() instead, passing the correct
  end pointer.

Mroonga UDFs can't be used without loading Mroonga.

Specifying filters (filter_status, filter_user, etc) in the mytop config
previously wouldn't work, because any filter specified here was added to
the config hash as a literal string.

This change fixes that - if filter_* is defined in the config and matches
an existing filter_* key, then run the value through StringOrRegex() and
assign to the config hash.

Item_exists_subselect::fix_length_and_dec() sets explicit_limit to 1.
In the exists2in transformation it resets select_limit to NULL. For
consistency we should reset explicity_limit too.

This fixes a bug where spider table returns wrong results for queries
that gets through exists2in transformation when semijoin is off.

To avoid off-by-one in spider_get_share.

And document conn key and conn key length.

Distinguish them in two place:

when constructing connection key in create_conn_key and
spider_create_conn for both ordinary queries and spider_direct_sql

For spider_create_conn and spider_udf_direct_sql_create_conn, the
created conn gets assigned a field from the source object if and only
if source->field is non-null.

For spider_create_conn_keys and spider_udf_direct_sql_create_conn_key,
we update the encoding so that absence of keyword and keyword with an
empty value result in different keys. More specifically, if the i-th
keyword has a value, the corresponding part in the conn key begins
with the char \i, followed by the possibly empty value and ends with a
\0. If the i-th keyword is not specified, then it does not get a
mention in the conn key.

As part of this change, we also update table param / option parsing to
recognise a singleton empty string when creating an string list,
instead of writing it off as NULL.

When connection parameters are specified in COMMENT, CONNECTION and
SERVER, they are overriden in the following order:

COMMENT > CONNECTION > SERVER

Blindly recursive chown is not way to do it.
This Workaround is not much better than just chown -R but
there is small adjustment just chown MariaDB statedir and logdir
then with find only chown those files that are not correctly
owned.

Fixes lintian warnings:
 * W: mariadb-server: recursive-privilege-change "chown -R" [postinst:*]
 * W: mariadb-server: recursive-privilege-change "chown -R" [postinst:*]

Based on the current logic, objects of classes Item_func_charset and
Item_func_coercibility (responsible for CHARSET() and COERCIBILITY()
functions) are always considered constant.
However, SQL syntax allows their use in a non-constant manner, such as
CHARSET(t1.a), COERCIBILITY(t1.a).

In these cases, the `used_tables()` parameter corresponds to table names
in the function parameters, creating an inconsistency: the item is marked
as constant but accesses tables. This leads to crashes when
conditions with CHARSET()/COERCIBILITY() are pushed into derived tables.

This commit addresses the issue by setting `used_tables()` to 0 for
`Item_func_charset` and `Item_func_coercibility`. Additionally, the items
now store the return values during the preparation phase and return
them during the execution phase. This ensures that the items do not call
its arguments methods during the execution and are truly constant.

Reviewer: Alexander Barkov <bar@mariadb.com>

ENGINE_SUBSTITUTION only applies to CREATE TABLE and ALTER TABLE, and
Storage_engine_name::resolve_storage_engine_with_error() could be
called when executing any sql command.

SPIDER_CONN::loop_check_meraged_first is useless, because all
SPIDER_CONN_LOOP_CHECKs are in SPIDER_CONN::loop_check_queue, which in
spider_db_conn::fin_loop_check() is iterated over.

This fixes the use-after-free issue when there are three spider tables
sharing the same remote, and their corresponding
SPIDER_CONN_LOOP_CHECKs getting merged in
spider_conn_queue_and_merge_loop_check()

This also fixes MDEV-34555

Same cause as MDEV-31996. This prevents reuse of conns freed
previously, e.g. during the commit of previous statement.

It does not occur in 10.4 because of the commit for MDEV-19002 removed
the call to spider_check_trx_and_get_conn().

This allows creation of SPIDER_CONN on demand, if the previous one was
freed.

Also, the first_link_idx's are reset during
spider_check_trx_and_get_conn(), which in the case of remote HANDLER
commands, might not match the link to use correct first_link_idx for
remote HANDLER statement that was later set in
ha_spider::rnd_handler_init() (causing testing regressions in the
spider/handler suite). Therefore we fix the first_link_idx there.

A SPIDER_TRX_HA associated with a SPIDER_TRX could have longer
lifetime than its associated SPIDER_SHARE. And it is identified with
the associated table name. When the SPIDER_SHARE no longer valid, e.g.
when the associated spider table has been dropped and recreated, the
SPIDER_TRX_HA should be reset too.

Since spider could create a new SPIDER_SHARE with the exact same
address of a freed SPIDER_SHARE, we try to mark all SPIDER_TRX_HAs
associated with a SPIDER_SHARE invalid when the SPIDER_SHARE is about
to be freed.

Some documentations come from commit for MDEV-29676, and the rest are
mainly related to SPIDER_TRX_HA.

Removed SPIDER_TRX_HA::trx as it is unused.

Reason for the error is that --defaults-group-suffix is passed twice
to the command line client, and option parser is not prepared for this.

Double occurence of comes from 2 invocations of parse_arguments(), which
appends unparsed arguments each time it is called.

Fixed by treating --defaults-group-suffix like other "--defaults-"
(--defaults-file/--defaults-extra-file). it will be parsed, and thus
passed only once to the command line client.

Address Sanitizer's know how to detect stack overrun, so there's
no point in us doing it.

As evidenced by perfschema tests where signficant test failures
because this function failed under ASAN (MDEV-33210).

Also, so since clang-16, we cannot assume much about how local
variables are allocated on the stack (MDEV-31605).

Disabling check idea thanks to Sanja.

MDEV-34490 get_copy() and build_clone() may return an instance of an ancestor class instead of a copy/clone

The `Item` class methods `get_copy()`, `build_clone()`, and `clone_item()`
face an issue where they may be defined in a descendant class
(e.g., `Item_func`) but not in a further descendant (e.g., `Item_func_child`).
This can lead to scenarios where `build_clone()`, when operating on an
instance of `Item_func_child` with a pointer to the base class (`Item`),
returns an instance of `Item_func` instead of `Item_func_child`.

Since this limitation cannot be resolved at compile time, this commit
introduces runtime type checks for the copy/clone operations.
A debug assertion will now trigger in case of a type mismatch.

`get_copy()`, `build_clone()`, and `clone_item()` are no more virtual,
but virtual `do_get_copy()`, `do_build_clone()`, and `do_clone_item()`
are added to the protected section of the class `Item`.

Additionally, const qualifiers have been added to certain methods
to enhance code reliability.

Reviewer: Oleksandr Byelkin <sanja@mariadb.com>

MDEV-34474 InnoDB: Failing assertion: stat_n_leaf_pages > 0 in ha_innobase::estimate_rows_upper_bound

- Fixing the compilation issue.

This fixes a valgrind failure where the bulk_size is used before
initialised in ha_spider::end_bulk_insert().

MDEV-34542  Assertion `lock_trx_has_sys_table_locks(trx) == __null' failed in void row_mysql_unfreeze_data_dictionary(trx_t*)

- During XA PREPARE, InnoDB releases the non-exclusive locks.
But it fails to remove the non-exclusive table lock from the
transaction table locks. In the mean time, main thread evicts
the table from the LRU cache. While rollbacking the XA transaction,
InnoDB iterates through the table locks to check whether it
holds lock on any system tables and wrongly assumes the
evicted table as system table since the table id is 0

Fix:
===
During XA PREPARE, remove the table locks of the transaction while
releasing the non-exclusive locks.

Problem:
========
- During shutdown, InnoDB tries to free the asynchronous
I/O slots and hangs. The reason is that InnoDB disables
asynchronous I/O before waiting for pending
asynchronous I/O to finish.

buf_load(): InnoDB aborts the buffer pool load due to
user requested shutdown and doesn't wait for the asynchronous
read to get completed. This could lead to debug assertion
in buf_flush_buffer_pool() during shutdown

Fix:
===
os_aio_free(): Should wait all read_slots and write_slots
to finish before disabling the aio.

buf_load(): Should wait for pending read request to complete
even though it was aborted.

Simplify in an attempt to avoid:

mysqltest: At line 275: File already exist: on the write_file
lines.

Using write_line as that's what a lot of other tests
do for writing small bits to a expect file.

Review thanks Valdislav Vaintroub

MDEV-34041 Display additional information for materialized subqueries in EXPLAIN/ANALYZE FORMAT=JSON

This commits adds the "materialization" block to the output of
EXPLAIN/ANALYZE FORMAT=JSON when materialized subqueries are involved
into processing. In the case of ANALYZE additional runtime information
is displayed, such as:
  - chosen strategy of materialization
  - number of partial match/index lookup loops
  - sizes of partial match buffers

from HAVING

The bug is caused by refixing of the constant subquery in pushdown from
HAVING into WHERE optimization.

Similarly to MDEV-29363 in the problematic query two references of the
constant subquery are used. After the pushdown one of the references of the
subquery is pushed into WHERE-clause and the second one remains as the part
of the HAVING-clause.
Before the represented fix, the constant subquery reference that was going to
be pushed into WHERE was cleaned up and fixed. That caused the changes of
the subquery itself and, therefore, changes for the second reference that
remained in HAVING. These changes caused a crash.

To fix this problem all constant objects that are going to be pushed into
WHERE should be marked with an IMMUTABLE_FL flag. Objects marked with this
flag are not cleaned up or fixed in the pushdown optimization.

Approved by Igor Babaev <igor@mariadb.com>

The version test on not_valgrind_build.inc was
broken as in BB the sp-no-valgrind.test was
executed.

The implication that it wouldn't work on ASAN
was also incorrect as ASAN tests show it running
fine there.

Correct sp-no-valgrind.test for not_valgrind.inc.

There are two problems.

First, replication fails when XA transactions are used where the
slave has replicate_do_db set and the client has touched a different
database when running DML such as inserts. This is because XA
commands are not treated as keywords, and are thereby not exempt
from the replication filter. The effect of this is that during an XA
transaction, if its logged “use db” from the master is filtered out
by the replication filter, then XA END will be ignored, yet its
corresponding XA PREPARE will be executed in an invalid state,
thereby breaking replication.

Second, if the slave replicates an XA transaction which results in
an empty transaction, the XA START through XA PREPARE first phase of
the transaction won’t be binlogged, yet the XA COMMIT will be
binlogged. This will break replication in chain configurations.

The first problem is fixed by treating XA commands in
Query_log_event as keywords, thus allowing them to bypass the
replication filter. Note that Query_log_event::is_trans_keyword() is
changed to accept a new parameter to define its mode, to either
check for XA commands or regular transaction commands, but not both.
In addition, mysqlbinlog is adapted to use this mode so its
--database filter does not remove XA commands from its output.

The second problem fixed by overwriting the XA state in the XID
cache to be XA_ROLLBACK_ONLY, so at commit time, the server knows to
rollback the transaction and skip its binlogging. If the xid cache
is cleared before an XA transaction receives its completion command
(e.g. on server shutdown), then before reporting ER_XAER_NOTA when
the completion command is executed, the filter is first checked if
the database is ignored, and if so, the error is ignored.

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>
Andrei Elkin <andrei.elkin@mariadb.com>

The server does not log errors after startup when it is started without the
--console parameter and not as a service. This issue arises due to an
undocumented behavior of FreeConsole() in Windows when only a single
process (mariadbd/mysqld) is attached to it, causing the window to close.

In this case stderr is redirected to a file before FreeConsole()
is called. Procmon shows FreeConsole closing file handle
subsequent writes to stderr fail with ERROR_INVALID_HANDLE because
WriteFile() cannot operate on the closed handle. This results in losing
all messages after startup, including warnings, errors, notes, and
crash reports.

Additionally, some users reported stderr being redirected to
multi-master.info and failing at startup, but this could not be reproduced
here.

The workaround involves calling FreeConsole() right before the redirection of
stdout/stderr. This fix has been tested with XAMPP and via cmd.exe using
"start mysqld". Automated testing using MTR is challenging for this case.

The fix is only applicable to version 10.5. In later versions, the
FreeConsole() call has been removed.

MDEV-34474 InnoDB: Failing assertion: stat_n_leaf_pages > 0 in ha_innobase::estimate_rows_upper_bound

- Column stat_value and sample_size in mysql.innodb_index_stats
table is declared as BIGINT UNSIGNED without any check constraint.
user manually updates the value of stat_value and sample_size
to zero. InnoDB aborts the server while reading the statistics
information because InnoDB expects at least one leaf
page to exist for the index.

- To fix this issue, InnoDB should interpret the value of
stat_n_leaf_pages, stat_index_size in innodb_index_stats
stat_clustered_index_size, stat_sum_of_other_index_sizes
in innodb_table_stats as valid one even though user
mentioned it as 0.

The test has been made more stable according to
the recommendations of the Codership team.

The problem was in error message suppression, which did not match
the actual warning messages, due to bad quotations.
Changed warnings message suppressions to more simple format.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

DML transactions on FK-child tables also get table locks
on FK-parent tables. If there is a DML transaction holding
such a lock, and a TOI transaction starts, the latter
BF-aborts the former and puts itself into a waiting state.
If at this moment another DML transaction on FK-child table
starts, it doesn't check that the transaction waiting on
a parent table lock is TOI, and it erroneously BF-aborts
the waiting TOI transaction.

The fix: don't roll back high-priority transaction waiting
on a lock in InnoDB, instead roll back an incoming DML
transaction.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

The IO thread can report error code 2013 into the error log when it
is stopped during the initial connection process to the primary, as
well as when trying to read an event. However, because the IO thread
is being stopped, its connection to the primary is force-killed by
the signaling thread (see THD::awake_no_mutex()), and thereby these
connection errors should be ignored.

Reviewed By:
============
Kristian Nielsen <knielsen@knielsen-hq.org>