- InnoDB tries to write FILE_CHECKPOINT marker during
early recovery when log file size is insufficient.
While updating the log checkpoint at the end of the recovery,
InnoDB must already have written out all pending changes
to the persistent files. To complete the checkpoint, InnoDB
has to write some log records for the checkpoint and to
update the checkpoint header. If the server gets killed
before updating the checkpoint header then it would lead
the logfile to be unrecoverable.

- This patch avoids FILE_CHECKPOINT marker during early
recovery and narrows down the window of opportunity to
make the log file unrecoverable.

There were erroneous calls for charpos() in key_hashnr() and key_buf_cmp().
These functions are never called with prefix segments.

The charpos() calls were wrong. Before the change BNHL joins
- could return wrong result sets, as reported in MDEV-34417
- were extremely slow for multi-byte character sets, because
  the hash was calculated on string prefixes, which increased
  the amount of collisions drastically.

This patch fixes the wrong result set as reported in MDEV-34417,
as well as (partially) the performance problem reported in MDEV-34352.

Changed error code for Galera unkillable threads to
be ER_KILL_DENIED_HIGH_PRIORITY giving message

This is a high priority thread/query and cannot be killed
without the compromising consistency of the cluster

also a warning is produced
  Thread %lld is [wsrep applier|high priority] and cannot be killed
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

MDEV-27966 Assertion `fixed()' failed and Assertion `fixed == 1' failed, both in Item_func_concat::val_str on SELECT after INSERT with collation utf32_bin on utf8_bin table

This problem was earlier fixed by this commit:

> commit 08c7ab40
> Author: Aleksey Midenkov <midenok@gmail.com>
> Date:   Mon Apr 18 12:44:27 2022 +0300
>
>    MDEV-24176 Server crashes after insert in the table with virtual
>    column generated using date_format() and if()

Adding an mtr test only.

After MDEV-4013, the maximum length of replication passwords was extended to
96 ASCII characters. After a restart, however, slaves only read the first 41
characters of MASTER_PASSWORD from the master.info file. This lead to slaves
unable to reconnect to the master after a restart.

After a slave restart, if a master.info file is detected, use the full
allowable length of the password rather than 41 characters.

Reviewed By:
============
Sergei Golubchik <serg@mariadb.com>

In the error case of THD::register_slave(), there is undefined
behavior of Slave_info si because it is allocated via malloc()
(my_malloc), and cleaned up via delete().

This patch makes these consistent by switching si's cleanup
to use my_free.

As all MariaDB Server errors now have a dedicated web page, the
perror utility is extended to include a link to the KB page of
the corresponding error code.

All new code of the whole pull request, including one or several
files that are either new files or modified ones, are contributed
under the BSD-new license. I am contributing on behalf of my
employer Amazon Web Services, Inc.

(It was accidentally removed by fix for MDEV-28846)

Review followup: RANGE_OPT_PARAM statement_should_be_aborted()
checks for thd->is_fatal_error and thd->is_error(). The first is
redundant when the second is present.

The optimizer deals with Rowid Filters this way:

1. First, range optimizer is invoked. It saves information
   about all potential range accesses.
2. A query plan is chosen. Suppose, it uses a Rowid Filter on
   index $IDX.
3. JOIN::make_range_rowid_filters() calls the range optimizer
again to create a quick select on index $IDX which will be used
to populate the rowid filter.

The problem: KILL command catches the query in step #3. Quick
Select is not created which causes a crash.

Fixed by checking if query was killed. Note: the problem also
affects 10.6, even if error handling for
SQL_SELECT::test_quick_select is different there.

Rowid Filter cannot be used with reverse-ordered scans, for the
same reason as IndexConditionPushdown cannot be.

test_if_skip_sort_order() already has logic to disable ICP when
setting up a reverse-ordered scan. Added logic to also disable
Rowid Filter in this case, factored out the code into
prepare_for_reverse_ordered_access(), and added a comment describing
the cause of this limitation.

This avoids a lot of SETVAL() results when applying a mysqldump with
sequences.

To make this possible, it was also necessary to enhance the mariadb
client with the option --print-query-on-error.
This option can also be very useful when running a batch of queries
through the mariadb client and one wants to find out where things goes
wrong.

TODO: It would be good to enhance mariadb_upgrade to not call the mariadb
client for executing queries but instead do this internally.  This
would have made this patch much easier!

Reviewed by: Sergei Golubchik <serg@mariadb.com>

- During recovery, InnoDB may fail to shrink the undo tablespaces
when there are no pages to recover while applying the redo log.
This issue exists only when innodb_undo_truncate is enabled.
trx_lists_init_at_db_start() could've applied the redo logs
for undo tablespace page0.

On an UBSAN clang-15 build, if running with UBSAN option
halt_on_error=1 (the issue doesn't show up without it),
MTR fails during mysqld --bootstrap with UBSAN error:

call to function io_callback(tpool::aiocb*) through pointer to incorrect function type 'void (*)(void *)'

This patch corrects the parameter type of io_callback
to match its expected type defined by callback_func,
i.e. (void*).

Reviewed By:
============
<TODO>

The bug can happens on macOS, if server closes the socket without sending
error packet to client. Closing the socket on server side is legitimate,
and happen e.g when write timeout occurs, perhaps also other situations.

However mysqltest is not prepared to handle mysql_errno 0, and erroneously
thinks connection was successfully established.

The fix/workaround in mysqltest is to treat client failure with
mysql_errno 0 the same as CR_SERVER_LOST (generic client-side
communication error)

The real fix in client library would ensure that mysql_errno is set
on errors.

Otherwise it may result in nonsensical values like 190 for a boolean.

Immediately close down the signal handler loop when we decide to
break connections as it's the start of process termination
anyway, and there's no need to wait once we've invoked break_connections.

--skip-not-found switch tells mtr to skip not found tests instead of aborting.
But it failed to skip the test if the suite name was not found.

This problem also made the *last-N-failed builbot builders fail
to run `mtr --skip-not-found` if the last commit removed a file in
the mysql-test/include/ directory.

This commit fixes it, now the not found test is properly skipped,
no matter what component of the test name was not found:

$ ./mtr main.foo --skip-not-found foo.main
...
==============================================================================
TEST                                  WORKER RESULT   TIME (ms) or COMMENT
--------------------------------------------------------------------------
foo.main                                 [ skipped ]  not found
main.foo                                 [ skipped ]  not found
--------------------------------------------------------------------------

In cmake -DWITH_UBSAN=ON builds with clang but not with GCC,
-fsanitize=undefined will flag several runtime errors on
function pointer mismatch related to the lock-free hash table LF_HASH.

Let us use matching function signatures and remove function pointer
casts in order to avoid potential bugs due to undefined behaviour.

These errors could be caught at compilation time by
-Wcast-function-type-strict, which is available starting with clang-16,
but not available in any version of GCC as of now. The old GCC flag
-Wcast-function-type is enabled as part of -Wextra, but it specifically
does not catch these errors.

Reviewed by: Vladislav Vaintroub

MDEV-34227 On startup: UBSAN: runtime error: applying non-zero offset in JOIN::make_aggr_tables_info in sql/sql_select.cc

Avoid undefined behaviour (applying offset to nullptr).
The reported scenario is covered in mysql-test/connect-no-db.test
No new tests needed.

MDEV-32376 SHOW CREATE DATABASE statement crashes the server when db name contains some unicode characters, ASAN stack-buffer-overflow

Adding the test for the length of lex->name into show_create_db().

Without this test writes beyond the end of db_name_buff were possible
upon a too long database name.

MDEV-34237: On Startup: UBSAN: runtime error: call to function MDL_lock::lf_hash_initializer lf_hash_insert through pointer to incorrect function type 'void (*)(st_lf_hash *, void *, const void *)'

A few different incorrect function type UBSAN issues have been
grouped into this patch.

The only real potentially undefined behavior is an error about
show_func_mutex_instances_lost, which when invoked in
sql_show.cc::show_status_array(), puts 5 arguments onto the stack;
however, the implementing function only actually has 3 parameters (so
only 3 would be popped). This was fixed by adding in the remaining
parameters to satisfy the type mysql_show_var_func.

The rest of the findings are pointer type mismatches that wouldn't
lead to actual undefined behavior. The lf_hash_initializer function
type definition is

typedef void (*lf_hash_initializer)(LF_HASH *hash, void *dst, const void *src);

but the MDL_lock and table cache's implementations of this function
do not have that signature. The MDL_lock has specific MDL object
parameters:

static void lf_hash_initializer(LF_HASH *hash __attribute__((unused)),
                                MDL_lock *lock, MDL_key *key_arg)

and the table cache has specific TDC parameters:

static void tdc_hash_initializer(LF_HASH *,
                                 TDC_element *element, LEX_STRING *key)

leading to UBSAN runtime errors when invoking these functions.

This patch fixes these type mis-matches by changing the
implementing functions to use void * and const void * for their
respective parameters, and later casting them to their expected
type in the function body.

Note too the functions tdc_hash_key and tc_purge_callback had
a similar problem to tdc_hash_initializer and was fixed
similarly.

Reviewed By:
============
Sergei Golubchik <serg@mariadb.com>

The code is slightly simplified taking into account
the fact that partition_ht() always returns a normal
hton when there is no partitioning.

This is regression from commit 3228c08f. Problem is that
when table storage engine is determined there should be
check is table partitioned and if it is then determine
partition implementing storage engine.

Reported bug is reproducible only with --log-bin so make
sure tests changed by 3228c08f and new test are run
with --log-bin and binlog disabled.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

The strncpy() wrapper that was introduced in
commit 567b6812
is checking whether the output was truncated even in cases
where the caller does not care about it.

Let us introduce a separate function safe_strcpy_truncated() that
indidates whether the output was truncated.

              number of non-user tablespace.

fil_space_t::try_to_close(): Don't try to close
the tablespace which is acquired by the caller of
the function

Added the suppression message in open_files_limit test case

"Process" sandbox short command put by masqldump to avoid an error.

In commit 9ec7819c the CRC-32 function
signatures had been unified somewhat, but not enough.

clang -fsanitize=undefined would flag a function pointer signature
mismatch between const char* and const void*, but not between
uint32_t and unsigned. We try to fix both inconsistencies anyway.

Reviewed by: Vladislav Vaintroub

		number of non-user tablespace.

- InnoDB only closes the user tablespace when the number of open
files exceeds innodb_open_files limit. In that case, InnoDB should
make sure that innodb_open_files value should be greater
than number of undo tablespace, system and temporary tablespace files.

wsrep_sst_mariabackup should use the tmpdir defined by
the user under the '[mysqld]' section of the configuration
file rather than the default '/tmp' directory.

Some galera tests starts 6 galera nodes. Each galera node requires
three ports: 6*3 = 18. Plus 6 ports are needed for 6 mariadbd servers.
Since the number of ports is rounded up to 10 everywhere in mtr, we
will take 30 as the default value for the port group size parameter.

Avoid starting transactions in wsrep-lib side when wsrep is
disabled. It is unnecessary, and causes spurious deadlock errors on
transaction clean up.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Problem was that updates to mysql.gtid_slave_pos table were
replicated even when they were newer used and because that
newer deleted. Avoid replication of mysql.gtid_slave_pos
table if wsrep_gtid_mode=OFF.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Problem:
=======
- This commit is a merge of mysql commit 129ee47ef994652081a11ee9040c0488e5275b14.
InnoDB FTS can be in inconsistent state when sync operation
terminates the server before committing the operation. This
could lead to incorrect synced doc id and incorrect query results.

Solution:
========
- During sync commit operation, InnoDB should pass
the sync transaction to update the max doc id
in the config table.

fts_read_synced_doc_id() : This function is used
to read only synced doc id from the config table.