Modern software (including text editors, static analysis software,
and web-based code review interfaces) often requires source code files
to be interpretable via a consistent character encoding, with UTF-8 or
ASCII (a strict subset of UTF-8) as the default. Several of the MariaDB
source files contain bytes that are not valid in either the UTF-8 or
ASCII encodings, but instead represent strings encoded in the
ISO-8859-1/Latin-1 or ISO-8859-2/Latin-2 encodings.

These inconsistent encodings may prevent software from correctly
presenting or processing such files. Converting all source files to
valid UTF8 characters will ensure correct handling.

Comments written in Czech were replaced with lightly-corrected
translations from Google Translate. Additionally, comments describing
the proper handling of special characters were changed so that the
comments are now purely UTF8.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer
Amazon Web Services, Inc.
Co-authored-by: Andrew Hutchings <andrew@linuxjedi.co.uk>

In commit 0b47c126 (MDEV-13542)
a few calls to mtr_t::memo_push() were moved before a write latch
on the page was acquired. This introduced a race condition:

1. is_block_dirtied() returned false to mtr_t::memo_push()
2. buf_page_t::write_complete() was executed, the block marked clean,
and a page latch released
3. The page latch was acquired by the caller of mtr_t::memo_push(),
and mtr_t::m_made_dirty was not set even though the block is in
a clean state.

The impact of this race condition is that crash recovery and backups
may fail.

btr_cur_latch_leaves(), btr_store_big_rec_extern_fields(),
btr_free_externally_stored_field(), trx_purge_free_segment():
Acquire the page latch before invoking mtr_t::memo_push().
This fixes the regression caused by MDEV-13542.

Side note: It would suffice to set mtr_t::m_made_dirty at the time
we set the MTR_MEMO_MODIFY flag for a block. Currently that flag is
unnecessarily set if a mini-transaction acquires a page latch on
a page that is in a clean state, and will not actually modify the block.
This may cause unnecessary acquisitions of log_sys.flush_order_mutex
on mtr_t::commit().

mtr_t::free(): If the block had been exclusively latched in this
mini-transaction, set the m_made_dirty flag so that the flush order mutex
will be acquired during mtr_t::commit(). This should have been part of
commit 4179f93d (MDEV-18976).
It was necessary to change mtr_t::free() so that
WriteOPT_PAGE_CHECKSUM::operator() would be able to avoid writing
checksums for freed pages.

buf_defer_drop_ahi(): Remove. Ever since
commit c7f8cfc9 (MDEV-27700)
it is safe to invoke btr_search_drop_page_hash_index(block, true)
to remove an orphan adaptive hash index.

Any attempt to upgrade page latches is prone to deadlocks. Recently,
we observed a few hangs that involved nothing more than a small table
consisting of one clustered index page, one secondary index page and
some undo pages.

followup for "remove invalid options from warning messages"

The issue is that trx_t::lock.was_chosen_as_deadlock_victim can be reset
before the transaction check it and set trx_t::error_state.

The fix is to reset trx_t::lock.was_chosen_as_deadlock_victim only in
trx_t::commit_in_memory(), which is invoked on full rollback. There is
also no need to have separate bit in
trx_t::lock.was_chosen_as_deadlock_victim to flag transaction it was
chosen as a victim of Galera conflict resolution, the same variable can be
used for both cases except debug build. For debug build we need to
distinguish deadlock and Galera's abort victims for debug checks. Also
there is no need to check for deadlock in lock_table_enqueue_waiting() for
Galera as the coresponding check presents in lock_wait().

Local variable "error_state" in lock_wait() was replaced with
trx->error_state, because before the replace
lock_sys_t::cancel<false>(trx, lock) and lock_sys.deadlock_check() could
change trx->error_state, which then could be overwritten with the local
"error_state" variable value.

The lock_wait_suspend_thread_enter DEBUG_SYNC point name is misleading,
because lock_wait_suspend_thread was eliminated in e71e6133. It was renamed
to lock_wait_start.

Reviewed by: Marko Mäkelä, Jan Lindström.

Removed use std::vector's ba push_back(), pop_back() to  make it more
obvious that memory in the vectors won't be reallocated.

Also, "borrowed" elements can be debugged a little better now,
they are put into the start of the m_cache vector.

MDEV-29319 Assertion failure size_in_header >= space.free_limit in fsp_get_available_space_in_free_extents()

- Don't remove the constant parameter in fsp_get_available_space_in_free_extents()

- InnoDB fts table initially added to LRU table cache
while creating the table. Later, table was marked
as non-evicted when we add the table to fts optimizer
list. Before marking the table as non-evicted, master
thread can try to evict the fts table.

MDEV-29319 Assertion failure size_in_header >= space.free_limit in fsp_get_available_space_in_free_extents()

- Race condition between fsp_get_available_space_in_free_extents()
and fsp_try_extend_data_file() while accessing space.free_limit.
Before calling fsp_get_available_space_in_free_extents(), take
shared lock on space->latch.

- Newly created InnoDB fulltext index does have only one column
and doesn't associate with primary key fields during DDL.
init_change_cols() has strict assertion that number of fields
should be greater than number of collation change columns.

Many Spider tests were broken by the merge commit, 36d173e5.

This reverts part of commit 212994f7
(MDEV-28974) and implements a better fix that works in that special case
while avoiding other failures.

fil_name_process(): Do not rename the tablespace in deferred_spaces;
it already contains the latest name for the space id.

deferred_spaces::create(): In mariadb-backup --prepare,
replace absolute data directory file path with short name
relative to the backup directory and store it as filename while
deferring tablespace file creation.

The index root page contains the fields BTR_SEG_TOP and BTR_SEG_LEAF
which keep track of allocated pages in the index tree. These fields
are normally protected by an Update latch, so that concurrent read
access to other parts of the page will be possible.

When the index root page is already exclusively latched in the
mini-transaction, we must not try to acquire a lower-grade Update latch.
In fact, when the root page is already X or U latched in the
mini-transaction, there is no point to acquire another latch.
Moreover, after a U latch was acquired on top of an X-latch,
mtr_t::defer_drop_ahi() would trigger an assertion failure or
lock corruption in block->page.lock.u_x_upgrade() because X locks
already exist on the block.

This problem may have been introduced in
commit 03ca6495 (MDEV-24142).

btr_page_alloc_low(), btr_page_free(): Initially buffer-fix the root page.
If it is already U or X latched, release the buffer-fix. Else, upgrade
the buffer-fix to a U latch.

mtr_t::u_lock_register(): Upgrade a buffer-fix to U latch.

mtr_t::have_u_or_x_latch(): Check if U or X latches are already
registered in the mini-transaction.

This problem was earlier fixed by MDEV-27101.
Adding INET6 tests only.

Problem:
========
When replicating SET DEFAULT ROLE, the pre-update check (i.e. that
in set_var_default_role::check()) tries to validate the existence of
the given rules/user even when the targeted tables are ignored. When
previously issued CREATE USER/ROLE commands are ignored by the
replica because of the replication filtering rules, this results in
an error because the targeted data does not exist.

Solution:
========
Before checking that the given roles/user exist of a SET DEFAULT
ROLE command, first ensure that the mysql.user and
mysql.roles_mapping tables are not excluded by replication filters.

Reviewed By:
============
Andrei Elkin <andrei.elkin@mariadb.com>
Sergei Golubchik <serg@mariadb.com>

Reason:
=======
Race condition between btr_search_drop_hash_index() and
btr_search_lazy_free(). One thread does resizing of buffer pool
and clears the ahi on all pages in the buffer pool, frees the
index and table while removing the last reference. At the same time,
other thread access index->heap in btr_search_drop_hash_index().

Solution:
=========
Acquire the respective ahi latch before checking index->freed()

btr_search_drop_page_hash_index(): Added new parameter to indicate
that drop ahi entries only if the index is marked as freed

btr_search_check_marked_free_index(): Acquire all ahi latches and
return true if the index was freed

TIMESTAMP columns were compared as strings in ALL/ANY comparison,
which did not work well near DST time change.

Changing ALL/ANY comparison to use "Native" representation to compare
TIMESTAMP columns, like simple comparison does.

Ever since commit 09177ead
the test innodb.row_format_redundant cannot work when the
data file was created with innodb_checksum_algorithm=full_crc32.

Backport of 10.5 commit a9d0bb12

As pointed out with MDEV-29308 there are issues with the code as is.
MariaDB is built as C++11 / C99. aligned_alloc() is not guarenteed
to be exposed when building with any mode other than C++17 / C11.
The other *BSD's have their stdlib.h header to expose the function
with C+11 anyway, but the issue exists in the C99 code too, the
build just does not use -Werror. Linux globally defines _GNU_SOURCE
hiding the issue as well.

Reason:
======
 This issue is caused by race condition between fulltext DDL and
purge thread. DDL sets the signal to stop the purge thread to
process the new undo log records and wait for the ongoing
processed FTS table undo log records to finish.

But in dict_acquire_mdl_shared(),InnoDB release all innodb
table related locks before acquiring the mdl. At the same time,
DDL assumes that there are no purge threads working on fts table.

There is a possiblity that purge thread can skip processing the
valid undo log records if it checks purge_sys.must_wait_FTS() twice
in different places.

Solution:
==========

Add the purge_sys.must_wait_FTS() check in dict_acquire_mdl_shared()
to avoid the purge thread processing undo log records.

dict_open_table_on_id(): return -1 if the purge thread has to
wait

dict_acquire_mdl_shared(): Added 1 new parameters to indicate that
purge thread invoking the function, return -1  if the purge
thread has to wait.

Even though commit b817afaa passed
the test mariabackup.compress_qpress, that test turned out to be
too small to reveal one more problem that had previously been prevented
by the existence of ctrl_mutex. I did not realize that there can be
multiple concurrent callers to compress_write(). One of them is the
log copying thread; further callers are data file copying threads
(default: --parallel=1).

By default, there is only one compression worker thread
(--compress-threads=1).

compress_write(): Fix a race condition between threads that would
use the same worker thread object. Make thd->data_avail contain the
thread identifier of the submitter, and add thd->avail_cond to
notify other compress_write() threads that are waiting for a slot.

In MySQL 5.7, rollback segments 1 to 32 are used for temporary tables,
which is an unnecessary file format change from MySQL 5.6.
This format change was avoided in MariaDB Server by
commit 124bae08 (MDEV-12289).

An upgrade from MySQL 5.7 would crash due to dereferencing a null pointer,
which is a regression due to
commit 0b47c126 (MDEV-13542).

trx_rseg_t::get(): Return nullptr if no tablespace exists. This is where
the upgrade would crash.

trx_rseg_mem_restore(): Return DB_TABLESPACE_NOT_FOUND if the
undo tablespace does not exist. This is likely dead code.

   - While creating a new InnoDB segment, allocates the extent
before allocating the inode or page allocation even though
the pages are present in fragment segment. This patch does
reserve the extent when InnoDB ran out of fragment pages
in the tablespace.

failures