PROBLEM
-------

1. The customer had presented a stack which had many threads waiting on
   multiple mutexes like LOCK_Status, srv_innodb_monitor_mutex,  ibuf_mutex etc.
2. The root cause was that the AHI latch was held in S (shared) mode by the a thread which was
   doing a truncate of a large table .
3. There was another thread which was trying to acquire the AHI latch in X (exclusive) mode
4. With our lock implementation any thread requesting a X lock ,blocks rest of the threads
   requesting S(shared) locks,this caused many threads to wait for this shared lock.
5. The main reason why we hold the latches in truncate is to avoid disabling of AHI
   during truncate

FIX

The easiest way to compile and test the server with UBSAN is to run:
./BUILD/compile-pentium64-ubsan
and then run mysql-test-run.
After this commit, one should be able to run this without any UBSAN
warnings. There is still a few compiler warnings that should be fixed
at some point, but these do not expose any real bugs.

The 'special' cases where we disable, suppress or circumvent UBSAN are:
- ref10 source (as here we intentionally do some shifts that UBSAN
  complains about.
- x86 version of optimized int#korr() methods. UBSAN do not like unaligned
  memory access of integers.  Fixed by using byte_order_generic.h when
  compiling with UBSAN
- We use smaller thread stack with ASAN and UBSAN, which forced me to
  disable a few tests that prints the thread stack size.
- Verifying class types does not work for shared libraries. I added
  suppression in mysql-test-run.pl for this case.
- Added '#ifdef WITH_UBSAN' when using integer arithmetic where it is
  safe to have overflows (two cases, in item_func.cc).

Things fixed:
- Don't left shift signed values
  (byte_order_generic.h, mysqltest.c, item_sum.cc and many more)
- Don't assign not non existing values to enum variables.
- Ensure that bool and enum values are properly initialized in
  constructors.  This was needed as UBSAN checks that these types has
  correct values when one copies an object.
  (gcalc_tools.h, ha_partition.cc, item_sum.cc, partition_element.h ...)
- Ensure we do not called handler functions on unallocated objects or
  deleted objects.
  (events.cc, sql_acl.cc).
- Fixed bugs in Item_sp::Item_sp() where we did not call constructor
  on Query_arena object.
- Fixed several cast of objects to an incompatible class!
  (Item.cc, Item_buff.cc, item_timefunc.cc, opt_subselect.cc, sql_acl.cc,
   sql_select.cc ...)
- Ensure we do not do integer arithmetic that causes over or underflows.
  This includes also ++ and -- of integers.
  (Item_func.cc, Item_strfunc.cc, item_timefunc.cc, sql_base.cc ...)
- Added JSON_VALUE_UNITIALIZED to json_value_types and ensure that
  value_type is initialized to this instead of to -1, which is not a valid
  enum value for json_value_types.
- Ensure we do not call memcpy() when second argument could be null.
- Fixed that Item_func_str::make_empty_result() creates an empty string
  instead of a null string (safer as it ensures we do not do arithmetic
  on null strings).

Other things:

- Changed struct st_position to an OBJECT and added an initialization
  function to it to ensure that we do not copy or use uninitialized
  members. The change to a class was also motived that we used "struct
  st_position" and POSITION randomly trough the code which was
  confusing.
- Notably big rewrite in sql_acl.cc to avoid using deleted objects.
- Changed in sql_partition to use '^' instead of '-'. This is safe as
  the operator is either 0 or 0x8000000000000000ULL.
- Added check for select_nr < INT_MAX in JOIN::build_explain() to
  avoid bug when get_select() could return NULL.
- Reordered elements in POSITION for better alignment.
- Changed sql_test.cc::print_plan() to use pointers instead of objects.
- Fixed bug in find_set() where could could execute '1 << -1'.
- Added variable have_sanitizer, used by mtr.  (This variable was before
  only in 10.5 and up).  It can now have one of two values:
  ASAN or UBSAN.
- Moved ~Archive_share() from ha_archive.cc to ha_archive.h and marked
  it virtual. This was an effort to get UBSAN to work with loaded storage
  engines. I kept the change as the new place is better.
- Added in CONNECT engine COLBLK::SetName(), to get around a wrong cast
  in tabutil.cpp.
- Added HAVE_REPLICATION around usage of rgi_slave, to get embedded
  server to compile with UBSAN. (Patch from Marko).
- Added #ifdef for powerpc64 to avoid a bug in old gcc versions related
  to integer arithmetic.

Changes that should not be needed but had to be done to suppress warnings
from UBSAN:

- Added static_cast<<uint16_t>> around shift to get rid of a LOT of
  compiler warnings when using UBSAN.
- Had to change some '/' of 2 base integers to shift to get rid of
  some compile time warnings.

Reviewed by:
- Json changes: Alexey Botchkov
- Charset changes in ctype-uca.c: Alexander Barkov
- InnoDB changes & Embedded server: Marko Mäkelä
- sql_acl.cc changes: Vicențiu Ciorbaru
- build_explain() changes: Sergey Petrunia

* Disallow setting wsrep_on = 1 if wsrep_provider is unset. Also, move
  wsrep_on_basic from sys_vars to wsrep suite: this test now requires
  to run with wsrep_provider set
* Disallow setting @@session.wsrep_on = 1 when @@global.wsrep_on = 0
* Handle the case where a new connection turns @@global.wsrep_on from
  off to on. In this case we would miss a call to wsrep_open, causing
  unexpected states in wsrep::client_state (causing assertions).
* Disable wsrep.MDEV-22443 because it is no longer possible to enable
  wsrep_on, if server is started with wsrep_provider='none'
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

MDEV-22757 Assertion !binlog || exist_hton_without_prepare' failed in MYSQL_BIN_LOG::unlog_xa_prepare

The assert fired falsely having not captured two more not apparent
possiblities in its condition.

They are masked out hton error out of REPLACE execution (so at later xa-prepare
that engine is still present as read-write) and a prepare-capable engine
which also may not be an actual participant in the xa transation. That
engine, such as SEQUENCE, though does create its own event block.

This was caused by MDEV-23634, which disabled index optimization for
index != constant.

* Table should have primary key
* Enable wsrep_sync_wait before final selects
* Enable autocommit before final selects.
* Fix joiner monitoring in case of mysqldump.
* Add wait_conditions to stabilize

When dropped all user key-parts we also drop key-parts of implicit system fields.

This patch sets the proper name resolution context for outer references
used in a subquery from an ON clause. Usually this context is more narrow
than the name resolution context of the parent select that were used before
this fix.
This fix revealed another problem that concerned ON expressions used in
from clauses of specifications of derived tables / views / CTEs. The name
resolution outer context for such ON expression must be set to NULL to
prevent name resolution beyond the derived table where it is used.
The solution to resolve this problem applied in sql_derived.cc was provided
by Sergei Petrunia <sergey@mariadb.com>.

The change in sql_parse.cc is not good for 10.4+. A corresponding diff for
10.4+ will be provided in JIRA entry for this bug.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

It seems some overly tolerant compilers (gcc) allow the structure
of IO_CACHE that is defined differently in libmaria to have
members equalivance to the iocache in mysys.

More strict Solaris compilers recognise that rc_pos really
isn't a structure member and won't compile.

Make Item_subselect::walk() walk the ON expressions, too.

This reverts commit f26e3259.

MDEV-22255 SIGABRT: Assertion `id' failed in trx_write_trx_id on INSERT | Assertion `id > 0' failed in trx_write_trx_id | Assertion `val > 0' failed in row_upd_index_entry_sys_field | Assertion `thr_get_trx(thr)->id || index->table->no_rollback()' failed.

InnoDB purge thread locks the root page of clustered index
while accessing the undo log records and later same thread
tries to open the table, initialize statistics and tries
to lock the clustered index root page while doing virtual
column computation.

Solution:
=========
InnoDB should prevent statistics initialization when the
table is being opened by purge thread

MDEV-25425 Useless message "If the mysqld execution user is authorized page cleaner thread priority can be changed."

On startup, InnoDB tried to increase the priority of the page cleaner
thread. This would usually fail, resulting in a server log message.

When the page cleaner code was refactored in MDEV-23855, the throughput
and latency were greatly improved even when the
buf_flush_page_cleaner thread always ran the same priority with other
threads in our benchmarks.

So, this code should have no benefit in MariaDB Server 10.5.7 or later.

Besides, starting with MariaDB Server 10.5, the executable is called
mariadbd, not mysqld.

When a column is added to an non-empty table, existing rows will have
a column's default value for existing rows. Or a "zero value" if the
column has no default.

But this check should be skipped when an existing column is altered.

Trigger `socket.ssl_reload` when FLUSH SSL is issued. To triger reloading
of certificate, key and CA, files needs to be physically changed.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Link with -pthread instead of -lpthread, and do that for all architectures.

Contributors:
Aurelien Jarno <aurelien@aurel32.net>

Closes: #1717

Link with -pthread instead of -lpthread, and do that for all architectures.

Contributors:
Aurelien Jarno <aurelien@aurel32.net>

Closes: #1717

Possible now since
https://github.com/mariadb-corporation/mariadb-connector-c/commit/29720950eeae75f1ea7ef1376a6149cabeb79d13

Upstreamed from https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/ca5e93c754209f9c1a729bb057332ab3b0b76363

Closes: #1722

When CMAKE_CROSSCOMPILING_EMULATOR is defined, a cross-compile
can be made, however with native (emulated) execution possible.

This commit takes those points in the build system that
execute built targets natively and allow these to be executed
in a crosscompile if CMAKE_CROSSCOMPILING_EMULATOR is defined.

Closes #1805

signal handler, display coredump file pattern similarly to MDEV-25294 but for FreeBSD, thankfully the sysctl OID is the same.

leveling up to some degree with linux's mallinfo* api for
 the memory usage display with debug build with malloc_zone
 Darwin api.

Closes: #1803

Part #2, specifically for the 10.5+ branch:

The auto-increment parameters can change sporadically during the
execution of the mtr test "galera_vote_rejoin_ddl", causing it to
fail. This patch creates an environment where unpredictable changes
to these auto-increment settings do not occur during the test.

also add =WARN as an alias for =OFF
and clarify the help text

In commit d25f806d (MDEV-22749)
the CRC-32C implementation of MariaDB was broken on some
IA-32 and AMD64 builds, depending on the compiler version and
build options. This was verified for IA-32 on GCC 10.2.1.

Even though we try to identify the SSE4.2 extensions and the
availaibility of the PCLMULQDQ instruction by executing CPUID,
the fall-back code could be generated with extended instructions,
because the entire file mysys/crc32/crc32c.c was being compiled
with -msse4.2 -mpclmul. This would cause SIGILL on a PINSRD
instruction on affected IA-32 targets (such as some Intel Atom
processors). This might also affect old AMD64 processors
(predating the 2007 Intel Nehalem microarchitecture), if some
compiler chose to emit the offending instructions.

While it is fine to pass a target-specific option to a target-specific
compilation unit (like -mpclmul to a PCLMUL-specific compilation unit),
that is not safe for mixed-architecture compilation units.

For mixed-architecture compilation units, the correct way is to set
target attributes on the target-specific functions.

There does not seem to be a way to pass target attributes to
function template instantiation. Hence, we must replace the
ExtendImpl template with plain functions crc32_sse42() and
crc32_slow().

We will also remove some inconsistency between
my_crc32_implementation() and mysys_namespace::crc32::Choose_Extend().

The function crc32_pclmul_enabled() will be moved to mysys/crc32/crc32c.cc
so that the detection code will be compiled without -msse4.2 -mpclmul.

The AMD64 PCLMUL accelerated crc32c_3way() will be moved to a new
file crc32c_amd64.cc. In this way, only a few functions that depend
on -msse4.2 in mysys/crc32/crc32c.cc can be declared with
__attribute__((target("sse4.2"))), and most of the file can be compiled
for the generic target.

Last, the file mysys/crc32ieee.cc will be omitted on 64-bit POWER,
because it was dead code (no symbols were exported).

Reviewed by: Vladislav Vaintroub