The background drop table queue in InnoDB is a work-around for
cases where the SQL layer is requesting DDL on tables on which
transactional locks exist.

One such case are XA transactions. Our test case exploits the
fact that the recovery of XA PREPARE transactions will
only resurrect InnoDB table locks, but not MDL that should
block any concurrent DDL.

srv_shutdown_t: Introduce the srv_shutdown_state=SRV_SHUTDOWN_INITIATED
for the initial part of shutdown, to wait for the background drop
table queue to be emptied.

srv_shutdown_bg_undo_sources(): Assign
srv_shutdown_state=SRV_SHUTDOWN_INITIATED
before waiting for the background drop table queue to be emptied.

row_drop_tables_for_mysql_in_background(): On slow shutdown, if
no active transactions exist (excluding ones that are in
XA PREPARE state), skip any tables on which locks exist.

row_drop_table_for_mysql(): Do not unnecessarily attempt to
drop InnoDB persistent statistics for tables that have
already been added to the background drop table queue.

row_mysql_close(): Relax an assertion, and free all memory
even if innodb_force_recovery=2 would prevent the background
drop table queue from being emptied.

This race condition was introduced by
commit ad6171b9 (MDEV-22456).

In the observed case, two threads were executing
btr_search_drop_page_hash_index() on the same block,
to free a stale entry that was attached to a dropped index.
Both threads were holding an S latch on the block.

We must prevent the double-free of block->index by holding
block->lock in exclusive mode.

btr_search_guess_on_hash(): Do not invoke
btr_search_drop_page_hash_index(block) to get rid of
stale entries, because we are not necessarily holding
an exclusive block->lock here.

buf_defer_drop_ahi(): New function, to safely drop stale
entries in buf_page_mtr_lock(). We will skip the call to
btr_search_drop_page_hash_index(block) when only requesting
bufferfixing (no page latch), because in that case, we should
not be accessing the adaptive hash index, and we might get
a deadlock if we acquired the page latch.

btr_search_sys_free(): Free btr_search_sys->hash_tables.

The leak was introduced in commit ad2bf112.

Allow targets for building "noinstall" zip, and debuginfo zip.

When acquiring SNW/SNRW/X MDL lock DDL/admin statements may abort pending
thr lock in concurrent connection with open HANDLER (or delayed insert
thread).

This may lead to a race condition when table->alias is accessed
concurrently by such threads. Either assertion failure or memory leak
is a practical consequence of this race condition.

Specifically HANDLER is opening a table and issuing alias.copy(), while
DDL executing get_lock_data()/alias.c_ptr()/realloc()/realloc_raw().

Fixed by perforimg table->init() before it is published via
thd->open_tables.

Added cost of sorting estimate to the optimizer trace

We are supposed to commit and restart the mini-transaction
between records. There is no point to store and restore the
persistent cursor position otherwise.

If buf_page_optimistic_get() is patched to always fail, the
debug build would fail to start up due to trying to re-acquire
an already S-latched block.

This bug (which should not have visible impact to users, because
the code is only executed during startup, while no other threads
are accessing B-trees or causing pages to be evicted from the
buffer pool) was caught as part of a debugging effort for
something else.

The debugging approach was: Make buf_page_optimistic_get()
always return FALSE, and add ut_a(block->lock.lock_word == X_LOCK_DECR)
to both buf_LRU_get_free_only() and buf_LRU_block_free_non_file_page().
This would catch misuse of the buffer pool. If it were not for
buf_page_optimistic_get(), no buf_block_t::lock of any BUF_BLOCK_NOT_USED
block would ever be acquired.

MDEV-16230: Server crashes when Analyze format=json is run with a window function with empty PARTITION BY and ORDER BY clauses

Currently when both PARTITION BY and ORDER BY clauses are empty then we create a Item
with the first field in the select list and sort with that field.
It should be created as an Item_temptable_field instead of Item_field because the
print() function continues to work even if the table has been dropped.

MDEV-22112 Assertion `tab_part_info->part_type == RANGE_PARTITION || tab_part_info->part_type == LIST_PARTITION' failed in prep_alter_part_table

Incorrect syntax for SYSTEM_TIME partition. work_part_info is detected
as HASH partition. We cannot add partition of different type neither
we cannot reorganize SYSTEM_TIME into/from different type
partitioning.

The sidefix for version until 10.5 corrects the message:
"For LIST partitions each partition must be defined"

Introduce a new ATTRIBUTE_NOINLINE to
ib::logger member functions, and add UNIV_UNLIKELY hints to callers.

Also, remove some crash reporting output. If needed, the
information will be available using debugging tools.

Furthermore, remove some fts_enable_diag_print output that included
indexed words in raw form. The code seemed to assume that words are
NUL-terminated byte strings. It is not clear whether a NUL terminator
is always guaranteed to be present. Also, UCS2 or UTF-16 strings would
typically contain many NUL bytes.

Problem:
========
  During buffer pool resizing, InnoDB recreates the dictionary hash
tables. Dictionary hash table reuses the heap of AHI hash tables.
It leads to memory corruption.

Fix:
====
- While disabling AHI, free the heap and AHI hash tables. Recreate the
AHI hash tables and assign new heap when AHI is enabled.

- btr_blob_free() access invalid page if page was reallocated during
buffer poolresizing. So btr_blob_free() should get the page from
buf_pool instead of using existing block.

- btr_search_enabled and block->index should be checked after
acquiring the btr_search_sys latch

- Moved the buffer_pool_scan debug sync to earlier before accessing the
btr_search_sys latches to avoid the hang of truncate_purge_debug
test case

- srv_printf_innodb_monitor() should acquire btr_search_sys latches
before AHI hash tables.

srv_purge_should_exit(): Report progress on slow shutdown
not only to systemd, but also to the error log.

btr_pcur_store_position(): Replace a too strict debug assertion.
It is possible to have a clustered index B-tree for a logically
empty table, which will consist of a node pointer from the root
page to a leaf page that contains the metadata record.

The too strict debug assertion was added in
commit 0e5a4ac2 (MDEV-15562).

MDEV-21546 main.backup_stages occasionally fails with lock wait timeout

commit f74023b9 (MDEV-15090)
inadvertently removed a mtr_t::commit() call from
trx_undo_report_rename(), causing an InnoDB hang if
we failed to log a RENAME operation.

It is unclear whether this condition is possible in practice.
The test case involved SET GLOBAL innodb_trx_rseg_n_slots_debug=1
and a failed CREATE TABLE...SELECT, whose error handling would
internally invoke RENAME in InnoDB.

Problem:
=======
  While evicting the uncompressed page from buffer pool, InnoDB writes
the checksum for the compressed page in buf_LRU_free_page().
So while flushing the compressed page, checksum validation fails
when innodb_checksum_algorithm variable changed to strict_none.

Solution:
========
- Calculate the checksum only during flushing of page. Removed the
checksum write in buf_LRU_free_page().

Compare to trx_roll_crash_recv_trx directly where needed.

innobase_init(): On every path to refused startup, log the reason
to refuse startup as an error, instead of a note.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected

Previously multiple threads were allowed to load histograms concurrently.
There were no known problems caused by this. But given amount of data
races in this code, it'd happen sooner or later.

To avoid scalability bottleneck, histograms loading is protected by
per-TABLE_SHARE atomic variable.

Whenever histograms were loaded by preceding statement (hot-path), a
scalable load-acquire check is performed.

Whenever histograms have to be loaded anew, mutual exclusion for loaders
is established by atomic variable. If histograms are being loaded
concurrently, statement waits until load is completed.

- Table_statistics::total_hist_size moved to TABLE_STATISTICS_CB: only
  meaningful within TABLE_SHARE (not used for collected stats).
- TABLE_STATISTICS_CB::histograms_can_be_read and
  TABLE_STATISTICS_CB::histograms_are_read are replaced with a tri state
  atomic variable.
- Simplified away alloc_histograms_for_table_share().

Note: there's still likely a data race if a thread attempts accessing
histograms data after it failed to load it (because of concurrent load).
It was there previously and goes out of the scope of this effort. One way
of fixing it could be reviving TABLE::histograms_are_read and adding
appropriate checks whenever it is needed.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected

Previously multiple threads were allowed to load statistics concurrently.
There were no known problems caused by this. But given amount of data
races in this code, it'd happen sooner or later.

To avoid scalability bottleneck, statistics loading is protected by
per-TABLE_SHARE atomic variable.

Whenever statistics were loaded by preceding statement (hot-path), a
scalable load-acquire check is performed.

Whenever statistics have to be loaded anew, mutual exclusion for loaders
is established by atomic variable. If statistics are being loaded
concurrently, statement waits until load is completed.

TABLE_STATISTICS_CB::stats_can_be_read and
TABLE_STATISTICS_CB::stats_is_read are replaced with a tri state atomic
variable.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected

Removed redundant loops, integrated logics into the caller instead.
Unified condition in read_statistics_for_tables(), less
"table_share != NULL" checks, no more potential "table_share == NULL"
dereferencing.

Part of MDEV-19061 - table_share used for reading statistical tables is
                     not protected

(The table already existed in the storage engine)

Wrong algorithm of closing partitions on error doesn't close last
partition.

MDEV-22744 *SAN: sql/item_xmlfunc.cc:791:43: runtime error: downcast of address ... which does not point to an object of type 'Item_func' note: object is of type 'Item_bool' (on optimized builds)

In Item_nodeset_func_predicate::val_nodeset, args[1] is not necessarily
an Item_func descendant. It can be Item_bool.

Removing a wrong cast. It was not really needed anyway.