In commit 03ca6495 (MDEV-24142)
we replaced a debug data structure that holds information about
S-latch holders with a std::set, which does not allow duplicates.

The assertion failed in btr_search_guess_on_hash() in an
s_lock_try() operation.

The reason why recursive S-latch requests are not normally allowed
is that if some other thread has enqueued a waiting X-lock, then
further S-latch requests will block until the exclusive lock has been
granted and released. If a thread were already holding one S-latch
while waiting for the X-latch to be granted and released by another
thread, the two threads would deadlock.

However, the nonblocking s_lock_try() is perfectly fine;
it will immediately return failure in case of conflict.

sux_lock::readers: Use std::unordered_multiset instead of std::set.

sux_lock::s_lock_register(): Allow 'duplicate' requests. Blocking-mode
latch acquisitions are already covered by !have_s() assertions.

sux_lock::s_unlock(): Erase only one element from readers.

buf_page_try_get(): Revert to s_lock_try(). It had been previously
changed to the more intrusive u_lock_try() in response to the
debug check failing.

We may end up with an empty leaf page (containing only an ADD COLUMN
metadata record) that is not the root page.

innobase_add_instant_try(): Disable an optimization for a non-canonical
empty table that contains a metadata record somewhere else than in
the root page.

btr_pcur_store_position(): Tolerate a non-canonical empty table.

MDEV-24659 Assertion !fsp_is_system_temporary(bpage->id().space()) failed in buf_flush_relocate_on_flush_list()

When commit 5eb53955 (MDEV-12227)
removed the pages of temporary tables from the buf_pool.flush_list,
an adjustment to the buffer pool resizing was forgotten.

buf_pool_t::realloc(): Do not invoke buf_flush_relocate_on_flush_list()
for pages that belong to the temporary tablespace. Also, deduplicate
some code at the end.

buf_page_t::set_corrupt_id(): Tolerate oldest_modification()==1
(the dummy value) for temporary tablespace pages. The revised
buf_pool_t::realloc() may invoke this on dirty temporary tablespace pages.

Ever since commit 947efe17
InnoDB no longer writes binlog position in one place.
It will not at all be written to the TRX_SYS page, and
instead it will be written to the undo log header page that
changes the transaction state.

trx_rseg_mem_restore(): Recover the information from the latest
written page.

For some reason, calls to trx_mark_sql_stat_end() or
ha_innobase::start_stmt() can be missed.

fts_optimize_request_sync_table() can avoid the repetitive
FTS SYNC request of the table if the table already has FTS_SYNC
message in fts_optimize_wq queue.

Reviewed-by: Marko Mäkelä

MDEV-24463 : galera.galera_sst_mysqldump_with_key MTR failed: 'INSERT failed: 1213: Deadlock found when trying to get lock

We need to complete SST if both new and old start positions are
not same as initial positions. If they are initial positions
just set local uuid and seqno.

FIXME: rpl.rpl_parallel_no_log_slave_updates still hangs!

While reusing the cached undo log block, mtr expects the page
write to change while writing the trx id. cached undo log block
could contain bytes which were originally written for some other
transaction. So InnoDB should make mtr to do MAYBE_NOP while reusing
cached undo log block.

Reviewed-by: Marko Mäkelä

trx_t::rollback_low(): On a partial rollback of an insert into an
empty table, release the exclusive table lock, because the transaction
will no longer have had any impact on that table.

lock_table_x_unlock(): Release an exclusive table lock.

fts_optimize_callback() could be called after processing
FTS_MSG_STOP due to timer initiated callback. This issue
is caused by commit 38fd7b7d
(MDEV-21452). In that case, fts_optimize_callback() should
check whether it processed FTS_MSG_STOP already.

Reviewed-by: Marko Mäkelä

reduces the weightage of the bulk insert because it generates less
undo log.

trx_mod_table_time_t: Add a BULK flag to the pre-existing field 'first'.
The maximum entropy of undo_no_t is only 7 bytes (the size of DB_ROLL_PTR),
so we can safely introduce flags in the 8th byte.

I_S tables were materialized too late, an attempt to use table
statistics before the table was created caused a crash.

Let's move table creation up. it only needs read_set to
be calculated properly, this happens in JOIN::optimize_inner(),
after semijoin transformation.

Note that tables are not populated at that point, so most of the
statistics would make no sense anyway. But at least field sizes
will be correct. And it won't crash.

The problem was that update_timing_fields_for_event() didn't release all
MDL locks it took.

FIXME: innodb.innodb_trx_weight reports deadlock

Naming: clear() instead of empty() or empty_table(), to align with STL

trx_mod_table_time_t: Add two new fields: first_is_bulk and
first_after_bulk. The latter field may be reset on partial rollback().

lock_rec_insert_check_and_lock(), lock_prdt_insert_check_and_lock():
Remove redundant parameter 'flags'.

trx_undo_page_report_insert(): Return early if bulk insert operation is
in progress. We will insert also temporary tables into trx->mod_tables
for the necessary bookkeeping.

MDEV-24596 : Assertion `state_ == s_exec || state_ == s_quitting' failed in wsrep::client_state::disable_streaming

There were multiple problems here
* wsrep_trx_fragment_size should not be set when wsrep is disabled or provider is not loaded
* wsrep_trx_fragment_unit should not be set when wsrep is disabled or provider is not loaded
* wsrep_debug has no effect if wsrep is disabled or provider is not loaded
* wsrep_start_position should not be set when wsrep is disabled or provider is not loaded any other value than default
* wsrep_start_position should be changed only when we are joiner or initialized
* wsrep_start_position should be allowed to set only a value that exits, thus
we need to add error handling to wsrep_sst_complete

Sample log error message generated:

mysql-test/var/log/mysqld.2.err:2021-01-21 13:02:30 8 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 329, master: 127.0.0.1:16000
mysql-test/var/log/mysqld.2.err:2021-01-21 13:02:30 7 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 329, master 127.0.0.1:16000
mysql-test/var/log/mysqld.2.err:2021-01-21 13:02:30 12 [Note] Slave SQL thread exiting, replication stopped in log 'master-bin.000001' at position 329; GTID position '', master: 127.0.0.1:16000

Reviewer: knielsen@knielsen-hq.org, Andrei and Sachin

row_sel_clust_sees(): Replaces lock_clust_rec_cons_read_sees().

lock_sec_rec_cons_read_sees(): Replaced with lower-level code.

row_merge_read_clustered_index(): Check bulk_trx_id at the correct spot.

row_sel(), row_sel_try_search_shortcut(),
row_sel_try_search_shortcut_for_mysql(): Check bulk_trx_id.

ReadView::changes_visible(trx_id_t) const: New accessor for the case
where the trx_id_t is not read from a potentially corrupted index page
but directly from the memory. In this case, we can skip a sanity check.

Fix for MDEV-23033 fixes a problem in replication applying of transactions, which contain cascading foreign key delete for a table, which has indexed virtual column.
This fix adds slave_fk_event_map flag for table, to mark when the prelocking is needed for applying of a transaction.
See commit 608b0ee5 for more details.
However, this fix is targeted for async replication only, Rows_log_event::do_apply_event() has condition to rule out galera replication from the fix domain, and use cases suffering from MDEV-23033 and related MDEV-21153 will fail in galera cluster.

The fix in this commit removes the condition to rule out the setting of slave_fk_event_map flag from galera replication, and makes the fix in MDEV-23033 effective for galera replication as well.

However, the above fix has caused regressions for some galera_sr suite tests, which run tests for streaming replication.
This regression can be observed e.g. by: /mtr galera_sr.galera_sr_multirow_rollback  --mysqld=--slave_run_triggers_for_rbr=yes
These galera_sr suite tests were failing in last phase of replication applying, where actual transaction is already applied, and streaming replication related meta data needs to be updated in wsrep system tables.
Opening the wsrep system tables failed for corrupt data in THD::lex:query_tables_list. The fix in this commit uses back query table list for the duration of fragment update operation.

Finally, a mtr test for virtual column support has been added. galera.galera_virtual_column.test has as first test a scenario from MDEV-21153

new fix
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Fix for MDEV-23033 fixes a problem in replication applying of transactions, which contain cascading foreign key delete for a table, which has indexed virtual column.
This fix adds slave_fk_event_map flag for table, to mark when the prelocking is needed for applying of a transaction.
See commit 608b0ee5 for more details.
However, this fix is targeted for async replication only, Rows_log_event::do_apply_event() has condition to rule out galera replication from the fix domain, and use cases suffering from MDEV-23033 and related MDEV-21153 will fail in galera cluster.

The fix in this commit removes the condition to rule out the setting of slave_fk_event_map flag from galera replication, and makes the fix in MDEV-23033 effective for galera replication as well.

Finally, a mtr test for virtual column support has been added. galera.galera_virtual_column.test has as first test a scenario from MDEV-21153
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

MDEV-24577: Fix warnings generated during compilation of plugin/auth_pam/testing/pam_mariadb_mtr.c on FreeBSD

Compiler warnings generated on building MariaDB server for BSD has the same
reason as in case building is performed on MacOS. Both platforms do use
clang as a C/C++ compiler. So, fix the compiler warnings in case the compiler
is clang doesn't matter what kind of building platform do we use for building.

This is a follow-up patch for the following bug reports:
  MDEV-23564: CMAKE failing due to deprecated Apple GSS method
  MDEV-23935: Fix warnings generated during compilation of
              plugin/auth_pam/testing/pam_mariadb_mtr.c on MacOS