ha_innobase::delete_table(): When the table that is being dropped
has a name starting with #sql, temporarily set
innodb_lock_wait_timeout=0 while attempting to lock the
persistent statistics tables. If the statistics tables cannot be locked,
pretend that statistics did not exist and carry on with dropping
the table. The SQL layer is not really prepared for failures of
this operation. This is what fixes the test case.

ha_innobase::rename_table(): When renaming a table from a name
that starts with #sql, try to lock the statistics tables with an
immediate timeout, and ignore the statistics if the locks were
not available. In fact, during any rename from a #sql name,
dict_stats_rename_table() should have no effect, because already
when an earlier rename to a #sql name took place we should have
deleted the statistics for the table using the non-#sql name.
This change is just analogous to the ha_innobase::delete_table().

As part of MDEV-26779 we first discovered the effect of enabling spinning for
some critical mutex. MDEV-26779 tried enabling it for lock_sys.wait_mutex and
observed a good gain in performance.

In yet another discussion, Mark Callaghan pointed a reference to pthread based
mutex spin using PTHREAD_MUTEX_ADAPTIVE_NP (MDEV-26769 Intel RTM).

Given the strong references, Marko Makela as part of his comment in #1923
pointed an idea to enable spinning for other mutexes. Based on perf profiling
we decided to explore spinning for log_sys_mutex and log_flush_order_mutex as
they are occupying the top slots in the contented mutex list.

The evaluation showed promising results for ARM64 but not for x86.
So a patch is here-by proposed to enable the spinning of the mutex for
ARM64-based platform.

The SQL layer never acquires metadata locks (MDL) on the tables
that the tables that DML statement accesses is modifying.

However, the storage engine must access the parent table in order to
ensure that the child table will not refer to a non-existing record
in the parent table.

During certain DDL operations, the InnoDB table metadata (dict_table_t)
may be be freed and rebuilt. This would cause a race condition with
a concurrent INSERT that is attempting to report a FOREIGN KEY violation.

We work around the insufficient MDL during DML by acquiring exclusive
InnoDB table locks on all child tables during DDL. To avoid deadlocks,
we will follow the following order of acquisition:

1. tables whose REFERENCES clauses point to the current table
2. the current table that is being subjected to DDL
3. mysql.innodb_table_stats
4. mysql.innodb_index_stats
5. the InnoDB dictionary tables (SYS_TABLES and so on)
6. exclusive dict_sys.latch

MDEV-26582 SIGSEGV in spider_db_bulk_insert and spider_db_connect and spider_db_before_query, and hang in "End of update loop" / "Reset for next command" query states

Spider accesses a freed connection in ha_spider::end_bulk_insert()
and results in SIGSEGV.

The cause of the bug is that ha_spider::is_bulk_insert_exec_period()
wrongly returns TRUE when the bulk insertion has not yet started.

Spider decides whether it is during the bulk insertion or not by
the value of insert_pos, but the variable is not reset in a case,
and this result in the bug.

The purpose of non-exclusive locks in a transaction is to guarantee
that the records covered by those locks must remain in that way until
the transaction is committed. (The purpose of gap locks is to ensure
that a record that was nonexistent will remain that way.)

Once a transaction has reached the XA PREPARE state, the only allowed
further actions are XA ROLLBACK or XA COMMIT. Therefore, it can be
argued that only the exclusive locks that the XA PREPARE transaction
is holding are essential.

Furthermore, InnoDB never preserved explicit locks across server restart.
For XA PREPARE transations, we will only recover implicit exclusive locks
for records that had been modified.

Because of the fact that XA PREPARE followed by a server restart will
cause some locks to be lost, we might as well always release all
non-exclusive locks during the execution of an XA PREPARE statement.

lock_release_on_prepare(): Release non-exclusive locks on XA PREPARE.

trx_prepare(): Invoke lock_release_on_prepare() unless the
isolation level is SERIALIZABLE or this is an internal distributed
transaction with the binlog (not actual XA PREPARE statement).

This has been discussed with Sergei Golubchik and Andrei Elkin.

Reviewed by: Sergei Golubchik

MDEV-26539 SIGSEGV in spider_check_and_set_trx_isolation and I_P_List_iterator from THD::drop_temporary_table (10.5.3 opt only) on ALTER

The server crashes if ALTER TABLE, which accesses physical data
placed at data nodes, is performed on a Spider table.

The cause of the bug is that spider_check_trx_and_get_conn() does
not allocate connections if sql_command == SQLCOM_ALTER_TABLE.
Some ALTER TABLE statements, like ALTER TABLE ... CHECK PARTITION,
access data nodes. So, we need to allocate a new connection before
performing ALTER TABLEs.

MDEV-26742 Assertion `field->type_handler() == this' failed in FixedBinTypeBundle<NATIVE_LEN, MAX_CHAR_LEN>::Type_handler_fbt::stored_field_cmp_to_item

Record correct result file.

Something initiates purge, while all purge THDs are destroyed.
tpool::waitable_task.disable() would not allow the task to be executed anymore

innodb_max_dirty_pages_pct_update(),
innodb_max_dirty_pages_pct_lwm_update():
Invoke buf_pool.page_cleaner_wakeup() in order to wake up
buf_flush_page_cleaner. This allows the test innodb.page_cleaner
to run without any occasional timeouts.

The occasional hangs were introduced by
commit 7b1252c0 (MDEV-24278).

Add wait_condition

10.5 version

The changes to galera.galear_var_replicate_myisam_on
in commit d9b933be
are omitted due to conflicts
with commit 27d66d64.

The test galera.galera_var_replicate_myisam_on
would trigger an assertion failure "mode() == m_local"
in wsrep-lib/src/client_state.cpp after the merge
of commit 3067ffc5 enabled it.

In commit 1cb218c3 (MDEV-26450)
we introduced the function log_write_and_flush(), which may
compete with log_checkpoint() invoking log_write_flush_to_disk_low()
from another thread.

The assertion n_pending_flushes==1 is too strict.
There is no possibility of a race condition here, because
fil_flush() is protected by fil_system->mutex and the
rest will be protected by log_sys->mutex.

log_write_flush_to_disk_low(), log_write_and_flush():
Relax the assertions to test for a nonzero count.

Before clang-10, asm goto was not supported, so we must use fetch_or().

Update wsrep-lib, and add MTR test case for MDEV-26707
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Also fixes MDEV-24467 Memory not freed after failed INSERT DELAYED

Description:

In case of an error (e.g. data truncation) during mysql_insert()
handling an INSERT DELAYED, the data type specific data in
fields (e.g. Field_blob::value) is not taken over by the delayed
writer thread.

All fields in table_list->table are freed by free_root()
immediately after mysql_insert(). To avoid a memory leak,
we need to free the specific data before exiting mysql_insert()
on error.

The crash happened because my_isalnum() does not support character
sets with mbminlen>1.

The value of "ft_boolean_syntax" is converted to utf8 in do_string_check().
So calling my_isalnum() is combination with "default_charset_info" was wrong.

Adding new parameters (size_t length, CHARSET_INFO *cs) to
ft_boolean_check_syntax_string() and passing self->charset(thd)
as the character set.

The crash happened because Item_aes_crypt::val_str() did not
set the character set of the result.

Co-authored-by: FX Coudert <fxcoudert@gmail.com>

Uninitialized ref_pointer_array[] because setup_fields() got empty
fields list.  mysql_multi_update() for some reason does that by
substituting the fields list with empty total_list for the
mysql_select() call (looks like wrong merge since total_list is not
used anywhere else and is always empty). The fix would be to return
back the original fields list. But this fails update_use_source.test
case:

  --error ER_BAD_FIELD_ERROR
  update v1 set t1c1=2 order by 1;

Actually not failing the above seems to be ok.

The other fix would be to keep resolve_in_select_list false (and that
keeps outer context from being resolved in
Item_ref::fix_fields()). This fix is more consistent with how SELECT
behaves:

  --error ER_SUBQUERY_NO_1_ROW
  select a from t1 where a= (select 2 from t1 having (a = 3));

So this patch implements this fix.

There are two fill_record() functions (lines 8343 and 8618). First one
is used when there are some explicit values, the second one is used
for all implicit values. First one does update_default_fields(), the
second one did not. Added update_default_fields() call to the implicit
version of fill_record().

Second alter subcommand correctly removed VERS_ROW_END flag. We throw
ER_VERS_PERIOD_COLUMNS in such case.

- Cleaned up Vers_parse_info::check_sys_fields();
- Renamed VERS_SYS_START_FLAG, VERS_SYS_END_FLAG to VERS_ROW_START,
  VERS_ROW_END.

If error_reported is not set upper caller open_table_from_share()
throws error ER_NOT_FORM_FILE itself via open_table_error().

On deadlock transaction is rolled back (and trx->state is cleared) but
SELECT continued the loop because evaluate_join_record() ignored the
error status returned from lower join evaluation. val_int() does not
return error status so it is checked by thd->is_error().

Test case was created by Thirunarayanan Balathandayuthapani
<thiru@mariadb.com>

* Handle the (rare) case where the ongoing transaction is aborted, if
  the ongoing transaction is considered orphaned. This happens if the
  node delivers two consecutive primary views with the
* Add ignorable warning to suite.pm
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>