MDEV-31098  InnoDB Recovery doesn't display encryption message when no encryption configuration passed

- InnoDB fails to report the error when encryption configuration
wasn't passed. This patch addresses the issue by adding
the error while loading the tablespace and deferring the
tablespace creation.

The problem was that RANGE_OPT_PARAM was not completely initialized in
some cases.
Added bzero() to ensure that all elements are always initialized.

Fixed hang when renaming index to original name

Use Dummy_error_handler in open_stat_tables() to ignore all errors
when opening statistics tables.

In commit 5ea5291d @sanja-byelkin for unknown reason switched the file mode
for 3 Galera tzinfo related test files from 644 -> 755. This exists only
from branch 10.6 onward:

    $ git checkout 10.5
    $ find mysql-test -executable -name *.test -or -executable -name *.result
    (no results)
    $ git checkout 10.6
    $ find mysql-test -executable -name *.test -or -executable -name *.result
    mysql-test/suite/galera/t/mysql_tzmysql-test/suite/galera/t/mysql_tzinfo_to_sql.test
    mysql-test/suite/galera/t/mariadb_tzinfo_to_sql.test
    mysql-test/suite/galera/r/mariadb_tzinfo_to_sql.resultinfo_to_sql.test

mysql-test/suite/galera/t/mariadb_tzinfo_to_sql.test
mysql-test/suite/galera/r/mariadb_tzinfo_to_sql.result

No test file nor test result file should be executable, so run chmod -x
on them.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.

log_t::create(): Return whether the initialisation succeeded.
It may fail if too large an innodb_log_buffer_size is specified.

copy_back(): Also copy the dummy empty ib_logfile0 so that
MariaDB Server 10.8 or later can be started after
--copy-back or --move-back.

Thanks to Daniel Black for reporting this.

This is a 10.5 version of
commit ebf36492

Table_cache_instance::operator new[](size_t): Reverted the changes
that were made in commit 8edef482
and move them to the only caller.

Use an std::atomic_flag to track thread creation in progress.
This is mainly a cleanup, the effect of this change was not measureable
in my tests.

Add threadpool functionality to restrict concurrency during "batch"
periods (where tasks are added in rapid succession).
This will throttle thread creation more agressively than usual, while
keeping performance at least on-par.

One of these cases is bufferpool load, where async read IOs are executed
without any throttling. There can be as much as 650K read IOs for
loading 10GB buffer pool.

Another one is recovery, where "fake read" IOs are executed.

Why there are more threads than we expect?
Worker threads are not be recognized as idle, until they return to the
standby list, and to return to that list, they need to acquire
mutex currently held in the submit_task(). In those cases, submit_task()
has no worker to wake, and would create threads until default concurrency
level (2*ncpus) is satisfied. Only after that throttling would happen.

The problem was that we did not handle errors properly in
JOIN::get_best_combination. In case an early error, JOIN->join_tab would
contain unintialized values, which would cause errors on cleanup().

The error in question was reported earlier, but not noticed until later.
One cause of this is that most of the sql_select.cc code just checks
thd->fatal_error and not thd->is_error().
Fixed by changing of checks of fatal_error to is_error().

This allows a user to to change the default value of MAX_SEL_ARGS (16000)
in the rare case where they neeed more generated SEL_ARGS (as part of
the range optimizer)

Raise notes if indexes cannot be used:
- in case of data type or collation mismatch (diferent error messages).
- in case if a table field was replaced to something else
  (e.g. Item_func_conv_charset) during a condition rewrite.

Added option to write warnings and notes to the slow query log for
slow queries.

New variables added/changed:

- note_verbosity, with is a set of the following options:
  basic            - All old notes
  unusable_keys    - Print warnings about keys that cannot be used
                     for select, delete or update.
  explain          - Print unusable_keys warnings for EXPLAIN querys.

The default is 'basic,explain'. This means that for old installations
the only notable new behavior is that one will get notes about
unusable keys when one does an EXPLAIN for a query. One can turn all
of all notes by either setting note_verbosity to "" or setting sql_notes=0.

- log_slow_verbosity has a new option 'warnings'. If this is set
  then warnings and notes generated are printed in the slow query log
  (up to log_slow_max_warnings times per statement).

- log_slow_max_warnings   - Max number of warnings written to
                            slow query log.

Other things:
- One can now use =ALL for any 'set' variable to set all options at once.
  For example using "note_verbosity=ALL" in a config file or
  "SET @@note_verbosity=ALL' in SQL.
- mysqldump will in the future use @@note_verbosity=""' instead of
  @sql_notes=0 to disable notes.
- Added "enum class Data_type_compatibility" and changing the return type
  of all Field::can_optimize*() methods from "bool" to this new data type.

Reviewer & Co-author: Alexander Barkov <bar@mariadb.com>
- The code that prints out the notes comes mainly from Alexander

The warning is given in case of table not found or if there is a lock
timeout. The warning is needed as in case of a lock timeout then the
persistent table stats are going to be wrong.

Updated ha_mroonga::storage_check_if_supported_inplace_alter to support
new ALTER TABLE flags.

This fixes failing tests:
mroonga/storage.alter_table_add_index_unique_duplicated
mroonga/storage.alter_table_add_index_unique_multiple_column_duplicated

- hostnames in hostname_cache added
- Some Galera (WSREP) allocations
- Table caches

This makes it easier to see how much memory MariaDB server has allocated.
(For all memory allocations that goes through mysys)

Example of what causes the problem:
T1: ANALYZE TABLE starts to collect statistics
T2: ALTER TABLE starts by deleting statistics for all changed fields,
    then creates a temp table and copies data to it.
T1: ANALYZE ends and writes to the statistics tables.
T2: ALTER TABLE renames temp table in place of the old table.

Now the statistics from analyze matches the old deleted tables.

Fixed by waiting to delete old statistics until ALTER TABLE is
the only one using the old table and ensure that rename of columns
can handle swapping of column names.

rename_columns_in_stat_table() (former rename_column_in_stat_tables())
now takes a list of columns to rename. It uses the following algorithm
to update column_stats to be able to handle circular renames

- While there are columns to be renamed and it is the first loop or
  last rename loop did change something.
  - Loop over all columns to be renamed
    - Change column name in column_stat
      - If fail because of duplicate key
      - If this is first change attempt for this column
         - Change column name to a temporary column name
         - If there was a conflicting row, replace it with the current row.
    else
     - Remove entry from column list

- Loop over all remaining columns in the list
 - Remove the conflicting row
 - Change column from temporary name to final name in column_stat

Other things:
- Don't flush tables for every operation. Only flush when all updates
  are done.
- Rename of columns was not handled in case of ALGORITHM=copy (old bug).
  - Fixed that we do not collect statistics for hidden hash columns
    used by UNIQUE constraint on long values.
  - Fixed that we do not collect statistics for blob columns referred by
    generated virtual columns. This was achieved by storing the fields for
    which we want to have statistics in table->has_value_set instead of
    in table->read_set.
- Rename of indexes was not handled for persistent statistics.
  - This is now handled similar as rename of columns. Renamed columns
    are now stored in 'rename_stat_indexes' and handled in
    Alter_info::delete_statistics() together with drooped indexes.
- ALTER TABLE .. ADD INDEX may instead of creating a new index rename
  an existing generated foreign key index. This was not reflected in
  the index_stats table because this was handled in
  mysql_prepare_create_table instead instead of in the mysql_alter() code.
  Fixed by adding a call in mysql_prepare_create_table() to drop the
  changed index.
  I also had to change the code that 'marked the index' to be ignored
  with code that would not destroy the original index name.

Reviewer: Sergei Petrunia <sergey@mariadb.com>

In `pseudo_slave_mode=1` aka "psedo-slave" mode any prepared XA
transaction disconnects from the user session, as if the user
connection drops. The xid of such transaction remains in the server,
and should the prepared transaction be read-only, it is marked.
The marking makes sure that the following termination of the
read-only transaction  ends up with ER_XA_RBROLLBACK.
This did not take place actually for  `pseudo_slave_mode=1` read-only.

Fixed with checking the read-only status of a prepared transaction
at time it disconnects from the `pseudo_slave_mode=1` session, to mark
its xid when that's the case.

Fix test

It is enough to just once, during connection phase

MDEV-30217 : Assertion `mode_ == m_local || transaction_.is_streaming()' failed in int wsrep::client_state::bf_abort(wsrep::seqno)

Problem was that brute force (BF) thread requested conflicting lock
and was trying to kill victim transaction, but this victim
was also brute force thread. However, this victim was not actually
holding conflicting lock, instead both brute force transaction
and victim transaction were had insert intention locks.
We should not kill brute force victim transaction if requesting
lock does not need to wait.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Spider is part of the server, and there's no need to check the
version.

All spider plugins are uninstalled in clean_up_spider.inc

DROP SERVER IF EXISTS makes things easier

Memory for type holders of the columns of a table value constructor must
be allocated only once.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

is_file_on_ssd() is more expensive than it should be.
It caches the results by volume name, but still calls GetVolumePathName()
every time, which, as procmon shows, opens multiple directories in
filesystem hierarchy (db directory, datadir, and all ancestors)

The fix is to cache SSD status by volume serial ID, which is cheap to
retrieve with GetFileInformationByHandleEx()

The counter is global so we do not need add backup to it
if we do not zero it after taking the backup.

Fix row counters to be able to get any possible value.

Follow-up to fix issue with access to probably not-initialized mutex/cond_var

Constructor of the class st_debug_sync_globals was changed to initialize
the data members dsp_hits, dsp_executed, dsp_max_active with zero.
Formerly, these data members were filled with zeroes by C-runtime since
the variable debug_sync_global was declared as static and according with C rules
the static variable initialized with zero bytes.

By the same reason, the data members
  debug_sync_global->ds_mutex
  debug_sync_global->ds_cond
were initialized by zeros before the patch for MDEV-31871. After this patch
the memory for the synch primitives debug_sync_global->ds_mutex
and debug_sync_global->ds_cond are initialized explicitly by calling
the functions mysql_mutex_init/mysql_cond_init so access to these synch
primitives should be done only after such initialization be completed.
Guarded access to these synch primitives has been added to the function
debug_sync_end_thread() that is called on clean up since that was single
problem place detected by MSAN. Theoretically problem places located in the
function debug_sync_execute were not protected with similar check since
it is not obvious that the variables debug_sync_global->ds_mutex
and debug_sync_global->ds_cond could be not initilialized for use cases where
the function debug_sync_execute() is called. It is required additional study
to conclude whether it does need or not.

mariadb-install crashes on start when the static variable debug_sync_global
of the class st_debug_sync_globals is initialized by constructor.

Definition of the class st_debug_sync_globals has a data member of the type
Hash_set whose implementation depends on thread-specific data associated
with the key THD_KEY_mysys. This dependency results from constructor of
the class Hash_set that runs my_hash_init2() which in turn invokes my_malloc.
The thread-specific data value associated with the key THD_KEY_mysys is used
by the function sf_malloc to get id of the current thread. The key
THD_KEY_mysys is defined as static variable at my_thr_init.c initialized
with the value -1. Proper initialization of the key THD_KEY_mysys is done
with the library call pthread_key_create but it happens at the  my_init()
that called much later after the first time the THD_KEY_mysys() has been
invoked. In according with Single Unix Specification, the effect of calling
pthread_setspecific() or pthread_getspecific() with a key value not obtained
from pthread_key_create() is undefined. That is the reason why mariadb-install
crashes on MacOS.

To fix the issue, the static variable debug_sync_global is converted to
a pointer to the class st_debug_sync_globals and its instantiation is done
explicitly at the function debug_sync_init() that is called at right time.

This is the follow-up patch to the commits
  8885225d
  f6ecadfe
where was introduced a statically instantiated object debug_sync_global of
the structure st_debug_sync_globals and the key THR_KEY_mysys for this
thread-specific data was initialized with the value -1.

trx_t::set_skip_lock_inheritance() must be invoked at the very beginning
of lock_release_on_prepare(). Currently  trx_t::set_skip_lock_inheritance()
is invoked at the end of lock_release_on_prepare() when lock_sys and trx
are released, and there can be a case when locks on prepare are released,
but "not inherit gap locks" bit has not yet been set, and page split
inherits lock to supremum.

Also reset supremum bit and rebuild waiting queue when XA is prepared.

Reviewed by: Marko Mäkelä

This fixes up commmit ed20e5b1
which fixed up the merge commit 202316a3

MDEV-29731 Assertion failure when HAVING in a correlated subquery references columns in the outer query

When resolving a column from the HAVING clause, a new Item_field
object may be created inside Item_ref::fix_fields().
But the object is created with an empty name resolution context,
which then leads to debug assertion failure during
Item_field::fix_fields().

The solution is to pass the correct name resolution context
when creating the Item_field object.

Reviewer: Oleksandr Byelkin (sanja@mariadb.com)