Before this patch mergeable derived tables / view used in a multi-table
update / delete were merged before the preparation stage.
When the merge of a derived table / view is performed the on expression
attached to it is fixed and ANDed with the where condition of the select S
containing this derived table / view. It happens after the specification of
the derived table / view has been merged into S. If the ON expression refers
to a non existing field an error is reported and some other mergeable derived
tables / views remain unmerged. It's not a problem if the multi-table
update / delete statement is standalone. Yet if it is used in a stored
procedure the select with incompletely merged derived tables / views may
cause a problem for the second call of the procedure. This does not happen
for select queries using derived tables / views, because in this case their
specifications are merged after the preparation stage at which all ON
expressions are fixed.
This patch makes sure that merging of the derived tables / views used in a
multi-table update / delete statement is performed after the preparation
stage.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

		to mysql interpreter

InnoDB returns uninitialized statistics to mysql interpreter
when background thread is opening the table. So it leads to
assertion failure. In that case, InnoDB avoid sending
innodb statistics information to mysql interpreter.

node->index was NULL.
But it's possible to get dict_table_t* from another source.

There was race between a committing transaction and the following in binlog
order FLUSH LOGS that could create a 2nd Binlog checkpoint (BCP) event
in the new file *before* the first logged-in-old-binlog transaction gets committed in
Innodb. That would cause the transaction loss at recovery, should
the server stop right after the BCP.

The race is tackled by enforcing the necessary set of mutexes to be acquired
by FLUSH-LOGS handler in the correct order (of the group commit leader
pattern).

Note, there remain two cases where a similar race is still possible:
  - the above race as it is when the server is run with ("unlikely")
    non-default `--binlog-optimize-thread-scheduling=0` (MDEV-24530), and
  - at unlikely event of bin-logging of Incident event (MDEV-24531) that
    also triggers binlog rotation,
    in both cases though with lesser chances after the current fixes.

Closes #1811

Since it ended up being a plain text file rather than a script
(post-fix for MDEV-25288)
Also removed main-test_sql_discovery.create from the list,
as a workaround for MDEV-25384. The bug only affects 10.3+,
but it will be simpler if 10.2 part of the tests is the same
in all branches

... to run upon building/packaging of MariaDB server

Eliminate a memory leak when init can fail by forgetting to delete the
Event_queue_element_for_exec object.

PROBLEM
-------

1. The customer had presented a stack which had many threads waiting on
   multiple mutexes like LOCK_Status, srv_innodb_monitor_mutex,  ibuf_mutex etc.
2. The root cause was that the AHI latch was held in S (shared) mode by the a thread which was
   doing a truncate of a large table .
3. There was another thread which was trying to acquire the AHI latch in X (exclusive) mode
4. With our lock implementation any thread requesting a X lock ,blocks rest of the threads
   requesting S(shared) locks,this caused many threads to wait for this shared lock.
5. The main reason why we hold the latches in truncate is to avoid disabling of AHI
   during truncate

FIX

When dropped all user key-parts we also drop key-parts of implicit system fields.

This patch sets the proper name resolution context for outer references
used in a subquery from an ON clause. Usually this context is more narrow
than the name resolution context of the parent select that were used before
this fix.
This fix revealed another problem that concerned ON expressions used in
from clauses of specifications of derived tables / views / CTEs. The name
resolution outer context for such ON expression must be set to NULL to
prevent name resolution beyond the derived table where it is used.
The solution to resolve this problem applied in sql_derived.cc was provided
by Sergei Petrunia <sergey@mariadb.com>.

The change in sql_parse.cc is not good for 10.4+. A corresponding diff for
10.4+ will be provided in JIRA entry for this bug.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

It seems some overly tolerant compilers (gcc) allow the structure
of IO_CACHE that is defined differently in libmaria to have
members equalivance to the iocache in mysys.

More strict Solaris compilers recognise that rc_pos really
isn't a structure member and won't compile.

Make Item_subselect::walk() walk the ON expressions, too.

MDEV-22255 SIGABRT: Assertion `id' failed in trx_write_trx_id on INSERT | Assertion `id > 0' failed in trx_write_trx_id | Assertion `val > 0' failed in row_upd_index_entry_sys_field | Assertion `thr_get_trx(thr)->id || index->table->no_rollback()' failed.

InnoDB purge thread locks the root page of clustered index
while accessing the undo log records and later same thread
tries to open the table, initialize statistics and tries
to lock the clustered index root page while doing virtual
column computation.

Solution:
=========
InnoDB should prevent statistics initialization when the
table is being opened by purge thread

When a column is added to an non-empty table, existing rows will have
a column's default value for existing rows. Or a "zero value" if the
column has no default.

But this check should be skipped when an existing column is altered.

Link with -pthread instead of -lpthread, and do that for all architectures.

Contributors:
Aurelien Jarno <aurelien@aurel32.net>

Closes: #1717

signal handler, display coredump file pattern similarly to MDEV-25294 but for FreeBSD, thankfully the sysctl OID is the same.

also add =WARN as an alias for =OFF
and clarify the help text

Between btr_pcur_store_position() and btr_pcur_restore_position()
it is possible that purge empties a table and enlarges
index->n_core_fields and index->n_core_null_bytes.
Therefore, we must cache index->n_core_fields in
btr_pcur_t::old_n_core_fields so that btr_pcur_t::old_rec can be
parsed correctly.

Unfortunately, this is a huge change, because we will replace
"bool leaf" parameters with "ulint n_core"
(passing index->n_core_fields, or 0 for non-leaf pages).
For special cases where we know that index->is_instant() cannot hold,
we may also pass index->n_fields.

- Fixing post-fix failure. In clean_new_vcol_index(), InnoDB has the wrong
offset to store the virtual column

MDEV-24135: Print warnings in XML, save test retries in XML, save the combinations in XML, replace the special symbols in the XML comment

At the second execution of the PS
1. mark_as_dependent() is called with the same parameters as at the first
   execution (select#4 and select#3)
2. as outer_select (select#3) has been already merged at the first
   execution of PS it cannot be reached using the outer_select() function
   anymore (and so can not stop iteration).
3. as a result all selects towards the top level select including the
   select for 'ca' are marked as uncacheable.
4. Marked uncacheable it executed incorrectly triggering filling its
   temporary table several times and using freed memory at the end.

To avoid the problem we use name resolution context to go "up".

NOTE: problem also exists in 10.2 but has no visible effect on execution.
That is why the problem is fixed in 10.2.

The patch also add debug logging of important procedures and
better specify parameters types of st_select_lex::mark_as_dependent.

MDEV-25108: Running of the EXPLAIN EXTENDED statement produces extra warning in case it is executed in PS (prepared statement) mode

The EXPLAIN EXTENDED statement run as a prepared statement can produce extra
warning comparing with a case when EXPLAIN EXTENDED statement is run as
a regular statement. For example, the following test case
  CREATE TABLE t1 (c int);
  CREATE TABLE t2 (d int);
  EXPLAIN EXTENDED SELECT (SELECT 1 FROM t2 WHERE d = c) FROM t1;

produces the extra warning
  "Field or reference 'c' of SELECT #2 was resolved in SELECT #1"
in case the above mentioned "EXPLAIN EXTENDED" statement is executed
in PS mode, that is by submitting the following statements:
   PREPARE stmt FROM "EXPLAIN EXTENDED SELECT (SELECT 1 FROM t2 WHERE d = c) FROM t1";
   EXECUTE stmt;

The reason of the extra warning emittion is in a way items
are handled (being fixed) during execution of the JOIN::prepare() method.
The method Item_field::fix_fields() calls the find_field_in_tables()
function in case a field hasn't been associated yet with the item.
Implementation of the find_field_in_tables() function first checks whether
a table containing the required field was already opened and cached.
It is done by checking the data member item->cached_table. This data member
is set on handling the PRERARE FROM statement and checked on executing
the EXECUTE statement. If the data member item->cached_table is set
the find_field_in_tables() function invoked and the
mark_select_range_as_dependent() function called if the field
is an outer referencee. The mark_select_range_as_dependent() function
calls the mark_as_dependent() function that finally invokes
the push_warning_printf() function that produces extra warning.

To fix the issue, calling of push_warning_printf() is elimited in case
it was run indirectly in result of hanlding already opened table from
the Item_field::fix_fields() method.

This commit removes the mtr test galera_sst_mariabackup_encrypt_with_key
from the list of disabled tests because the problem with it has already
been fixed.

Problem:
========
 InnoDB fails to clean the index stub if it fails to add the
virtual index which contains new virtual column. But it clears
the newly virtual column from index in clear_added_indexes()
during inplace_alter_table. On commit, InnoDB evicts and
reload the table. In case of rollback, it doesn't happen.
InnoDB clears the ABORTED index while opening the table
or doing the DDL. In the mean time, InnoDB can access
the dropped virtual index columns while creating prebuilt
or rollback of concurrent DML.

Solution:
==========
(1) InnoDB should maintain newly added virtual column while
rollbacking the newly added virtual index.
(2) InnoDB must not defer the index removal
if the alter table is executed with LOCK=EXCLUSIVE.
(3) For LOCK=SHARED, InnoDB should check whether the table
has any other transaction lock other than alter transaction
before deferring the index stub.

Replaced has_new_v_col with dict_add_vcol_info in dict_index_t to
indicate whether the index has any new virtual column.

dict_index_t::has_new_v_col(): Returns whether the index has
newly added virtual column, it doesn't say which columns are
newly added virtual column

ha_innobase_inplace_ctx::is_new_vcol(): Return whether the
given column is added as a part of the current alter.

ha_innobase_inplace_ctx::clean_new_vcol_index(): Copy the newly
added virtual column to new_vcol_info in dict_index_t. Replace
the column in the index fields with virtual column stored
in new_vcol_info.

dict_index_t::assign_new_v_col(): Store the number of virtual
column added in index as a part of alter table.

dict_index_t::get_n_new_vcol(): Get the number of newly added
virtual column

dict_index_t::assign_drop_v_col(): Allocate the memory for
adding new virtual column in new_vcol_info.

dict_index_t::add_drop_v_col(): Add the newly added virtual
column in new_vcol_info.

dict_table_t::has_lock_for_other_trx(): Whether the table has
any other transaction lock than given transaction.

row_merge_drop_indexes(): Add parameter alter_trx and check
whether the table has any other lock than alter transaction.

When a table has been evicted from dict_sys and reloaded internally by
InnoDB for FOREIGN KEY processing, statistics may not be initialized,
but nevertheless row_update_cascade_for_mysql() could invoke
dict_stats_update_if_needed(). In that case, we cannot really update
the statistics. For tables that have STATS_PERSISTENT=1 and
STATS_AUTO_RECALC=1, ANALYZE TABLE might have to be executed later.

dict_stats_update_if_needed(): Replace the assertion with
a conditional early return.

trx_sys_any_active_transactions(): Remove a bogus debug assertion.
In trx_commit_in_memory() and trx_erase_lists(), we will remove
the transaction from trx_sys->rw_trx_list and set the state to
TRX_STATE_COMMITTED_IN_MEMORY.

MariaDB Server still supports Ubuntu 16.04 "Xenial" until it goes EOL
in April 30, 2021. Thus we need to include a customization for backwards
compatibility.

This change is intended to be applied for all MariaDB versions still
supported, i.e. 10.2 to 10.6.