Field_iterator_table_ref::set_field_iterator

Several functions that processed different prepare statements missed
the DT_INIT flag in last parameter of the open_normal_and_derived_tables()
calls. It made context analysis of derived tables dependent on the order in
which the derived tables were processed by mysql_handle_derived(). This
order was induced by the order of SELECTs in all_select_list.
In 10.4 the order of SELECTs in all_select_list became different and lack
of the DT_INIT flags in some open_normal_and_derived_tables() call became
critical as some derived tables were not identified as such.

The SELECT with the INNER JOIN is executed with one of the two tables being
optimized as a constant table, which is pre-read.  Spider nevertheless attempts
to push down the join to the data node.  The crash occurs because the constant
table is excluded from the optimized query that Spider attempts to push down.

In order for Spider to be able to push down a join, the following conditions
need to be met:
- All of the tables involved in the join need to be included in the optimized
  query that Spider pushes down.  When any of the tables involved in the join
  is a constant table, it is excluded from the optimized query that Spider
  attempts to push down.
- All fields involved in the query need to be members of tables included in the
  optimized query.

I fixed the problem by preventing Spider from pushing down queries that include
a field that is not a member of a table included in the optimized query.  This
solution fixes the reported problem and also fixes other potential problems.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 4885baf6 on branch bb-10.3-MDEV-16889

During a server restart that follows a server crash, there may be error
messages that indicate that certain system tables are marked as crashed and
may be corrupted.  Upon checking the system tables that are marked as crashed,
it may be found that there is no corruption that needs repair.  However, the
error messages that are issued imply that the user needs to nevertheless
manually repair the system tables that were marked as crashed.  These issues
have been moved to a separate bug MDEV-17068.

MDEV-16250 addresses the work to make the Spider system tables crash safe.
This work involves the following changes in 10.4:
- Changes to the install_spider.sql script to change the storage engine for
  the Spider system tables to Aria.  This is implemented in such a way as to
  allow the script to be run repeatedly without any harm or errors.
- Changes to the init_spider.inc script that is run during initialization of
  every test in the Spider test suites.  This script now uses the Aria
  storage engine for the Spider system tables in MariaDB Server 10.4 and
  later releases.
- Added a test to the Spider test suite to display the storage engine of each
  Spider system table, to verify that the correct storage engine is used.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Merged:
  Commit 73fac2a5 from branch bb-10.4-MDEV-16250

MDEV-17068 mysql system table is marked as crashed and should be repaired after the server crashes or is killed

- Changed ERROR to WARNING for MyISAM/Aria message
  that are warnings in the check utilities.
  This affects for example "client is using or
  hasn't closed the table properly".
- Print "Table is fixed" if check succeded in
  fixing the table.

During a server restart that follows a server crash, there may be error
messages that indicate that certain system tables are marked as crashed and
may be corrupted.  Upon checking the system tables that are marked as crashed,
it may be found that there is no corruption that needs repair.  However, the
error messages that are issued imply that the user needs to nevertheless
manually repair the system tables that were marked as crashed.  These issues
have been moved to a separate bug MDEV-17068.

MDEV-16250 addresses the work to make the Spider system tables crash safe.
This work involves the following changes in 10.4:
- Changes to the install_spider.sql script to change the storage engine for
  the Spider system tables to Aria.  This is implemented in such a way as to
  allow the script to be run repeatedly without any harm or errors.
- Changes to the init_spider.inc script that is run during initialization of
  every test in the Spider test suites.  This script now uses the Aria
  storage engine for the Spider system tables in MariaDB Server 10.4 and
  later releases.
- Added a test to the Spider test suite to display the storage engine of each
  Spider system table, to verify that the correct storage engine is used.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Moved the checks for arguments validation of Item_name_const from the constructor
to Create_func_name_const::create_2_arg
Also reverted the fix bf1c53e9

The problem was that when a mysql.proc table was
opened for reading it was added to the current Aria
transaction context but never properly deleted from
it. Normally this isn't a problem, except if the
mysql.proc table is closed before the transaction
ended, which happened in this test case.

Fixed by removing mysql.proc from the transaction
context before closing the table.

For Item name_const , we should never do typecast it to Item_field because we
always expect it to be a constant value.
So instead of checking the type() its better to introduce a function in the
Item class get_item_field, which would return the item_field object for the item
which have type of FIELD_ITEM

This was introduced by two pointers I added to TRN
as part of MDEV-16421 Make system tables crash safe

- Added code to ensure that trn_prev is not pointing
  to wrong object
- A lot of new asserts and more code comments
- Simplified code in _ma_trnman_end_trans_hook()
  - New back link allowed me to remove a loop

Changed bootstrap scripts, adjusted result files.

Amend commit b853b4fd
that was reverted in commit 29150e23.

recv_parse_log_recs(): Do check for corrupted redo log or file
system before checking for len==0, but only read *ptr if
it is not past the end of the buffer (end_ptr).

recv_parse_log_rec(): Report incorrect redo log type
in a consistent way with recv_parse_or_apply_log_rec_body().
This is a follow-up to commit f30c5af4.

The Pool poisoning that was introduced in MDEV-15030 introduced
race conditions in AddressSanitizer builds, because concurrent
poisoning and unpoisoning were not prevented by any synchronization
primitive.

Pool::get(): Protect the unpoisoning by m_lock_strategy.

Pool::mem_free(): Protect the poisoning by m_lock_strategy.

Pool::putl(): Renamed from put(), because now the caller is
responsible for invoking m_lock_strategy.

If trx_free() and trx_create_low() were called while a call to
trx_reference() was pending, we could get a reference to a wrong
transaction object.

trx_reference(): Return NULL if the trx->id no longer matches.

lock_trx_release_locks(): Assign trx->id = 0, so that trx_reference()
will not return a reference to this object.

trx_cleanup_at_db_startup(): Assign trx->id = 0.

assert_trx_is_free(): Assert !trx->n_ref. Assert trx->id == 0,
now that it will be cleared as part of a transaction commit.

Also, replace reverse iteration with forward iteration.

lock_table_has(): Remove a redundant condition.

Allocate trx->lock.rec_pool and trx->lock.table_pool directly from trx_t.
Remove unnecessary use of std::vector.

In order to do this, move some definitions from lock0priv.h to
lock0types.h, so that ib_lock_t will not be an opaque type.

The patch for MDEV-16928 added a few new asserts to check that
time, date, datetime values are valid and consistent after initialization.
One of the new asserts caught an improper initialization in
Time::make_from_datetime_with_days_diff()
(the former function calc_datetime_days_diff()).
If the YYYYMM part is not zero after unpack time
we have an out-of-range TIME value.

Fixing the bug (in the code from MDEV-16910)

row_log_table_get_pk_col(): Replace a condition that was inadvertently
removed in MDEV-16365. PRIMARY KEY columns are never allowed to be NULL,
and failure to enforce the constraint caused a null pointer to be
dereferenced in mem_heap_dup().

Make all system tables in mysql directory of type
engine=Aria

Privilege tables are using transactional=1
Statistical tables are using transactional=0, to allow them
to be quickly updated with low overhead.
Help tables are also using transactional=0 as these are only
updated at init time.

Other changes:
- Aria store engine is now a required engine
- Update comment for Aria tables to reflect their new usage
- Fixed that _ma_reset_trn_for_table() removes unlocked table
  from transaction table list. This was needed to allow one
  to lock and unlock system tables separately from other
  tables, for example when reading a procedure from mysql.proc
- Don't give a warning when using transactional=1 for engines
  that is using transactions. This is both logical and also
  to avoid warnings/errors when doing an alter of a privilege
  table to InnoDB.
- Don't abort on warnings from ALTER TABLE for changes that
  would be accepted by CREATE TABLE.
- New created Aria transactional tables are marked as not movable
  (as they include create_rename_lsn).
- bootstrap.test was changed to kill orignal server, as one
  can't anymore have two servers started at same time on same
  data directory and data files.
- Disable maria.small_blocksize as one can't anymore change
  aria block size after system tables are created.
- Speed up creation of help tables by using lock tables.
- wsrep_sst_resync now also copies Aria redo logs.

aws_key_management needs current directory to be datadir during
initalization, it scans current directory for encrypted keys.

Fix is to ensure, that plugin initialization in mariabackup happens
after the call to my_setwd(mysql_real_data_home).

The problem occurs because the statement generated by Spider used an
internal function name, ADD_TIME.

This problem has been corrected by the fix for bug MDEV-16878 within the
server, which enables Spider to generate the statement using the actual
SQL function name.  I have made some additional changes within Spider to fix
related problems that I observed while testing.

Author:
  Jacob Mathew.

First Reviewer:
  Alexander Barkov.

Second Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 4b6dccc8 on branch bb-10.3-MDEV-16398

The problem occurs because the statement generated by Spider used an
internal function name, ADD_TIME.

This problem has been corrected by the fix for bug MDEV-16878 within the
server, which enables Spider to generate the statement using the actual
SQL function name.  I have made some additional changes within Spider to fix
related problems that I observed while testing.

Author:
  Jacob Mathew.

First Reviewer:
  Alexander Barkov.

Second Reviewer:
  Kentoku Shiba.

Merged:
  Commit 4b6dccc8 on branch bb-10.3-MDEV-16398

Fix a race condition in the test.