After MDEV-17772 table existence check is performed much earlier, so
create_table_select_before_check_if_exists debug sync point is not
reachable when table exists.

Moved debug sync point to appropriate place.

The result is updated here because by changing the defaults for use_stat_tables
EITs statistcs are providing more accurate statistics than INNODB statistics.

row_undo_ins_remove_clust_rec(): When rolling back the insert
of a metadata record, also reset the root page state within the
same mini-transaction.

dict_table_t::rollback_instant(): Do not allow index->n_core_fields
to be larger than index->n_fields. This logic exists in the 10.3
branch but was incorrectly removed as part of MDEV-15562 in 10.4.0.

mariabackup.lock_ddl_per_table was broken, after MDEV-17772 got fixed.
Fix removes MDL waiter killer, it is is not needed anymore.

This regression was introduced in MDEV-11369 instant ADD COLUMN.

prepare_inplace_alter_table_dict(): Avoid dereferencing ctx->new_table
after row_create_table_for_mysql() returns a failure code, because
it will have freed the memory.

There was a race condition in the error handling of ALTER TABLE when
the table contains FULLTEXT INDEX.

During the error handling of an erroneous ALTER TABLE statement,
when InnoDB would drop the internally created tables for FULLTEXT INDEX,
it could happen that one of the hidden tables was being concurrently
accessed by a background thread. Because of this, InnoDB would defer
the drop operation to the background.

However, related to MDEV-13564 backup-safe TRUNCATE TABLE and its
prerequisite MDEV-14585, we had to make the background drop table queue
crash-safe by renaming the table to a temporary name before enqueueing it.
This renaming was introduced in a follow-up of the MDEV-13407 fix.
As part of this rename operation, we were unnecessarily parsing the
current SQL statement, because the same rename operation could also be
executed as part of ALTER TABLE via ha_innobase::rename_table().

If an ALTER TABLE statement was being refused due to incorrectly formed
FOREIGN KEY constraint, then it could happen that the renaming of the hidden
internal tables for FULLTEXT INDEX could also fail, triggering a host of
error log messages, and causing a subsequent table-rebuilding ALTER TABLE
operation to fail due to the tablespace already existing.

innobase_rename_table(), row_rename_table_for_mysql(): Add the parameter
use_fk for suppressing the parsing of FOREIGN KEY constraints. It
will only be passed as use_fk=true by ha_innobase::rename_table(),
which can be invoked as part of ALTER TABLE...ALGORITHM=COPY.

Removing the unused THD* parameter.

MDEV-17772 - 3 way lock : ALTER, MDL, BACKUP STAGE BLOCK_DDL

While waiting for a (potentially long) RO transaction or SELECT, DDL and
LOCK TABLES ... WRITE hold protection against FTWRL and BACKUP STAGE.

This effectively makes FTWRL/BACKUP STAGE indirectly wait for this RO
transaction or SELECT to finish. Which is not great, as otherwise we
could do something useful meanwhile.

With this patch BACKUP lock is attempted to be acquired after TABLE/SCHEMA
locks. If this attempt fails, TABLE/SCHEMA locks gets released and we
start waiting for BACKUP lock. When wait finishes, BACKUP lock is released
(to avoid deadlocks) and we attempt to acquire all locks once again.

Other changes:
- Take MDL lock before testing if table exists as part of
  CREATE TABLE ... IF EXISTS.  This change was an effect of changes in
  lock_table_name and removes an inconsistency where one could get
  different error messages from CREATE TABLE .. IF EXISTS depending on
  active mdl locks.
  One effect of this change is that we don't binary log CREATE TABLE IF
  EXISTS if the table exists.  This was done because old code was sometimes
  behaving inconsistenly (it was logged some time and not other times)
  and sending the query to the slave could make the slave even more
  inconsistent as there is not guarantee that the new table will have
  the same definition as the old table on the master.

Part of MDEV-5336 Implement LOCK FOR BACKUP

- Changed check of Global_only_lock to also include BACKUP lock.
- We store latest MDL_BACKUP_DDL lock in thd->mdl_backup_ticket to be able
  to downgrade lock during copy_data_between_tables()

Part of MDEV-5336 Implement LOCK FOR BACKUP

- Added new locks to MDL_BACKUP for all stages of backup locks and
  a new MDL lock needed for backup stages.
- Renamed MDL_BACKUP_STMT to MDL_BACKUP_DDL
- flush_tables() takes a new parameter that decides what should be flushed.
- InnoDB, Aria (transactional tables with checksums), Blackhole, Federated
  and Federatedx tables are marked to be safe for online backup. We are
  using MDL_BACKUP_TRANS_DML instead of MDL_BACKUP_DML locks for these
  which allows any DML's to proceed for these tables during the whole
  backup process until BACKUP STAGE COMMIT which will block the final
  commit.

- Call delete_statistics_tables() after lock_table_names in drop tables.
  This avoids a deadlock issue with FTWRL and future backup locks.
- Added some missing clear_error()
- Ensure we don't clear error caused by the caller
- Updated function comments

- Re-numbered enum_table_category to make some tests easier
- Moved TABLE_CATEGORY_INFORMATION to be first CATEGORY of virtual tables
- Don't take MDL locks for not updateable table category's

Part of MDEV-5336 Implement LOCK FOR BACKUP

FLUSH TABLE table_names have changed slighty as we are now opening
tables before taking the MDL lock. The difference is that FLUSH TABLE
table_name will now be blocked by a table that is waiting for FTWRL.
There should not be any new deadlocks as part of this change.

The end result is still better in most cases as FTWRL is now only
waiting for write statements to end, not for read only statements and
it's not flushing tables in use from the table cache.

Share will be needed to be able to determine if table supports online
backup. Appropriate metadata lock type in BACKUP namespace will be
acquired basing on this information.

Also made pending global read lock request to be preferred victim of MDL
deadlock detector. This allows us to hide some non-fatal deadlocks and
make FTWRL less likely to break concurrent queries.

Part of MDEV-5336 Implement LOCK FOR BACKUP

Other things:
- Added printing of MDL locks to DBUG.

Part of MDEV-5336 Implement LOCK FOR BACKUP

Originally both table metadata lock and global read lock protection
were acquired before getting TABLE from table cache. This will be
reordered in a future commit with MDL_BACKUP_XXX locks so that we
first take table metadata lock, then get TABLE from table cache, then
acquire analogue of global read lock.

This patch both simplifies FLUSH TABLES code, makes FLUSH TABLES to
lock less and also enables FLUSH TABLES code to be used with backup
locks.

The usage of FLUSH TABLES changes slightly:
- FLUSH TABLES without any arguments will now only close not used tables
  and tables locked by the FLUSH TABLES connection.  All not used table
  shares will be closed.
  Tables locked by the FLUSH TABLES connection will be reopened and
  re-locked after all others has stoped using the table (as before).
  If there was no locked tables, then FLUSH TABLES is instant and will
  not cause any waits.
  FLUSH TABLES will not wait for any in use table.
- FLUSH TABLES with a table list, will ensure that the tables are closed
  before statement returns. The code is now only using MDL locks and not
  table share versions, which simplices the code greatly. One visible
  change is that the server will wait for the end of the transaction that
  are using the tables. Before FLUSH TABLES only waited for the statements
  to end.
Signed-off-by: Monty <monty@mariadb.org>

FLUSH TABLES kills all insert delayed treads. If this happens at same time
as a newly started insert delayed query, the insert may fail with either
and error ("Query interrupted") or the row may be lost.
This patch fixes this by changing a failed attempt of get_delay_table() to
convert the query to use a normal insert.

The test case for this patch can be found in the FLUSH TABLES commit after
this one.

Part of MDEV-5336 Implement LOCK FOR BACKUP

The idea is that instead of waiting in close_cached_tables() for all
tables to be closed, we instead call flush_tables() that does:
- Flush not used objects in table cache to free memory
- Collect all tables that are open
- Call HA_EXTRA_FLUSH on the objects, to get them into "closed state"
- Added HA_EXTRA_FLUSH support to archive and CSV
- Added multi-user protection to HA_EXTRA_FLUSH in MyISAM and Aria

The benefit compared to old code is:
- FTWRL doesn't have to wait for long running read operations or
  open HANDLER's

Part of MDEV-5336 Implement LOCK FOR BACKUP

Added support for backup calls in Aria to protect
removal of redo logs.

- The old code used the original create_info from lex, not the new one
  that includes more information (like OPT_OR_REPLACE).
  The bug was not discovered as the code in lock_table_named() only
  checked for OPT_OR_REPLACE in case of timeout errors.
  As lock_table_names will be fixed as part of BACKUP STAGE's, there
  is no changes in lock_table_names() in this commit.
- Removed also the 'temporary' copy of statement flags to thd for
  lock_table_names()

- Moved tests depending on utf8 characters from create to create_utf8
- Fixed some tests in create and sp that wrongly tried to create table
  from non existing table on existing table.  In a later patch we may
  first check if table exists, in which case the error message would
  change.
- Updated results for partition_debug_tokudb

Bascially this means that all builds with BUILD scripts are done with
--with-jemalloc=NO      # Required by tokudb
--with-ssl              # Required on OpenSuse 10.5 to get galera to work

- Remove obsolete documentation files
- Removed old obsolete information from some documentation files

Added to new values to the server variable use_stat_tables.
The values are COMPLEMENTARY_FOR_QUERIES and PREFERABLY_FOR_QUERIES.
Both these values don't allow to collect EITS for queries like
    analyze table t1;
To collect EITS we would need to use the syntax with persistent like
   analyze table t1 persistent for columns (col1,col2...) index (idx1, idx2...) / ALL

Changing the default value from NEVER to PREFERABLY_FOR_QUERIES.

use_stat_tables= PREFERABLY
optimizer_use_condition_selectivity= 4

MDEV-17928 Conversion from TIMESTAMP to VARCHAR SP variables does not work well on fractional digits