MDEV-25683 Atomic DDL: With innodb_force_recovery=3 InnoDB: Trying to load index but the index tree has been freed

The purpose of the parameter innodb_force_recovery is to allow some
data to be dumped from a corrupted database. Its values used to be
as follows:

innodb_force_recovery=0: normal (default)

innodb_force_recovery=1: ignore (skip log for) corrupted pages or
missing data files when applying the redo log

innodb_force_recovery=2: additionally, disable background tasks
(such as the purge of committed undo logs)

innodb_force_recovery=3: additionally, disable the rollback of
recovered incomplete (not committed or XA PREPARE) transactions

innodb_force_recovery=4: same as 3 (since MDEV-19514 in MariaDB 10.5)

innodb_force_recovery=5: additionally, do not process any undo log,
disallow any writes, and force READ UNCOMMITTED isolation level

innodb_force_recovery=6: additionally, pretend that ib_logfile0 does
not exist (prevent any recovery). Never use this!

The bad thing that happens with innodb_force_recovery=3 and
innodb_force_recovery=4 is that also the rollback of any recovered
DDL transaction will be skipped. This would break the DDL log recovery
that was introduced in MDEV-17567.

For one data directory sample, the DDL log recovery would hangs due to
a conflict on the InnoDB SYS_TABLES table, because the lock holder
transaction was not rolled back due to innodb_force_recovery=3.

Fix: Make innodb_force_recovery=3 skip the DML transaction rollback only,
and make innodb_force_recovery=4 (renamed to SRV_FORCE_NO_DDL_UNDO)
behave like innodb_force_recovery=3 used to (skip the rollback of all
recovered transactions, both DML and DDL).

Startup with innodb_force_recovery=4 will be unaffected by this change.
(There may be hangs, possibly preceded by messages about failing to
load an index.)

Side note: With innodb_force_recovery=5, any DDL log for InnoDB tables
will be essentially ignored by InnoDB, but the server will start up.

Fix by removing the trigger. It does not do anything useful anyway.

The InnoDB changes in MySQL 5.7.36 that were applicable to MariaDB
were covered by MDEV-26864, MDEV-26865, MDEV-26866.

The initial test case for MySQL Bug #33053297 is based on
mysql/mysql-server@27130e25078864b010d81266f9613d389d4a229b.

innobase_get_field_from_update_vector is not a suitable function to fetch
updated row info, as well as parent table's update vector is not always
suitable. For instance, in case of DELETE it contains undefined data.

castade->update vector seems to be good enough to fetch all base columns
update data, and besides faster, and less error-prone.

In the replication-related code, in the exec_relay_log_event() (slave.cc)
function, where the "data_lock" mutex is captured, this mutex is then not
released on one of the early return branches within a specific insert for
WSREP, namely under the branch: "if (wsrep_before_statement(thd))". As a
result, the mutex remains captured, resulting in errors or hangs.

This commit fixes this issue, which is now showing up as intermittent
failures in mtr tests for galera and galera_sr suites.

Similar to commit f7684f0c (MDEV-26855)
we will try to enable the adaptive spinloop for lock_sys.wait_mutex
on ARMv8.

Enabling any form of spinloop for lock_sys.wait_mutex did not show a
significant improvement in our tests on AMD64.

Spinning can be argued to be a hack to reduce the impact on mutex
contention. It would be better to adjust the code to reduce
contention in the first place.

ha_rocksdb.h:459:15: warning: 'table_type' overrides a member
function but is not marked 'override' [-Winconsistent-missing-override]

The assert inside String::copy() prevents copying from from "str"
if its own String::Ptr also points to the same memory.

The idea of the assert is that copy() performs memory reallocation,
and this reallocation can free (and thus invalidate) the memory pointed by Ptr,
which can lead to further copying from a freed memory.

The assert was incomplete: copy() can free the memory pointed by its Ptr
only if String::alloced is true!

If the String is not alloced, it is still safe to copy even from
the location pointed by Ptr.

This scenario demonstrates a safe copy():
  const char *tmp= "123";
  String str1(tmp, 3);
  String str2(tmp, 3);
  // This statement is safe:
  str2.copy(str1->ptr(), str1->length(), str1->charset(), cs_to, &errors);

Inside the copy() the parameter "str" is equal to String::Ptr in this example.
But it's still ok to reallocate the memory for str2, because str2
was a constant before the copy() call. Thus reallocation does not
make the memory pointed by str1->ptr() invalid.

Adjusting the assert condition to allow copying for constant strings.

Adding 10.4 specific tests.

Also fixes:
MDEV-25399 Assertion `name.length == strlen(name.str)' failed in Item_func_sp::make_send_field

Also fixes a problem that in this scenario:

SET NAMES binary;
SELECT 'some not well-formed utf8 string';

the auto-generated column name copied the binary string value directly
to the Item name, without checking utf8 well-formedness.

After this change auto-generated column names work as follows:
- Zero bytes 0x00 are copied to the name using HEX notation
- In case of "SET NAMES binary", all bytes sequences that do not make
  well-formed utf8 characters are copied to the name using HEX notation.

ALTER TABLE IMPORT doesn't properly handle instant alter metadata.
This patch makes IMPORT read, parse and apply instant alter metadata at the
very beginning of operation. So, cases when source table has some metadata
and destination table doesn't have it now works fine.

DISCARD already removes instant metadata so importing normal table into
instant table worked fine before this patch.

decrypt_decompress(): decrypts and decompresses page if needed

handle_instant_metadata(): this should be the first thing to read source
table. Basically, it applies instant metadata to a destination
dict_table_t object. This is the first thing to read FSP flags so
all possible checks of it were moved to this function.

PageConverter::update_index_page(): it doesn't now read instant metadata.
This logic were moved into handle_instant_metadata()

row_import::match_flags(): this is a first part row_import::match_schema().
As a separate function it's used by handle_instant_metadata().

fil_space_t::is_full_crc32_compressed(): added convenient function

ha_innobase::discard_or_import_tablespace(): do not reload table definition
to read instant metadata because handle_instant_metadata() does it better.
The reverted code was originally added in
4e7ee166

ANONYMOUS_VAR: this is a handy thing to use along with make_scope_exit()

full_crc32_import.test shows different results, because no
dict_table_close() and dict_table_open_on_id() happens.
Thus, SHOW CREATE TABLE shows a little bit older table definition.

Also fixes MDEV-24619 Wrong result or Assertion `0' in Item::val_native / Type_handler_inet6::Item_val_native_with_conversion

Type_handler_inet6::create_item_copy() created a generic Item_copy_string,
which does not implement val_native() - it has a dummy implementation
with DBUG_ASSERT(0), which made the server crash.

Fix:

- Adding a new class Type_handler_inet6
  which implements val_native().
- Fixing Type_handler_inet6::create_item_copy()
  to make Item_copy_inet6 instead of Item_copy_string.

Take into account client capabilities.

Sometimes, although not often, it would timeout after 3 minutes.

Happens with Innodb engine.

Move unlock_locked_table() past drop_open_table(), and
rollback current statement, so that we can actually unlock the table.

Anything else results in assertions, in drop, or unlock, or in close_table.

MDEV-25702(commit 696de6d0) should've
closed the fts table further. This patch closes the table after
finishing the bulk insert operation.

rollback_inplace_alter_table(): Tolerate a case where the transaction
is not in an active state. If ha_innobase::commit_inplace_alter_table()
failed with a deadlock, the transaction would already have been
rolled back. This omission of error handling was introduced in
commit 1bd681c8 (MDEV-25506 part 3).

After commit c3c53926 (MDEV-26554)
it became easier to trigger DB_DEADLOCK during exclusive table lock
acquisition in ha_innobase::commit_inplace_alter_table().

lock_table_low(): Add DBUG injection "innodb_table_deadlock".

DBUG_ASSERT removed as the AUTO INCREMENT can actually be 0 when the
SET insert_id= 0; was done.

Get rid of the global big_buffer.

SysTablespace::file_not_found(): If the system tablespace cannot be
found and innodb_force_recovery has been specified, refuse to start up.
The system tablespace is necessary for accessing any InnoDB tables,
because it contains the TRX_SYS page (the state of transactions)
and the InnoDB data dictionary.

This is similar to our handling of innodb_read_only except that
we will happily create the InnoDB temporary tablespace even if
innodb_force_recovry is set.

`mytop` and `my_print_defaults` for RPM

- Add `mytop` to client package
- Add man page of `my_print_defaults` to client package
- Add dependencies for RPMs
- Remove old comment
- Remove dead link

Reviewed by: serg@mariadb.com

Make `my_print_defaults` a client app (required by `mytop`).

Closes #1566

Reviewed by: serg@mariadb.com, anel@mariadb.org

We have observed hangs of the io_uring subsystem when using a
Linux kernel newer than 5.10. Also 5.15-rc6 is affected by this.

The exact cause of the hangs has not been diagnosed yet.
As a safety measure, we will disable innodb_use_native_aio by default
when the server has been configured with io_uring and the kernel
version is between 5.11 and 5.15.

If the start-up parameter innodb_use_native_aio=ON is set, then
we will issue a warning to the server error log.

This implements memory transaction support for:

* Intel Restricted Transactional Memory (RTM), also known as TSX-NI
(Transactional Synchronization Extensions New Instructions)
* POWER v2.09 Hardware Trace Monitor (HTM) on GNU/Linux

transactional_lock_guard, transactional_shared_lock_guard:
RAII lock guards that try to elide the lock acquisition
when transactional memory is available.

buf_pool.page_hash: Try to elide latches whenever feasible.
Related to the InnoDB change buffer and ROW_FORMAT=COMPRESSED
tables, this is not always possible.
In buf_page_get_low(), memory transactions only work reasonably
well for validating a guessed block address.

TMLockGuard, TMLockTrxGuard, TMLockMutexGuard: RAII lock guards
that try to elide lock_sys.latch and related latches.

Since commit bd5a6403 (MDEV-26033)
we can actually calculate the buf_pool.page_hash cell and latch
addresses while not holding buf_pool.mutex.

buf_page_alloc_descriptor(): Remove the MEM_UNDEFINED.
We now expect buf_page_t::hash to be zero-initialized.

buf_pool_t::hash_chain: Dedicated data type for buf_pool.page_hash.array.

buf_LRU_free_one_page(): Merged to the only caller
buf_pool_t::corrupted_evict().