Compiling hashicorp plugin statically got many InnoDB
encryption test to fail with errors like:
Error: InnoDB: The page [page id: space=5, page number=3] in file './test/t1.ibd' cannot be decrypted.

In commit f80deb95 (MDEV-27868)
a fix for a correctness regression caused a performance regression
by increasing the amount of work that is executed while holding
buf_pool.flush_list_mutex.

buf_page_t::set_temp_modified(): Relax an assertion, to allow an
already dirty block to be marked as dirty.

buf_page_t::flush_list_requests: Note that the variable is not
always protected by buf_pool.flush_list_mutex. Already dirty blocks
that are being written to will increment the counter without
holding buf_pool.flush_list_mutex.

mtr_t::process_freed_pages(): Handle pages that were freed during
the execution of the mini-transaction.

ReleaseUnlogged, mtr_t::release_unlogged(): Release modified pages when
no log was written. This is for pages of the temporary tablespace, or for
IMPORT TABLESPACE.

ReleaseModified: Renamed from ReleaseBlocks.
Assume that buf_pool.flush_list_mutex was acquired by the caller.

ReleaseSimple: A combination of ReleaseLatches and ReleaseModified,
for the case that for any modified pages, some earlier modifications
are already waiting to be written.

mtr_t::commit(): Invoke one of release_unlogged(), ReleaseModified,
ReleaseSimple, ReleaseAll. Acquire and release buf_pool.flush_list_mutex
at most once.

memo_slot_release(): Simplify the code.

mtr_t::sx_latch_at_savepoint(), mtr_t::x_latch_at_savepoint():
Reduce the size of the critical section.

fil_space_t::update_last_freed_lsn(), fil_space_t::clear_freed_ranges(),
fil_space_t::add_free_range(): Assume that freed_range_mutex is held
by the caller.

buf_pool_t::prepare_insert_into_flush_list(): Determine the insert
position for buf_pool_t::insert_into_flush_list(). Remove any clean
blocks from buf_pool.flush_list that were encountered while searching.

buf_pool_t::insert_into_flush_list(): Insert the block at the
predetermined position.

PageConverter::update_header(): Remove an unnecessary write.
The field that was originally called FIL_PAGE_FILE_FLUSH_LSN only
made sense for the first page of the system tablespace
(initially, for the first page of each file of the system tablespace).
It never had any meaning for .ibd files, and it lost its original
meaning in MariaDB Server 10.8.1 when
commit b07920b6 (MDEV-27199)
removed the ability to start without ib_logfile0.

If the most significant 32 bits of the LSN are nonzero, this
unnecessary write would write the wrong encryption key identifier
to the page. The first page of any file is never encrypted,
so normally those bytes should be 0 for any .ibd file.

The zoneinfo directory is littered with non-timezone information files.

These frequently contain extensions, not present in real timezone files.

Alo leapseconds is frequently there and is not a timezone file.

The else condition is meant to be here to define the functions
if the Red Hat include file isn't there.

Fixes: commit 467011bc / MDEV-26614

RedHat -> Red Hat by Daniel Black

Continue the effort of a previous commit (PR#2114) which changed the man
pages titles from MariaDB to MySQL, to further update the man pages.

Update the man page NAME sections to use mariadb-* instead of mysql* for
MariaDB binaries that are drop-in replacements for MySQL equivalents,
indicating that the commands are actually of the MariaDB version.

Before:
    NAME
        mysql_upgrade - check tables for MariaDB upgrade
    ...

After:
    NAME
        mariadb-upgrade - check tables for MariaDB upgrade (mysql_upgrade
        is now a symlink to mariadb-upgrade)
    ...

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.

ha_innobase::build_template may initialize m_prebuilt->idx_cond
even if there is no valid pushed_idx_cond_keyno.
This potentially problematic piece of code was found while
working on MDEV-27366

A prominent remaining source of crashes on corrupted index pages
is page directory corruption.

A frequent caller of page_dir_find_owner_slot() is page_rec_get_prev().
Some of those calls can be replaced with simpler logic that is less
prone to fail.

page_dir_find_owner_slot(),
page_rec_get_prev(), page_rec_get_prev_const(),
btr_pcur_move_to_prev(), btr_pcur_move_to_prev_on_page(),
btr_cur_upd_rec_sys(),
page_delete_rec_list_end(),
rtr_page_copy_rec_list_end_no_locks(),
rtr_page_copy_rec_list_start_no_locks(): Return an error code on failure.

fil_space_t::io(), buf_page_get_low(): Use DB_CORRUPTION for
out-of-bounds page reads.

PageBulk::getSplitRec(), PageBulk::copyOut(): Simplify the code.

btr_validate_level(): Prevent some more CHECK TABLE crashes on
corrupted pages.

btr_block_get(), btr_pcur_move_to_next_page(): Implement some checks that
were previously only part of IndexPurge::next().

IndexPurge::next(): Use btr_pcur_move_to_next_page().

MariaDB never supported this form of preemption via high-priority
transactions. This error code shold not have been added in the
first place, in commit 2e814d47.

and failing spider partition test.

With some small datatype changes to the Linux/Solaris my_gethwaddr implementation
the hardware address of AIX can be returned. This is an important aspect
in Spider (and UUID).

Spider test change reviewed by Nayuta Yanagisawa.

my_gethwaddr review by Monty in #2081

fil_page_type_validate(): Remove. This debug check was mostly redundant
and added little value to the code paths that deal with page_compressed
or encrypted pages.

fil_get_page_type_name(): Remove; unused function.

fil_space_decrypt(): Return an error if the page is not
supposed to be encrypted. It is possible that an unencrypted page
contains a nonzero key_version field even though it is not supposed
to be encrypted. Previously we would crash in such a situation.

buf_page_decrypt_after_read(): Simplify the code. Remove some
unnecessary error message about temporary tablespace corruption.
This is where we would usually invoke fil_space_decrypt().

Even after commit 0b47c126
there are a few ib::fatal() calls in non-debug code
that can be replaced easily.

btr_page_reorganize_low(): On size invariant violation, return
an error code instead of crashing.

btr_check_blob_fil_page_type(): On an invalid page type, report
an error but do not crash.

btr_copy_blob_prefix(): Truncate the output if a page type is invalid.

dict_load_foreign_cols(): On an error, return DB_CORRUPTION instead
of crashing.

fil_space_decrypt_full_crc32(), fil_space_decrypt_for_non_full_checksum():
On error, return DB_DECRYPTION_FAILED instead of crashing.

fil_set_max_space_id_if_bigger(): Replace ib::fatal() with an
equivalent ut_a() assertion.

Spider is now a part of the server. So, it doesn't make sense for Spider
to have its own version number apart from the server's one.

This patch fixes the following issues in Aria error reporting in case
of read errors & crashed tables:
- Added the table name to the most error messages, including in case of
  read errors or when encrypting/decrypting a table. The format for
  error messages was changed sligtly to accomodate logging of errors
  from lower level routines.
- If we got an read error from storage (hard disk, ssd, S3 etc) we only
  reported 'table is crashed'. Now the error number from the storage
  is reported.
- Added checking of read failure from records_in_range()
- Calls to ma_set_fatal_error() did not inform the SQL level of
  errors (to not spam the user with multiple error messages).
  Now the first error message and any fatal error messages are reported
  to the user.

- Print correct server version for header
- Updated version number
- One can now specify file name last (without -f)

Main-author: Sergei Petrunia

Part of:
MDEV-28073 Slow query performance in MariaDB when using many tables

s->key_dependent has a list of tables that are compared with key fields
in the current table.  However it does not take into account if a key
field could be resolved by another table.
This is because MariaDB expands 'join_tab->keyuse' to include all generated
comparisons.
For example:
SELECT * from t1,t2,t3 where t1.key=t2.key and t2.key=t3.key
In this case keyuse for t1 includes t2.key and t3.key and key_dependent
contains 't2.map | t3.map'
If we in best_extension_by_limited_search() consider t2,t1 then t1's
key is fully defined, but we cannot do any prune of plans as
s->key_dependent indicates that t3 is still needed.

Fixed by calculating in best_access_patch the current key_dependent map
of tables that is needed to satisfy all keys. This allows us to prune
more bad plans earlier as soon as all keys can be used.

We also set key_dependent to 0 if we found an EQ_REF key, as this an
optimal key for the table and there is no reason to check more keys.

best_extension_by_limited_search() assumes that tables should be sorted
according to size to be able to quickly disregard bad plans. However the
current usage of swap_variables() will change the table order to a not
sorted one for the next recursive call. This breaks the assumtion and
causes performance issues when using many tables (we have to examine
many more plans).

This patch fixes this by ensuring that the original table order is kept
for the not yet used tables when best_extension_by_limited_search() is
called.

This was done by always calling swap_variables() for each table and
restoring the original table order at exit.

Some test changed:
- In a majority of the test the change was that two "identical tables"
  where swapped and the optimzer is now using the first/smaller table
- In few test the table order was changed. The new plan looks identical
  or slighly better than the original.

(Try 2)

The code that updates semi-join optimization state for a join order prefix
had several bugs. The visible effect was bad optimization for FirstMatch or
LooseScan strategies: they either weren't considered when they should have
been, or considered when they shouldn't have been.

In order to hit the bug, the optimizer needs to consider several different
join prefixes in a certain order. Queries with "obvious" query plans which
prune all join orders except one are not affected.

Internally, the bugs in updates of semi-join state were:
1. restore_prev_sj_state() assumed that
  "we assume remaining_tables doesnt contain @tab"
  which wasn't true.
2. Another bug in this function: it did remove bits from
   join->cur_sj_inner_tables but never added them.
3. greedy_search() adds tables into the join prefix but neglects to update
   the semi-join optimization state. (It does update nested outer join
   state, see this call:
     check_interleaving_with_nj(best_table)
   but there's no matching call to update the semi-join state.
   (This wasn't visible because most of the state is in the POSITION
    structure which is updated. But there is also state in JOIN, too)

The patch:
- Fixes all of the above
- Adds JOIN::dbug_verify_sj_inner_tables() which is used to verify the
  state is correct at every step.
- Renames advance_sj_state() to optimize_semi_joins().
  = Introduces update_sj_state() which ideally should have been called
    "advance_sj_state" but I didn't reuse the name to not create confusion.

Main fix was replacing read_time+= with read_time

I also did updated the 'identical' code in optimize_straight_join) and
best_extension_by_limited_search() to make them eaiser to compare.

Reviewer: Sergei Petrunia <sergey@mariadb.com>

otherwise following tests that crash the server will see them
corrupted

(Try 2) (Cherry-pick back into 10.3)

The code that updates semi-join optimization state for a join order prefix
had several bugs. The visible effect was bad optimization for FirstMatch or
LooseScan strategies: they either weren't considered when they should have
been, or considered when they shouldn't have been.

In order to hit the bug, the optimizer needs to consider several different
join prefixes in a certain order. Queries with "obvious" query plans which
prune all join orders except one are not affected.

Internally, the bugs in updates of semi-join state were:
1. restore_prev_sj_state() assumed that
  "we assume remaining_tables doesnt contain @tab"
  which wasn't true.
2. Another bug in this function: it did remove bits from
   join->cur_sj_inner_tables but never added them.
3. greedy_search() adds tables into the join prefix but neglects to update
   the semi-join optimization state. (It does update nested outer join
   state, see this call:
     check_interleaving_with_nj(best_table)
   but there's no matching call to update the semi-join state.
   (This wasn't visible because most of the state is in the POSITION
    structure which is updated. But there is also state in JOIN, too)

The patch:
- Fixes all of the above
- Adds JOIN::dbug_verify_sj_inner_tables() which is used to verify the
  state is correct at every step.
- Renames advance_sj_state() to optimize_semi_joins().
  = Introduces update_sj_state() which ideally should have been called
    "advance_sj_state" but I didn't reuse the name to not create confusion.

In any files that were created in the
innodb_checksum_algorithm=full_crc32 format
(commit c0f47a4a)
any unused data fields will have been zero-initialized
(commit 3926673c).

A subsequently executed test could fail due to warning messages
that some mysql.help_% tables were not closed properly.