otherwise following tests that crash the server will see them
corrupted

(Try 2) (Cherry-pick back into 10.3)

The code that updates semi-join optimization state for a join order prefix
had several bugs. The visible effect was bad optimization for FirstMatch or
LooseScan strategies: they either weren't considered when they should have
been, or considered when they shouldn't have been.

In order to hit the bug, the optimizer needs to consider several different
join prefixes in a certain order. Queries with "obvious" query plans which
prune all join orders except one are not affected.

Internally, the bugs in updates of semi-join state were:
1. restore_prev_sj_state() assumed that
  "we assume remaining_tables doesnt contain @tab"
  which wasn't true.
2. Another bug in this function: it did remove bits from
   join->cur_sj_inner_tables but never added them.
3. greedy_search() adds tables into the join prefix but neglects to update
   the semi-join optimization state. (It does update nested outer join
   state, see this call:
     check_interleaving_with_nj(best_table)
   but there's no matching call to update the semi-join state.
   (This wasn't visible because most of the state is in the POSITION
    structure which is updated. But there is also state in JOIN, too)

The patch:
- Fixes all of the above
- Adds JOIN::dbug_verify_sj_inner_tables() which is used to verify the
  state is correct at every step.
- Renames advance_sj_state() to optimize_semi_joins().
  = Introduces update_sj_state() which ideally should have been called
    "advance_sj_state" but I didn't reuse the name to not create confusion.

DBUG_PUSH_EMPTY is used by thr_mutex.cc.
If there are 4G of DBUG_PUSH_EMPTY calls, then DBUG_POP_EMPTY will
cause a crash when DBUGCloseFile() will try to free an object that
was never allocated.

The assert happens in 10.6 with the following command:

./mtr --no-reorder --verbose-restart main.update_ignore_216 main.upgrade_MDEV-19650 main.upgrade_MDEV-23102-1 main.upgrade_MDEV-23102-2 main.upgrade_geometrycolumn_procedure_definer main.upgrade_mdev_24363 main.varbinary sys_vars.aria_log_file_size_basic

Reviewer: Oleksandr Byelkin <sanja@mariadb.com>

comp_thread_ctxt_t: Remove ctrl_mutex, ctrl_cond, started. We do not
actually need them for anything.

destroy_worker_thread(): Split from destroy_worker_threads().

create_worker_threads(): We already initialize
thd->data_avail=FALSE and thd->cancelled=FALSE before
invoking pthread_create(). If any thread creation fails,
clean up by destroy_worker_thread().

compress_worker_thread_func(): Assume that thd->started and
thd->data_avail are already initialized.

Reviewed by: Vladislav Vaintroub

ER_CHECK_NO_SUCH_TABLE was raised because a view does not have
the corresponding TABLE instance connected to TABLE_LIST and the
server interprets the absence as the absence of the table itself.

To fix the problem, we add a check to ensure that the target table
to be swapped with a partition is not a view.

Reviewed by: Nayuta Yanagisawa

If the connecting user doesn't have alter table privilege this isn't
allowed.

This patch removes enable / disable key commands that should never have been here

Closes #2002

For compatibility reasons, add the option to the MariaDB client without
any functional changes besides simply accepting the option and emitting
a warning that it is obsolete.

In MySQL this security related option is compulsory in certain use
cases. When users switch to MariaDB, this client command that used to
work starts failing without a sensible error message. In worst case
users resort to re-installing the mysql client from MySQL.

In MariaDB the option is obsolete and should simply be ignored. Users
however don't have any opportunity to learn that unless the client
program tells them so.

Before:

    mysql --enable-cleartext-plugin ...
    mysql: unknown option '--enable-cleartext-plugin'

    (program terminates)

After:

    mysql --enable-cleartext-plugin ...
    WARNING: option '--enable-cleartext-plugin' is obsolete.

    (program executes)

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.

- InnoDB fails to create a fts cache while loading the innodb fts
table which is stored in system tablespace. InnoDB should create
the fts cache while loading FTS_DOC_ID column from system column.

Cause: a copy of the joined TABLE_LIST is created during multi_update::prepare
and TABLE::pos_in_table_list of the tables are set to point to the new
TABLE_LIST object. This prevents some optimization steps to perform correctly.
Solution: do not update pos_in_table_list during multi_update::prepare

MDEV-14642 Assertion 'table->s->db_create_options == part_table->s->db_create_options' failed in compare_table_with_partition

When trying to execute ALTER TABLE EXCHANGE PARTITION with different
definitions, assertion

    table->s->db_create_options == part_table->s->db_create_options

failed in compare_table_with_partition().

However, this execution should not be allowed since executing
'exchange partition' requires the identical structure of the two tables.

To fix the problem, I deleted the assertion code and added code that
returns an error that indicates tables have different definitions.

Reviewed By: Nayuta Yanagisawa

commit 84984b79 is null-merged

This commit sends a flag indicating the presence of the "--bypass"
option from the donor node to the joiner nodes during rsync IST,
because without such a flag it is impossible to distinguish IST
from the SST on the joiner nodes (in IST/SST scripts, because the
"--bypass" option is still not passed to scripts from server code).
Specifically, this fixes an issue with binary logs disappearing
after IST (via rsync). There are also changes to diagnostic messages
here that will make it easier to diagnose script-related problems
in the future when debugging and when checking the logs. This commit
also adds more robust signal handlers - to handle exceptions during
script execution. These handlers won't mask some crashes and it
also unifies exit codes between different scripts. These changes
have already been helpful to debugging "bypass" flag handling.

This commit fixes an issue with IST handling in
version 10.9 which is a regression after MDEV-26971
and related to trying to get a non-existent "total"
tag on the IST branch (this tag is only defined in
SST mode).

We cannot permanently change bits in read_partitions in the middle of
processing because ha_rnd_init()/ha_rnd_end() depends on that.

prefer if/skip over require
(works better with debugging, not affected by query log)

followup for c9b5a053

This reverts commit 17e0f522.

The counter srv_stats.key_rotation_list_length is never updated, and
therefore Innodb_encryption_key_rotation_list_length will always be 0.

The view INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION comes close
to reporting this information.

The counters were added in commit 5e55d1ce
and any code to update them was
inadvertently removed in commit 2e814d47
when applying InnoDB changes from MySQL 5.7.

Let us remove these counters that never reported anything useful. If such
statistics are really needed in a special case, they can be obtained by
instrumenting the code by some means, such as eBPF or a source code patch.

- In best_extension_by_limited_search(), do not check for
  "(remaining_tables & real_table_bit)", it is guaranteed to be true.
  Make it an assert.
- In (!idx || check_interleaving_with_nj())", remove the !idx part.
  This check made sense only in the original version of this function.
- "micro optimization" in check_interleaving_with_nj().

The condition of the if statements are always true.

If a slave received a fake GLLE event after a GTID event
it would terminate the group. This adds a test for the
previous commit which fixed this issue (939672a).

Review by Andrei Elkin <andrei.elkin@mariadb.com>

GTID_LIST_EVENT or INCIDENT_EVENT.

It's legal to have either of the two inside a group. E.g
  Gtid_event, Gtid_log_list_event, Query_1, ... Xid_log_event
is permitted.
However, the slave IO thread treated both
as the terminal even when the group represents a DDL query.
That causes a premature Gtid state update so the slave IO would think
the whole group has been collected while in fact Query_1 etc are yet to process.

Fixed with correcting a condition to compute the terminal event
of the group.
Tested with rpl_mysqlbinlog_slave_consistency (of 10.9) and
rpl_gtid_errorlog.test.

Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Make sure that nodes have correct auto_increment_offset when
they start and when control is turned on.

Fixed failing main.default on Windows
(to trigger an assert the test needed a debug build without
safemalloc, as 0xa5 happened to have the important bit set "correctly")

Test compat/oracle.sp-package-mysqldump needed re-record.