This will avoid issues like MDEV-32486

IDs used in
- spider_alloc_calc_mem_init()
- spider_string::init_calc_mem()
- spider_malloc()
- spider_bulk_alloc_mem()
- spider_bulk_malloc()

This fixes MDEV-30014, MDEV-29456, MDEV-29667, and MDEV-30049.

The server may ask storage engines to unlock when the original sql
command is not UNLOCK. This patch makes sure that spider honours these
requests, so that the server has the correct idea which tables are
locked and which are not.

MDEV-29456, MDEV-29667, MDEV-30049: a later LOCK statement would, as
the first step, unlock locked tables and clear the OPTION_TABLE_LOCK
bit in thd->variables.option_bits, as well as locked_tables_list,
indicating no tables are locked. If Spider does not unlock because the
sql command is not UNLOCK, and if after this the LOCK statement fails
to lock any tables, these indications that no tables are locked
remains, so a later UNLOCK TABLES; statement would not try to unlock
any table. Causing later statements requiring mdl locks to hang on
waiting until lock_wait_timeout (default 1h) has passed.

MDEV-30014: when a LOCK statement tries to lock more than one tables,
say t2 and t3 as in mdev_30014.test, and t2 succeeds but t3 fails, the
sql layer would try to undo by unlocking t2, and again, if Spider does
not honour this request, the sql layer would assume t2 has been
unlocked, but later actions on t2 or t2's remote table could hang on
waiting for the mdl.

Spider populates its lock lists (a hash) in store_lock(), and normally
clears them in the actual lock_tables(). However, if lock_tables()
fails, there's no reset_lock() method for storage engine handlers,
which can cause bad things to happen. For example, if one of the table
involved is dropped and recreated, or simply TRUNCATEd, when executing
LOCK TABLES again, the lock lists would be queried again in
store_lock(), which could cause access to freed space associated with
the dropped table.

Spider GBH's query rewrite of table joins is overly complex and
error-prone. We replace it with something closer to what
dbug_print() (more specifically, print_join()) does, but catered to
spider.

More specifically, we replace the body of
spider_db_mbase_util::append_from_and_tables() with a call to
spider_db_mbase_util::append_join(), and remove downstream append_X
functions.

We make it handle const tables by rewriting them as (select 1). This
fixes the main issue in MDEV-26247.

We also ban semijoin from spider gbh, which fixes MDEV-31645 and
MDEV-30392, as semi-join is an "internal" join, and "semi join" does
not parse, and it is different from "join" in that it deduplicates the
right hand side

Not all queries passed to a group by handler are valid (MDEV-32273),
for example, a join on expr may refer outer fields not in the current
context. We detect this during the handler creation when walking the
join. See also gbh_outer_fields_in_join.test.

It also skips eliminated tables, which fixes MDEV-26193.

Spider gbh query rewrite should get table for fields in a simple way.
Add a method spider_fields::find_table that searches its table holders
to find table for a given field. This way we will be able to get rid
of the first pass during the gbh creation where field_chains and
field_holders are created.

We also check that the field belongs to a spider table while walking
through the query, so we could remove
all_query_fields_are_query_table_members(). However, this requires an
earlier creation of the table_holder so that tables are added before
checking. We do that, and in doing so, also decouple table_holder and
spider_fields

Remove unused methods and fields. Add comments.

Two methods from spider_fields. There are probably more of these
conn_holder related methods that can be removed

reappend_tables_part()
reappend_tables()

MDEV-32811 Potentially broken crash recovery if a mini-transaction frees a page, not modifying previously clean pages

- The 11.2 test innodb.sys_truncate_debug fails while executing insert statement.
Reason for the failure is that same mini-transaction does freeing, allocating
and freeing the same page. Page initialization clears the FIL_PAGE_LSN
on the page, fails to set the FIL_PAGE_LSN after freeing the same page.
This issue is caused by commit f46efb44

mtr_t::commit(): Should set the FIL_PAGE_LSN even though page is freed

recv_ring::copy_if_needed(): If len==0, do not decrypt anything,
to avoid unnecessary work and a possible crash.

This is similar to the condition recv_buf::copy_if_needed() where
a similar condition was present since the beginning
(commit 685d958e).

Tested by: Matthias Leich

Checking for kill with thd_kill_level() or check_killed() runs apc
requests, which takes the LOCK_thd_kill mutex. But this is dangerous,
as checking for kill needs to be called while holding many different
mutexes, and can lead to cyclic mutex dependency and deadlock.

But running apc is only "best effort", so skip running the apc if the
LOCK_thd_kill is not available. The apc will then be run on next check
of kill signal.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

recv_dblwr_t::find_page(): Tablespace flags validity should be
checked only for page 0.

Correctly supress error issuing when saving value in field for comporison

Fix the issue introduced in ec2574fd, fix for MDEV-31983:

get_quick_record_count() must set quick_count=0 when it got
IMPOSSIBLE_RANGE from test_quick_select.

Failure to do so will cause an assertion in 11.0, when the number of
quick select rows (0) is checked to be lower than the number of
found_records (which is capped up to 1).

specify algorithm/lock explicitly, don't depend on server settings

recv_dblwr_t::find_page(): Tablespace flags validity should be
checked only for page 0.

fil_space_t::drop(): If the caller is not interested in a
detached handle, close it immediately.

fil_delete_tablespace(): Invoke fil_space_free_low() directly.
This fixes up commit 39e3ca8b

Added missing initializer

According to the standart draft UUIDv6 and UUIDv7 values
must be compared as opaque raw bytes.

Let's only compare with byte-swapping if both values need
byte swapping.

* modify the test to use different and not monotonous timestamps
* rename methods to be unambiguous (for IDE challenged devs)
* move byte swap checks into helpers

Fixup for MDEV-31983, incorrect test for checking ability to use quick select.

Approved by Sergei Petrunia

For some reason, in embedded server, a command

let $a=`$query`

ignores local context. Make a workaround: use SET STATEMENT to set
debug_dbug in the same statement.

ref->null_rejecting is a key_part_map. we need to check
the bit corresponding to the particular store_key.
Note that there are no store_key objects for const ref parts.

Make ANALYZE FORMAT=JSON print block-nl-join.r_unpack_ops when
analyze_print_r_unpack_ops debug flag is set.

Then, add a testcase.

Also fix it to work with hashed join (MDEV-30830).

Reviewed by: Monty <monty@mariadb.org>

This patch fixes a performance regression introduced in the patch for the
bug MDEV-21104. The performance regression could affect queries for which
join buffer was used for an outer join such that its on expression from
which a conjunctive condition depended only on outer tables can be
extracted. If the number of records in the join buffer for which this
condition was false greatly exceeded the number of other records the
slowdown could be significant.

If there is a conjunctive condition extracted from the ON expression
depending only on outer tables this condition is evaluated when interesting
fields of each survived record of outer tables are put into the join buffer.
Each such set of fields for any join operation is supplied with a match
flag field used to generate null complemented rows. If the result of the
evaluation of the condition is false the flag is set to MATCH_IMPOSSIBLE.
When looking in the join buffer for records matching a record of the
right operand of the outer join operation the records with such flags
are not needed to be unpacked into record buffers for evaluation of on
expressions.

The patch for MDEV-21104 fixing some problem of wrong results when
'not exists' optimization by mistake broke the code that allowed to
ignore records with the match flag set to MATCH_IMPOSSIBLE when looking
for matching records. As a result such records were unpacked for each
record of the right operand of the outer join operation. This caused
significant execution penalty in some cases.

One of the test cases added in the patch can be used only for demonstration
of the restored performance for the reported query. The second test case is
needed to demonstrate the validity of the fix.

When mysql/mysql-server@0c954c2289a75d90d1088356b1092437ebf45a1d
added a plugin interface for FULLTEXT INDEX tokenization to MySQL 5.7,
fts_tokenize_ctx::processed_len got a second meaning, which is only
partly implemented in row_merge_fts_doc_tokenize().

This inconsistency could cause a crash when using FULLTEXT...WITH PARSER.
A test case that would crash MySQL 8.0 when using an n-gram parser and
single-character words would fail to crash in MySQL 5.7, because the
buf_full condition in row_merge_fts_doc_tokenize() was not met.

This change is inspired by
mysql/mysql-server@38e9a0779aeea2d197c727e306a910c56b26a47c
that appeared in MySQL 5.7.44.

A recently added by MDEV-32593 assert conditions are corrected.

After two concurrent FTWRL/UNLOCK TABLES, the node stays in paused state
and the following CREATE TABLE fails with

  ER_UNKNOWN_COM_ERROR (1047): Aborting TOI: Replication paused on
  node for FTWRL/BACKUP STAGE.

The cause is the use of global `wsrep_locked_seqno` to determine
if the node should be resumed on UNLOCK TABLES. In some executions
the `wsrep_locked_seqno` is cleared by the first UNLOCK TABLES
after the second FTWRL gets past `make_global_read_lock_block_commit()`.

As a fix, use `thd->wsrep_desynced_backup_stage` to determine
if the thread should resume the node on UNLOCK TABLES.

Add MTR test galera.galera_ftwrl_concurrent to reproduce the
race. The test contains also cases for BACKUP STAGE which
uses similar mechanism for desyncing and pausing the node.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

remove the hack where NO_DEFAULT_VALUE_FLAG was temporarily removed
from a field to initialize DEFAULT() functions in CHECK constraints
while disabling self-reference field checks.

Instead, initialize DEFAULT() functions in CHECK explicitly,
don't call check_field_expression_processor() for CHECK at all.