set the proper charset for a temporary result

This is a 10.5 backport of 10.6
commit d4265fbd.

page_cleaner_flush_pages_recommendation(): If dirty_pct is
between innodb_max_dirty_pages_pct_lwm
and innodb_max_dirty_pages_pct,
scale the effort relative to how close we are to
innodb_max_dirty_pages_pct.

The previous formula was missing a multiplication by 100.

This is a 10.5 backport from 10.6
commit 9593cccf.

Adaptive flushing is enabled by setting innodb_max_dirty_pages_pct_lwm>0
(not default) and innodb_adaptive_flushing=ON (default).
There is also the parameter innodb_adaptive_flushing_lwm
(default: 10 per cent of the log capacity). It should enable some
adaptive flushing even when innodb_max_dirty_pages_pct_lwm=0.
That is not being changed here.

This idea was first presented by Inaam Rana several years ago,
and I discussed it with Jean-François Gagné at FOSDEM 2023.

buf_flush_page_cleaner(): When we are not near the log capacity limit
(neither buf_flush_async_lsn nor buf_flush_sync_lsn are set),
also try to move clean blocks from the buf_pool.LRU list to buf_pool.free
or initiate writes (but not the eviction) of dirty blocks, until
the remaining I/O capacity has been consumed.

buf_flush_LRU_list_batch(): Add the parameter bool evict, to specify
whether dirty least recently used pages (from buf_pool.LRU) should
be evicted immediately after they have been written out. Callers outside
buf_flush_page_cleaner() will pass evict=true, to retain the existing
behaviour.

buf_do_LRU_batch(): Add the parameter bool evict.
Return counts of evicted and flushed pages.

buf_flush_LRU(): Add the parameter bool evict.
Assume that the caller holds buf_pool.mutex and
will invoke buf_dblwr.flush_buffered_writes() afterwards.

buf_flush_list_holding_mutex(): A low-level variant of buf_flush_list()
whose caller must hold buf_pool.mutex and invoke
buf_dblwr.flush_buffered_writes() afterwards.

buf_flush_wait_batch_end_acquiring_mutex(): Remove. It is enough to have
buf_flush_wait_batch_end().

page_cleaner_flush_pages_recommendation(): Avoid some floating-point
arithmetics.

buf_flush_page(), buf_flush_check_neighbor(), buf_flush_check_neighbors(),
buf_flush_try_neighbors(): Rename the parameter "bool lru" to "bool evict".

buf_free_from_unzip_LRU_list_batch(): Remove the parameter.
Only actual page writes will contribute towards the limit.

buf_LRU_free_page(): Evict freed pages of temporary tables.

buf_pool.done_free: Broadcast whenever a block is freed
(and buf_pool.try_LRU_scan is set).

buf_pool_t::io_buf_t::reserve(): Retry indefinitely.
During the test encryption.innochecksum we easily run out of
these buffers for PAGE_COMPRESSED or ENCRYPTED pages.

Tested by Matthias Leich and Axel Schwenke

trx_purge_truncate_history(): Do not prematurely mark dirty pages
as clean. This will be done in mtr_t::commit_shrink() as part of
Shrink::operator()(mtr_memo_slot_t*). Also, register each dirty page
only once in the mini-transaction.

fsp_page_create(): Adjust and simplify the page creation during
undo tablespace truncation. We can directly reuse pages that are
already in buf_pool.page_hash.

This fixes a regression that was caused by
commit f5794e1d (MDEV-26445).

Tested by: Matthias Leich
Reviewed by: Thirunarayanan Balathandayuthapani

Commit Removed Ubuntu Bionic from
debian/autobake-debs.sh as it's not used
anymore to build official MariaDB images

REMINDER TO MERGER: This commit should not
be merged up to 10.6 or forward

innodb_preshutdown(): Only wait for active transactions to be terminated
if InnoDB was started and innodb_force_recovery=3 or larger does not
prevent a rollback.

This fixes the following:

./mtr --parallel=auto --mysqld=--innodb-fast-shutdown=0 \
innodb.log_file_size innodb.innodb_force_recovery \
innodb.read_only_recovery innodb.read_only_recover_committed \
mariabackup.apply-log-only-incr

MDEV-32737 innodb.log_file_name fails on Assertion `after_apply || !(blocks).end in recv_sys_t::clear

recv_group_scan_log_recs(): Set the debug flag recv_sys.after_apply
after actually completing the log scan.

In the test, suppress some errors that may be reported when
the crash recovery of RENAME TABLE t1 TO t2 is preceded by
copying t2.ibd to t1.ibd.

recv_dblwr_t::find_page(): Tablespace flags validity should be
checked only for page 0.

The test case can get extra rows in its output from table
performance_schema.table_handles, left there by an earlier test case (for
example main.long_unique_delayed). So force a server restart at the
beginning of the test.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Partial revert of this commit:

  commit 6b685ea7
  Author: Sergei Golubchik <serg@mariadb.org>
  Date:   Wed Sep 28 18:55:15 2022 +0200

Don't hold LOCK_thd_data over run_commit_ordered(). Holding the mutex
is unnecessary and will deadlock if any code in a commit_ordered
handlerton call tries to take the mutex to change THD local data.

Instead, set the current_thd for the duration of the call to keep
asserts happy around LOCK_thd_data.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

- Rename files as requested by Vicentiu:
```
 mysql_json_mysql_upgrade.test                    -> mysql_upgrade_mysql_json.test
 mysql_json_mysql_upgrade_with_plugin_loaded.test -> mysql_upgrade_mysql_json_with_plugin_loaded.test
 mysql_json_mysql_upgrade_system_tables.test      -> mysql_upgrade_mysql_json_system_tables.test
```
- Related to PR #2790
- Reviewer: <daniel@mariadb.org>, <vicentiu@mariadb.org>

- Prevent opening of any user tables in case `upgrade-system-table`
option is used.
- Still there may be uninstalled data types in `mysql` system table so
allow it to perform.
- Closes PR #2790
- Reviewer: <daniel@mariadb.org>, <vicentiu@mariadb.org>

- Regression introduced in 957cb7b7
- Patch 4abb8216 change `mysql.user` to `mysql.global_priv` for
`add_anonymous.inc`, update `delete_anonymous.inc`.
- Added test case with `--skip-name-resolve`
- Add test case with anonymous user
  - Disable this test for windows, assignes current user to anonymous
  user.

Reviewed by: <serg@mariadb.com>

For some reason, in embedded server, a command

let $a=`$query`

ignores local context. Make a workaround: use SET STATEMENT to set
debug_dbug in the same statement.

Make ANALYZE FORMAT=JSON print block-nl-join.r_unpack_ops when
analyze_print_r_unpack_ops debug flag is set.

Then, add a testcase.

Also fix it to work with hashed join (MDEV-30830).

Reviewed by: Monty <monty@mariadb.org>

This patch fixes a performance regression introduced in the patch for the
bug MDEV-21104. The performance regression could affect queries for which
join buffer was used for an outer join such that its on expression from
which a conjunctive condition depended only on outer tables can be
extracted. If the number of records in the join buffer for which this
condition was false greatly exceeded the number of other records the
slowdown could be significant.

If there is a conjunctive condition extracted from the ON expression
depending only on outer tables this condition is evaluated when interesting
fields of each survived record of outer tables are put into the join buffer.
Each such set of fields for any join operation is supplied with a match
flag field used to generate null complemented rows. If the result of the
evaluation of the condition is false the flag is set to MATCH_IMPOSSIBLE.
When looking in the join buffer for records matching a record of the
right operand of the outer join operation the records with such flags
are not needed to be unpacked into record buffers for evaluation of on
expressions.

The patch for MDEV-21104 fixing some problem of wrong results when
'not exists' optimization by mistake broke the code that allowed to
ignore records with the match flag set to MATCH_IMPOSSIBLE when looking
for matching records. As a result such records were unpacked for each
record of the right operand of the outer join operation. This caused
significant execution penalty in some cases.

One of the test cases added in the patch can be used only for demonstration
of the restored performance for the reported query. The second test case is
needed to demonstrate the validity of the fix.

When mysql/mysql-server@0c954c2289a75d90d1088356b1092437ebf45a1d
added a plugin interface for FULLTEXT INDEX tokenization to MySQL 5.7,
fts_tokenize_ctx::processed_len got a second meaning, which is only
partly implemented in row_merge_fts_doc_tokenize().

This inconsistency could cause a crash when using FULLTEXT...WITH PARSER.
A test case that would crash MySQL 8.0 when using an n-gram parser and
single-character words would fail to crash in MySQL 5.7, because the
buf_full condition in row_merge_fts_doc_tokenize() was not met.

This change is inspired by
mysql/mysql-server@38e9a0779aeea2d197c727e306a910c56b26a47c
that appeared in MySQL 5.7.44.

A recently added by MDEV-32593 assert conditions are corrected.

After two concurrent FTWRL/UNLOCK TABLES, the node stays in paused state
and the following CREATE TABLE fails with

  ER_UNKNOWN_COM_ERROR (1047): Aborting TOI: Replication paused on
  node for FTWRL/BACKUP STAGE.

The cause is the use of global `wsrep_locked_seqno` to determine
if the node should be resumed on UNLOCK TABLES. In some executions
the `wsrep_locked_seqno` is cleared by the first UNLOCK TABLES
after the second FTWRL gets past `make_global_read_lock_block_commit()`.

As a fix, use `thd->wsrep_desynced_backup_stage` to determine
if the thread should resume the node on UNLOCK TABLES.

Add MTR test galera.galera_ftwrl_concurrent to reproduce the
race. The test contains also cases for BACKUP STAGE which
uses similar mechanism for desyncing and pausing the node.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

remove the hack where NO_DEFAULT_VALUE_FLAG was temporarily removed
from a field to initialize DEFAULT() functions in CHECK constraints
while disabling self-reference field checks.

Instead, initialize DEFAULT() functions in CHECK explicitly,
don't call check_field_expression_processor() for CHECK at all.

Semisync ack (master side) receiver thread is made to report
details of faced errors.
In case of 'magic byte' error, a hexdump of the received packet
is always (level) NOTEd into the error log.
In other cases an exact server level error is print out
as a warning (as it may not be critical) under log_warnings > 2.

An MTR test added for the magic byte error. For others existing mtr
tests cover that, provided log_warnings > 2 is set.

test_if_skip_sort_order() should catch the join types JT_EQ_REF,
JT_CONST and JT_SYSTEM and skip sort order for these.

Such join types imply retrieving of a single row of data, and sorting
of a single row can always be skipped.

Blaming hardware and poor libraries seems on the rare end of the
scale of things that go wrong. Accept the blame, apologize, and
kindly request a bug report.

Also url change on stack traces is changed to include mariadbd.

Thanks Vlad for also raising that blaming was wrong.

This commit checks the validity of value change of wsrep_sst_method variable.
The validity check is same as happens in donor node when incoming SST request
is parsed.

The commit has also a mtr test: wsrep.wsrep_variables_sst_method which verifies
that wsrep_sst_method can be succesfully changed to acceptable values and that
the SET command results in error if invalid value was entered.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Also fixes: MDEV-30050 Inconsistent results of DISTINCT with NOPAD

Problem:

Key segments for CHAR columns where compared using strnncollsp()
for engines MyISAM and Aria.

This did not work correct in case if the engine applyied trailing
space compression.

Fix:

Replacing ha_compare_text() calls to new functions:

- ha_compare_char_varying()
- ha_compare_char_fixed()
- ha_compare_word()
- ha_compare_word_prefix()
- ha_compare_word_or_prefix()

The code branch corresponding to comparison of CHAR column keys
(HA_KEYTYPE_TEXT segment type) now uses ha_compare_char_fixed()
which calls strnncollsp_nchars().

This patch does not change the behavior for the rest of the code:
- comparison of VARCHAR/TEXT column keys
  (HA_KEYTYPE_VARTEXT1, HA_KEYTYPE_VARTEXT2 segments types)
- comparison in the fulltext code

Changing the code handling sql_mode-dependent function DECODE():

- removing parser tokens DECODE_MARIADB_SYM and DECODE_ORACLE_SYM
- removing the DECODE() related code from sql_yacc.yy/sql_yacc_ora.yy
- adding handling of DECODE() with help of a new Create_func_func_decode