buf_page_t::set_os_unused(): Remove the system call that had been added in
commit 16c97187 and revised in
commit c1fd082e for Microsoft Windows.

buf_pool_t::garbage_collect(): A new function to collect any garbage
from the InnoDB buffer pool that can be removed without writing any
log or data files. This will also invoke madvise() for all of buf_pool.free.

To trigger this the following MDEV is implemented:
MDEV-24670 avoid OOM by linux kernel co-operative memory management

To avoid frequent triggers that caused the MDEV-31953 regression, while
still preserving the 10.11 functionality of non-greedy kernel memory
usage, memory triggers are used.

On the triggering of memory pressure, if supported in the Linux kernel,
trigger the garbage collection of the innodb buffer pool.

The hard coded triggers occur where there is:
* some memory pressure in 5 of the last 10 seconds
* a full stall on memory pressure for 10ms in the last 2 seconds

The kernel will trigger only one in each of these time windows. To avoid
mariadb being in a constant state of memory garbage collection, this has
been limited to once per minute.

For a small set of kernels in 2023 (6.5, 6.6), there was a limit requiring
CAP_SYS_RESOURCE that was lifted[1] to support the use case of user
memory pressure. It not currently possible to set CAP_SYS_RESOURCES in
a systemd service as its setting a capability inside a usernamespace.

Running under systemd v254+ requires the default MemoryPressureWatch=auto
(or alternately "on").

Functionality was tested in a 6.4 kernel Fedora successfully under a
systemd service.

Running in a container requires that (unmask=)/sys/fs/cgroup be writable
by the mariadbd process.

To aid testing, the buf_pool_resize was a convient trigger point on
which to trigger garbage collection.

ref [1]: https://lore.kernel.org/all/CAMw=ZnQ56cm4Txgy5EhGYvR+Jt4s-KVgoA9_65HKWVMOXp7a9A@mail.gmail.com/T/#m3bd2a73c5ee49965cb73a830b1ccaa37ccf4e427

Co-Author: Daniel Black (on memory pressure trigger)

Reviewed by: Marko Mäkelä, Vladislav Vaintroub, Vladislav Lesin,
   Thirunarayanan Balathandayuthapani

Tested by: Matthias Leich

MDEV-32811 Potentially broken crash recovery if a mini-transaction frees a page, not modifying previously clean pages

- The 11.2 test innodb.sys_truncate_debug fails while executing insert statement.
Reason for the failure is that same mini-transaction does freeing, allocating
and freeing the same page. Page initialization clears the FIL_PAGE_LSN
on the page, fails to set the FIL_PAGE_LSN after freeing the same page.
This issue is caused by commit f46efb44

mtr_t::commit(): Should set the FIL_PAGE_LSN even though page is freed

recv_ring::copy_if_needed(): If len==0, do not decrypt anything,
to avoid unnecessary work and a possible crash.

This is similar to the condition recv_buf::copy_if_needed() where
a similar condition was present since the beginning
(commit 685d958e).

Tested by: Matthias Leich

Checking for kill with thd_kill_level() or check_killed() runs apc
requests, which takes the LOCK_thd_kill mutex. But this is dangerous,
as checking for kill needs to be called while holding many different
mutexes, and can lead to cyclic mutex dependency and deadlock.

But running apc is only "best effort", so skip running the apc if the
LOCK_thd_kill is not available. The apc will then be run on next check
of kill signal.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

recv_dblwr_t::find_page(): Tablespace flags validity should be
checked only for page 0.

Correctly supress error issuing when saving value in field for comporison

Fix the issue introduced in ec2574fd, fix for MDEV-31983:

get_quick_record_count() must set quick_count=0 when it got
IMPOSSIBLE_RANGE from test_quick_select.

Failure to do so will cause an assertion in 11.0, when the number of
quick select rows (0) is checked to be lower than the number of
found_records (which is capped up to 1).

specify algorithm/lock explicitly, don't depend on server settings

recv_dblwr_t::find_page(): Tablespace flags validity should be
checked only for page 0.

fil_space_t::drop(): If the caller is not interested in a
detached handle, close it immediately.

fil_delete_tablespace(): Invoke fil_space_free_low() directly.
This fixes up commit 39e3ca8b

Added missing initializer

According to the standart draft UUIDv6 and UUIDv7 values
must be compared as opaque raw bytes.

Let's only compare with byte-swapping if both values need
byte swapping.

* modify the test to use different and not monotonous timestamps
* rename methods to be unambiguous (for IDE challenged devs)
* move byte swap checks into helpers

Fixup for MDEV-31983, incorrect test for checking ability to use quick select.

Approved by Sergei Petrunia

For some reason, in embedded server, a command

let $a=`$query`

ignores local context. Make a workaround: use SET STATEMENT to set
debug_dbug in the same statement.

ref->null_rejecting is a key_part_map. we need to check
the bit corresponding to the particular store_key.
Note that there are no store_key objects for const ref parts.

Make ANALYZE FORMAT=JSON print block-nl-join.r_unpack_ops when
analyze_print_r_unpack_ops debug flag is set.

Then, add a testcase.

Also fix it to work with hashed join (MDEV-30830).

Reviewed by: Monty <monty@mariadb.org>

This patch fixes a performance regression introduced in the patch for the
bug MDEV-21104. The performance regression could affect queries for which
join buffer was used for an outer join such that its on expression from
which a conjunctive condition depended only on outer tables can be
extracted. If the number of records in the join buffer for which this
condition was false greatly exceeded the number of other records the
slowdown could be significant.

If there is a conjunctive condition extracted from the ON expression
depending only on outer tables this condition is evaluated when interesting
fields of each survived record of outer tables are put into the join buffer.
Each such set of fields for any join operation is supplied with a match
flag field used to generate null complemented rows. If the result of the
evaluation of the condition is false the flag is set to MATCH_IMPOSSIBLE.
When looking in the join buffer for records matching a record of the
right operand of the outer join operation the records with such flags
are not needed to be unpacked into record buffers for evaluation of on
expressions.

The patch for MDEV-21104 fixing some problem of wrong results when
'not exists' optimization by mistake broke the code that allowed to
ignore records with the match flag set to MATCH_IMPOSSIBLE when looking
for matching records. As a result such records were unpacked for each
record of the right operand of the outer join operation. This caused
significant execution penalty in some cases.

One of the test cases added in the patch can be used only for demonstration
of the restored performance for the reported query. The second test case is
needed to demonstrate the validity of the fix.

When mysql/mysql-server@0c954c2289a75d90d1088356b1092437ebf45a1d
added a plugin interface for FULLTEXT INDEX tokenization to MySQL 5.7,
fts_tokenize_ctx::processed_len got a second meaning, which is only
partly implemented in row_merge_fts_doc_tokenize().

This inconsistency could cause a crash when using FULLTEXT...WITH PARSER.
A test case that would crash MySQL 8.0 when using an n-gram parser and
single-character words would fail to crash in MySQL 5.7, because the
buf_full condition in row_merge_fts_doc_tokenize() was not met.

This change is inspired by
mysql/mysql-server@38e9a0779aeea2d197c727e306a910c56b26a47c
that appeared in MySQL 5.7.44.

A recently added by MDEV-32593 assert conditions are corrected.

After two concurrent FTWRL/UNLOCK TABLES, the node stays in paused state
and the following CREATE TABLE fails with

  ER_UNKNOWN_COM_ERROR (1047): Aborting TOI: Replication paused on
  node for FTWRL/BACKUP STAGE.

The cause is the use of global `wsrep_locked_seqno` to determine
if the node should be resumed on UNLOCK TABLES. In some executions
the `wsrep_locked_seqno` is cleared by the first UNLOCK TABLES
after the second FTWRL gets past `make_global_read_lock_block_commit()`.

As a fix, use `thd->wsrep_desynced_backup_stage` to determine
if the thread should resume the node on UNLOCK TABLES.

Add MTR test galera.galera_ftwrl_concurrent to reproduce the
race. The test contains also cases for BACKUP STAGE which
uses similar mechanism for desyncing and pausing the node.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

remove the hack where NO_DEFAULT_VALUE_FLAG was temporarily removed
from a field to initialize DEFAULT() functions in CHECK constraints
while disabling self-reference field checks.

Instead, initialize DEFAULT() functions in CHECK explicitly,
don't call check_field_expression_processor() for CHECK at all.

Semisync ack (master side) receiver thread is made to report
details of faced errors.
In case of 'magic byte' error, a hexdump of the received packet
is always (level) NOTEd into the error log.
In other cases an exact server level error is print out
as a warning (as it may not be critical) under log_warnings > 2.

An MTR test added for the magic byte error. For others existing mtr
tests cover that, provided log_warnings > 2 is set.

buf_flush_LRU_list_batch(): Do not skip pages that are actually clean
but in buf_pool.flush_list due to the "lazy removal" optimization of
commit 22b62eda, but try to evict them.
After acquiring buf_pool.flush_list_mutex, reread oldest_modification
to ensure that the block still remains in buf_pool.flush_list.

In addition to server hangs, this bug could also cause
InnoDB: Failing assertion: list.count > 0
in invocations of UT_LIST_REMOVE(flush_list, ...).

This fixes a regression that was caused by
commit a55b951e
and possibly made more likely to hit due to
commit aa719b50.

InnoDB was violating the write-ahead-logging protocol when a file
was being deleted, like this:

1. fil_delete_tablespace() set the fil_space_t::STOPPING flag
2. The buf_flush_page_cleaner() thread discards some changed pages for
this tablespace advances the log checkpoint a little.
3. The server process is killed before fil_delete_tablespace() wrote
a FILE_DELETE record.
4. Recovery will try to apply log to pages of the tablespace, because
there was no FILE_DELETE record. This will fail, because some pages
that had been modified since the latest checkpoint had not been written
by the page cleaner.

Page writes must not be stopped before a FILE_DELETE record has been
durably written.

fil_space_t::drop(): Replaces fil_space_t::check_pending_operations().
Add the parameter detached_handle, and return a tablespace pointer
if this thread was the first one to stop I/O on the tablespace.

mtr_t::commit_file(): Remove the parameter detached_handle, and
move some handling to fil_space_t::drop().

fil_space_t: STOPPING_READS, STOPPING_WRITES: Separate flags for STOPPING.
We want to stop reads (and encryption) before stopping page writes.

fil_space_t::is_stopping_writes(), fil_space_t::get_for_write():
Special accessors for the write path.

fil_space_t::flush_low(): Ignore the STOPPING_READS flag and only
stop if STOPPING_WRITES is set, to avoid an infinite loop in
fil_flush_file_spaces(), which was occasionally repeated by
running the test encryption.create_or_replace.

Reviewed by: Vladislav Lesin
Tested by: Matthias Leich

...errors, then failing ASSERT.

UPDATE queries treat warnings as errors. In this case, an invalid
condition "datetime_key_col >= '2012-01'" caused warning-as-error inside
SQL_SELECT::test_quick_select().

The code that called test_quick_select() ignored this error and continued
join optimization. Then it eventually reached a thd->is_error() check
and failed to setup SJ-Materialization which failed an assert.

Fixed this by making SQL_SELECT::test_quick_select() return error in
its return value, and making any code that calls it to check for error
condition and abort the query if the error is returned.

Places in the code that didn't check for errors from
SQL_SELECT::test_quick_select but now do:
- get_quick_record_count() call in make_join_statistics(),
- test_if_skip_sort_order(),
- "Range checked for each record" code.

Extra error handling fixes and commit text wording by Sergei Petrunia,

Reviewed-by: Sergei Petrunia, Oleg Smirnov

This commits only adds  --verbose-bootstrap to mysql_install_db.exe
call to have more error information dumped in case of an error.