buf_flush_LRU_list_batch(): When evicting clean pages,
release and reacquire the buf_pool.mutex after every 32 pages.
Also, eliminate some conditional branches.

fil_space_t::create(), fil_space_t::add(): Expect the caller to
acquire and release fil_system.mutex. In this way, creating a tablespace
and adding the first (usually only) data file will be atomic.

recv_sys_t::recover_deferred(): Correctly protect some changes by
holding fil_system.mutex.

Tested by: Matthias Leich

trx_assign_rseg_low(): Let us restore the debug variable look_for_rollover
to avoid assertion failures when a server that was created with
multiple undo tablespaces is being started with innodb_undo_tablespaces=0.

trx_assign_rseg_low(): Correct a debug injection condition.

trx_assign_rseg_low(): Simplify the debug check.

trx_rseg_t::reinit(): Reset the skip_allocation() flag.
This logic was broken in the merge
commit 3e2ad0e9
of commit 0de3be8c
(that is, innodb_undo_log_truncate=ON would never be "completed").

Tested by: Matthias Leich

If SQL_MODE contains ANSI_QUOTES (https://mariadb.com/kb/en/sql-mode/), then
the double-quote character (") is not a legal string delimiter.

In https://github.com/MariaDB/server/commit/13e77930e615f05cc74d408110e887b00e1abcc9#diff-a333d4ebb2d73b6361ef7dfebc86d883f7e19853b4a9eb85984b039058fae47cR2431-R2435,
Daniel Black introduced a case where the double-quote character would be used as
a string delimiter in the SQL queries generated by mariadb-tzinfo-to-sql.

This tool tool generates SQL queries which should be able to run on any
MariaDB server of the matching version.  Therefore, it should be extremely
conservative in the SQL that it outputs, in order to maximize the chance
that it can run regardless of the build or execution environment of the
server.

See MDEV-18778, MDEV-28263, and MDEV-28782 for previous cases where MariaDB
has FAILED TO ENSURE that the generated timezone.sql actually works in
different build and execution environments.  More test coverage is clearly
needed here.

All new code of the whole pull request, including one or several files that are
either new files or modified ones, are contributed under the BSD-new license. I
am contributing on behalf of my employer Amazon Web Services, Inc.

The 2013 error was right to catch the case B of the test unprepared
for an expected simulated crash.

The test gets refined to SELECT a (type of) bool value before the
crash is invoked.

Appending to 'eatmydata' will obviously cause an executable that
doesn't exist. Use an array to create the entire executable.

Also while we are at it, check the fakeroot actually works before
using it.

MDEV-31049 fil_delete_tablespace() returns wrong file handle if tablespace was closed by parallel thread

fil_delete_tablespace() stores file handle in local variable and calls
mtr_t::commit_file()=>fil_system_t::detach(..., detach_handle=true), which
sets space->chain.start->handle = OS_FILE_CLOSED. fil_system_t::detach()
is invoked under fil_system.mutex.

But before the mutex is acquired some parallel thread can change
space->chain.start->handle. fil_delete_tablespace() returns value, stored
in local variable, i.e. wrong value.

File handle can be closed, for example, from buf_flush_space() when the
limit of innodb_open_files exceded and fil_space_t::get() causes
fil_space_t::try_to_close() call.

fil_space_t::try_to_close() is executed under fil_system.mutex. And
mtr_t::commit_file() locks it for fil_system_t::detach() call.
fil_system_t::detach() returns detached file handle if its argument
detach_handle is true. The fix is to let mtr_t::commit_file() to pass
that detached file handle to fil_delete_tablespace().

Post-push fix.

10.5 MDEV-30775 fix inserts just opened tablespace just after the element
which fil_system.space_list_last_opened points to.

In MDEV-25223 fil_system_t::space_list was changed from UT_LIST to
ilist. ilist<...>::insert(iterator pos, reference value) inserts element
to list before pos.

But it was not taken into account during 10.5->10.6 merge in
85cbfaef, and the fix
does not work properly, i.e. it inserted just opened tablespace to the
position preceding fil_system.space_list_last_opened.

Currently autobake-debs.sh does not pass shellcheck
it fails making errors:

 * SC1091 when using shellcheck -x it needs to know where to
   find ./VERSION. As this is not needed we just specify it
   as /dev/null as mentioned in shellcheck documentation:
   https://www.shellcheck.net/wiki/SC1091

 * SC2086 make sure that there is no globbing or word splitting
   in dpkg-buidpackage string. This not big problem or about to happen
   but now extra parameter parsing is more Bash compliant with
   using array.
   Change BUILDPACKAGE_PREPEND to BUILDPACKAGE_DPKGCMD which holds
   'eatmydata' if it's available and needed 'dpkg-buildpackage'
   https://www.shellcheck.net/wiki/SC2086

Fix small script indentation problem.

It is used in the out-of date Ubuntu 21.10 Impish.

Github PR #2424 regressed Ubuntu 18.04 building
other than x86_64 machines. Architecture that are
impacted are PPC64 and ARM64.
This was because of changes in debian/rules file
which caused removing dependency to package 'libpmem-dev'
and CMake which '-DWITH_PMEM' removing not working
correctly. Package libpmem-dev was removed but
it still required to have PMEM with CMake which.
Commit make change that -DWITH_PMEM is correctly removed
if it's not wanted.

tpool::cache::m_mtx: Add PERFORMANCE_SCHEMA instrumentation
(wait/synch/mutex/innodb/tpool_cache_mutex). This covers the
InnoDB read_slots and write_slots for asynchronous data page I/O.

fil_name_process(): Starting with
commit 212994f7
the name is not guaranteed to be NUL terminated.

- This issue caused by race condition between drop thread
and fil_encrypt_thread. fil_encrypt_thread closes
the tablespace if the number of opened files
exceeds innodb_open_files. fil_node_open_file()
closes the tablespace which are open and it doesn't
have pending operations. At that time, InnoDB drop tries
to write the redo log for the file delete operation.
It throws the bad file descriptor error.

- When trying to close the file, InnoDB should check
whether the table is going to be dropped.

os_aio_pending_reads_approx(), os_aio_pending_reads(): Replaces
buf_pool.n_pend_reads.

os_aio_pending_writes(): Replaces buf_dblwr.pending_writes().

buf_dblwr_t::write_cond, buf_dblwr_t::writes_pending: Remove.

buf_flush_try_neighbors(): Tolerate count<2 in case the tablespace
is being dropped.

The problem was earlier fixed by the patch for MDEV-30034.
Adding MTR tests only.

Spider system tables should be created so that wsrep_on=OFF.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

Query cache should be invalidated if we are not in applier. For some
reason this condition was incorrect starting from 10.5 but it is
correct in 10.4.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>

After MDEV-30830 has added block-nl-join.r_unpack_time_ms, it became
apparent that there is some unaccounted-for time in BNL join operation,
namely the time that is spent after unpacking the join buffer record.

Fix this by adding a Gap_time_tracker to track the time that is spent
after unpacking the join buffer record and before any next time tracking.
The collected time is printed in block-nl-join.r_other_time_ms.

Reviewed by: Monty <monty@mariadb.org>

Also fix it to work with hashed join (MDEV-30830).

Reviewed by: Monty <monty@mariadb.org>

- Adding a new argument "flag" to MY_COLLATION_HANDLER::strnncollsp_nchars()
  and a flag MY_STRNNCOLLSP_NCHARS_EMULATE_TRIMMED_TRAILING_SPACES.
  The flag defines if strnncollsp_nchars() should emulate trailing spaces
  which were possibly trimmed earlier (e.g. in InnoDB CHAR compression).
  This is important for NOPAD collations.

  For example, with this input:
   - str1= 'a '    (Latin letter a followed by one space)
   - str2= 'a  '   (Latin letter a followed by two spaces)
   - nchars= 3
  if the flag is given, strnncollsp_nchars() will virtually restore
  one trailing space to str1 up to nchars (3) characters and compare two
  strings as equal:
  - str1= 'a  '  (one extra trailing space emulated)
  - str2= 'a  '  (as is)

  If the flag is not given, strnncollsp_nchars() does not add trailing
  virtual spaces, so in case of a NOPAD collation, str1 will be compared
  as less than str2 because it is shorter.

- Field_string::cmp_prefix() now passes the new flag.
  Field_varstring::cmp_prefix() and Field_blob::cmp_prefix() do
  not pass the new flag.

- The branch in cmp_whole_field() in storage/innobase/rem/rem0cmp.cc
  (which handles the CHAR data type) now also passed the new flag.

- Fixing UCA collations to respect the new flag.
  Other collations are possibly also affected, however
  I had no success in making an SQL script demonstrating the problem.
  Other collations will be extended to respect this flags in a separate
  patch later.

- Changing the meaning of the last parameter of Field::cmp_prefix()
  from "number of bytes" (internal length)
  to "number of characters" (user visible length).

  The code calling cmp_prefix() from handler.cc was wrong.
  After this change, the call in handler.cc became correct.

  The code calling cmp_prefix() from key_rec_cmp() in key.cc
  was adjusted according to this change.

- Old strnncollsp_nchar() related tests in unittest/strings/strings-t.c
  now pass the new flag.
  A few new tests also were added, without the flag.

The tests innodb.import_tablespace_race, innodn.restart, and innodb.innodb-wl5522 move
the tablespace file between the data directory and the tmp directory specified by
global environment variables. However this is risky because it's not unusual that the
set tmp directory (often under /tmp) is mounted on another disk partition or device,
and 'move_file' command may fail with "Errcode: 18 'Invalid cross-device link.'"

For innodb.import_tablespace_race and innodb.innodb-wl5522, moving files
across directories is not necessary. Modify the tests so they rename
files under the same directory. For innodb.restart, instead of moving
between datadir and MYSQL_TMPDIR, move the files under MYSQLTEST_VARDIR.

All new code of the whole pull request, including one or several files that
are either new files or modified ones, are contributed under the BSD-new license.
I am contributing on behalf of my employer Amazon Web Services, Inc.

The tests innodb.import_tablespace_race, innodn.restart, and innodb.innodb-wl5522 move
the tablespace file between the data directory and the tmp directory specified by
global environment variables. However this is risky because it's not unusual that the
set tmp directory (often under /tmp) is mounted on another disk partition or device,
and 'move_file' command may fail with "Errcode: 18 'Invalid cross-device link.'"

To stabilize mysqltest in the described scenario, and prevent such
behavior in the future, let make_file() check both from file path and to
file path and make sure they are either both under MYSQLTEST_VARDIR or
MYSQL_TMP_DIR.

All new code of the whole pull request, including one or several files that
are either new files or modified ones, are contributed under the BSD-new license.
I am contributing on behalf of my employer Amazon Web Services, Inc.

Add vital missing step to MariaDB 10.5 upgrade job to actually install
the new binary being built. Without this the test was happily passing all
the time but actually not testing the upgrade.

Also stop using oneliner syntax for the install step to make the debugging
of failing installs/upgrades from build logs easier.

NOTE TO MERGERS: This commit is made on 10.6 branch and can be merged to
all later branches (10.7, 10.8, ..., 11.0). If/when some jobs break, they
will be fixed per branch on follow-up commits.

Tests that try to upgrade MariaDB 10.6 in Debian Sid can no longer work
properly on versions < 10.11 as MariaDB 10.11 in now Debian Sid.

Change RELEASE to use Bullseye and refactor builds and upgrade tests to
use primarily Bullseye, as it has MariaDB 10.5 and thus testing upgrades
to MariaDB 10.6 and higher can work. Add on new 'build sid' job to continue
at least on build on Sid as well, even though it is not tested in other
jobs.

Due to this many Sid specific workarounds are can also be dropped, and
since Bullseye is now used for everything, the old bullseye-backports jobs
are obsolete and removed.

Tests that upgrade MySQL in Sid to MariaDB are also removed, as no test
in Debian Sid with MariaDB < 10.11 can reliably work anymore.

Also disable reprotest as unnecessary on old branches, refactor the naming
of autobake-deb.sh and native Debian build jobs to be more clear.

NOTE TO MERGERS: This commit is made on 10.6 branch and can be merged to
all later branches (10.7, 10.8, ..., 11.0). If/when some jobs break, they
will be fixed per branch on follow-up commits.

Compare to Debian packaging of MariaDB 1:10.6.11-2 release at commit
https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/2934e8a795cbd1eba79e33b8d6c82ed2b0897128
and sync upstream everything that is relevant for upstream and safe
to import on a stable 10.6 release.

* Use OpenSSL 1.1 from Debian Snapshots
  https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/0040c272bf8a9a253fcd393b235b18cd5cdc9289
  Related: https://jira.mariadb.org/browse/MDEV-30322

* Prefer using bullseye-backports in mosts tests over buster-bpo
  https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/daa827ecded9dc1f096ecb05242116173298d602

* Add new upgrade test for MySQL Community Cluster 8.0
  https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/3c71bec9b7626d25a55e23b2e570b4e125d5d5df

* Enable automatic datadir move also on upgrades from MySQL.com packages
  https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/4cbbcb7e56d9604a67230d11a68543d5a83960e3

* Update Breaks/Replaces
  https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/2cab13d05959fe8f6ea56b866df51f3bfe17dd3f

* Normalize apt-get and curl commands
  https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/8754ea2578bd214d75a2a9e6a0f7b4b74c062b41

* Standardize on using capitalized 'ON' in CMake build options
  https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/938757a85aee44c727f3f6996677ef1126c2f5aa

* Apply wrap-and-sort -av and other minor tweaks and inline documentation

NOTE TO MERGERS: This commit is made on 10.6 branch and can be merged to
all later branches (10.7, 10.8, ..., 11.0).

This is allowed:

  STRING_WITH_LEN("string literal")

This is not:

  char *str = "pointer to string";
  ... STRING_WITH_LEN(str) ..

In C++ this is also allowed:

  const char str[] = "string literal";
  ... STRING_WITH_LEN(str) ...

Test fails sporadically and very rarely on this:
```
let $org_queries= `SHOW STATUS LIKE 'Queries'`;
SELECT f1();
CALL p1();
let $new_queries= `SHOW STATUS LIKE 'Queries'`;
let $diff= `SELECT SUBSTRING('$new_queries',9)-SUBSTRING('$org_queries',9)`;
```
if COM_QUIT from one of the earlier (in the test) disconnect's
happens between the two SHOW STATUS commands.
Because COM_QUIT increments "Queries".

The directly previous test uses wait_condition to wait for
its disconnects to complete. But there are more disconnects earlier
in the test file and nothing waits for them.

Let's change wait_condition to wait for *all* disconnect to complete.

MariaDB server prints the stack information if a crash happens.

It traverses the stack frames in function `print_with_addr_resolve`.
For *EACH* frame, it tries to parse the file name and line number of the
frame using `addr2line`, or prints `backtrace_symbols_fd` if `addr2line`
fails.

1. Logic in `addr_resolve` function uses addr2line to get the file name
   and line numbers. It has a timeout of 500ms to wait for the response
   from addr2line. However, that's not enough on small instances
   especially if the debug information is in a separate file or
   compressed.

   Increase the timeout to 5 seconds to support some edge cases, as
   experiments showed addr2line may take 2-3 seconds on some frames.

2. While parsing a frame inside of a shared library using `addr2line`,
   the file name and line numbers could be `??`, empty or `0` if the
   debug info is not loaded.
   It's easy to reproduce when glibc-debuginfo is not installed.

   Instead of printing a meaningless frame like:

       :0(__GI___poll)[0x1505e9197639]
       ...
       ??:0(__libc_start_main)[0x7ffff6c8913a]

   We want to print the frame information using `backtrace_symbols_fd`,
   with the shared library name and a hexadecimal offset.
   Stacktrace example on a real instance with this commit:

       /lib64/libc.so.6(__poll+0x49)[0x145cbf71a639]
       ...
       /lib64/libc.so.6(__libc_start_main+0xea)[0x7f4d0034d13a]

   `addr_resolve` has considered the case of meaningless combination of
   file name and line number returned by `addr2line`. e.g. `??:?`
   However, conditions like `:0` and `??:0` are not handled. So now the
   function will rollback to `backtrace_symbols_fd` in above cases.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.

In block-nl-join, add:

- r_loops - this shows how many incoming record combinations this
  query plan node had.

- r_effective_rows - this shows the average number of matching rows
  that this table had for each incoming record combination. This is
  comparable with r_rows in non-blocked access methods.
  For BNL-joins, it is always equal to
   $.table.r_rows * $.table.r_filtered
  For BNL-H joins the value cannot be computed from other values

Reviewed by: Monty <monty@mariadb.org>