MDEV-27557 InnoDB unnecessarily commits mtr during secondary index search to preserve clustered index latching order

New function to release latches till savepoint was added in mtr_t. As
there is no longer need to limit MDEV-20605 fix usage for locking reads
only, the limitation is removed.

While moving to a prescribed dependencies in MDEV-28011, an error was made
in the merge. The Ubuntu and Debian supported architectures of rocksdb-tools
are different and need to be treated as such.

This actually had no effect as our support of mariadb-plugin-rocksdb was never
different to the distro support of rocksdb-tools. Some notes where added
to this affect.

There is also nothing to do for Debian sid, and never should be.

The differentiation and grouping of distro codenames is for convenience in
merging upwards as more dependencies change across distro versions.

The fixing of versions rather than relying on apt-cache to be correct prevents
unstable changes between releases, and potentially uninstallable packages like
happened in MDEV-28014.

Correct comment about zstd to MDEV-16525

The call mtr.add_suppression() that was added
in commit 75b7cd68
for MemorySanitizer and Valgrind runs is causing
a result difference for the test rpl.rpl_gtid_stop_start.
Let us disable the binlog for executing that statement.

Also, the test perfschema.statement_program_lost_inst
would fail due to the changes to have_innodb.inc in this commit.
To compensate for that, we will make more --suite=perfschema
tests run without InnoDB, and explicitly enable InnoDB in
those tests that depend on a transactional storage engine.

S3 engine and libmarias3 don't compile on Windows

also fixes:
MDEV-28104 Typo in storage/maria/s3_func.c: 'FN_DECVHAR

A few regression tests invoke heavy flushing of the buffer pool
and may trigger warnings that tablespaces could not be deleted
because of pending writes. Those warnings are to be expected
during the execution of such tests.

The warnings are also frequently seen with Valgrind or MemorySanitizer.
For those, the global suppression in have_innodb.inc does the trick.

This was suggested by Sergei Golubhick.
Fixes up commit b91a123d

Since Debian Sid now has MariaDB 10.6, we can't do any upgrade tests in
Debian Sid for the 10.5 branch anymore. It would just fail with downgrade
errors.

Also, since MariaDB 10.5 is no longer in Sid, we can't even test 10.5.x
to 10.5.y upgrades in Sid.

Instead the 10.5 branch salsa-ci.yml should run all builds and tests based
on Debian Bullseye, which has MariaDB 10.5 (only).

To achieve this, essentially sync most the the salsa-ci.yml contents from
https://salsa.debian.org/mariadb-team/mariadb-10.5/-/tree/bullseye

Also add a couple Lintian overrides to make Salsa-CI pass.

NOTE TO MERGERS: This commit is intended for the 10.5 branch only, do not
merge anything from it on 10.6 or any other branch.

In commit ab38b751
an added "goto err" would seemingly cause a read of
an uninitialized variable old_info if errpos>=5.

However, because we would have errpos=0 at that point,
there was no real error.

extra2_read_len resolved by keeping the implementation
in sql/table.cc by exposed it for use by ha_partition.cc

Remove identical implementation in unireg.h
(ref: bfed2c7d)

Per Marko's comment in JIRA, sql_kill is passing the thread id
as long long. We change the format of the error messages to match,
and cast the thread id to long long in sql_kill_user.

The 10.5 test error main.grant_kill showed up a incorrect
thread id on a big endian architecture.

The cause of this is the sql_kill_user function assumed the
error was ER_OUT_OF_RESOURCES, when the the actual error was
ER_KILL_DENIED_ERROR. ER_KILL_DENIED_ERROR as an error message
requires a thread id to be passed as unsigned long, however a
user/host was passed.

ER_OUT_OF_RESOURCES doesn't even take a user/host, despite
the optimistic comment. We remove this being passed as an
argument to the function so that when MDEV-21978 is implemented
one less compiler format warning is generated (which would
have caught this error sooner).

Thanks Otto for reporting and Marko for analysis.

buf_flush_freed_pages(): Assert that neither buf_pool.mutex
nor buf_pool.flush_list_mutex are held. Simplify the loops.
Return the tablespace and the number of pages written or punched.

buf_flush_LRU_list_batch(), buf_do_flush_list_batch():
Release buf_pool.mutex before invoking buf_flush_space().

buf_flush_list_space(): Acquire the mutexes only after invoking
buf_flush_freed_pages().

Reviewed by: Thirunarayanan Balathandayuthapani

fil_space_t::try_to_close(): Tolerate a tablespace that has no
data files attached. The function fil_ibd_create() initially
creates and attaches a tablespace with no files, and invokes
fil_space_t::add() later.

fil_node_open_file(): After releasing and reacquiring fil_system.mutex,
check if the file was already opened by another thread. This avoids
an assertion failure !node->is_open() in fil_node_open_file_low().

These failures were reproduced with the test
innodb.table_definition_cache_debug and the fix of MDEV-27985.

commit '6de482a6'

10.3 no longer errors in truncate_notembedded.test
but per comments, a non-crash is all that we are after.

- InnoDB fails to skip newly created column while checking for
change column when table is in redundant row format. This issue
is caused the MDEV-18035 (ccb1acbd)

MDEV-23210 Assertion `(length % 4) == 0' failed in my_lengthsp_utf32 on ALTER TABLE, SELECT and INSERT

Problem:
Parse-time conversion from binary to tricky character sets like utf32
produced ill-formed strings. So, later a chash happened in debug builds,
or a wrong SHOW CREATE TABLE was returned in release builds.

Fix:

1. Backporting a few methods from 10.3:
  - THD::check_string_for_wellformedness()
  - THD::convert_string() overloads
  - THD::make_text_string_connection()

2. Adding a new method THD::reinterpret_string_from_binary(),
   which makes sure to either returns a well-formed string
   (optionally prepending with zero bytes), or returns an error.

Let us remove the redundant typedef.
This problem was revealed by
commit 77c184df

For some reason, the tests of the MemorySanitizer build on 10.5 failed
with both clang 13 and clang 14 with SIGSEGV. On 10.6 where it worked
better, some more places to work around were identified.

mysqltest allows leading spaces before `--`, so mtr should too

whenever possible, partitioning should use the full
partition plugin name, not the one byte legacy code.

Normally, ha_partition can get the engine plugin from
table_share->default_part_plugin.

But in some cases, e.g. in DROP TABLE, the table isn't
opened, table_share is NULL, and ha_partition has to parse
the frm, much like dd_frm_type() does.

temporary_tables.cc, sql_table.cc:

When dropping a table, it must be deleted in the engine
first, then frm file. Because frm can be the only true
source of metadata that the engine might need for DROP.

table.cc:

when opening a partitioned table, if the engine for
partitions is not found, do not fallback to MyISAM.

The MemorySanitizer implementation in clang includes some built-in
instrumentation (interceptors) for GNU libc. In GNU libc 2.33, the
interface to the stat() family of functions was changed. Until the
MemorySanitizer interceptors are adjusted, any MSAN code builds
will act as if that the stat() family of functions failed to initialize
the struct stat.

A fix was applied in
https://reviews.llvm.org/rG4e1a6c07052b466a2a1cd0c3ff150e4e89a6d87a
but it fails to cover the 64-bit variants of the calls.

For now, let us work around the MemorySanitizer bug by defining
and using the macro MSAN_STAT_WORKAROUND().

not every index-using plan sets bits in table->quick_keys.
QUICK_ROR_INTERSECT_SELECT, for example, doesn't.

Use the fact that select->quick is set instead.

Also allow EXPLAIN to work.

As btrfs showed, a partial read of data in AIO /O_DIRECT circumstances can
really confuse MariaDB.

Filipe Manana (SuSE)[1] showed how database programmers can assume
O_DIRECT is all or nothing.

While a fix was done in the kernel side, we can do better in our code by
requesting that the rest of the block be read/written synchronously if
we do only get a partial read/write.

Per the APIs, a partial read/write can occur before an error, so
reattempting the request will leave the caller with a concrete error to
handle.

[1] https://lore.kernel.org/linux-btrfs/CABVffENfbsC6HjGbskRZGR2NvxbnQi17gAuW65eOM+QRzsr8Bg@mail.gmail.com/T/#mb2738e675e48e0e0778a2e8d1537dec5ec0d3d3a

Also spell synchronously correctly in other files.

Let us explicitly wait for purge before invoking a slow shutdown,
so that instrumented builds (such as ASAN or UBSAN) will not
exceed the 60-second timeout during shutdown.

This fixes commit 77c184df.

The virtual member function that was added in
commit 1766a18e
needs to be declared "override".

Externally kill and restart the server, and remove the
unreliable crash_after_checkpoint.