In commit 118e258a (part of MDEV-23855)
we inadvertently broke crash recovery, reintroducing MDEV-11556.

fil_system_t::extend_to_recv_size(): Extend all open tablespace files
to the recovered size.

recv_sys_t::apply(): Invoke fil_system.extend_to_recv_size() at the
start of each batch. In this way, any fil_space_t::recv_size
changes that were parsed after the file was opened will be applied.

page_apply_insert_redundant(): Replace a too strict condition
hdr_c > pextra_size. It turns out that page_cur_insert_rec_low()
is not even computing the extra_size of cur->rec when it is trying
to reuse header bytes of the preceding record.

- Currently page cleaner thread will stop flushing if
  dirty_pct < innodb_max_dirty_pages_pct_lwm.

- If the server is not performing any activity then said resources/time
  could be used to flush the pending dirty pages and keep buffer pool
  clean for the next burst of the cycle. This flushing is called idle flushing.

- flushing logic underwent a complete revamp in 10.5.7/8
  and as part of the revamp idle flushing logic got removed.

- New proposed logic of idle flushing is based on updated logic of the
  page cleaner that will enable idle flushing if
  - buf page cleaner is idle
  - there are dirty pages (< innodb_max_dirty_pages_pct_lwm)
  - server is not performing any activity

  Logic will kickstart the idle flushing bounded by innodb_io_capacity.

(Thanks to Marko Makela for reviewing the patch and idea
 right from the its inception).

The problem was that the CONNECT engine is trying to open the .frm file
during drop_table(), which the code did not take into account.
Fixed by adding the HA_REUSES_FILE_NAMES table flag to CONNECT.

Other things:
- Fixed a wrong test of HA_REUSE_FILE_NAMES of in mysql_alter_table()
  (Comment was correct, no the code)
- Added a test in the connect engine that if the .frm it tries to use in
  delete is not made for connect, it will generate an error instead of
  crash.

Do not use --source directive as we don't want to load the plugin when
the test starts.

- Reduce Build-Depends
  https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/150bf990c6059a2fca95ea9bc4d108c9789b179d

  Dependencies chrpath, dh-apparmor and libarchive-dev are not needed.
  Fixes buildbot sid failures that error on:
    Unmet build dependencies: chrpath dh-apparmor libarchive-dev

- Salsa-CI: Remove mysql-5.7 upgrade in Sid test as package was removed
  https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/6f55ac620ce26046475b2a430838367732b47d54

  Also clean away extra Salsa-CI markup not needed anymore.

- Autopkgtest: Simplify autopkgtest 'smoke' to be easier to debug
  https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/836907989ad8d355b2f88f8013a6acb8a64cf7d9

- Autopkgtest: Skip main.failed_auth_unixsocket on armhf and i386
  https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/74601f8b31a6c59e825089c52a1ca21545062813

This is fixup for commit f06a0b53.

Since we do not stop at corrupted page error, there is no reason to log
a backup error.

row_prebuilt_t::m_no_prefetch: Remove (it was always false).
row_prebuilt_t::m_read_virtual_key: Remove (it was always false).

Only ha_innopart ever set these fields.

innobase_rename_table(): Invoke dict_stats_wait_bg_to_stop_using_table()
to ensure that dict_stats_update() cannot be accessing the table name
that we will be modifying. If we are executing RENAME rather than TRUNCATE,
reset the flag at the end so that persistent statistics can be calculated
again.

The race condition was encountered with ASAN and rr.
Sorry, there is no test case, like there is for nothing related to
dict_stats_wait_bg_to_stop_using_table(). The entire code is an ugly
work-around for the failure of dict_stats_process_entry_from_recalc_pool()
to acquire MDL.

Note: It appears that an ALTER TABLE that is not rebuilding the table
will fail to reset the flag that blocks the processing of statistics.

row_number() over () window function can be used without any column in the OVER
clause. Additionally, the item doesn't reference any tables, as it's not
effectively referencing any table. Rather it is specifically built based
on the end temporary table used for window function computation.

This caused remove_const function to wrongly drop it from the ORDER
list. Effectively, we shouldn't be dropping any window function from the
ORDER clause, so adjust remove_const to account for that.

Reviewed by: Sergei Petrunia sergey@mariadb.com

The bug caused crashes of the server when processing queries with nested
table value constructors (TVC) . It happened because the grammar rules to
parse TVC used the same global lists for both nested TVC and nesting TVC.
As a result invalid select trees were constructed for queries with nested
TVC and this led to crashes at the prepare stage.
This patch provides its own lists structures for each TVC nest level.

Besides the patch fixes a bug in the function wrap_tvc() that missed
inheritance of the SELECT_LEX::exclude_from_table_unique_test for
selects that wrapped TVCs. This inheritance is critical for specifications
of derived tables that employ nested TVCs.

Approved by dmitry.shulga@mariadb.com

In btr_index_rec_validate(), externally stored column
check is missing while matching the length of the field
with the length of the field data stored in record.
Fetch the length of the externally stored part and compare it
with the fixed field length.

- This issue is caused by commit deadec4e
(MDEV-24569). InnoDB fails to read the change buffer bitmap page
from dropped tablespace. In ibuf_bitmap_get_map_page_func(), InnoDB
should fetch the page using BUF_GET_POSSIBLY_FREED mode. Callers of
ibuf_bitmap_get_map_page() should be adjusted in that case.

This is after-merge fix of f33e57a9.
In btr_search_drop_page_hash_index(), InnoDB should take
the exclusive lock on the AHI latch if index is already
freed to avoid the freed memory access during buf_pool_resize()

MDEV-20648 InnoDB: Failing assertion: !(*node)->being_extended, innodb.log_data_file_size failed in buildbot, assertion `!space->is_stopping()'

InnoDB should check whether the tablespace is being deleted
while extending the tablespace.

This is a backport of commit 18535a40
from 10.6.

lock_release(): Implement innodb_evict_tables_on_commit_debug.
Before releasing any locks, collect the identifiers of tables to
be evicted. After releasing all locks, look up for the tables and
evict them if it is safe to do so.

trx_commit_in_memory(): Invoke trx_update_mod_tables_timestamp()
before lock_release(), so that our locks will protect the tables
from being evicted.

When doing a truncate on an Innodb under lock tables, InnoDB would rename
the old table to #sql-... and recreate a new 't1' table. The table lock
would still be on the #sql-table.

When doing ALTER TABLE, Innodb would do the changes on the #sql table
(which would disappear on close).
When the SQL layer, as part of inline alter table, would close the
original t1 table (#sql in InnoDB) and then reopen the t1 table, Innodb
would notice that this does not match it's own (old) t1 table and
generate an error.

Fixed by adding code in truncate table that if we are under lock tables
and truncating an InnoDB table, we would close, reopen and lock the
table after truncate. This will remove the #sql table and ensure that
lock tables is using the new empty table.

Reviewer: Marko Mäkelä

Fixed binary logging in ANALYZE TABLE to work as optimize table

Attempt to execute EXPLAIN statement on multi-table DELETE statement
leads to firing firing of the assertion
  DBUG_ASSERT(! is_set());
in the method Diagnostics_area::set_eof_status.

For example, above mentioned assertion failure happens
in case any of the following statements
  EXPLAIN DELETE FROM t1.* USING t1
  EXPLAIN DELETE b FROM t1 AS a JOIN t1 AS b
are executed in prepared statement mode provided the table t1
does exist.

This assertion is hit by the reason that a status of
Diagnostics_area is set twice. The first time it is set from
the function do_select() when the method multi_delete::send_eof()
called. The second time it is set when the method
Explain_query::send_explain() calls the method select_send::send_eof
(this method invokes the method Diagnostics_area::set_eof_status that
finally hits assertion)

The second invocation for a setter method of the class Diagnostics_area
is correct and run to send a response containing explain data.

But first invocation of a setter method of the class Diagnostics_area
is wrong since the function do_select() shouldn't be called at all
for handling of the EXPLAIN statement.

The reason by that the function do_select() is called during handling of
the EXPLAIN statement is that the flag SELECT_DESCRIBE not set in the
data member JOIN::select_options. The flag SELECT_DESCRIBE
if is copied from values select_lex->options.

During parsing of EXPLAIN statement this flag is set but latter reset
from the function reinit_stmt_before_use() that is called on
execution of prepared statement.
  void reinit_stmt_before_use(THD *thd, LEX *lex)
  {
    ...
    for (; sl; sl= sl->next_select_in_list())
    {
      if (sl->changed_elements & TOUCHED_SEL_COND)
      {
        /* remove option which was put by mysql_explain_union() */
        sl->options&= ~SELECT_DESCRIBE;
      ...
      }
   ...
  }

So, to fix the issue the flag SELECT_DESCRIBE is set forcibly at the
mysql_select() function in case thd->lex->describe set,
that is in case EXPLAIN being executed.

The test case encryption.innodb_encrypt_freed was failing in
MemorySanitizer builds.

recv_recover_page(): Mark non-recovered pages as freed.

fil_crypt_rotate_page(): Before comparing the block->frame contents,
check if the block was marked as freed.

Other places: Whenever using BUF_GET_POSSIBLY_FREED, check the
block->page.status before accessing the page frame.

(Both uses of BUF_GET_IF_IN_POOL should be correct now.)

           used in set function

If a subselect is formed by a table value constructor (TVC) then the
following transformation is applied at the prepare stage:
  VALUES (v1), ... (vn) => SELECT * FROM (VALUES (v1), ... (vn)) tvc_x.
The transformation is performed by the function wrap_tvc() that resets
THD::LEX::current select to the top level select of the result of the
transformation. After the call of wrap_tvc() in the function
Item_subselect::wrap_tvc_into_select() the field THD::LEX::current must be
reset to the same select as before the call. It was not done. As a result
if the subselect formed by a TVC was an argument of a set function then
an assertion was hit in the function Item_sum::check_sum_func().

Approved by Oleksandr Byelkin <sanja@mariadb.com>

Before the changes two things could happen:
- "path required name explain_filename path" error
- unit test never finishead (as it tried to execute just /bin/sh as
  a test case)

Fixes also:
MDEV-24942 Server crashes in _ma_rec_pack... with DEFAULT() on BLOB

This was caused by two different bugs, both related to that the default
value for the blob was not calculated before it was used:
- There where now Item_default_value::..result() wrappers, which is
  needed as item in HAVING uses these.  This causes crashes when
  using a reference to a DEFAULT(blob_field) in HAVING. It also
  caused wrong results when used with other fields with default value
  expressions that are not constants.
- create_tmp_field() did not take into account that blob fields with
  default expressions are not yet initialized. Fixed by treating
  Item_default_value(blob) like a normal item expression.