MDEV-24659 Assertion !fsp_is_system_temporary(bpage->id().space()) failed in buf_flush_relocate_on_flush_list()

When commit 5eb53955 (MDEV-12227)
removed the pages of temporary tables from the buf_pool.flush_list,
an adjustment to the buffer pool resizing was forgotten.

buf_pool_t::realloc(): Do not invoke buf_flush_relocate_on_flush_list()
for pages that belong to the temporary tablespace. Also, deduplicate
some code at the end.

buf_page_t::set_corrupt_id(): Tolerate oldest_modification()==1
(the dummy value) for temporary tablespace pages. The revised
buf_pool_t::realloc() may invoke this on dirty temporary tablespace pages.

fts_optimize_request_sync_table() can avoid the repetitive
FTS SYNC request of the table if the table already has FTS_SYNC
message in fts_optimize_wq queue.

Reviewed-by: Marko Mäkelä

While reusing the cached undo log block, mtr expects the page
write to change while writing the trx id. cached undo log block
could contain bytes which were originally written for some other
transaction. So InnoDB should make mtr to do MAYBE_NOP while reusing
cached undo log block.

Reviewed-by: Marko Mäkelä

The problem was that update_timing_fields_for_event() didn't release all
MDL locks it took.

Do not init encryption threads if shutdown is in progress.

- Fix long lines to be <= 80 character
- Trivial changes (no logic changes)

- Create_tmp_table::finalize didn't clear file after delete which
  could cause a double free. This is however not a likely problem as
  this code path is very unlikely to happen
- free_tmp_table() could do handler calls even if the table was never
  opened. Fixed by adding a test if the table is opened.

The table performance_schema.events_transactions_history_long that
was imported from MySQL 5.7.28 in
commit 0ea717f5
may report bogus trx_id values for InnoDB transactions.

innobase_register_trx(): Pass trx->id to trans_register_ha(),
even if it is 0. It is more appropriate to report NULL than some
arbitrary value that has been constructed from the address of a
transaction identifier.

closes #1737

Reviewers: Marko Mäkelä, Krunal Bauskar

Fixes build on powerpc64 and powerpc64le.

Closes: #1710

Starting with MDEV-15528, we will avoid writing freed pages back
to data files. During stress testing, the assertion
mach_read_from_4(frame + 4U) == block.page.id().page_no()
failed in log_phys_t::apply(), because before the server was
killed and restarted, change buffer merge was executed on a
previously freed page.

During recovery, the assertion would fail when InnoDB would apply
a FREE_PAGE and a WRITE record to the page.

ibuf_merge_or_delete_for_page(): Before applying any changes, check whether
the secondary index leaf page has already been freed according to
the allocation bitmap page. If it is freed, then we must reset the bits
in change buffer bitmap page.

ibuf_reset_bitmap(): Auxiliary function for repeated code.

mtr_t::sx_lock_space(): Replaces mtr_t::x_lock_space(). Due to the
lazy approach of the change buffer, The function
ibuf_merge_or_delete_for_page() may be executed in buf_page_create()
while the tablespace is already X-latched. An S-latch must not be
acquired while the thread already holds an X-latch, but a redundant
SX-latch is fine.

The fix was developed by Thirunarayanan Balathandayuthapani.

The parameter innodb_idle_flush_pct that was introduced in
MariaDB Server 10.1.2 by MDEV-6932 has no effect ever since
the InnoDB changes from MySQL 5.7.9 were applied in
commit 2e814d47.

Let us declare the parameter as MARIADB_REMOVED_OPTION.
For earlier versions, commit ea9cd97f
declared the parameter deprecated.

4. update columnstore

3. Embed plugin version into the DEB package version

this assumes that -DDEB is only used from within autobake-deb.sh

and not ${binary:Version}, which is the version of the current
package, whatever it is.

2. Embed plugin version into the RPM package version

introduce SERVER_VERSION, because plugins can overwrite VERSION
(and columnstore actually does)

1. specify plugin version in MYSQL_ADD_PLUGIN

The fix of MDEV-17556 (commit e25623e7
and commit 61a362c9) has been
omitted due to conflicts and will have to be applied separately later.

Reviewer: Daniel Black

- Galera cluster must be in node 1 and 2, and acts as slave for node 3
- Sync point commit_monitor_enter_sync was renamed to
  commit_monitor_master_enter_sync in 4.x
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

MDEV-24546 : AddressSanitizer: initialization-order-fiasco on address ... in Sys_var_integer from __static_initialization_and_destruction_0, possibly inside global var wsrep_gtid_server

Galera parameter wsrep_gtid_domain_id was defined using a class where
actual parameter was not a first member. Fixed this by using normal
variable and assigning this value to class member value.

A race condition may occur between the execution of transaction commit,
and an execution of a KILL statement that would attempt to abort that
transaction.

MDEV-17092 worked around this race condition by modifying InnoDB code.
After that issue was closed, Sergey Vojtovich pointed out that this
race condition would better be fixed above the storage engine layer:

If you look carefully into the above, you can conclude that
thd->free_connection() can be called concurrently with
KILL/thd->awake(). Which is the bug. And it is partially fixed in
THD::~THD(), that is destructor waits for KILL completion:

Fix: Add necessary mutex operations to THD::free_connection()
and move WSREP specific code also there. This ensures that no
one is using THD while we do free_connection(). These mutexes
will also ensures that there can't be concurrent KILL/THD::awake().

innobase_kill_query
  We can now remove usage of trx_sys_mutex introduced on MDEV-17092.

trx_t::free()
  Poison trx->state and trx->mysql_thd

This patch is validated with an RQG run similar to the one that
reproduced MDEV-17092.

The assertion failed in handler::ha_reset upon SELECT under
READ UNCOMMITTED from table with index on virtual column.

This was the debug-only failure, though the problem is mush wider:
* MY_BITMAP is a structure containing my_bitmap_map, the latter is a raw
 bitmap.
* read_set, write_set and vcol_set of TABLE are the pointers to MY_BITMAP
* The rest of MY_BITMAPs are stored in TABLE and TABLE_SHARE
* The pointers to the stored MY_BITMAPs, like orig_read_set etc, and
 sometimes all_set and tmp_set, are assigned to the pointers.
* Sometimes tmp_use_all_columns is used to substitute the raw bitmap
 directly with all_set.bitmap
* Sometimes even bitmaps are directly modified, like in
TABLE::update_virtual_field(): bitmap_clear_all(&tmp_set) is called.

The last three bullets in the list, when used together (which is mostly
always) make the program flow cumbersome and impossible to follow,
notwithstanding the errors they cause, like this MDEV-17556, where tmp_set
pointer was assigned to read_set, write_set and vcol_set, then its bitmap
was substituted with all_set.bitmap by dbug_tmp_use_all_columns() call,
and then bitmap_clear_all(&tmp_set) was applied to all this.

To untangle this knot, the rule should be applied:
* Never substitute bitmaps! This patch is about this.
 orig_*, all_set bitmaps are never substituted already.

This patch changes the following function prototypes:
* tmp_use_all_columns, dbug_tmp_use_all_columns
 to accept MY_BITMAP** and to return MY_BITMAP * instead of my_bitmap_map*
* tmp_restore_column_map, dbug_tmp_restore_column_maps to accept
 MY_BITMAP* instead of my_bitmap_map*

These functions now will substitute read_set/write_set/vcol_set directly,
and won't touch underlying bitmaps.

MDEV-16272 rpl.rpl_semisync_ali_issues failed in buildbot, SHOW variable was done instead of waiting for the value of that variable

Ensure consistent use of logging macros in wsrep-related code
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

buf_LRU_get_free_block(): Increment the counter after acquiring
buf_pool.mutex.

buf_pool.stat.LRU_waits: Replaces export_vars.innodb_buffer_pool_wait_free
and srv_stats.buf_pool_wait_free.

Reads of the counter will remain dirty, not acquiring the mutex.

Use the same variable in both scopes (from where we have "goto error" and target of the goto)

Problem: Assertion `transactional_table || !changed ||
thd->transaction.stmt.modified_non_trans_table' failed due REPLACE into a
versioned table.

It is not specific to system versioning/pertitioning/heap, but this
combination makes it much easier to reproduce.

The thing is to make first ha_update_row call succeed to make
info->deleted != 0. And then make REPLACE fail by any reason.

In this scenario we overflow versioned partition, so next ha_update_row
succeeds, but corresponding ha_write_row fails to insert history record.

Fix: modified_non_trans_table is set in one missed place

fixing windows build.

In commit 3a9a3be1 (MDEV-23855)
some previous logic was replaced with the condition
dirty_pct < srv_max_dirty_pages_pct_lwm, which caused
the default value of the parameter innodb_max_dirty_pages_pct_lwm=0
to lose its special meaning: 'refer to innodb_max_dirty_pages_pct instead'.

This implicit special meaning was visible in the function
af_get_pct_for_dirty(), which was removed in
commit f0c295e2 (MDEV-24369).

page_cleaner_flush_pages_recommendation(): Restore the special
meaning that was removed in MDEV-24369.

buf_flush_page_cleaner(): If srv_max_dirty_pages_pct_lwm==0.0,
refer to srv_max_buf_pool_modified_pct. This fixes the observed
performance regression due to excessive page flushing.

buf_pool_t::page_cleaner_wakeup(): Revise the wakeup condition.

innodb_init(): Do initialize srv_max_io_capacity in Mariabackup.
It was previously constantly 0, which caused mariadb-backup --prepare
to hang in buf_flush_sync(), making no progress.