Histogram_json_hb::range_selectivity() may return small negative
numbers due to rounding errors in the histogram.

Make sure the returned value is non-negative.
Add an assert to catch negative values that are not small.

(attempt #2)

On GNU/Linux, even though the C11 aligned_alloc() appeared in
GNU libc early on, some custom memory allocators did not
implement it until recently. For example, before
gperftools/gperftools@d406f2285390c402e824dd28e6992f7f890dcdf9
the free() in tcmalloc would fail to free memory that was
returned by aligned_alloc(), because the latter would map to the
built-in allocator of libc. The Linux specific memalign() has a
similar interface and is safer to use, because it has been
available for a longer time. For AddressSanitizer, we will use
aligned_alloc() so that the constraint on size can be enforced.

buf_tmp_reserve_compression_buf(): When HAVE_ALIGNED_ALLOC holds,
round up the size to be an integer multiple of the alignment.

pfs_malloc(): In the unit test stub, round up the size to be an
integer multiple of the alignment.

Table_cache_instance: Define the structure aligned at
the CPU cache line, and remove a pad[] data member.
Krunal Bauskar reported this to improve performance on ARMv8.

aligned_malloc(): Wrapper for the Microsoft _aligned_malloc()
and the ISO/IEC 9899:2011 <stdlib.h> aligned_alloc().
Note: The parameters are in the Microsoft order (size, alignment),
opposite of aligned_alloc(alignment, size).
Note: The standard defines that size must be an integer multiple
of alignment. It is enforced by AddressSanitizer but not by GNU libc
on Linux.

aligned_free(): Wrapper for the Microsoft _aligned_free() and
the standard free().

HAVE_ALIGNED_ALLOC: A new test. Unfortunately, support for
aligned_alloc() may still be missing on some platforms.
We will fall back to posix_memalign() for those cases.

HAVE_MEMALIGN: Remove, along with any use of the nonstandard memalign().

PFS_ALIGNEMENT (sic): Removed; we will use CPU_LEVEL1_DCACHE_LINESIZE.

PFS_ALIGNED: Defined using the C++11 keyword alignas.

buf_pool_t::page_hash_table::create(),
lock_sys_t::hash_table::create():
lock_sys_t::hash_table::resize(): Pad the allocation size to an
integer multiple of the alignment.

Reviewed by: Vladislav Vaintroub

There was a race condition between log_checkpoint_low() and
deleting or renaming data files. The scenario is as follows:

1. The buffer pool does not contain dirty pages.
2. A FILE_DELETE or FILE_RENAME record is written.
3. The checkpoint LSN will be moved ahead of the write of the record.
4. The server is killed before the file is actually renamed or deleted.

We will prevent this race condition by ensuring that a log checkpoint
cannot occur between the durable write and the file system operation:

1. Durably write the FILE_DELETE or FILE_RENAME record.
2. Perform the file system operation.
3. Allow any log checkpoint to proceed.

mtr_t::commit_file(): Implement the DELETE or RENAME logic.

fil_delete_tablespace(): Delegate some of the logic to
mtr_t::commit_file().

fil_space_t::rename(): Delegate some logic to mtr_t::commit_file().
Remove the debug injection point fil_rename_tablespace_failure_2
because we do test RENAME failures without any debug injection.

fil_name_write_rename_low(), fil_name_write_rename(): Remove.

Tested by Matthias Leich

buf_page_create_low(): Before retrying, release the exclusive page latch
in order to prevent an infinite loop in buf_pool_t::corrupted_evict().

recv_recover_page(): Correct a debug assertion to refer to recv_sys.lsn,
which may be ahead of log_sys.lsn during non-final recovery batches.
In commit 685d958e (MDEV-14425)
when some redundant LSN fields were removed,
log_sys.log.scanned_lsn had been replaced with a reference to
log_sys.lsn instead of the more appropriate recv_sys.lsn.

recv_scan_log(): Remove a redundant call to log_sys.set_recovered_lsn().
It suffices to adjust the log_sys.lsn after parsing (and before starting
to apply) records for the last batch.

Note: Normally, log_sys.lsn must be the latest log sequence number.
Before the final recovery batch, this may be safely violated, because
log_write_up_to() will be a no-op. That function will be invoked by the
buf_flush_page_cleaner thread to initiate writes of recovered pages.

Work around MDEV-28718 for now, but also optimize the interation
of information_schema.SYSTEM_VARIABLES.

Add test case to show that tzinfo data into bootstrap is
desired functionality.

Bug report thanks to Dan Lenski of AWS.

ha_innobase::prepare_inplace_alter_table(): If the ALTER TABLE
operation is no-op for InnoDB, do reset m_prebuilt->trx_id
so that ha_innobase::table_version() will always report either 0
or the identifier of the transaction that would commit changes
to the InnoDB data dictionary.

The failure scenario involved a completed DROP INDEX followed by
a no-op ALTER TABLE during which the server was killed.
An effort to create a reproducible test failed.

Since commit 0b47c126 (MDEV-13542)
we treat all-zero pages as corrupted ones.

During a stress test, a read-ahead of an all-zero page was triggered
and the page read was completed concurrently with buf_page_create_low().
This caused the assertion to fail, because buf_page_create_low() was
waiting for the page latch.

buf_page_get_low(): Only invoke buf_pool_t::corrupted_evict()
if the block was not already marked as corrupted.

buf_page_create_low(): On page identifier mismatch, retry the
buf_pool.page_hash lookup.

buf_pool_t::corrupted_evict(): Set the state of the block to FREED
so that a concurrent buf_page_get_low() will refuse to load the page.
Wait for the page latch to be vacant before proceeding to remove
the block from buf_pool.page_hash and buf_pool.LRU.

page_id_t::set_corrupted(), page_id_t::is_corrupted(): Accessors
for indicating a corrupted page identifier.

Tested by Matthias Leich

Avoid the loop with getting rid of back and forth jumping.

In commit c4c88307 (MDEV-28111) we disabled
the file system cache on the InnoDB write-ahead log file (ib_logfile0)
by default on Linux.

It turns out that especially with innodb_flush_trx_log_at_commit=2,
writing to the log via the file system cache typically improves throughput,
especially on slow storage or at a small number of concurrent transactions.
For other values of innodb_flush_log_at_trx_commit, direct writes were
observed to be mostly but not always faster. Whether it pays off to
disable the file system cache on the log may depend on the type of storage,
the workload, and the operating system kernel version.

On Linux and Microsoft Windows, we will introduce the settable Boolean
global variable innodb_log_file_buffering that indicates whether the
file system cache on the redo log file is enabled. The default value is
innodb_log_file_buffering=OFF. If the server is started up with
innodb_flush_log_at_trx_commit=2, the value will be changed to
innodb_log_file_buffering=ON.

When a persistent memory interface is being used for the log,
the value cannot be changed from innodb_log_file_buffering=OFF.
On Linux, when the physical block size cannot be determined
to be a power of 2 between 64 and 4096 bytes, the file system cache
cannot be disabled, and innodb_log_file_buffering=ON cannot be changed.

Server log messages will indicate whether the file system cache is
enabled for the redo log:

[Note] InnoDB: Buffered log writes (block size=512 bytes)
[Note] InnoDB: File system buffers for log disabled (block size=512 bytes)

After this change, the startup parameter innodb_flush_method will no
longer control whether O_DIRECT will be set on the redo log on Linux.

On other operating systems that support O_DIRECT, no interface has been
implemented for controlling the file system cache for the redo log.
The innodb_flush_method values O_DIRECT, O_DIRECT_NO_FSYNC, O_DSYNC
will enable O_DIRECT for data files, not the log.

Tested by: Matthias Leich, Axel Schwenke

trx_purge_free_segment(): Do mark that the block will be modified.

It seems possible that this regression was introduced by the
changes to the page-freeing logic
in commit 4179f93d (MDEV-18976).

Tested by: Matthias Leich

- innodb_fts.sync_block doesn't make sense after MDEV-25581's patch
because fts cache syncing is done as a part of insert operation
and it leads to completion of select over insert sometimes.
This test case is not relevant any more

In commit 73fee39e (MDEV-27985)
a regression was introduced that would cause bpage=nullptr to
be referenced.

buf_flush_LRU_list_batch(): Always terminate the loop upon
encountering a null pointer.

MDEV-26127 Assertion `err != DB_DUPLICATE_KEY' failed or InnoDB: Failing assertion: id != 0 on ALTER ... REBUILD PARTITION

During rebuild of partition, the partitioning engine calls
alter_close_table(), which does not unlock and close some table
instances of the target table.
Then, the engine fails to rename partitions because there are table
instances that are still locked.

Closing all the table instance of the target table fixes the bug.

innodb_drop_database(): Use explicit TO_BINARY casts on
SYS_TABLES.NAME, which for historical reasons uses the wrong collation
latin1_swedish_ci instead of BINARY.

These had been damaged during some merge which caused --embedded
test to fail

MariaDB codebase is huge and Lintian has lots of test than
can fire false-positive warnings which leads to situation
where real problems can't be spotted.

Suspend obvious false-positive Lintian warnings and
let Lintian problems that needs some love shine
out.

Suspends in package mariadb-test-data

Supporting BSD family needs to use '/usr/bin/env perl' and not '/usr/bin/perl'
Perl script are for testing and not for production in mariadb-test-data
package:
 * incorrect-path-for-interpreter

There is several files with national-encoding which are test file so they
can't be in unicode charset
 * national-encoding

Serveral test paths are intentionally repeated:
 * repeated-path-segment

Suspends in package mariadb-test

Supporting BSD family needs to use '/usr/bin/env perl' and not '/usr/bin/perl'
Perl script are for testing and not for production in mariadb-test-data
package:
 * incorrect-path-for-interpreter

Suspends in package source package

Remade some 'version-substvar-for-external-package' to use
regex.

MGroonga is missing source file 'jquery-ui-1.8.18.custom.js' correct
lintian suspend with regex:
 * source-is-missing

There is several files with very long line lenghts. Add suspends
for those that can't be corrected in several places. Most
of them are test result files, SQL test files or intentional
long lines that can't be splitted.
  * very-long-line-length-in-source-file

There is several autogenerated C++ files which probably should not
be there but they should not do any harm:
 * source-contains-autogenerated-visual-c++-file

Problem:
========
  InnoDB FTS requesting the fts sync of the table once the fts
cache size reaches 1/10 of innodb_ft_cache_size. But fts_sync()
releases cache lock when writing the word. By doing this, InnoDB
insert thread increases the innodb fts cache memory and
SYNC operation will take more time to complete.

Solution:
=========
  Remove the fts sync operation(FTS_MSG_SYNC_TABLE) from
the fts optimize background thread. Instead of that,
allow user thread to sync the InnoDB fts cache when
the cache size exceeds 512 kb. User thread holds
cache lock while doing cache syncing, it make sure that
other threads doesn't add the docs into the cache.

Removed FTS_MSG_SYNC_TABLE and its related function
because we do remove the FTS_MSG_SYNC_TABLE message
itself.

Removed fts_sync_index_check() and all related
function because other threads doesn't add while
cache operation going on.

This fixes up commit 3d241eb9

In commit f80deb95 (MDEV-27868)
a fix for a correctness regression caused a performance regression
by increasing the amount of work that is executed while holding
buf_pool.flush_list_mutex.

buf_page_t::set_temp_modified(): Relax an assertion, to allow an
already dirty block to be marked as dirty.

buf_page_t::flush_list_requests: Note that the variable is not
always protected by buf_pool.flush_list_mutex. Already dirty blocks
that are being written to will increment the counter without
holding buf_pool.flush_list_mutex.

mtr_t::process_freed_pages(): Handle pages that were freed during
the execution of the mini-transaction.

ReleaseUnlogged, mtr_t::release_unlogged(): Release modified pages when
no log was written. This is for pages of the temporary tablespace, or for
IMPORT TABLESPACE.

ReleaseModified: Renamed from ReleaseBlocks.
Assume that buf_pool.flush_list_mutex was acquired by the caller.

ReleaseSimple: A combination of ReleaseLatches and ReleaseModified,
for the case that for any modified pages, some earlier modifications
are already waiting to be written.

mtr_t::commit(): Invoke one of release_unlogged(), ReleaseModified,
ReleaseSimple, ReleaseAll. Acquire and release buf_pool.flush_list_mutex
at most once.

memo_slot_release(): Simplify the code.

mtr_t::sx_latch_at_savepoint(), mtr_t::x_latch_at_savepoint():
Reduce the size of the critical section.

fil_space_t::update_last_freed_lsn(), fil_space_t::clear_freed_ranges(),
fil_space_t::add_free_range(): Assume that freed_range_mutex is held
by the caller.

buf_pool_t::prepare_insert_into_flush_list(): Determine the insert
position for buf_pool_t::insert_into_flush_list(). Remove any clean
blocks from buf_pool.flush_list that were encountered while searching.

buf_pool_t::insert_into_flush_list(): Insert the block at the
predetermined position.