buf_pool_t::page_hash_contains(): Check if a page is cached.

buf_read_ahead_random(), buf_read_page_background(),
buf_read_ahead_linear(): Before invoking buf_read_page_low(),
preallocate a buffer page for the read request.

buf_read_page(), buf_page_init_for_read(), buf_read_page_low():
Add a parameter for the buf_pool.page_hash chain, to avoid duplicated
computations.

buf_page_t::read_complete(): Only attempt recovery if an uncompressed
page frame has been allocated.

buf_page_init_for_read(): Before trying to acquire buf_pool.mutex, acquire
an exclusive buf_pool.page_hash latch and check if the page is already
located in the buffer pool. If the buf_pool.mutex is not immediately
available, release both latches and acquire them in the correct order,
and then recheck if the page is already in the buffer pool. This should
hopefully reduce some contention on buf_pool.mutex.

buf_page_init_for_read(), buf_read_page_low(): Input the "recovery needed"
flag in the least significant bit of zip_size.

buf_read_acquire(), buf_read_release(): Interface for allocating and
freeing buffer pages for reading.

buf_read_recv_pages(): Set the flag that recovery is needed.
Other ROW_FORMAT=COMPRESSED reads during recovery
will not need any recovery.

Fall back to using .mysql_history if .mariadb_history isn't
present and .mysql_history is present.

Also replace the use of MYSQL_HISTFILE as an environment variable
with MARIADB_HISTFILE and fall back to using MYSQL_HISTFILE if
MARIADB_HISTFILE isn't present.

In commit 24648768, some use of
O_DIRECT was added without proper #ifdef guard. That broke the
compilation in environments that do not define O_DIRECT, such as
OpenBSD.

log_slow_filter=admin as been available for a long time.

Uses can migrate from log_slow_statements_statements=OFF by removing
'admin' from the default log_slow_filter variable setting.

We do not want people to try to rebase their patch on anything
prior to the earliest *maintained* version. (Currently, 10.3)

try to make them less confusing for users.
Hopefully, if the version string will be changed like

- mariadb Ver 15.1 Distrib 10.11.2-MariaDB for Linux (x86_64)
+ mariadb from 10.11.2-MariaDB, client 15.1 for Linux (x86_64)

users will be less inclined to reply "15.1" to the question
"what mariadb version are you using?"

it should now always be

/path/to/exe Ver <tool version> Distrib <server version> for <OS> (<ARCH>)

in all tools and clients

Partial revert of d29d9157.

It seems from CI that lower case RPM names are generated without
explictly setting the package name to mixed case.

Starting with commit baf276e6 (MDEV-19229)
the parameter innodb_undo_tablespaces can be increased from its
previous default value 0 while allowing an upgrade from old databases.

We will change the default setting to innodb_undo_tablespaces=3
so that the space occupied by possible bursts of undo log records
can be reclaimed after SET GLOBAL innodb_undo_log_truncate=ON.

We will not enable innodb_undo_log_truncate by default, because it
causes some observable performance degradation.

Special thanks to Thirunarayanan Balathandayuthapani for diagnosing
and fixing a number of bugs related to this new default setting.

Tested by: Matthias Leich, Axel Schwenke, Vladislav Vaintroub
(with both values of innodb_undo_log_truncate)

node->is_delete was incorrectly set to NO_DELETE for a set of operations.

In general we shouldn't rely on sql_command and look for more abstract ways
to control the behavior.

trg_event_map seems to be a suitable way. To mind replica nodes, it is ORed
with slave_fk_event_map, which stores trg_event_map when replica has
triggers disabled.

If installation directory is not empty, MSI would shows a popup informing
user about it. The error message is stored as MSI property
INSTALLDIRERROR.

Fixed the bug that the error was not cleared, when the user changes
directory in installer UI.

Signed-off-by: lilinjie <lilinjie@uniontech.com>

This only happens with 'timestamp_column IN (select ...)

The reason was a missing assignment in Item_cache_timestamp::cache_value()

MDEV-25277: mysqlbinlog --verbose cannot read row events with compressed columns: Don't know how to handle column type: 140

Problem:
=======
Mysqlbinlog cannot show the type of a compressed
column when two levels of verbosity is provided.

Solution:
========
Extend the log event printing logic to handle and
tag compressed types.

Behavioral Changes:
==================
  Old: When mysqlbinlog is called in verbose mode and
the database uses compressed columns, an error is
returned to the user.

  New: The output will append “ COMPRESSED” on the
type of compressed columns

Reviewed By
===========
Andrei Elkin <andrei.elkin@mariadb.com>

This issue was caused by the bug fix for
MDEV-30325 Wrong result upon range query using index condition

The bug could happen in the case of several overlapping key ranges
with OR

InnoDB tables that lack a primary key (and any UNIQUE INDEX whose
all columns are NOT NULL) will use an internally generated index,
called GEN_CLUST_INDEX(DB_ROW_ID) in the InnoDB data dictionary,
and hidden from the SQL layer.

The 48-bit (6-byte) DB_ROW_ID is being assigned from a
global sequence that is persisted in the DICT_HDR page.

There is absolutely no reason for the DB_ROW_ID to be globally
unique across all InnoDB tables.

A downgrade to earlier versions will be prevented by the file format
change related to removing the InnoDB change buffer (MDEV-29694).

DICT_HDR_ROW_ID, dict_sys_t::row_id: Remove.

dict_table_t::row_id: The per-table sequence of DB_ROW_ID.

commit_try_rebuild(): Copy dict_table_t::row_id from the old table.

btr_cur_instant_init(), row_import_cleanup(): If needed, perform
the equivalent of SELECT MAX(DB_ROW_ID) to initialize
dict_table_t::row_id.

row_ins(): If needed, obtain DB_ROW_ID from dict_table_t::row_id.
Should it exceed the maximum 48-bit value, return DB_OUT_OF_FILE_SPACE
to prevent further inserts into the table.

dict_load_table_one(): Move a condition to btr_cur_instant_init_low()
so that dict_table_t::row_id will be restored also for
ROW_FORMAT=COMPRESSED tables.

Tested by: Matthias Leich

The purpose of the change buffer was to reduce random disk access,
which could be useful on rotational storage, but maybe less so on
solid-state storage.
When we wished to
(1) insert a record into a non-unique secondary index,
(2) delete-mark a secondary index record,
(3) delete a secondary index record as part of purge (but not ROLLBACK),
and the B-tree leaf page where the record belongs to is not in the buffer
pool, we inserted a record into the change buffer B-tree, indexed by
the page identifier. When the page was eventually read into the buffer
pool, we looked up the change buffer B-tree for any modifications to the
page, applied these upon the completion of the read operation. This
was called the insert buffer merge.

We remove the change buffer, because it has been the source of
various hard-to-reproduce corruption bugs, including those fixed in
commit 5b9ee8d8 and
commit 165564d3 but not limited to them.

A downgrade will fail with a clear message starting with
commit db14eb16 (MDEV-30106).

buf_page_t::state: Merge IBUF_EXIST to UNFIXED and
WRITE_FIX_IBUF to WRITE_FIX.

buf_pool_t::watch[]: Remove.

trx_t: Move isolation_level, check_foreigns, check_unique_secondary,
bulk_insert into the same bit-field. The only purpose of
trx_t::check_unique_secondary is to enable bulk insert into an
empty table. It no longer enables insert buffering for UNIQUE INDEX.

btr_cur_t::thr: Remove. This field was originally needed for change
buffering. Later, its use was extended to cover SPATIAL INDEX.
Much of the time, rtr_info::thr holds this field. When it does not,
we will add parameters to SPATIAL INDEX specific functions.

ibuf_upgrade_needed(): Check if the change buffer needs to be updated.

ibuf_upgrade(): Merge and upgrade the change buffer after all redo log
has been applied. Free any pages consumed by the change buffer, and
zero out the change buffer root page to mark the upgrade completed,
and to prevent a downgrade to an earlier version.

dict_load_tablespaces(): Renamed from
dict_check_tablespaces_and_store_max_id(). This needs to be invoked
before ibuf_upgrade().

btr_cur_open_at_rnd_pos(): Specialize for use in persistent statistics.
The change buffer merge does not need this function anymore.

btr_page_alloc(): Renamed from btr_page_alloc_low(). We no longer
allocate any change buffer pages.

btr_cur_open_at_rnd_pos(): Specialize for use in persistent statistics.
The change buffer merge does not need this function anymore.

row_search_index_entry(), btr_lift_page_up(): Add a parameter thr
for the SPATIAL INDEX case.

rtr_page_split_and_insert(): Specialized from btr_page_split_and_insert().

rtr_root_raise_and_insert(): Specialized from btr_root_raise_and_insert().

Note: The support for upgrading from the MySQL 3.23 or MySQL 4.0
change buffer format that predates the MySQL 4.1 introduction of
the option innodb_file_per_table was removed in MySQL 5.6.5
as part of mysql/mysql-server@69b6241a79876ae98bb0c9dce7c8d8799d6ad273
and MariaDB 10.0.11 as part of 1d0f70c2.

In the tests innodb.log_upgrade and innodb.log_corruption, we create
valid (upgraded) change buffer pages.

Tested by: Matthias Leich

We introduce the following settable Boolean global variables:

innodb_log_file_write_through: Whether writes to ib_logfile0 are
write-through (disabling any caching, as in O_SYNC or O_DSYNC).

innodb_data_file_write_through: Whether writes to any InnoDB data files
(including the temporary tablespace) are write-through.

innodb_data_file_buffering: Whether the file system cache is enabled
for InnoDB data files.

All these parameters are OFF by default, that is, the file system cache
will be disabled, but any hardware caching is enabled, that is,
explicit calls to fsync(), fdatasync() or similar functions are needed.

On systems that support FUA it may make sense to enable write-through,
to avoid extra system calls.

If the deprecated read-only start-up parameter is set to one of the
following values, then the values of the 4 Boolean flags (the above 3
plus innodb_log_file_buffering) will be set as follows:

O_DSYNC:
innodb_log_file_write_through=ON, innodb_data_file_write_through=ON,
innodb_data_file_buffering=OFF, and
(if supported) innodb_log_file_buffering=OFF.

fsync, littlesync, nosync, or (Microsoft Windows specific) normal:
innodb_log_file_write_through=OFF, innodb_data_file_write_through=OFF,
and innodb_data_file_buffering=ON.

Note: fsync() or fdatasync() will only be disabled if the separate
parameter debug_no_sync (in the code, my_disable_sync) is set.

In mariadb-backup, the parameter innodb_flush_method will be ignored.

The Boolean parameters can be modified by SET GLOBAL while the
server is running. This will require reopening the ib_logfile0
or all currently open InnoDB data files.

We will open files straight in O_DSYNC or O_SYNC mode when applicable.
Data files we will try to open straight in O_DIRECT mode when the
page size is at least 4096 bytes. For atomically creating data files,
we will invoke os_file_set_nocache() to enable O_DIRECT afterwards,
because O_DIRECT is not supported on some file systems. We will also
continue to invoke os_file_set_nocache() on ib_logfile0 when
innodb_log_file_buffering=OFF can be fulfilled.

For reopening the ib_logfile0, we use the same logic that was developed
for online log resizing and reused for updates of
innodb_log_file_buffering.

Reopening all data files is implemented in the new function
fil_space_t::reopen_all().

Reviewed by: Vladislav Vaintroub
Tested by: Matthias Leich

Before commit 6112853c in MySQL 4.1.1
introduced the parameter innodb_file_per_table, all InnoDB data was
written to the InnoDB system tablespace (often named ibdata1).
A serious design problem is that once the system tablespace has grown to
some size, it cannot shrink even if the data inside it has been deleted.

There are also other design problems, such as the server hang MDEV-29930
that should only be possible when using innodb_file_per_table=0 and
innodb_undo_tablespaces=0 (storing both tables and undo logs in the
InnoDB system tablespace).

The parameter innodb_change_buffering was deprecated
in commit b5852ffb.
Starting with commit baf276e6
(MDEV-19229) the number of innodb_undo_tablespaces can be increased,
so that the undo logs can be moved out of the system tablespace
of an existing installation.

If all these things (tables, undo logs, and the change buffer) are
removed from the InnoDB system tablespace, the only variable-size
data structure inside it is the InnoDB data dictionary.

DDL operations on .ibd files was optimized in
commit 86dc7b4d (MDEV-24626).
That should have removed any thinkable performance advantage of
using innodb_file_per_table=0.

Since there should be no benefit of setting innodb_file_per_table=0,
the parameter should be deprecated. Starting with MySQL 5.6 and
MariaDB Server 10.0, the default value is innodb_file_per_table=1.

This commit adds support for the --aria-log-dir-path
option on the command line and for the aria-log-dir-path
option in the configuration file to the SST scripts, since
before this change these parameters were completely ignored
during SST - SST scripts assumed that aria logs files are
always located in the same directory as logs for innodb.

Tests for this change will be added as a separate commit,
along with tests for MDEV-30157 and MDEV-28669.

This fix adds separate handling for "undo*" files that contain undo
logs as part of innodb files and adds a filter for undo* to the main
filter used when initially transferring files with rsync.

This commit adds even more correct handling of parameters
with paths when they contain leading or trailing spaces and/or
slashes. Also it fixes problems that occur when the user specified
explicit paths to additional directories, but these paths match
the specified path of the data directory - in this case, additional
subdirectories should be treated (in relation to the data directory)
in the same way as if these paths were not specified or as if they
are implicitly specified as "." or "./". But prior to this fix,
existing code treated any values as if they were completely
separate directories, whether or not they actually point to the
same location to which datadir points to - and this sometimes
resulted in incorrect file transfers.

This fix does not contain separate tests, as tests will be
part of the main commit(s). This fix has been made as a separate
commit to facilitate review for major substantive fixes related
to MDEV-30157 and MDEV-28669.

(Variant 3, initial variant was by Rex Jonston)

A LEFT JOIN with a constant as a column of the inner table produced wrong
query result if the optimizer had to write the inner table column into a
temp table. Query pattern:

  SELECT ...
  FROM (SELECT /*non-mergeable select*/
        FROM t1 LEFT JOIN (SELECT 'Y' as Val) t2 ON ...) as tbl

Fixed this by adding Item_direct_view_ref::save_in_field() which follows
the pattern of Item_direct_view_ref's save_org_in_field(),
save_in_result_field() and val_XXX() functions:
* call check_null_ref() and handle NULL value
* if we didn't get a NULL-complemented row, call Item_direct_ref's function.

lock_rec_has_to_wait(): Remove the unused parameter for_locking
that had been originally added
in commit df4dd593