This task deals with packing the non-sorted fields (or addon fields).
This would lead to efficient usage of the memory allocated for the sort buffer.
The changes brought by this feature are
  1) Sort buffers would have records of variable length
  2) Each record in the sort buffer would be stored like
     <sort_key1><sort_key2>....<addon_length><null_bytes><field1><field2>....
     addon_length is the extra bytes that are required to store the variable
     length of addon field across different records.
  3) Changes in rr_unpack_from_buffer and rr_from_tempfile to take into account
     the variable length of records.

Ported  WL#1509 Pack values of non-sorted fields in the sort buffer from
MySQL by Tor Didriksen

This was forgotten in e9de6386

Add mutex protection while we calculate required slave
thread change and create them. Add error handling.

The MDEV-17062 fix in commit c4195305
was omitted.

row_search_idx_cond_check with rowid_filter upon concurrent access to table

This bug has nothing to do with the concurrent access to table. Rather it
concerns queries for which the optimizer decides to employ a rowid filter
when accessing an InnoDB table by a secondary index, but later when
calling test_if_skip_sort_order() changes its mind to access the table by
the primary key.
Currently usage of rowid filters is not supported in InnoDB if the table
is accessed by the primary key. So in this case usage of a rowid filter
to access the table must be prohibited.

followup for 9d18b624

strangely enough it only failed upgrade test on eoan

This bug could happen when both optimizer switches 'mrr' and 'mrr_sort_keys'
are enabled and the optimizer decided to use a rowid filter when
accessing an InnoDB table by a secondary key. With the above setting
any access by a secondary is converted to the rndpos access. In InnoDB the
rndpos access uses the primary key.
Currently usage of a rowid filter within InnoDB engine is not supported
if the table is accessed by the primary key.

Do not use pushed rowid filter if the table is accessed actually by the
primary key. Use the rowid filter outside the egine code instead.

log_group_max_size: is not needed because redo log do not use fil_io() now

Make chown error non-fatal. Just print a warning.

don't let mysql_install_db set SUID bit for auth_pam_tool in rpm/deb
packages - instead package files with correct permissions and
only fix the ownership of auth_pam_tool_dir (which can only be done
after mysql user is created, so in post-install).

keep old mysql_install_db behavior for bintars

if my_realpath() fails, don't return the error code, get_defaults_options()
returns a number of options consumed, not 0=ok/1=error.

instead, ignore the error from my_realpath. If it fails it internally
falls back to my_load_path, which restores 10.4- behavior

- Fixed a warning visible in optimized build related to calling
memcpy with length parameters larger than ptrdiff_t max.

rb#23333 approved by Annamalai Gurusami <annamalai.gurusami@oracle.com>

IndexPurge::next(): Replace btr_pcur_move_to_next_user_rec()
with some equivalent code that performs sanity checks without
killing the server. Perform some additional sanity checks as well.

This change is motivated by
mysql/mysql-server@48de4d74f4d2f10cd01b129753c7dfa908cf36b5
which unnecessarily introduces storage overhead to btr_pcur_t
and uses a test case that injects a fault somewhere else,
not in the code path that was modified.

MySQL 5.7.29 includes the following fix:
Bug #30287668 INNODB: A LONG SEMAPHORE WAIT
mysql/mysql-server@5cdbb22b51cf2b35dbdf5666a251ffbec2f84dec

There is no test case. It seems that the problem could occur when
a spatial index is large and peculiar enough so that multiple R-tree
leaf pages will have the exactly same maximum bounding rectangle (MBR).

The commit message suggests that the hang can occur when R-tree
non-leaf pages are being merged, which should only be possible
during transaction rollback or the purge of transaction history,
when the R-tree index is at least 2 levels high and very many records
are being deleted. The message says that a comparison result that two
spatial index node pointer records are equal will cause an infinite loop
in rtr_page_copy_rec_list_end_no_locks(). Hence, we must include the
child page number in the comparison to be consistent with
mysql/mysql-server@2e11fe0e152e34d73579e1a9ec19aedc3f6010f6.

We fix this bug in a simpler way, involving fewer code changes.

cmp_rec_rec(): Renamed from cmp_rec_rec_with_match().
Assert that rec2 always resides in an index page.
Treat non-leaf spatial index pages specially.

Now that we will be invoking dtuple_get_n_ext() instead of
letting btr_push_update_extern_fields() update an already
calculated value, it is unnecessary to calculate the n_ext
upfront.

row_rec_to_index_entry(), row_rec_to_index_entry_low():
Remove the output parameter n_ext.

During update, rollback, or MVCC read, we may miscalculate
the number of off-page columns, and thus the size of the
clustered index record. The function btr_push_update_extern_fields()
is mostly redundant, because the off-page columns would also be
moved by row_upd_index_replace_new_col_val(), which is invoked
via row_upd_index_replace_new_col_vals().

btr_push_update_extern_fields(): Remove.

This is based on
mysql/mysql-server@1fa475b85d24de4b9ce2958c0eed738c221fc82c
which refines a fix for a recovery bug fix
mysql/mysql-server@ce0a1e85e24e48b8171f767b44330da635a6ea0a
in MySQL 5.7.5.

No test case was provided by Oracle.
Some of the changed code is being covered by the existing test
innodb.blob-crash.

WL#6326 in MariaDB 10.2.2 introduced a potential hang on purge or rollback
when an index tree is being shrunk by multiple levels.

This fix is based on
mysql/mysql-server@f2c58526300c0d84837effa26d37cbd5d2694967
with the main difference that our version of the test case uses
DEBUG_SYNC instrumentation on ROLLBACK, not on purge.

btr_cur_will_modify_tree(): Simplify the check further.
This is the actual bug fix.

row_undo_mod_remove_clust_low(), row_undo_mod_clust(): Add DEBUG_SYNC
instrumentation for the test case.

Add mutex protection while we calculate required slave
thread change and create them. Add error handling.

Old temporal data types (created with a pre-10.0 version of MariaDB)
are now displayed with a /* mariadb-5.3 */ comment in:

- SHOW CREATE TABLE
- DESCRIBE
- INFORMATION_SCHEMA.COLUMNS.COLUMN_TYPE

For example:

CREATE TABLE `t1` (
  `t0` datetime /* mariadb-5.3 */ DEFAULT NULL,
  `t6` datetime(6) /* mariadb-5.3 */ DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1

Note, new temporal data types are displayed without a format comment.

# Conflicts:
#	mysql-test/suite/galera/r/MW-388.result
#	mysql-test/suite/galera/t/MW-388.test
#	mysql-test/suite/innodb/r/truncate_inject.result
#	mysql-test/suite/innodb/t/truncate_inject.test
#	mysql-test/suite/rpl/r/rpl_stop_slave.result
#	mysql-test/suite/rpl/t/rpl_stop_slave.test
#	sql/sp_head.cc
#	sql/sp_head.h
#	sql/sql_lex.cc
#	sql/sql_yacc.yy
#	storage/xtradb/buf/buf0dblwr.cc

# Conflicts:
#	sql/sp_head.cc
#	sql/sql_select.cc
#	sql/sql_trigger.cc

followup for 3a3605f4

* It uses stack a lot, limit the recursion depth like
  we used to do for pcre.
* But don't do that for newer pcre2 that uses stack very little
  (and doesn't support limiting recursion depth anyway).
* Fix tests to verify only that deep recursion doesn't crash, but
  ignore results and warnings (which depend on pcre2 version)
* Also different pcre2 versions report different offset in error messages,
  take that into account.

Debian is apparently offended that pcre2-posix implements POSIX API,
thus it renames all posix-compatible symbols in libpcre2-posix to have the
PCRE2 prefix. But Debian doesn't do anything to pcre2posix.h header,
so any unaware application will get POSIX compatible type names
and function prototypes from pcre2, but actual symbols will come
from libc.

To remedy this enormous incongruity we have to redefine POSIX-compatible
function names in pcre2posix to match Debian's hack.

prevent potential stalls for when all slots are used up, and all io threads
are used up, and io callbacks are slow.

Add wait condition for event creation.