Made the test order_by_pack_big stable

An assertion added in commit c20bf8fd
includes a sign mismatch. Make the affected data members unsigned.

Since commit f52bf920 the type
Sql_sort is non-trivial, because it includes a data member
Bounds_checked_array<SORT_FIELD> local_sortorder.
There still is no vtable, so memset() is safe to invoke, but
we must add a cast to silence a warning in GCC 8 or later.

MDEV-21541: main.sum_distinct-big fails with Assertion `m_buffer_end == __null || end <= m_buffer_end'

For Merge_chuck structures first set the start and end positions of the buffer and
then adjust the end positions of the buffer if the records are dynamic in nature.

- Adding a way to define I_S columns without DEFAULT.
- Fixing all ENUM columns in I_S.INNODB* table definitions to have no DEFAULT.

maybe_wake_or_create_thread()

A task that is executed,could be counted as waiting (after wait_begin()
before wait_end()) or as long-running (callback runs for a long time).

If task is both marked waiting and long running, then calculation of
current concurrency (# of executing tasks - # of long tasks - #of waiting tasks)
is wrong, as task is counted twice.

Thus current concurrency could go negative, but with unsigned arithmetic
it will become a huge number.

As a result, maybe_wake_or_create_thread() would neither wake or create
a thread, when it should. Which may result in a deadlock.

Someone tried -DBUILD_SHARED_LIBS:BOOL=ON. This did not work well

threadpool has to know about long waits, to be able to create or wakeup
new threads, or it can become slow, if it just relies on maintainence
timer.

The merge commit ded128aa failed to adjust
commit 9d18b624 in debian/rules.
We must adjust the build path, which is derived from the package name,
which contains the major version number.

This task deals with packing the non-sorted fields (or addon fields).
This would lead to efficient usage of the memory allocated for the sort buffer.
The changes brought by this feature are
  1) Sort buffers would have records of variable length
  2) Each record in the sort buffer would be stored like
     <sort_key1><sort_key2>....<addon_length><null_bytes><field1><field2>....
     addon_length is the extra bytes that are required to store the variable
     length of addon field across different records.
  3) Changes in rr_unpack_from_buffer and rr_from_tempfile to take into account
     the variable length of records.

Ported  WL#1509 Pack values of non-sorted fields in the sort buffer from
MySQL by Tor Didriksen

This was forgotten in e9de6386

Add mutex protection while we calculate required slave
thread change and create them. Add error handling.

The MDEV-17062 fix in commit c4195305
was omitted.

row_search_idx_cond_check with rowid_filter upon concurrent access to table

This bug has nothing to do with the concurrent access to table. Rather it
concerns queries for which the optimizer decides to employ a rowid filter
when accessing an InnoDB table by a secondary index, but later when
calling test_if_skip_sort_order() changes its mind to access the table by
the primary key.
Currently usage of rowid filters is not supported in InnoDB if the table
is accessed by the primary key. So in this case usage of a rowid filter
to access the table must be prohibited.

followup for 9d18b624

strangely enough it only failed upgrade test on eoan

This bug could happen when both optimizer switches 'mrr' and 'mrr_sort_keys'
are enabled and the optimizer decided to use a rowid filter when
accessing an InnoDB table by a secondary key. With the above setting
any access by a secondary is converted to the rndpos access. In InnoDB the
rndpos access uses the primary key.
Currently usage of a rowid filter within InnoDB engine is not supported
if the table is accessed by the primary key.

Do not use pushed rowid filter if the table is accessed actually by the
primary key. Use the rowid filter outside the egine code instead.

log_group_max_size: is not needed because redo log do not use fil_io() now

Make chown error non-fatal. Just print a warning.

don't let mysql_install_db set SUID bit for auth_pam_tool in rpm/deb
packages - instead package files with correct permissions and
only fix the ownership of auth_pam_tool_dir (which can only be done
after mysql user is created, so in post-install).

keep old mysql_install_db behavior for bintars

if my_realpath() fails, don't return the error code, get_defaults_options()
returns a number of options consumed, not 0=ok/1=error.

instead, ignore the error from my_realpath. If it fails it internally
falls back to my_load_path, which restores 10.4- behavior

- Fixed a warning visible in optimized build related to calling
memcpy with length parameters larger than ptrdiff_t max.

rb#23333 approved by Annamalai Gurusami <annamalai.gurusami@oracle.com>

IndexPurge::next(): Replace btr_pcur_move_to_next_user_rec()
with some equivalent code that performs sanity checks without
killing the server. Perform some additional sanity checks as well.

This change is motivated by
mysql/mysql-server@48de4d74f4d2f10cd01b129753c7dfa908cf36b5
which unnecessarily introduces storage overhead to btr_pcur_t
and uses a test case that injects a fault somewhere else,
not in the code path that was modified.

MySQL 5.7.29 includes the following fix:
Bug #30287668 INNODB: A LONG SEMAPHORE WAIT
mysql/mysql-server@5cdbb22b51cf2b35dbdf5666a251ffbec2f84dec

There is no test case. It seems that the problem could occur when
a spatial index is large and peculiar enough so that multiple R-tree
leaf pages will have the exactly same maximum bounding rectangle (MBR).

The commit message suggests that the hang can occur when R-tree
non-leaf pages are being merged, which should only be possible
during transaction rollback or the purge of transaction history,
when the R-tree index is at least 2 levels high and very many records
are being deleted. The message says that a comparison result that two
spatial index node pointer records are equal will cause an infinite loop
in rtr_page_copy_rec_list_end_no_locks(). Hence, we must include the
child page number in the comparison to be consistent with
mysql/mysql-server@2e11fe0e152e34d73579e1a9ec19aedc3f6010f6.

We fix this bug in a simpler way, involving fewer code changes.

cmp_rec_rec(): Renamed from cmp_rec_rec_with_match().
Assert that rec2 always resides in an index page.
Treat non-leaf spatial index pages specially.

Now that we will be invoking dtuple_get_n_ext() instead of
letting btr_push_update_extern_fields() update an already
calculated value, it is unnecessary to calculate the n_ext
upfront.

row_rec_to_index_entry(), row_rec_to_index_entry_low():
Remove the output parameter n_ext.

During update, rollback, or MVCC read, we may miscalculate
the number of off-page columns, and thus the size of the
clustered index record. The function btr_push_update_extern_fields()
is mostly redundant, because the off-page columns would also be
moved by row_upd_index_replace_new_col_val(), which is invoked
via row_upd_index_replace_new_col_vals().

btr_push_update_extern_fields(): Remove.

This is based on
mysql/mysql-server@1fa475b85d24de4b9ce2958c0eed738c221fc82c
which refines a fix for a recovery bug fix
mysql/mysql-server@ce0a1e85e24e48b8171f767b44330da635a6ea0a
in MySQL 5.7.5.

No test case was provided by Oracle.
Some of the changed code is being covered by the existing test
innodb.blob-crash.

WL#6326 in MariaDB 10.2.2 introduced a potential hang on purge or rollback
when an index tree is being shrunk by multiple levels.

This fix is based on
mysql/mysql-server@f2c58526300c0d84837effa26d37cbd5d2694967
with the main difference that our version of the test case uses
DEBUG_SYNC instrumentation on ROLLBACK, not on purge.

btr_cur_will_modify_tree(): Simplify the check further.
This is the actual bug fix.

row_undo_mod_remove_clust_low(), row_undo_mod_clust(): Add DEBUG_SYNC
instrumentation for the test case.