MDEV-27919 mariabackup --log-copy-interval is measured in millisecondss in 10.5 and in microseconds in 10.6

Multiply polling interval by 1000.

I missed include/no_protocol.inc already existed.

aarch64 RHEL7 and Centos7 internal compiler error here, so we carry
another pragma enchantment like buf0lru.cc and row0ins.cc.

Remove Warning that occured by doing an ALTER TABLE ... ORDER
BY on an InnoDB table.

Reviewed by Brandon Nesterenko

The --skip-write-binlog message was confusing that it only had
an effect if the galera was enabled. There are uses beyond galera
so we apply SET SESSION SQL_LOG_BIN=0 as implied by the option
without being conditional on the wsrep status.

We also with --skip-write-binlog actually check the session @@WSREP_ON
variable rather than the global server variable.

Since 10.6, the wsrep_mode could replicate Aria and MyISAM, in which
case no change to innodb and back is needed.

By removing the conditions, we can use LOCK TABLES in a general case
improving the load speed of Aria (MDEV-23326), regardless of the
skip-write-binlog flag. The only case where we don't use LOCK TABLES is
when we are replicating via Innodb, because wsrep_on=1 and wsrep_mode
doesn't contain REPLICATE_ARIA{,MYISAM}. This uses an Innodb transaction
instead. When replicating via InnoDB we change the table engine type
back to what it was originally.

By removing the \d and other syntax that requires parsing by
the mariadb client, we can use the generated SQL more generally, like
in the embedded server.

We also save and restore the SQL_LOG_BIN and WSREP_ON session server
variables so this can be included in the same session as other data
without taking into changes in state.

Remove wsrep.mysql_tzinfo_to_sql_symlink{,_skip} tests as they offered
no additional coverage beyond main.mysql_tzinfo_to_sql_symlink (no
server testing was done).

Add galera.mariadb_tzinfo_to_sql to actually test the replication
of tzinfo data through galera.

The conditional executable comment around /*M!100602 ...
START TRANSACTION .. LOCK TABLES.. */ is so that we can provide tzinfo
files (MDEV-27113, MDBF-389) and in the case that a user uses it on a
pre-10.6 server version it will still work. Both START TRANSACTION and
LOCK TABLES are not supported in prepared statements in MariaDB versions
earlier than 10.6.2.

Reviewed by Brandon Nesterenko

- While loading the indexes, InnoDB fails to clear the index if the
system record has TEMP_INDEX_PREFIX_STR. This lead to ASAN failure.
The leak was introduced by commit cc2ddde4
(MDEV-18518)

Let us use the common pthread_t wrapper for Microsoft Windows.

This fixes up commit dbe941e0

MDEV-9948 Failing assertion: new_state->key_version != ENCRYPTION_KEY_VERSION_INVALID in fil0crypt.cc

The test encryption.innodb-redo-nokeys did not actually test
recovery without valid keys, because due to the setting
innodb_encrypt_tables, InnoDB refused to start up at all,
without even attempting any crash recovery.

fil_ibd_load(): If the encryption key is not available,
refuse to load the file.

Let us replace all use of MY_ALIGNED in InnoDB with C++11 alignas.

CACHE_LINE_SIZE: Replaced with CPU_LEVEL1_DCACHE_LINESIZE.

A few PERFORMANCE_SCHEMA instrumentation keys were not exposed
in all_innodb_mutexes[]. Let us remove them.

The keys fts_pll_tokenize_mutex_key and read_view_mutex_key were
internally used. Let us make ReadView::m_mutex use the simpler
and smaller srw_mutex, hoping to improve memory access patterns.

The element mutex is unnecessarily large. The PERFORMANCE_SCHEMA
instrumentation was not even enabled.

trx_lock_t: Remove byte pad[256] and use
alignas(CPU_LEVEL1_DCACHE_LINESIZE) instead.

trx_t: Declare n_ref (the first member) aligned at cache line.

Pool: Assert that the sizes are multiples of
CPU_LEVEL1_DCACHE_LINESIZE, and invoke an aligned allocator.

per Marko request

Access the all_charsets[] array directly in a hot loop.
This avoids get_charset() that increments a shared counter via
my_collation_statistics_inc_use_count(), causing a scalability issue.

Instead, call get_charset() when a table is opened (and InnoDB
fills in dtype_t values) - this is enough, as charset is marked
ready (MY_CS_READY) only once, on the first get_charset() call,
after that it can be accessed directly.

This also fixes a potential bug in InnoDB. It used to access charsets
directly when filling dtype_t values. This wasn't preceded by any
get_charset() call inside InnoDB. Normally the server would call
get_charset() on reading the frm, but if the table was first opened
from a purge thread InnoDB could, theoretically, access a not-ready
charset in innobase_get_cset_width().

on_table_fill_finished() should always be done at the end of open()
even if result is not Select_materialize but (for example)
Select_fetch_into_spvars.

The problem was not with the server behavior,
it was with wrong test results recorded.

Enabling both type_set and type_enum.

type_set.result was OK. The test did not fail.

type_enum.result was incorrectly recorded in this commit:
  eb483c51

Recording correct results.

Remove unndeeded paths from mariadb-server-10.6.postinst as
it's already in export PATH and it fires 'command-with-path-in-maintainer-script'
in lintian checks

- Purge thread is trying to access the table name when other thread
does rename of the table name. It leads to heap use after free error
by purge thread. purge thread should check whether the table name
is temporary while holding dict_sys.mutex.

dict_acquire_mdl_shared(): Invoke the correct variant
dict_table_t::parse_name<true>() also when trylock=true,
that is, we are being called from fts_optimize_sync_table().

Ever since commit 82b7c561 (MDEV-24258)
this code was prone to a hang. If another thread requested an
exclusive dict_sys.latch between the time
dict_acquire_mdl_shared<trylock=true>() acquired a shared dict_sys.latch
and dict_table_t::parse_name<false>() was trying to acquire another
shared dict_sys.latch, InnoDB would get into an infinite livelock
of threads, because the futex-based srw-lock implementation prioritizes
exclusive latch requests.

dict_table_t::parse_name(): Rename the template parameter dict_locked
into dict_frozen.

buf_page_t::set_state(): Relax a debug assertion. It is fine to update
a read-fixed block descriptor to be both read-fixed and buffer-fixed.

buf_pool_t::watch_unset(): Fix some incorrect logic that was implemented
in commit e9e6db93.

Thanks to Elena Stepanova for the test case.

The implementation calls SetServiceStatus() with updated
SERVICE_STATUS::dwHint and SERVICE_STATUS::dwCheckpoint

10.5 tests for MDEV-26507 Assertion `tmp != ((long long) 0x8000000000000000LL)' failed in TIME_from_longlong_datetime_packed

The fix for MDEV-27673 (in 10.3) fixed MDEV-26507 as well.
This patch only adds MTR tests.

10.6 tests for MDEV-26507 Assertion `tmp != ((long long) 0x8000000000000000LL)' failed in TIME_from_longlong_datetime_packed

The patch for MDEV-27673 (in 10.3) fixed MDEV-26507 as well.

This patch only adds 10.6 specific MTR tests for MDEV-26507.

use vio_type_names[] values as in MySQL

Noticed by Monty

MDEV-27239 Spider: Assertion `thd->transaction->stmt.ha_list == __null || trans == &thd->transaction->stmt' failed in ha_commit_trans on BEGIN WORK after FTWRL

The check on the SQL command type, in ha_spider::external_lock() is deleted
by e954d9de. This resulted in the wrong call of spider_internal_start_trx()
(and thus Ha_trx_info::register_ha()).

I reverted the check and refactored ha_spider::external_lock().

Rather than Debian logs containing a barely decipherable mix
of build command and the output of some other command, we
use the make option --output-sync=target. This make the compile
line and the output stay together in the output stream.

This option exists even in the make version in debian;stretch
so should work everywhere.

Test on debian:stretch, ubuntu:18.04, the two lowest version that
use the debian/rules.

Without doing the full build.

Autobake now includes a dependency on lsb-release.

As the BB CI images
(https://github.com/MariaDB/mariadb.org-tools/blob/master/buildbot.mariadb.org/ci_build_images/debian.Dockerfile)
have explicit dependencies, there's no point maintaining them in
two places. We don't want do the full autobake-deb.sh there, just enough
to have the control file containing the correct dependencies.

Helps: https://github.com/MariaDB/mariadb.org-tools/pull/130

This failure was caused by MDEV-25975, which removed the parameter
innodb_disallow_writes.

Added a check for wsrep_sst_disable_writes to the function
ibuf_merge_in_background().

As discovered by tracing, but also presenting in AIX fseeko
documentation, seeking beyond the EOF is acceptable, as you can write
there.

To display the same error in AIX to other implementations that return
errors on seek, we take the EFBIG error code on reading and error the
same way.

An AIX truss of an aspect of the test:

truss extra/innochecksum --page=18446744073709551615 $MYSQLD_DATADIR/test/tab1.ibd

  statx("./mysql-test/var/log/innodb_zip.innochecksum_3-4k,crc32,innodb/mysqld.1/data//test/tab1.ibd", 0x0FFFFFFFFFFFF610, 176, 010) = 0
  kopen("./mysql-test/var/log/innodb_zip.innochecksum_3-4k,crc32,innodb/mysqld.1/data//test/tab1.ibd", O_RDONLY|O_LARGEFILE) = 3
  kfcntl(3, 12, 0x00000001100006C8)               = 0
  kfcntl(3, F_GETFL, 0x00000001100A6CF8)          = 67108864
  kioctl(3, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
  klseek(3, 0, 1, 0x0FFFFFFFFFFFF3F0)             = 0
  kioctl(3, 22528, 0x0000000000000000, 0x0000000000000000) Err#25 ENOTTY
  kread(3, "DEADBEEF\0\0\0\0FFFFFFFF".., 4096)    = 4096
  klseek(3, 0, 1, 0x0FFFFFFFFFFFF450)             = 0
  klseek(3, 17592186040320, 0, 0x0FFFFFFFFFFFF450) = 0
  klseek(3, 0, 1, 0x0FFFFFFFFFFFF3F0)             = 0
  kread(3, "DEADBEEF\0\0\0\0FFFFFFFF".., 4096)    Err#27 EFBIG

An equivalent Linux trace:

ltrace extra/innochecksum --page=18446744073709551615 $MYSQLD_DATADIR/test/tab1.ibd

  stat64(0x7fff10ea2dc3, 0x7fff10ea0670, 88, 0x8026be41)         = 0
  open64("./mysql-test/var/log/innodb_zip."..., 0, 02072403160)  = 3
  fcntl64(3, 6, 0x139f180, 1)                                    = 0
  fgetpos64(0x615000000080, 0x7fff10ea0760, 1, 0)                = 0
  fseeko64(0x615000000080, 0xffffffff000, 0, 5 <unfinished ...>
  pthread_getspecific(0, 0x4d0eb8, 0x7fff10ea0490, 0)            = 0x7f7b2806d000
  <... fseeko64 resumed> )                                       = 0
  fgetpos64(0x615000000080, 0x7fff10ea0760, 1, 1)                = 0
  feof(0x615000000080)                                           = 0
  feof(0x615000000080)                                           = 1
  Error: Unable to seek to necessary offset

MDEV-27783 InnoDB: Failing assertion: table->get_ref_count() == 0 upon ALTER TABLE ... MODIFY COLUMN

- There is a race condition occurs between purge thread and DDL.
So purge thread can increment n_ref_count even after DDL does
purge_sys_t::stop_FTS().

- dict_table_open_on_id for purge thread should check
purge_sys.must_wait_FTS() before acquring the table.

- purge_sys.stop_FTS() does acquire dict_sys.latch for setting
the purge system flag and check table ref count on auxilary tables.

The Shared/Update/Exclusive locks that were introduced in
commit 03ca6495 (MDEV-24142)
did not work correctly when a futex-like system call interface
was not available.

On all tested implementations (IBM AIX as well as FreeBSD and GNU/Linux
with the futex interface artificially disabled), the old implementation
would cause hangs in some SPATIAL INDEX tests (innodb_gis suite).
On FreeBSD, a hang was also observed in an encryption test.

We will simply emulate the futex system calls with a single mutex
and two condition variables, one for each wait queue. The condition
variables basically shadow the lock words and are used as wait queues,
just like the futex system calls would be.

The storage overhead of ssux_lock_impl will be increased by 32 bits
when using SUX_LOCK_GENERIC. Compared to the futex-based implementation,
the SUX_LOCK_GENERIC implementation has an overhead of
sizeof(pthread_mutex_t)+2*sizeof(pthread_cond_t).

rw_lock: Remove all SUX_LOCK_GENERIC extensions.

pthread_mutex_wrapper: A simple wrapper of pthread_mutex that
implements srw_spin_mutex and srw_mutex for SUX_LOCK_GENERIC.

srw_mutex_impl: Define this also for SUX_LOCK_GENERIC, but in
that case add the fields mutex, cond.

ssux_lock_impl: Define for SUX_LOCK_GENERIC with a minimal difference:
adding readers_cond.