MDEV-15948 Fix error "Lost connection to MySQL server..." in test gal…

* Increased timeout counter in galera_wait_ready.inc

* Replaced useless include/wait_until_ready.inc after start_mysqld.inc
  in galera_st_*.inc with wait_condition on cluster size.

MDEV-15929 Fix lock wait timeout on `SELECT @@GLOBAL.WSREP_ON`

Test galera_sst_mysqldump often fails with error "2013: Lost connection
to MySQL server during query". The connection is lost after the test
restart one of the nodes. This happens because the server closes client
connections if it is joining a cluster through SST method mysqldump.
On unlucky runs of the test it is possible that mysqld is restarted,
and then mtr client is disconnected while it tries to determine if
galera is ready before going on with the test.
This patch rewrites galera_wait_ready.inc so that it is immune to
being disconnected.

This patch fixes a lock wait timeout error on `SELECT @@GLOBAL.WSREP_ON`
in `wait_wsrep_ready.inc`:

```
--connection node_2
...
--source include/kill_galera.inc

--connection node_1
--source include/wait_until_connected_again.inc # This includes wait_wsrep_ready.inc

```

The problem is that on node_2, kill_galera.inc may return before
the node is killed. So node_1 may still see that node_1 is alive
and will attempt to sync wait when doing those `SELECT` statements.
But sync wait is doomed to fail given that node_1 is killed, hence
the lock wait timeout.
One possible fix is to disable wsrep_sync_wait before including
wait_until_connected_again.
However, it appears that including wait_until_connected_again is
not necessary at all in node_1, so this patch removes it altogether.

fil_crypt_rotate_pages
	If tablespace is marked as stopping stop also page rotation

fil_crypt_flush_space
	If tablespace is marked as stopping do not try to read
	page 0 and write it back.

Fixed typo on select that is executed only when something unexpected
happens.

MDEV-12632: Source and destination overlap in memcpy, encryption.innodb-discard-import-change fails in buildbot with valgrind

Use block->page.offset for checking page number.

Problem was that we did not delete explain information
when Galera must replay a query.

Could not find easily repeatable test case that would not
cause other problems.

CIFS does not like O_DIRECT flag (it is set successfully, but pread would
fail).

The fix is not to use O_DIRECT, there is not need for it.
posix_fadvise() was used already that should prevent buffer cache
pollution on Linux.

As recommended by documentation of posix_fadvise(), we'll also fsync()
tablespaces after a batch of writes.

System tablespace can be specified with absolute path, if innodb_data_home_dir
is empty.

This was not handled well  by mariabackup

1. In backup phase, empty innodb_data_home_dir variable from server was
not recognized. full paths were stored in backup-my.ini, even if
it stored all files locally. thus prepare phase would not find the system
tablespace files.

2. In copy-back phase, copy would not be done to the absolute destination
path, as path would be stripped with trim_dotslash

This patch fixes the above defects.

MDEV-12632: Source and destination overlap in memcpy, encryption.innodb-discard-import-change fails in buildbot with valgrind

Problem was that if tablespace was encrypted we try to copy
also page 0 from read buffer to write buffer that are in
that case the same memory area.

fil_iterate
	When tablespace is encrypted or compressed its
        first page (i.e. page 0) is not encrypted or
	compressed and there is no need to copy buffer.

Wait until rotation has ended and shutdown before grep to make sure
that dirty pages are on datafiles.

Return early if plugin is not built.

MDEV-15804 Fix and re-enable MTR test galera.pxc-421

MDEV-15808 Fix and re-enable test galera.galera_gra_log

This patch makes two changes:

* Remove unnecessary `set SESSION wsrep_sync_wait=0`.
  Disabling `wsrep_sync_wait` caused the test to fail occasionally,
  due to subsequent `SELECT`s observing stale values.

* Remove redundant `--source include/galera_wait_ready.inc`,
  `galera_wait_ready.inc` is already included in the above
  `wait_until_connected_again.inc`

Test would occasionally fail as follows:

```
mysqltest: At line 20: query 'SELECT COUNT(*) = 0 FROM t1' failed:
1317: Query execution was interrupted

```

This was due to a `CREATE TABLE` applied concurrently that would
occasionally cause the `SELECT` to be BF aborted, due to MDL
conflict.

The `SELECT` is executed while `wsrep_on=OFF`. The change makes sure
it is executed only after wsrep is enabled again.

There was a missing argument to service_manager_extend_timeout call
and the signness of arguments did not match.

Wait drop table to be replicated before continuing.

If innodb_fast_shutdown<2, all transactions of active connections
will be rolled back on shutdown. This can take a long time, and
the systemd shutdown timeout should be extended during the wait.

logs_empty_and_mark_files_at_shutdown(): Extend the timeout when
waiting for other threads to complete.

This reverts commit 76ec37f5.

This behaviour change will be done separately in:
MDEV-15832 With innodb_fast_shutdown=3, skip the rollback
of connected transactions

Table_open_cache gets adjusted on server startup to prevent an out of
file descriptor error. However this means that when we reset its value
to default, it does not get re-adjusted. This leads to the mtr
consistency check to fail with different server status at the end of the
test case as opposed to when it started.

To fix the problem, do not make use of the DEFAULT keyword to set the
variable back, instead save the value before any modifications and
restore it from there.

Disable occasionally failing test case galera.galera_var_slave_threads.

Wait until create table is replicated before continuing.

Wait until create tables are replicated to the slave before trying
lock table.

Disable more occasionally failing test cases.
	galera.galera_lock_table
	galera.galera_gra_log
	galera.MW-44
	galera.lp1376747
	galera.lp1376747-2
	galera.galera_pc_ignore_sb

Make sure DDL's (create tables) are replicated before continuing to
lock table.

Disable occasionally failing test cases
	pxc-421
	query_cache
	galera_parallel_simple

fil_crypt_read_crypt_data(): Do not attempt to read the tablespace
if the file is unaccessible due to a pending DDL operation, such as
renaming the file or DROP TABLE or TRUNCATE TABLE. This is only
reducing the probability of the race condition, not completely
preventing it.

MDEV-13549 Fix test galera.galera_wsrep_desync_wsrep_on

The test tends to fail if many parallel instances of it are executed:

```
mysqltest: At line 23: query 'ALTER TABLE t1 ADD PRIMARY KEY (f1)' failed:
1317: Query execution was interrupted
```

The `ALTER` fails because it is BF aborted due to an earlier `INSERT SELECT`
that is being applied:

```
INSERT INTO t1 (f1) SELECT ...

--connection node_2
SET GLOBAL wsrep_desync = TRUE;
SET SESSION wsrep_on = FALSE;

ALTER TABLE t1 ADD PRIMARY KEY (f1);

SET SESSION wsrep_on = TRUE;
SET GLOBAL wsrep_desync = FALSE;
```

And because the `ALTER` is executed with `wsrep_on = OFF`, it does not
run in total order isolation.
To avoid the problem it must be ensured that the `ALTER` only after the
large `INSERT SELECT` is done. To do so it is sufficient to issue
`SELECT COUNT(*) FROM t1;` from `node_2` before turning off wsrep.
The `SELECT` will trigger `wsrep_sync_wait` and proceed only after the
`INSERT SELECT` from node_1 is done.

Problem was that key rotation from encrypted to unecrypted was skipped
when encryption is disabled (i.e. set global innodb-encrypt-tables=OFF).

fil_crypt_needs_rotation
        If encryption is disabled (i.e. innodb-encrypt-tables=off)
	and there is tablespaces using default encryption (e.g.
	system tablespace) that are still encrypted state we need
	to rotate them from encrypted state to unencrypted state.