Wait until rotation has ended and shutdown before grep to make sure
that dirty pages are on datafiles.

Return early if plugin is not built.

MDEV-15804 Fix and re-enable MTR test galera.pxc-421

MDEV-15808 Fix and re-enable test galera.galera_gra_log

This patch makes two changes:

* Remove unnecessary `set SESSION wsrep_sync_wait=0`.
  Disabling `wsrep_sync_wait` caused the test to fail occasionally,
  due to subsequent `SELECT`s observing stale values.

* Remove redundant `--source include/galera_wait_ready.inc`,
  `galera_wait_ready.inc` is already included in the above
  `wait_until_connected_again.inc`

Test would occasionally fail as follows:

```
mysqltest: At line 20: query 'SELECT COUNT(*) = 0 FROM t1' failed:
1317: Query execution was interrupted

```

This was due to a `CREATE TABLE` applied concurrently that would
occasionally cause the `SELECT` to be BF aborted, due to MDL
conflict.

The `SELECT` is executed while `wsrep_on=OFF`. The change makes sure
it is executed only after wsrep is enabled again.

There was a missing argument to service_manager_extend_timeout call
and the signness of arguments did not match.

Wait drop table to be replicated before continuing.

If innodb_fast_shutdown<2, all transactions of active connections
will be rolled back on shutdown. This can take a long time, and
the systemd shutdown timeout should be extended during the wait.

logs_empty_and_mark_files_at_shutdown(): Extend the timeout when
waiting for other threads to complete.

This reverts commit 76ec37f5.

This behaviour change will be done separately in:
MDEV-15832 With innodb_fast_shutdown=3, skip the rollback
of connected transactions

Table_open_cache gets adjusted on server startup to prevent an out of
file descriptor error. However this means that when we reset its value
to default, it does not get re-adjusted. This leads to the mtr
consistency check to fail with different server status at the end of the
test case as opposed to when it started.

To fix the problem, do not make use of the DEFAULT keyword to set the
variable back, instead save the value before any modifications and
restore it from there.

Disable occasionally failing test case galera.galera_var_slave_threads.

Wait until create table is replicated before continuing.

Wait until create tables are replicated to the slave before trying
lock table.

Disable more occasionally failing test cases.
	galera.galera_lock_table
	galera.galera_gra_log
	galera.MW-44
	galera.lp1376747
	galera.lp1376747-2
	galera.galera_pc_ignore_sb

Make sure DDL's (create tables) are replicated before continuing to
lock table.

Disable occasionally failing test cases
	pxc-421
	query_cache
	galera_parallel_simple

fil_crypt_read_crypt_data(): Do not attempt to read the tablespace
if the file is unaccessible due to a pending DDL operation, such as
renaming the file or DROP TABLE or TRUNCATE TABLE. This is only
reducing the probability of the race condition, not completely
preventing it.

MDEV-13549 Fix test galera.galera_wsrep_desync_wsrep_on

The test tends to fail if many parallel instances of it are executed:

```
mysqltest: At line 23: query 'ALTER TABLE t1 ADD PRIMARY KEY (f1)' failed:
1317: Query execution was interrupted
```

The `ALTER` fails because it is BF aborted due to an earlier `INSERT SELECT`
that is being applied:

```
INSERT INTO t1 (f1) SELECT ...

--connection node_2
SET GLOBAL wsrep_desync = TRUE;
SET SESSION wsrep_on = FALSE;

ALTER TABLE t1 ADD PRIMARY KEY (f1);

SET SESSION wsrep_on = TRUE;
SET GLOBAL wsrep_desync = FALSE;
```

And because the `ALTER` is executed with `wsrep_on = OFF`, it does not
run in total order isolation.
To avoid the problem it must be ensured that the `ALTER` only after the
large `INSERT SELECT` is done. To do so it is sufficient to issue
`SELECT COUNT(*) FROM t1;` from `node_2` before turning off wsrep.
The `SELECT` will trigger `wsrep_sync_wait` and proceed only after the
`INSERT SELECT` from node_1 is done.

Problem was that key rotation from encrypted to unecrypted was skipped
when encryption is disabled (i.e. set global innodb-encrypt-tables=OFF).

fil_crypt_needs_rotation
        If encryption is disabled (i.e. innodb-encrypt-tables=off)
	and there is tablespaces using default encryption (e.g.
	system tablespace) that are still encrypted state we need
	to rotate them from encrypted state to unencrypted state.

Fix test case galera_toi_ddl_nonconflicting as we need to wait
until both alter tables are finished on that Galera node.

Boost includes sys/param.h on FreeBSD, which in turn defines setbit()
macro. This macro is conflicting with open_query::judy_bitset::setbit().

Reordered includes such that oqgraph_judy.h never sees this macro.
Also removed duplicate includes of graphcore-config.h, which is included
by graphcore-graph.h/oqgraph_shim.h/oqgraph_thunk.h.

buf_flush_remove(): Disable the output for now, because we
certainly do not want this after every page flush on shutdown.
It must be rate-limited somehow. There already is a timeout
extension for waiting the page cleaner to exit in
logs_empty_and_mark_files_at_shutdown().

log_write_up_to(): Use correct format.

srv_purge_should_exit(): Move the timeout extension to the
appropriate place, from one of the callers.

Fix test case galera.pxc-421 to reset auto_increment_offset
correctly.

Use systemd EXTEND_TIMEOUT_USEC to advise systemd of progress

Move towards progress measures rather than pure time based measures.

Progress reporting at numberious shutdown/startup locations incuding:
* For innodb_fast_shutdown=0 trx_roll_must_shutdown() for rolling back incomplete transactions.
* For merging the change buffer (in srv_shutdown(bool ibuf_merge))
* For purging history, srv_do_purge

Thanks Marko for feedback and suggestions.

Suggested by Marko on github pr #576

buf_all_freed only needs to be called once, not 3 times.

buf_all_freed will always return TRUE if it returns.
It will crash if any page was not flushed so its effectively
an assert anyway.

The following calls are likely redundant and could be removed:

		fil_flush_file_spaces(FIL_TYPE_TABLESPACE);
		fil_flush_file_spaces(FIL_TYPE_LOG);

row_undo_step(): If fast shutdown has been requested, abort the
rollback of any non-DDL transactions. Starting with MDEV-12323,
we aborted the rollback of recovered transactions. These
transactions would be rolled back on subsequent server startup.

trx_roll_report_progress(): Renamed from trx_roll_must_shutdown(),
now that the shutdown check has been moved to the only caller.

Enable tests that pass on 10.1:
       galera_var_auto_inc_control_on
       galera_var_retry_autocommit
       pxc-421
       lp1376747-2
       lp1376747
       galera_toi_ddl_nonconflicting
       galera_parallel_simple
       galera_admin
       galera_pc_ignore_sb
       galera_lock_table
       galera_unicode_identifiers
       galera.galera_gcs_fc_limit
       galera.galera_gtid

Fix test case MW-44 by disabling Galera replication of MyISAM
tables (mysql.general_log especially) to avoid MDL lock
wait later.

Test did not reset modified variables correctly.

galera_wsrep_desync_wsrep_on test does work on 10.1

Fix test failure on galera_gcs_fragment. Test requires Galera debug
library with debug_sync functionality.