Problem was that key rotation from encrypted to unecrypted was skipped
when encryption is disabled (i.e. set global innodb-encrypt-tables=OFF).

fil_crypt_needs_rotation
        If encryption is disabled (i.e. innodb-encrypt-tables=off)
	and there is tablespaces using default encryption (e.g.
	system tablespace) that are still encrypted state we need
	to rotate them from encrypted state to unencrypted state.

Fix test case galera_toi_ddl_nonconflicting as we need to wait
until both alter tables are finished on that Galera node.

buf_flush_remove(): Disable the output for now, because we
certainly do not want this after every page flush on shutdown.
It must be rate-limited somehow. There already is a timeout
extension for waiting the page cleaner to exit in
logs_empty_and_mark_files_at_shutdown().

log_write_up_to(): Use correct format.

srv_purge_should_exit(): Move the timeout extension to the
appropriate place, from one of the callers.

Fix test case galera.pxc-421 to reset auto_increment_offset
correctly.

Use systemd EXTEND_TIMEOUT_USEC to advise systemd of progress

Move towards progress measures rather than pure time based measures.

Progress reporting at numberious shutdown/startup locations incuding:
* For innodb_fast_shutdown=0 trx_roll_must_shutdown() for rolling back incomplete transactions.
* For merging the change buffer (in srv_shutdown(bool ibuf_merge))
* For purging history, srv_do_purge

Thanks Marko for feedback and suggestions.

Suggested by Marko on github pr #576

buf_all_freed only needs to be called once, not 3 times.

buf_all_freed will always return TRUE if it returns.
It will crash if any page was not flushed so its effectively
an assert anyway.

The following calls are likely redundant and could be removed:

		fil_flush_file_spaces(FIL_TYPE_TABLESPACE);
		fil_flush_file_spaces(FIL_TYPE_LOG);

row_undo_step(): If fast shutdown has been requested, abort the
rollback of any non-DDL transactions. Starting with MDEV-12323,
we aborted the rollback of recovered transactions. These
transactions would be rolled back on subsequent server startup.

trx_roll_report_progress(): Renamed from trx_roll_must_shutdown(),
now that the shutdown check has been moved to the only caller.

Enable tests that pass on 10.1:
       galera_var_auto_inc_control_on
       galera_var_retry_autocommit
       pxc-421
       lp1376747-2
       lp1376747
       galera_toi_ddl_nonconflicting
       galera_parallel_simple
       galera_admin
       galera_pc_ignore_sb
       galera_lock_table
       galera_unicode_identifiers
       galera.galera_gcs_fc_limit
       galera.galera_gtid

Fix test case MW-44 by disabling Galera replication of MyISAM
tables (mysql.general_log especially) to avoid MDL lock
wait later.

Test did not reset modified variables correctly.

galera_wsrep_desync_wsrep_on test does work on 10.1

Fix test failure on galera_gcs_fragment. Test requires Galera debug
library with debug_sync functionality.

MDEV-13549 Fix and re-enable MTR test galera.MW-284

The following changes are committed:

* `RESET MASTER` at the end of the test. This was necessary to allow the test
  to run on repeated runs.

* `--source include/galera_wait_ready.inc` after setting `gmcast.isolate=0` to
  get back to a primary component.

* Fix for assertion in `Protocol::end_statement()`. The assertion is due to
  the fact that function `do_command()` calls `thd->protocol->end_statement()`,
  without setting an error, when it is detected that galera is not ready yet.
  Following line somehow disappeared in a past merge:
  ```
  my_message(ER_UNKNOWN_COM_ERROR,
             "WSREP has not yet prepared node for application use", MYF(0));
  ```

Fix test failure on galera_toi_dll_fk_insert. DEFAULT value seems
to be 3 but test expects it to be 1.

MDEV-13549 Fix and re-enable MTR test galera.mysql-wsrep#90

MDEV-13549 Fix and re-enable MTR test galera.galera_gra_log

Test galera checks that a `GRA_x_x.log` file is created whenever
wsrep applier fails to apply some replication event. The file
contains the corresponding binlog event that failed to apply.
The test creates a new file by concatenating a pre-recorded
file containing the binlog header (see `std-data/binlog-header.log`)
and the `GRA_x_x.log` file. The test then checks that the resulting
file, containing the binlog header and the event that failed to
apply, is correctly read by `mysqlbinlog` program.
The test fails in MariaDB because the GRA_x_x.log file created
by MariaDB already contains the binlog header (see MDEV-7867).
This patch fixes/simplifies test `galera.galera_gra_log` so that
it doesn't concatenate `std-data/binlog-header.log` with the
`GRA_x_x.log` file. File `std-data/binlog-header.log` is deleted
altoghether, because not used by any other test.

(Currently) LibreSSL doesn't calculate the string length of the hostname
that's passed to X509_check_host automatically in case namelen/chklen is 0.
This causes server certificate validation to fail when building MariaDB with
LibreSSL.

The proposed fix makes MariaDB determine the string length passed to
X509_check_host. As there are no ill side-effects (OpenSSL's X509_check_host
also simply calls strlen if namelen == 0, see also X509_check_host(3)), this
wasn't wrapped in any #ifdef like constructs.

Please see here for a proposed patch to modify LibreSSL's behavior:
https://github.com/libressl-portable/openbsd/pull/87

MariaDB adjustments.

mysqltest.cc : Allow 12 error codes at --error

wait_until_connected_again.inc: Replace numeric error codes with symbols

mysqltest.test: Add error codes to test that tests too many errorcodes

kill_galera.inc can no longer rely on wait_until_connected_again.inc.
This is because wait_until_connected_again now tries to make sure
that the server it is connected eventually transition to ready
state. Whereas some tests may need to kill galera while the server
is in a non-primary view.

Test galera_3nodes.galera_pc_weight started to fail because it
expects to use wait_until_connected_again while remaining in
non-primary view. Hopefully this is the only test which makes
this assumption, and fortunately those wait_until_connected_again
seem unnecessary, so this patch removes them.

Remove clause on `thd->variables.wsrep_on` in the following code:

if (WSREP(thd))
{
...
        if (thd->variables.wsrep_on &&
            ...

In the above snippet, `WSREP(thd)` already ensures thd->variables.wsrep_on

wait_until_connected_again issues 'SHOW STATUS' query repeatedly
until mysqld replies without errors.
However, SHOW STATUS is treated specially by wsrep in that it is
allowed to proceed even if wsrep is not yet in ready state. As a
consequence, after returning from wait_until_connected_again,
wsrep may not be ready yet and subsequent queries may fail with
error "1047 WSREP has not yet prepared node for application use".
To avoid those errors, the patch includes wait_wsrep_ready.inc at
the end of the wait_until_connected_again.

Test was missing DEBUG_SYNC cleanup `SET DEBUG_SYNC ='RESET'`.

MDEV-13549 Fix and re-enable MTR test galera.galera_gtid_slave

MDEV-13549 Fix and re-enable MTR test galera.galera_as_master

This patch makes two changes:

* It replaces `--sleep 1` with appropriate wait_conditions, and
  removes another `--sleep 1` which is no longer necessary, after
  MDEV-14144 has been fixed.

* It moves the `RESET MASTER` from node_1 at the very end, when it
  it sure that the other nodes have applied the the final cleanup
  `DROP TABLE t1,t2.`
  This solves the problem with the `DROP TABLE` not being replicated
  to the asynchronous slaves, which would make the test fail with:
  `Timeout in wait_condition.inc for SELECT COUNT(*) = 0 FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_NAME = 't1';`

MDEV-13549 Fix and re-enable test galera.galera_suspend_slave

Was failing due to missing RESET MASTER

Test galera_suspend_slave checks that after suspending a galera
node, like this:
```
--perl
        my $pid_filename = $ENV{'NODE_2_PIDFILE'};
        my $mysqld_pid = `cat $pid_filename`;
        chomp($mysqld_pid);
        system("kill -SIGSTOP $mysqld_pid");
        exit(0);
EOF
```

the remaining one node cluster is no longer able to successfully
INSERT new rows:
```
--error ER_UNKNOWN_COM_ERROR,ER_LOCK_WAIT_TIMEOUT,ER_LOCK_DEADLOCK,ER_ERROR_DURING_COMMIT
INSERT INTO t1 VALUES (1);
```

On rare occasions when the system is overloaded, it appears that
suspending the process with ```system("kill -SIGSTOP $mysqld_pid")```
takes some time, enough for the subsequent INSERT to succeed. In which
case the test fails because it rightly expects the INSERT to fail.

To fix the problem, the patch makes sure that the cluster has shrinked
to one node, before trying to INSERT the new value.

MDEV--15609 engines/funcs.crash_manytables_number crashes with error 24
           (too many open files)
MDEV-10286  Adjustment of table_open_cache according to system limits
            does not work when open-files-limit option is provided

Fixed by adjusting tc_size downwards if there is not enough file
descriptors to use.

Other changes:
- Ensure that there is 30 (was 10) extra file descriptors for other usage
- Decrease TABLE_OPEN_CACHE_MIN to 200 as it's better to have a smaller
  table cache than getting error 24
- Increase minimum of max_connections and table_open_cache from 1 to 10
  as 1 is not usable for any real application, only for testing.

and remove redundant have_innodb.inc, it's included in
galera_cluster.inc anyway.

and remove redundant have_innodb.inc, it's included in
galera_cluster.inc anyway.