MDEV-22617 Galera node crashes when trying to log to slow_log table in
streaming replication mode

Other things:
- Changed name of wsrep_after_row(two arguments) to
  wsrep_after_row_internal(one argument) to not depended on the
  function signature with unused arguments.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>
	     Added test case

Small postfix to MDEV-23175 to ensure faster option on FreeBSD
and compatibility to Solaris that isn't high resolution.

ftime is left as a backup in case an implementation doesn't
contain any of these clocks.

FreeBSD
    $ ./unittest/mysys/my_rdtsc-t
    1..11
    # ----- Routine ---------------
    # myt.cycles.routine          :             5
    # myt.nanoseconds.routine     :            11
    # myt.microseconds.routine    :            13
    # myt.milliseconds.routine    :            11
    # myt.ticks.routine           :            17
    # ----- Frequency -------------
    # myt.cycles.frequency        :    3610295566
    # myt.nanoseconds.frequency   :    1000000000
    # myt.microseconds.frequency  :       1000000
    # myt.milliseconds.frequency  :           899
    # myt.ticks.frequency         :           136
    # ----- Resolution ------------
    # myt.cycles.resolution       :             1
    # myt.nanoseconds.resolution  :             1
    # myt.microseconds.resolution :             1
    # myt.milliseconds.resolution :             7
    # myt.ticks.resolution        :             1
    # ----- Overhead --------------
    # myt.cycles.overhead         :            26
    # myt.nanoseconds.overhead    :         19140
    # myt.microseconds.overhead   :         19036
    # myt.milliseconds.overhead   :           578
    # myt.ticks.overhead          :         21544
    ok 1 - my_timer_init() did not crash
    ok 2 - The cycle timer is strictly increasing
    ok 3 - The cycle timer is implemented
    ok 4 - The nanosecond timer is increasing
    ok 5 - The nanosecond timer is implemented
    ok 6 - The microsecond timer is increasing
    ok 7 - The microsecond timer is implemented
    ok 8 - The millisecond timer is increasing
    ok 9 - The millisecond timer is implemented
    ok 10 - The tick timer is increasing
    ok 11 - The tick timer is implemented

vsnprintf takes the space need for trailing '\0' in consideration, and copies only n-1 characters to destination buffer.
With the old code, only sizeof(buf)-2 characters were copied, this caused that last character of message could be lost.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

This commit contains a fix, where the replication write set for a CREATE TABLE
will contain, as certification keys, table names for all FK references.
With this, all DML for the FK parent tables will conflict with the CREATE TABLE
statement.

There is also new test galera.MDEV-27276 to verify the fix.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

This commit has a mtr test where two two transactions delete a row from
two separate tables, which will cascade a FK delete for the same row in
a third table. Second replica node is configured with 2 applier threads,
and the test will fail if these two transactions are applied in parallel.

The actual fix, in this commit, is to mark a transaction as unsafe for
parallel applying when it traverses into cascade delete operation.
Reviewed-by: Jan Lindström <jan.lindstrom@mariadb.com>

Example build: ./BUILD/compile-pentium64-valgrind-max

Fixes:
- sp-no-valgrind failed if binary was built for valgrind as in this case
  mem_root is allocated in very small hunks which the test cannot handle.
  Fixed by testing of valgrind build
- truncate_notembedded failed in reap because of more memory used.
  Fixed by allowing reap to fail too

This enables optimizer_trace output for the next SQL command.
Identical as if one would have done:
- Store value of @@optimizer_trace
- Set @optimizer_trace="enabled=on"
- Run query
- SELECT * from OPTIMIZER_TRACE
- Restore value of @@optimizer_trace

This is a great time saver when one wants to quickly check the optimizer
trace for a query in a mtr test.

MDEV-18187 Aria engine: Redo phase failed with "error 192 when executing record redo_index_new_page" upon startup on a restored datadir

The issue is that when recovery is about to create a new data or index
page it check if the page already exits.
If the page does not exists (file is too short) or contains wrong checksum,
then the recovery code will recreate the page.
The bug was that the code that checked if the page existed didn't take
into account encrypted pages.

Fixed by adding a check if page could not be encrypted solved the issue.
I also added some code to silence decryption errors for new pages.

Test case and some inspiration for how to solve this come from
the pull request by alexandr.miloslavsky

1. Galera SST scripts should use ssl_capath (not ssl_ca) for CA
   directory. The current implementation tries to automatically
   detect the path using the trailing slash in the ssl_ca variable
   value, but this approach is not compatible with the server
   configuration. Now, by analogy with the server, SST scripts
   also use a separate ssl_capath variable. In addition, a similar
   tcapath variable has been added for the old-style configuration
   (in the "sst" section).
2. Openssl utility detection made more reliable.
3. Removed extra spaces in automatically generated command lines -
   to simplify debugging of the SST scripts.
4. In general, the code for detecting the presence or absence of
   auxiliary utilities has been improved - it is made more reliable
   in some configurations (and for shells other than bash).

mtr test galera.galera_UK_conflict, has rather distorted logic in test scenario 2.

The test would fail always with:

CURRENT_TEST: galera.galera_UK_conflict
mysqltest: In included file "./include/galera_wait_sync_point.inc":
included from /home/seppo/work/wsrep/mariadb-server/mysql-test/suite/galera/t/galera_UK_conflict.test at line 216:
At line 3: query 'SET SESSION wsrep_on = 0' failed: 1179: You are not allowed to execute this command in a transaction

This happens because wait_condition is called in wrong connection (node_1, which is excuting MST transaction)

This test is fixed by using contl connection for wait contdition check, and new result is recorded as well

Fixes to make the galera.MDEV-20793 test deterministic.
Specifically, after --send COMMIT, there is now a sync point to catch a known state of the COMMIT execution

Don't switch state to DONOR in `wsrep_sst_donate()` - wsrep-lib does it now

A part of the test main.long_unique attempts to insert records
with two 60,000,001-byte columns. Let us move that test into
a separate file main.long_unique_big, declared as big test,
so that it can be skipped in environments with limited memory.

How 'bout know?

Hi,

if the pid-file option is configured more than once (e.g. multiple times in different files), my_print_defaults prints it twice, resulting in the logrotate postrotate script failing because of a syntax error. Debian fixed this already (https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=830976#42).

Perhaps you could implement this small change in the other branches as well?

Thanks,

Christopher

Include gronnga and groonga-normalizer-mysql install path

* galera_kill_applier : we should make sure that node has
  correct number of wsrep appliers
* galera_bad_wsrep_new_cluster: This test restarts both nodes,
so it is bad on mtr. Make sure it is run alone
* galera_update_limit : Make sure we have PK when needed
galera_as_slave_replay : bf abort was not consistent
* galera_unicode_pk : Add wait_conditions so that all nodes
are part of cluster and DDL and INSERT has replicated before
any further operations are done.
* galera_bf_abort_at_after_statement : Add wait_conditions
to make sure all nodes are part of cluster and that DDL
and INSERT has replicated. Make sure we reset DEBUG_SYNC.

Per bug report, cycles was woefully insufficient to
detect any implementation error.

MariaDB server crashes on ARM (weak memory model architecture) while
concurrently executing l_find to load node->key and add_to_purgatory
to store node->key = NULL. l_find then uses key (which is NULL), to
pass it to a comparison function.

The specific problem is the out-of-order execution that happens on a
weak memory model architecture. Two essential reorderings are possible,
which need to be prevented.

a) As l_find has no barriers in place between the optimistic read of
the key field lf_hash.cc#L117 and the verification of link lf_hash.cc#L124,
the processor can reorder the load to happen after the while-loop.

In that case, a concurrent thread executing add_to_purgatory on the same
node can be scheduled to store NULL at the key field lf_alloc-pin.c#L253
before key is loaded in l_find.

b) A node is marked as deleted by a CAS in l_delete lf_hash.cc#L247 and
taken off the list with an upfollowing CAS lf_hash.cc#L252. Only if both
CAS succeed, the key field is written to by add_to_purgatory. However,
due to a missing barrier, the relaxed store of key lf_alloc-pin.c#L253
can be moved ahead of the two CAS operations, which makes the value of
the local purgatory list stored by add_to_purgatory visible to all threads
operating on the list. As the node is not marked as deleted yet, the
same error occurs in l_find.

This change three accesses to be atomic.

* optimistic read of key in l_find lf_hash.cc#L117
* read of link for verification lf_hash.cc#L124
* write of key in add_to_purgatory lf_alloc-pin.c#L253

Reviewers: Sergei Vojtovich, Sergei Golubchik

Fixes: MDEV-23510 / d30c1331a18d875e553f3fcf544997e4f33fb943

Print this piece when we've just made the choice to convert to semi-join.
Also, print it when we've already made that choice before:

  transformation": {
     "select_id": 2,
     "from": "IN (SELECT)",
     "to": "semijoin",
     "chosen": true
   }

Let us create and drop a separate database for getting rid of the
default database in the MDEV-22781 test.

This bug was introduced by commit be00e279
The commit was applied for the task MDEV-6480 that allowed to remove top
level disjuncts from WHERE conditions if the range optimizer evaluated them
as always equal to FALSE/NULL.
If such disjuncts are removed the WHERE condition may become an AND formula
and if this formula contains multiple equalities the field JOIN::item_equal
must be updated to refer to these equalities. The above mentioned commit
forgot to do this and it could cause crashes for some queries.

Approved by Oleksandr Byelkin <sanja@mariadb.com>

restore the old behavior where without a debugger mtr does not
wait for mysqld to start. It was broken in feacc0aa

close_connections() in mysqld.cc sends a signal to all threads.
But InnoDB is too busy purging, doesn't react immediately.
close_connections() waits 20 seconds, which isn't enough in this
particular case, and then unlinks all threads from
the list and forcibly closes their vio connection.

InnoDB background  threads have no vio connection to close, but
they're unlinked all the same. So when later they finally notice
the shutdown request and try to unlink themselves, they fail to
assert that they're still linked.

Fix: don't assert_linked, as another thread can unlink this THD anytime

MDEV-20330 Combination of "," (comma), cross join and left join fails to parse

From 10.4.13, the `mariadb.sys` user was created to replace `root` definers.
 - In commit 0253ea7f, definer of
   Add/DropGeometryColumn procedures was changed to `mariadb.sys`, in
   `scripts/maria_add_gis_sp.sql.in`.
   However, maria_add_gis_sp.sql only applies to new databases created by
   installation script. Databases upgraded from old versions will miss this
   change.
 - In addition, according to commit
   0d6d801e(MDEV-23102), in some scenarios
   when root user is replaced it will skip creating `mariadb.sys` user.

This commit is to update the definer from `root` to `mariadb.sys` during
upgrade. It only makes the change if the original definers are root.

Doesn't choose to execute `maria_add_gis_sp.sql` in upgrade script to
recreate the procedures is because of considering the scenarios of
MDEV-23102 that `root` user is replaced and `mariadb.sys` is not created.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.

Fix a potential deadlock bug between locks ctrl_mutex and entry->mutex

Sys_lc_messages, Sys_lc_time_names: Use offsetof(MY_LOCALE, ...)
because the type has standard layout.

mdl_hash_function(): Use plain offsetof; MDL_key has standard layout.

MySQL documentation isn't going to help our
users and we shouldn't refer to it.

The bug occurs where the float token containing a dot with an 'e'
notation was dropped from the request completely.

This causes a manner of invalid SQL statements like:

select id 1.e, char 10.e(id 2.e), concat 3.e('a'12356.e,'b'1.e,'c'1.1234e)1.e, 12 1.e*2 1.e, 12 1.e/2 1.e, 12 1.e|2 1.e, 12 1.e^2 1.e, 12 1.e%2 1.e, 12 1.e&2 from test;

To be parsed correctly as if it was:

select id, char(id), concat('a','b','c'), 12*2, 12/2, 12|2, 12^2, 12%2, 12&2 from test.test;

This correct parsing occurs when e is followed by any of:

( ) . , | & % * ^ /