The original code was correct. mysql_upgrade calls the mysql client to
talk with MariaDB. It doesn't call itself!

Appoligies, had a dirty branch before pushing:

This reverts commit 053653a2.

This reverts commit 0ff89780.

This reverts commit 85b08597.

This reverts commit f3f45e46.

This reverts commit a470b357.

This reverts commit f8b8d202.

This reverts commit 6b6f066f.

This reverts commit a701e9e6.

This reverts commit c1698386.

Leave debian/additions/mysqlreport as #!/usr/bin/perl

Acknowledge that `env perl` is a hack, a complete fix
needs to consider which path perl is at and insert into
these scripts.

The usefulness of these scripts is questionable.

This is just to make sure no ExecStartPre/Post actions from the
multi-instance MariaDB service definition are executed
when a user attempts to start mariadb@bootstrap.

Fixes: 3723c70a

* Maintain coding style in sql_yacc.yy in regards to optional clauses.
* Remove unused variable from sql_acl.cc.
* Update test case

Update test to account for the new SHOW CREATE USER command.

Adding an extra statement requires test cases update.

Implemented the alter user syntax. Also tested that create user
creates users accordingly.

Extend the syntax accepted by the grammar to account for the new create user
and alter user syntax.

Problem:
=======
SHOW BINLOG EVENTS FROM <"random"-pos> caused a variety of failures as
reported in MDEV-18046. They are fixed but that approach is not future-proof
as well as is not optimal to create extra check for being constructed event
parameters.

Analysis:
=========
"show binlog events from <pos>" code considers the user given position as a
valid event start position. The code starts reading data from this event start
position onwards and tries to map it to a set of known events. Each event has
a specific event structure and asserts have been added to ensure that, read
event data, satisfies the event specific requirements. When a random position
is supplied to "show binlog events command" the event structure specific
checks will fail and they result in assert.

For example: https://jira.mariadb.org/browse/MDEV-18046
In the bug description user executes CREATE TABLE/INSERT and ALTER SQL
commands.

When a crazy offset like "SHOW BINLOG EVENTS FROM 365" is provided code
assumes offset 365 as valid event begin and proceeds to EVENT_LEN_OFFSET reads
some random length and comes up with a crazy event which didn't exits in the
binary log. In this quoted example scenario, event read at offset 365 is
considered as "Update_rows_log_event", which is not present in binary log.
Since this is a random event its validation fails and code results in
assert/segmentation fault, as shown below.

mysqld: /data/src/10.4/sql/log_event.cc:10863: Rows_log_event::Rows_log_event(
    const char*, uint, const Format_description_log_event*):
    Assertion `var_header_len >= 2' failed.
    181220 15:27:02 [ERROR] mysqld got signal 6 ;
#7  0x00007fa0d96abee2 in __assert_fail () from /lib/x86_64-linux-gnu/libc.so.6
#8  0x000055e744ef82de in Rows_log_event::Rows_log_event (this=0x7fa05800d390,
    buf=0x7fa05800d080 "", event_len=254, description_event=0x7fa058006d60) at
/data/src/10.4/sql/log_event.cc:10863
#9  0x000055e744f00cf8 in Update_rows_log_event::Update_rows_log_event

Since we are reading random data repeating the same command SHOW BINLOG EVENTS
FROM 365 produces different types of crashes with different events. MDEV-18046
reported 10 such crashes.

In order to avoid such scenarios user provided starting offset needs to be
validated for its correctness. Best way of doing this is to make use of
checksums if they are available. MDEV-18046 fix introduced the checksum based
validation.

The issue still remains in cases where binlog checksums are disabled. Please
find the following bug reports.

MDEV-22473: binlog.binlog_show_binlog_event_random_pos failed in buildbot,
            server crashed in read_log_event
MDEV-22455: Server crashes in Table_map_log_event,
            binlog.binlog_invalid_read_in_rotate failed in buildbot

Fix:
====
When binlog checksum is disabled, perform scan(via reading event by event), to
validate the requested FROM <pos> offset. Starting from offset 4 read the
event_length of next_event in the binary log. Using the next_event length
advance current offset to point to next event. Repeat this process till the
current offset is less than or equal to crazy offset. If current offset is
higher than crazy offset provide appropriate invalid input offset error.

Problem:- Test case uses socket which does not work on windows, So mysqlslap
falls back to default connection which is defined in my.cnf and connects
to master instead of slave. Since start slave/stop slave is executed on
master we get error MASTER_HOST was not set

Solution:- Use Port instead of socket

MDEV-9501: rpl.rpl_binlog_index, rpl.rpl_gtid_crash, rpl.rpl_stm_multi_query fail sporadically in buildbot with Master command COM_REGISTER_SLAVE failed

Analysis:
========
Slave server will send COM_REGISTER_SLAVE command at the time of establishing
a connection to master. If master is down, then the command will fail and
COM_REGISTER_SLAVE failed warning is reported.

'rpl_binlog_index.test' shutsdown the master and it relocates binary logs to a
new location and attempts to start master by pointing 'log-bin' to new
location. During this process the slave threads are active. IO thread actively
checks for the presence of master when it finds that the connection is lost it
attempts a reconnect, as master is down COM_REGISTER_SLAVE command fails.

As part of fix, stop the slave threads and then shutdown the master and do the
binlog relocation. Once master is restarted start the slave threads and sync
them with the master. In test binary logs and index files on master are
relocated to /tmpdir but during master restart only --log-bin option is
provided, this is incorrect. Even --log-bin-index also should be pointed to
/tmpdir otherwise upon master server restart two index files will be created.
One master-bin.index in /tmpdir and a new master-bin.index as per log_basename
in datadir. Due to this slave will fail to connect to master.

'rpl_gtid_crash.test' tests following scenario "crashing master, causing slave
IO thread to reconnect while SQL thread is running". When IO thread tries to
connect to crashed master on slow platforms COM_REGISTER_SLAVE command fails.
This is expected hence the warning should be added to suppression list.

MDEV-7098 spider/bg.spider_fixes failed in buildbot with safe_mutex: Trying to unlock mutex conn->mta_conn_mutex that wasn't locked at storage/spider/spd_db_conn.cc, line 671

On some libc (like musl[1]), posix_fallocate() is a fallocate() syscall
wrapper, and does not include fallback code like glibc does. In that
case, EOPNOTSUPP is returned if underlying filesystem does not
support fallocate() with mode = 0.

This patch enables falling back to writing zeros when EOPNOTSUPP, fixes
some cases like running on filesystem without proper fallocate support
on Alpine.

[1]: https://git.musl-libc.org/cgit/musl/tree/src/fcntl/posix_fallocate.c?h=v1.2.1

MDEV-23535 SIGSEGV, SIGABRT and SIGILL in typeinfo for Item_func_set_collation (on optimized builds)

This piece of the code in Item_func_or_sum::agg_item_set_converter:

 if (!conv && ((*arg)->collation.repertoire == MY_REPERTOIRE_ASCII))
   conv= new (thd->mem_root) Item_func_conv_charset(thd, *arg, coll.collation, 1);

was wrong because:

1. It could change Item_cache to Item_func_conv_charset
  (with the old Item_cache in args[0]).
  Such Item type change is not always supported, e.g.
  the code in Item_singlerow_subselect::reset() expects only Item_cache,
  to be able to call Item_cache::set_null(). So it erroneously
  reinterpreted Item_func_conv_charset to Item_cache and called
  a non-existing method set_null(), which crashed the server.

2. The 1 in the last parameter to Item_func_conv_charset() was also
  a problem. In MariaDB versions where the reported query did not crash,
  it erroneously returned "empty set" instead of one row, because
  the 1 made subselects execute too earlier and return NULL.

Fix:

1. Removing the above two lines from Item_func_or_sum::agg_item_set_converter()

2. Adding the repertoire test inside the constructor of Item_func_conv_charset,
   so it now detects itself as "safe" in more cases than before.
   This is needed to avoid new "Illegal mix of collations" after
   removing the wrong code in various scenarios when character set
   conversion from pure ASCII happens, including the reported scenario.

So now this sequence:

   Item_cache -> Item_func_concat

is replaced to this compatible sequence (the top Item is still Item_cache):

   new Item_cache -> Item_func_conv_charset -> Item_func_concat

Before the fix it was replaced to this incompatible sequence:

   Item_func_conv_charset -> old Item_cache -> Item_func_concat

Passing a null pointer to a nonnull argument is not only undefined
behaviour, but it also grants the compiler the permission to optimize
away further checks whether the pointer is null. GCC -O2 at least
starting with version 8 may do that, potentially causing SIGSEGV.

These problems were caught in a WITH_UBSAN=ON build with the
Bug#7024 test in main.view.

Passing a null pointer to the "%s" argument of a printf-like
function is undefined behaviour. In the GNU libc implementation
of the printf() family of functions, it happens to work.

GCC 10.2.0 would diagnose this with -Wformat-overflow -Og.
In -fsanitize=undefined (WITH_UBSAN=ON) builds, a runtime error
would be generated. In some other builds, GCC 8 or later might infer
that the parameter is nonnull and optimize away further checks whether
the parameter is null, leading to SIGSEGV.

Backporting fixes for:

MDEV-22317: SIGSEGV in my_free/delete_dynamic in optimized builds (ARIA)
  Backported following commits from: 10.5.3
  commit 77e1b0c3 -- Post push fix.
  commit 2e6b21be

MDEV-22059: MSAN report at replicate_ignore_table_grant
  Backported following  commits from: 10.5.4
  commit 840fb495

The InnoDB index fields store bytes, not characters.
Remove some unnecessary conversions from characters to bytes.

This also fixes MDEV-20422 and the wrong-result bug MDEV-12486.

(This commit is exclusively for 10.1 branch, do not merge it to upper ones)

In case of a pattern of non-STMT_END-marked Rows-log-event (A) followed by
a STMT_END marked one (B) mysqlbinlog mixes up the base64 encoded rows events
with their pseudo sql representation produced by the verbose option:
      BINLOG '
        base64 encoded data for A
        ### verbose section for A
        base64 encoded data for B
        ### verbose section for B
      '/*!*/;
In effect the produced BINLOG '...' query is not valid and is rejected with the error.
Examples of this way malformed BINLOG could have been found in binlog_row_annotate.result
that gets corrected with the patch.

The issue is fixed with introduction an auxiliary IO_CACHE to hold on the verbose
comments until the terminal STMT_END event is found. The new cache is emptied
out after two pre-existing ones are done at that time.
The correctly produced output now for the above case is as the following:
      BINLOG '
        base64 encoded data for A
        base64 encoded data for B
      '/*!*/;
        ### verbose section for A
        ### verbose section for B

Thanks to Alexey Midenkov for the problem recognition and attempt to tackle,
Venkatesh Duggirala who produced a patch for the upstream whose
idea is exploited here, as well as to MDEV-23077 reporter LukeXwang who
also contributed a piece of a patch aiming at this issue.

Extra: mysqlbinlog_row_minimal refined to not produce mutable numeric values into the result file.

The issue here was that the query was using ORDER BY LIMIT optimzation where
the access method was changed from EQ_REF access to an index scan (index that would
resolve the ORDER BY clause).
But the parameter READ_RECORD::unlock_row was not reset to rr_unlock_row, which is
used when the access method is not EQ_REF access.

for internal temporary tables: don't use realpath(),
and let them overwrite whatever orphan temp files might've
left in the tmpdir (see main.error_simulation test).

for user created temporary tables: we have to use realpath(),
(see 3a726ab6, remember DATA/INDEX DIRECTORY). don't allow
them to overwrite existing files.

This bug was reported by RACK911 LABS

This reverts commit 57960211.

Remove __bss_start & Co, because systen call "write" check buffer address and return EFAULT if it is wrong.

Adding any unknown option to the "[mysqld_safe]" section makes
mysqld impossible to start with mysqld_multi. For example, after
adding the unknown option "numa_interleave" to the "[mysqld_safe]"
section, mysqld_multi exits with the following diagnostics:

[ERROR] /usr/local/mysql/bin/mysqld: unknown option '--numa_interleave'

To get rid of this behavior, this patch by default adds the "--loose-"
prefix to all unknown (for mysqld_safe) options. This behavior can be
enabled explicitly with the --ignore-unknown option and disabled with
the --no-ignore-unknown option.

Since MDEV-18778, timezone tables get changed to innodb
to allow them to be replicated to other galera nodes.

Even without galera, timezone tables could be declared innodb.
With the standalone innodb tables, the mysql_tzinfo_to_sql takes
approximately 27 seconds.

With the transactions enabled in this patch, 1.2 seconds is
the approximate load time.

While explicit checks for the engine of the time zone tables could be
done, or checks against !opt_skip_write_binlog, non-transactional
storage engines will just ignore the transactional state without
even a warning so its safe to enact globally.

Leap seconds are pretty much ignored as they are a single insert
statement and have gone out of favour as they have caused MariaDB
stalls in the past.

fil_page_decompress(): Remove a rather useless debug check.
We should have test coverage for reading page_compressed pages
from files, either due to buffer pool page eviction or due to
server restarts.

A similar check was removed from fil_space_encrypt() in
commit 0b36c27e (MDEV-20307).

The usage message for the innodb_compression_algorithm system variable
did not list snappy, which was added as an optional compression algorithm
in MariaDB 10.1.3 and might actually work since
commit 90c52e52 (MDEV-12615)
in MariaDB 10.1.24.

Unfortunately, we will include also unavailable compression algorithms
in the list, because ENUM parameters allow numeric values, and we do
not want innodb_compression_algorithm=3 to change meaning depending on
the way how the source code was compiled.

InnoDB only reserves 13 bits for the heap number in the record header,
limiting the heap number to be at most 8191. But, when using
innodb_page_size=64k and secondary index records of 7 bytes each,
it is possible to exceed the maximum heap number.

btr_cur_optimistic_insert(): Let the operation fail if the
maximum number of records would be exceeded.

page_mem_alloc_heap(): Move to the same compilation unit with the
only caller, and let the operation fail if the maximum heap number
has been allocated already.

The problem is caused by the fact that adding the
--defaults-group-suffix option to fix MDEV-18863 causes
mysqld to read all options from the appropriate sections
of the config file, including options specific to mysqld_multi.
Reading unknown options (which are not supported by mysqld)
causes mysqld to terminate with an error.

However, the MDEV-18863 problem has been completely fixed
by passing options on the command line, and now there is no
need to specify the --defaults-group-suffix option (we just
need to give priority to options passed through the command
line, so as not to break MDEV-18863).

Some tests relied on the fact that DATETIME->DATE conversion
always produce a truncation (with a warning). This is not the case
when the SQL statement is executed at current time '00:00:00' sharp.

Adding a new SET TIMESTAMP statements to make sure time is not '00:00:00'.

The test encryption.create_or_replace would occasionally fail,
because some fil_space_t::n_pending_ops would never be decremented.

fil_crypt_find_space_to_rotate(): If rotate_thread_t::should_shutdown()
holds due to innodb_encryption_threads having been reduced, do
release the reference.

fil_space_remove_from_keyrotation(), fil_space_next(): Declare the
functions static, simplify a little, and define in the same compilation
unit with the only caller, fil_crypt_find_space_to_rotate().

fil_crypt_key_mutex: Remove (unused).