Fatal error: mysql.user table is damaged or in unsupported 3.20 format

The problem stems from MySQL 5.7.6. According to MySQL documentation:
In MySQL 5.7.6, the Password column was removed and all credentials are
stored in the authentication_string column.

If opening a MySQL 5.7.6 (and up) datadir with MariaDB 10.2, the user table
appears corrupted. In order to fix this, the server must be started with
--skip-grant-tables and then a subsequent mysql_upgrade command must be
issued.

This patch updates the mysql_upgrade command to also add the removed
Password column. The password column is necessary, otherwise
the mysql_upgrade script fails due to the Event_scheduler not being able
to start, as it can't find Event_priv in the table where it ought to be.
MySQL's version has column position 28 (0 index) vs our datadir version
expects position 29.

This reverts commit 1f372cf1.
Wasn't supposed to be pushed just yet.

remove CAST(... AS JSON) from the grammar for 10.2.4

relax innodb assertion, because Field_blob::store() clearly says
that a data pointer can be zero if the length is zero.

change the parser not to allow SERIAL as a normal data type.
make a special rule for it, where it could be used for define
fields, but not generated fields, not return type of a stored function, etc.

JOIN_CACHE's were initialized in  check_join_cache_usage()
from make_join_readinfo(). After that make_join_readinfo() was looking
whether it's possible to use keyread. Later, after make_join_readinfo(),
optimizer decided whether to use filesort. And even later, at the
execution time, from join_read_first(), keyread was actually enabled.

The problem is, that if a query uses a vcol, base columns that it
depends on are automatically added to the read_set - because they're
needed to calculate the vcol. But if we're doing keyread, vcol is taken
from the index, not calculated, and base columns do not need to  be
in the read set (even should not be - as they aren't getting values).

The bug was that JOIN_CACHE used read_set with base columns,
they were not read because of keyread, so it was caching garbage.

So read_set is only known after the keyread was decided. And after the
filesort was decided, as filesort doesn't use keyread. But
check_join_cache_usage() needs to be done in make_join_readinfo(),
as the code below depends on these checks,

Fix: keep JOIN_CACHE checks where they were, but move initialization
down to the very end of JOIN::optimize_inner. If keyread was enabled,
update the read_set to include only columns that are part of the index.
Copy the keyread logic from join_read_first() to happen at optimize time.

make init_read_record() to detect enabled keyread
and use index_* access methods, not rnd_*

this makes MariaDB to use keyread a lot more often than before

because it doesn't work, vcols are never calculated for ICP

with --sorted_result

* rename to "keyread" (to avoid conflicts with tokudb),
* change from bool to uint and store the keyread index number there
* provide a bool accessor to check if keyread is enabled

Filesort temporarily changes read_set to be tmp_set and marks only
fields needed for filesort. Add an assert to ensure that it doesn't
overwrite the old value of tmp_set, that is that read_set was *not*
already tmp_set when filesort was invoked.

Fix sql_update.cc that was was doing exactly that - changing read_set to
tmp_set, configuring tmp_set for keyread, and then invoking filesort.

mark_columns_used_by_index used to do
reset + mark_columns_used_by_index_no_reset + start keyread + set bitmaps

Now prepare_for_keyread does that, while mark_columns_used_by_index
does only reset + mark_columns_used_by_index_no_reset,
just as its name suggests.

old code didn't calculate vcols that were part of keyread,
but calculated other vcols. It was wrong - there was no guarantee
that vcol's base columns were part of keyread.

Technically it's possible for the vcol not be a part of keyread,
but all its base columns being part of keyread. But currently the
optimizer doesn't do that, keyread is only used if it covers
all columns used in the query.

This fixes crashes of vcol.vcol_trigger_sp_innodb

TABLE::add_read_columns_used_by_index() is conceptually wrong,
it *adds* columns used by index to the bitmap, without clearing
it first. But it also enables keyread, meaning that *only* columns
from the index will be read. It is supposed to be used to
add columns used by an index to a bitmap that already has columns
of a primary key - for engines where a primary key is part of every
index.

The correct fix is to change mark_columns_used_by_index() to
take into account extended keys.

this reverts 1d0acc77 and cf97cbd1

it should not mark base columns that a vcol depends on, because
keyread (on a vcol) will not read them

use table->mark_columns_used_by_index, don't copy it

use table->mark_columns_used_by_index, don't copy it

Do *not* modify write_set.
keyread only affects what columns are *read*, UPDATE statement can
*write* into columns that aren't part of the keyread.

move the check where it make sense, remove incorrect comment

EINVAL means that the filesystem doesn't support posix_fallocate().

There were two places where this error was issued, one checked for
EINVAL, the other did not. This commit fixed the other place
to also check for EINVAL.

Also, remove the space after the REFMAN to get the valid url
with no space in the middle.

Also don't say "Make sure the file system supports this function."
when posix_fallocate() fails, because this message is only shown
when the filesystem does support this function.

fix Item_default_value not to pretend being const_item
if the field's default_value expression isn't parsed yet

MDEV-11750 Assertion `vfield' failed in TABLE::update_virtual_fields after crash recovery on corrupted MyISAM table

Adjust the length of the BIT field
(same as in _mi_put_key_in_record())

move TABLE::key_read into handler. Because in index merge and DS-MRR
there can be many handlers per table, and some of them use
key read while others don't. "keyread" is really per handler,
not per TABLE property.

MDEV-11784 View is created with invalid definition which causes ERROR 1241 (21000): Operand should contain 1 column(s)

set the correct print precedence for IN subqueries

Item_default_value::calculate was updating table->s->default_values,
but it is supposed to be read-only

Optionally do table->update_default_fields() even for INSERT
that supposedly provides values for all column. Because these
"values" might be DEFAULT, which would need table->update_default_fields()
at the end.

Also set Item_default_value::used_tables() from the default expression.
Non-zero used_field() means that mysql_insert() will initialize all
fields to their default values (with restore_record()) even if
all columns are later provided with values. Because default expressions
may refer to other columns and they must be initialized.

Remove also the field trx_t::read_write that was only used by
the Memcached hooks.

Oracle introduced a Memcached plugin interface to the InnoDB
storage engine in MySQL 5.6. That interface is essentially a
fork of Memcached development snapshot 1.6.0-beta1 of an old
development branch 'engine-pu'.

To my knowledge, there have not been any updates to the Memcached code
between MySQL 5.6 and 5.7; only bug fixes and extensions related to
the Oracle modifications.

The Memcached plugin is not part of the MariaDB Server. Therefore it
does not make sense to include the InnoDB interfaces for the Memcached
plugin, or to have any related configuration parameters:

    innodb_api_bk_commit_interval
    innodb_api_disable_rowlock
    innodb_api_enable_binlog
    innodb_api_enable_mdl
    innodb_api_trx_level

Removing this code in one commit makes it possible to easily restore
it, in case it turns out to be needed later.

log_crypt_101_read_checkpoint(): Read the encryption information
from a MariaDB 10.1 checkpoint page.

log_crypt_101_read_block(): Attempt to decrypt a MariaDB 10.1
redo log page.

recv_log_format_0_recover(): Only attempt decryption on checksum
mismatch. NOTE: With the MariaDB 10.1 innodb_encrypt_log format,
we can actually determine from the cleartext portion of the redo log
whether the redo log is empty. We do not really have to decrypt the
redo log here, if we did not want to determine if the checksum is valid.

We can use log_sys->buf instead of recv_sys->last_block during
crash recovery. Remove the redundant buffer.

innodb_shutdown(), trx_sys_close(): Startup may be aborted between
purge_sys and trx_sys creation. Therefore, purge_sys must be freed
independently of trx_sys.

innobase_start_or_create_for_mysql(): Remember to free purge_queue if
it was not yet attached to purge_sys.

These are different bugs, but the fixing code is the same:
if window functions are used over implicit grouping then
now the execution should follow the general path calling
the function set in JOIN::first_select.