We introduce the following settable Boolean global variables:

innodb_log_file_write_through: Whether writes to ib_logfile0 are
write-through (disabling any caching, as in O_SYNC or O_DSYNC).

innodb_data_file_write_through: Whether writes to any InnoDB data files
(including the temporary tablespace) are write-through.

innodb_data_file_buffering: Whether the file system cache is enabled
for InnoDB data files.

All these parameters are OFF by default, that is, the file system cache
will be disabled, but any hardware caching is enabled, that is,
explicit calls to fsync(), fdatasync() or similar functions are needed.

On systems that support FUA it may make sense to enable write-through,
to avoid extra system calls.

If the deprecated read-only start-up parameter is set to one of the
following values, then the values of the 4 Boolean flags (the above 3
plus innodb_log_file_buffering) will be set as follows:

O_DSYNC:
innodb_log_file_write_through=ON, innodb_data_file_write_through=ON,
innodb_data_file_buffering=OFF, and
(if supported) innodb_log_file_buffering=OFF.

fsync, littlesync, nosync, or (Microsoft Windows specific) normal:
innodb_log_file_write_through=OFF, innodb_data_file_write_through=OFF,
and innodb_data_file_buffering=ON.

Note: fsync() or fdatasync() will only be disabled if the separate
parameter debug_no_sync (in the code, my_disable_sync) is set.

In mariadb-backup, the parameter innodb_flush_method will be ignored.

The Boolean parameters can be modified by SET GLOBAL while the
server is running. This will require reopening the ib_logfile0
or all currently open InnoDB data files.

We will open files straight in O_DSYNC or O_SYNC mode when applicable.
Data files we will try to open straight in O_DIRECT mode when the
page size is at least 4096 bytes. For atomically creating data files,
we will invoke os_file_set_nocache() to enable O_DIRECT afterwards,
because O_DIRECT is not supported on some file systems. We will also
continue to invoke os_file_set_nocache() on ib_logfile0 when
innodb_log_file_buffering=OFF can be fulfilled.

For reopening the ib_logfile0, we use the same logic that was developed
for online log resizing and reused for updates of
innodb_log_file_buffering.

Reopening all data files is implemented in the new function
fil_space_t::reopen_all().

Reviewed by: Vladislav Vaintroub
Tested by: Matthias Leich

Before commit 6112853c in MySQL 4.1.1
introduced the parameter innodb_file_per_table, all InnoDB data was
written to the InnoDB system tablespace (often named ibdata1).
A serious design problem is that once the system tablespace has grown to
some size, it cannot shrink even if the data inside it has been deleted.

There are also other design problems, such as the server hang MDEV-29930
that should only be possible when using innodb_file_per_table=0 and
innodb_undo_tablespaces=0 (storing both tables and undo logs in the
InnoDB system tablespace).

The parameter innodb_change_buffering was deprecated
in commit b5852ffb.
Starting with commit baf276e6
(MDEV-19229) the number of innodb_undo_tablespaces can be increased,
so that the undo logs can be moved out of the system tablespace
of an existing installation.

If all these things (tables, undo logs, and the change buffer) are
removed from the InnoDB system tablespace, the only variable-size
data structure inside it is the InnoDB data dictionary.

DDL operations on .ibd files was optimized in
commit 86dc7b4d (MDEV-24626).
That should have removed any thinkable performance advantage of
using innodb_file_per_table=0.

Since there should be no benefit of setting innodb_file_per_table=0,
the parameter should be deprecated. Starting with MySQL 5.6 and
MariaDB Server 10.0, the default value is innodb_file_per_table=1.

don't include my_progname in the error message, my_error starts from it
automatically, resulting in, like

/usr/bin/mysqladmin: Notice: /usr/bin/mysqladmin is deprecated and will be removed in a future release, use command 'mariadb-admin'

and remove "Notice" so that the problem description would directly
follow the executable name.

make the check to work when the executable is in the PATH
(so, invoked simply like 'mysql' and thus readlink cannot find it)

fix the check in mysql_install_db and mysql_secure_installation to not
print the warning if the intermediate path contains "mysql" substring

add this message also to
* mysql_waitpid
* mysql_convert_table_format
* mysql_find_rows
* mysql_setpermissions
* mysqlaccess
* mysqld_multi
* mysqld_safe
* mysqldumpslow
* mysqlhotcopy
* mysql_ldb

Closes #2273

Eventually mysql symlinks will go away, as MariaDB and MySQL keep
diverging and we do not want to make it impossible to install
MariaDB and MySQL side-by-side when users want it.

It also useful if people start using MariaDB tools with MariaDB.

If the exe doesn't begine with "mariadb" or is a symlink,
print a warning to use the resolved name.

In my_readlink, add check on my_thread_var as its used by comp_err
and other build utils that also use my_init.

including patches from Andrei Elkin

try to make them less confusing for users.
Hopefully, if the version string will be changed like

- mariadb Ver 15.1 Distrib 10.11.2-MariaDB for Linux (x86_64)
+ mariadb from 10.11.2-MariaDB, client 15.1 for Linux (x86_64)

users will be less inclined to reply "15.1" to the question
"what mariadb version are you using?"

it should now always be

/path/to/exe Ver <tool version> Distrib <server version> for <OS> (<ARCH>)

in all tools and clients

SUPER privilege used to allow various actions that were alternatively
allowed by one of BINLOG ADMIN, BINLOG MONITOR, BINLOG REPLAY,
CONNECTION ADMIN, FEDERATED ADMIN, REPL MASTER ADMIN, REPL SLAVE ADMIN,
SET USER, SLAVE MONITOR.

Now SUPER no longer does that, one has to grant one of the fine-grained
privileges above to be to perform corresponding actions.

On upgrade from MariaDB versions 10.11 and below all the privileges
above are granted automatically if the user has SUPER.

As a side-effect, such an upgrade will allow SUPER-user to run SHOW
BINLOG EVENTS, SHOW RELAYLOG EVENTS, SHOW SLAVE HOSTS, even if he wasn't
able to do it before the upgrade.

no longer needed, MySQL replication was fixed meanwhile.

client code still can recognize and strip the prefix though.

The conn_kind, which stands for "connection kind", is no longer useful
because the HandlerSocket support is deleted and Spider now has only
one connection kind, SPIDER_CONN_KIND_MYSQL. Remove conn_kind and
related code.
Signed-off-by: Yuchen Pei <yuchen.pei@mariadb.com>
Reviewed-by: Nayuta Yanagisawa <nayuta.yanagisawa@mariadb.com>

Produced using the following command

unifdef -UITEM_FUNC_TIMESTAMPDIFF_ARE_PUBLIC -m storage/spider/spd_*
Signed-off-by: Yuchen Pei <yuchen.pei@mariadb.com>
Reviewed-by: nayuta.yanagisawa@mariadb.com

unifdef -DSPIDER_FIELD_FIELDPTR_REQUIRES_THDPTR -m
storage/spider/*.{cc,h}

and removing the definition manually.
Signed-off-by: Yuchen Pei <yuchen.pei@mariadb.com>
Reviewed-by: Nayuta Yanagisawa <nayuta.yanagisawa@mariadb.com>

spider_rewrite plugin is not in 11.0. spider initialization code used
"11.0" for "in the distant future", but suddenly it's now

spider tests didn't expect anything beyond 10.x

commit_try_rebuild(): Only invoke trx_t::drop_table_statistics()
if both InnoDB statistics tables are accessible (and exclusively
locked by the current transaction). This avoids a crash due to
ut_a(sym_node->table != NULL) in pars_retrieve_table_def().

The crash was repeated on a partial copy of a MariaDB 10.3 data
directory that lacked the *.ibd files for the statistics tables.

fil_space_t::flush<false>(): If the CLOSING flag is set,
the file may already have been closed, resulting in EBADF
being returned by fdatasync(). In any case, the
thread that had set the flag should take care of invoking
os_file_flush_func().

The crash occurred during the execution of FLUSH TABLES...FOR EXPORT.

Tested by: Matthias Leich