Related to FIXME here. Now check_bulk_buffer() returns false.

3389                    if (auto t= trx->check_bulk_buffer(index->table)) {
3390                            /* MDEV-25036 FIXME:
3391                            row_ins_check_foreign_constraint() check
3392                            should be done before buffering the insert
3393                            operation. */
3394                            ut_ad(index->table->skip_alter_undo
3395                                  || !trx->check_foreigns);
3396                            return t->bulk_insert_buffered(*entry, *index, trx);
3397                    }

This commit is not intended to be pushed into main branch. Its mission
is to simplify the review process. The comments here are in different
style: // and without indentation. The reviewer may ask for some of
them to be included into the task code. In that case they will be
reformatted and moved to the task commit.

The patch adds the ability to run foreign keys in partitioned tables with
limitations.

Example:

  create or replace table t1 (id int primary key) engine innodb;
  create or replace table t2 (fk int references t1(id)) engine innodb
  partition by hash (fk) partitions 2;

Limitations:

  1. Foreign keys cannot refer partitioned table even SYSTEM_TIME-partitioned
  still. create_foreign_keys() at InnoDB layer receives Foreign_key about
  referenced table T, but no such table exists in InnoDB layer, only partition
  tables T#P#p0, T#P#p1, ..., T#P#pn. Finding out it is SYSTEM_TIME partitioning
  and current partition at that point is impossible without modification of
  SYS_TABLES or opening TABLE object of referenced table. Both should be avoided
  as this is superseded by MDEV-12483.

  2. CASCADE and SET NULL actions are disabled in partitioned foreign table as
  these actions update row data and this is the subject of row placement into
  another partition but it cannot be done in InnoDB layer. DELETE CASCADE for
  SYSTEM_TIME partitioning requires the row to be moved from current to history
  partition.

The task is basically divided into 3 parts:

  1. Remove prohibiting code, allow FKs to be created for partitions;
  2. Allow partitioned FKs at SQL layer for such functions as SHOW CREATE;
  3. Implement correct handling of FKs when partitioning configuration changes.

1. Remove prohibiting code, allow FKs to be created for partitions

  In SYS_FOREIGN table foreign key records are unique by ID which was taken from
  constraint name or automatically generated. Normally foreign ID and constraint
  name are identical, but that's not the case for partitioned table as InnoDB
  holds foreign keys per dict_table_t object and each partition is a different
  table for InnoDB. So for each foreign key in SQL layer there is a set of foreign
  keys in InnoDB layer per each partition (except SYSTEM_TIME partitioning where
  we keep foreign keys only for current partition). To constitute unique foreign
  ID at InnoDB layer we concatenate constraint name with partition suffix, the one
  what is added to partition table name beginning with #P# and optionally
  containing #SP# for subpartitions. Constraint name and partitioning suffix are
  separated by \xFF character which is the safe character code non-clashing with
  identifier character set.

  When we return back foreign ID to SQL layer this partitioning suffix is stripped
  off the constraint name and SQL output receives the name similar to
  non-partitioned table.

  User may see a bit more truthful version of foreign ID in
  INFORMATION_SCHEMA.INNODB_SYS_FOREIGN with \xFF replaced by ':' and #P# or
  everything starting from #P# and ending by #SP# chopped out. So he may see
  corresponding partition name or subpartition name in ID of INNODB_SYS_FOREIGN.

2. Allow partitioned FKs at SQL layer for such functions as SHOW CREATE

  Through standard handler interface get_foreign_key_list() foreign keys are
  returned to SQL layer. For SYSTEM_TIME partitioning from current partition, for
  any other partitioning from first read-marked partition.

3. Implement correct handling of FKs when partitioning configuration changes

  ALTER operations such as ADD PARTITION, DROP PARTITION, REMOVE PARTITIONING,
  etc. are reflected into correct configuration of foreign keys in InnoDB.
  Handling of foreign key ID for temporary tables in ALTER was done based on
  MDEV-28933.

Use temporary constraint names for temporary tables. The constraints
are not added to cache (skipped in dict_table_rename_in_cache()).

The scheme for temporary constraint names is as follows:

    for old table: db_name/\xFFconstraint_name
    for new table: db_name/\xFF\xFFconstraint_name

normalize_table_name_c_low(): wrong comparison "less than FN_REFLEN -
1". Somewhere array of FN_REFLEN includes the trailing 0, somewhere
array of FN_REFLEN + 1 includes trailing 0, but nowhere array of
FN_REFLEN - 1 must include trailing 0.

Change from original MDEV-28933 fix:

As temporary FK is now required for any ALTER operation (especially
partitioning operations) row_rename_table_for_mysql() does FK rename
on both RENAME_FK and RENAME_ALTER_COPY (see do_rename_fk).

All InnoDB internal SQL functions should be moved to sql_funcs.h

  Based on Marko Makela's patch 4a591d4:

  dict_get_referenced_table(): Let the caller convert names when needed.

  row_rename_table_for_mysql(): Specify the treatment of FOREIGN KEY
  constraints in a 4-valued enum parameter. In cases where FOREIGN KEY
  constraints cannot exist (partitioned tables, or internal tables of
  FULLTEXT INDEX), we can use the mode RENAME_IGNORE_FK.
  The mod RENAME_REBUILD is for any DDL operation that rebuilds the
  table inside InnoDB, such as TRUNCATE and native ALTER TABLE
  (or OPTIMIZE TABLE). The mode RENAME_ALTER_COPY is used solely
  during non-native ALTER TABLE in ha_innobase::rename_table().
  Normal ha_innobase::rename_table() will use the mode RENAME_FK.

  CREATE OR REPLACE will rename the old table (if one exists) along
  with its FOREIGN KEY constraints into a temporary name. The replacement
  table will be initially created with another temporary name.
  Unlike in ALTER TABLE, all FOREIGN KEY constraints must be renamed
  and not inherited as part of these operations, using the mode RENAME_FK.

  dict_get_referenced_table(): Let the callers convert names when needed.

  create_table_info_t::create_foreign_keys(): CREATE OR REPLACE creates
  the replacement table with a temporary name table, so for
  self-references foreign->referenced_table will be a table with
  temporary name and charset conversion must be skipped for it.

Return true for temporary partitions.

When there are GCC-incompatible compiler flags dtrace fails like this:

gcc: error: unrecognized command-line option ‘-fno-limit-debug-info’
gcc: error: unrecognized command-line option ‘-mbranches-within-32B-boundaries’
"gcc .dtrace-temp.3fd6bacf.c" failed
Usage /usr/bin/dtrace [--help] [-h | -G] [-C [-I<Path>]] -s File.d [-o <File>]

Alexey Botchkov complains there are problems with empty curly brackets
initializer in old-ish gcc(4.9)

Can't verify, but making all members are initialized by
default constructor is easy enough.

The recent commit 4ca355d8 (MDEV-33894)
caused a serious regression for online InnoDB ib_logfile0 resizing,
breaking crash-safety unless the memory-mapped log file interface is
being used. However, the log resizing was broken also before this.

To prevent such regressions in the future, we extend the test
innodb.log_file_size_online with a kill and restart of the server
and with some writes running concurrently with the log size change.
When run enough many times, this test revealed all the bugs that
are being fixed by the code changes.

log_t::resize_start(): Do not allow the resized log to start before
the current log sequence number. In this way, there is no need to
copy anything to the first block of resize_buf. The previous logic
regarding that was incorrect in two ways. First, we would have to
copy from the last written buffer (buf or flush_buf). Second, we failed
to ensure that the mini-transaction end marker bytes would be 1
in the buffer. If the source ib_logfile0 had wrapped around an odd number
of times, the end marker would be 0. This was occasionally observed
when running the test innodb.log_file_size_online.

log_t::resize_write_buf(): To adjust for the resize_start() change,
do not write anything that would be before the resize_lsn.
Take the buffer (resize_buf or resize_flush_buf) as a parameter.
Starting with commit 4ca355d8
we no longer swap buffers when rewriting the last log block.

log_t::append(): Define as a static function; only some debug
assertions need to refer to the log_sys object.

innodb_log_file_size_update(): Wake up the buf_flush_page_cleaner()
if needed, and wait for it to complete a batch while waiting for
the log resizing to be completed. If the current LSN is behind the
resize target LSN, we will write redundant FILE_CHECKPOINT records to
ensure that the log resizing completes. If the buf_pool.flush_list is
empty or the buf_flush_page_cleaner() is stuck for some reason, our wait
will time out in 5 seconds, so that we can periodically check if the
execution of SET GLOBAL innodb_log_file_size was aborted. Previously,
we could get into a busy loop here while the buf_flush_page_cleaner()
would remain idle.

Added missing method of Item_static_float_func

  --quick-max-column-width parameter added to limit field
    width in --quick mode.

recv_recovery_from_checkpoint_start(): Abort startup due to log
corruption if we were unable to parse the entire log between
the latest log checkpoint and the corresponding FILE_CHECKPOINT record.

Also, reduce some code bloat related to log output and log_sys.mutex.

Reviewed by: Debarun Banerjee

Changing the alias LOCALTIME->CURRENT_TIMESTAMP to LOCALTIME->CURRENT_TIME.

This changes the return type of LOCALTIME from DATETIME to TIME,
according to the SQL Standard.

This is the CS part of the implementation of MENT-2070.

In commit fa8a46eb (MDEV-33613)
the parameter innodb_lru_flush_size ceased to have any effect.

Let us declare the parameter as deprecated and additionally as
MARIADB_REMOVED_OPTION, so that there will be a warning written
to the error log in case the option is specified in the command line.

Let us also do the same for the parameter
innodb_purge_rseg_truncate_frequency
that was deprecated&ignored earlier in MDEV-32050.

Reviewed by: Debarun Banerjee

Add Oracular to the allowed Ubuntu names.

Let us avoid EXTENDED in the CHECK TABLE after a defragmentation,
because it would occasionally report an orphan delete-marked record
in the index "third". That error does not seem to be reproducible
when using the regular OPTIMIZE TABLE.

Also, let us make the test --repeat safe by removing the defragmentation
related statistics after DROP TABLE.

The defragmentation feature was removed in later releases in
commit 7ca89af6 (MDEV-30545)
along with this test case.

Also restored a change that resulted in off-by-one, as well as
appending the correctly indexed key_hint.

purge_sys_t::stop_FTS(): Fix an incorrect debug assertion that
commit d58734d7 added.
The assertion would fail if there had been prior invocations of
purge_sys.stop_SYS() without purge_sys.resume_SYS().
The intention of the assertion is to check that number of pending
stop_FTS() stays below 65536.

The test was expecting the I/O thread to be in a specific state, but thread
scheduling may cause it to not yet have reached that state. So just have a
loop that waits for the expected state to occur.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Remove the test for MDEV-14528. This is supposed to test that parallel
replication from pre-10.0 master will update Seconds_Behind_Master. But
after MDEV-12179 the SQL thread is blocked from even beginning to fetch
events from the relay log due to FLUSH TABLES WITH READ LOCK, so the test
case is no longer testing what is was intended to. And pre-10.0 versions are
long since out of support, so does not seem worthwhile to try to rewrite the
test to work another way.

The root cause of the test failure is MDEV-34778. Briefly, depending on
exact timing during slave stop, the rli->sql_thread_caught_up flag may end
up with different value. If it ends up as "true", this causes
Seconds_Behind_Master to be 0 during next slave start; and this caused test
case timeout as the test was waiting for Seconds_Behind_Master to become
non-zero.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Depending on timing, an extra event run could start just when the event
scheduler is shut down and delay running until after the table has been
dropped; this would cause the test to fail with a "table does not exist"
error in the log.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

(Revert a change done by mistake when XtraDB was removed.)
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>

Before doing mark_start_commit(), check that there is no pending deadlock
kill. If there is a pending kill, we won't commit (we will abort, roll back,
and retry). Then we should not mark the commit as started, since that could
potentially make the following GCO start too early, before we completed the
commit after the retry.

This condition could trigger in some corner cases, where InnoDB would take
temporarily table/row locks that are released again immediately, not held
until the transaction commits. This happens with dict_stats updates and
possibly auto-increment locks.

Such locks can be passed to thd_rpl_deadlock_check() and cause a deadlock
kill to be scheduled in the background. But since the blocking locks are
held only temporarily, they can be released before the background kill
happens. This way, the kill can be delayed until after mark_start_commit()
has been called. Thus we need to check the synchronous indication
rgi->killed_for_retry, not just the asynchroneous thd->killed.
Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>