failed

The bug appeared as in MDEV-12387 setup_jtbm_semi_joins() procedure had been
devided into two functions, one called before optimization of WHERE clause
and another after this optimization. When the second function was called for
a degenerated jtbm semi join equalities connecting the subselect and
the parent select were created but invocation of fix_fields() for these
equalities was missing.

Adding new methods:

- virtual void Type_handler::Column_definition_reuse_fix_attributes()
  according to the MDEV description

- virtual uint32 Field::character_octet_length()
  To simplify handling of Column_definition::length for
  TEXT and VARCHAR columns (with and without compression).

The problem occurred because the Spider node was incorrectly handling
timestamp values sent to and received from the data nodes.

The problem has been corrected as follows:
- Added logic to set and maintain the UTC time zone on the data nodes.
  To prevent timestamp ambiguity, it is necessary for the data nodes to use
  a time zone such as UTC which does not have daylight savings time.
- Removed the spider_sync_time_zone configuration variable, which did not
  solve the problem and which interfered with the solution.
- Added logic to convert to the UTC time zone all timestamp values sent to
  and received from the data nodes.  This is done for both unique and
  non-unique timestamp columns.  It is done for WHERE clauses, applying to
  SELECT, UPDATE and DELETE statements, and for UPDATE columns.
- Disabled Spider's use of direct update when any of the columns to update is
  a timestamp column.  This is necessary to prevent false duplicate key value
  errors.
- Added a new test spider.timestamp to thoroughly test Spider's handling of
  timestamp values.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Merged:
  Commit 97cc9d34 on branch bb-10.3-MDEV-16246

Field_varstring::sql_type() did not calculate character length correctly.
Using char_length() instead of the bad code.

The problem occurs on Ubuntu where a Spider package is installed on the system
separately from the MariaDB package.  MariaDB and Spider upgrades leave the
Spider plugin improperly installed.  Spider is present in the mysql.plugin
table but is not present in information_schema.

The problem has been corrected in Spider's installation script.  Logic has
been added to check for Spider entries in both information_schema and
mysql.plugin.  If Spider is present in mysql.plugin but is not present in
information_schema, then Spider is first removed from mysql.plugin.  The
subsequent plugin install of Spider will insert entries in both mysql.plugin
and information_schema.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 0897d81 on branch bb-10.3-MDEV-15786

The problem occurs on Ubuntu where a Spider package is installed on the system
separately from the MariaDB package.  MariaDB and Spider upgrades leave the
Spider plugin improperly installed.  Spider is present in the mysql.plugin
table but is not present in information_schema.

The problem has been corrected in Spider's installation script.  Logic has
been added to check for Spider entries in both information_schema and
mysql.plugin.  If Spider is present in mysql.plugin but is not present in
information_schema, then Spider is first removed from mysql.plugin.  The
subsequent plugin install of Spider will insert entries in both mysql.plugin
and information_schema.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 0897d81 on branch bb-10.3-MDEV-15786

This helped fix MDEV-16779.

Declare all fields of purge_vcol_info_t private, and add
accessor functions.

During a table-rebuilding operation, the function table_name_parse()
can encounter a table name that starts with #sql. Here is an example
of a failure:

CURRENT_TEST: gcol.innodb_virtual_basic
mysqltest: At line 1204: query 'alter table t drop column d ' failed:
2013: Lost connection to MySQL server during query

Let us just remove these bogus debug assertions.

If the final renaming phase during ALTER TABLE never fails, it
should not do any harm to skip the purge. If it does fail, then
we might end up 'leaking' some delete-marked records in the
indexes on virtual columns of the original table, and these
garbage records would keep consuming space until the indexes are
dropped or the table is successfully rebuilt.

This is a regression caused by the fix of MDEV-15855.

purge_vcol_info_t::set_used(): Add a missing condition.

row_purge_poss_sec(): Invoke set_used() in order to
have !is_first_fetch() when retrying.

This should affect at least rec_printer() output.

Replace pairs of rw_lock_own() calls with
calls to rw_lock_own_flagged().

These calls are only present in debug builds.

There is only one temporary tablespace. Reserving a data member in
each read-write lock object for a Boolean flag seems like a huge waste,
especially because this field was only actually used in debug builds.

LatchDebug::check_order(): Compare to fil_system.temp_space->latch.

as a separate source for data

Actually MDEV-15867 and MDEV-16192 are same, Slave adds "or replace" to create
table stmt. So create table t1 is create or replace on slave. So this bug
is not because of replication, We can get this bug on general server if we
manually add or replace to create query.

Problem:- So if we try to create table t1 (same name as of temp table t1 ) via
   CREATE or replace TABLE t AS SELECT * FROM t;
Since in this query we are creating table from select * from t1 , we call
unique_table function to see whether if source and destination table are same.
But there is one issue unique_table does not account if source table is tmp table
in this case source and destination table can be same.

Solution:- We will change find_dup_table to not to look for temp table if
CHECK_DUP_SKIP_TEMP_TABLE flag is on.

mysql_install_db.sh script fixed.

Problem was as part of SET PASSWORD FOR ROOT, mysql.user table changed
compared to how it was originally created. (plugin changed value)

Use alloc() if we don't need original string (avoid copy)
Removed not needed test of str_length in sql_string.cc

This is a post-fix for MDEV-16542

The problem occurred because the Spider node was incorrectly handling
timestamp values sent to and received from the data nodes.

The problem has been corrected as follows:
- Added logic to set and maintain the UTC time zone on the data nodes.
  To prevent timestamp ambiguity, it is necessary for the data nodes to use
  a time zone such as UTC which does not have daylight savings time.
- Removed the spider_sync_time_zone configuration variable, which did not
  solve the problem and which interfered with the solution.
- Added logic to convert to the UTC time zone all timestamp values sent to
  and received from the data nodes.  This is done for both unique and
  non-unique timestamp columns.  It is done for WHERE clauses, applying to
  SELECT, UPDATE and DELETE statements, and for UPDATE columns.
- Disabled Spider's use of direct update when any of the columns to update is
  a timestamp column.  This is necessary to prevent false duplicate key value
  errors.
- Added a new test spider.timestamp to thoroughly test Spider's handling of
  timestamp values.

Author:
  Jacob Mathew.

Reviewer:
  Kentoku Shiba.

Cherry-Picked:
  Commit 97cc9d34 on branch bb-10.3-MDEV-16246

The parameter innodb_lock_schedule_algorithm was introduced in
MariaDB Server 10.1.19, 10.2.13, 10.3.4 as part of MDEV-11039.
In MariaDB 10.1, the default value of the parameter is 'fcfs',
that is, the existing algorithm is used by default. But in
later versions of MariaDB Server, the parameter was 'vats',
enabling the new algorithm.

Because the new algorithm is triggering a debug assertion failure
that suggests corruption of the transactional lock data structures,
we will revert to the old algorithm by default until we have
resolved the problem.

Assign "SELECT" to the table before it usage.

Problem:
========
Truncate operation holds MDL on the table (t1) and tries to
acquire InnoDB dict_operation_lock. Purge holds dict_operation_lock
and tries to acquire MDL on the table (t1) to evaluate virtual
column expressions for indexed virtual columns.
It leads to deadlock of purge and truncate table (DDL).

Solution:
=========
If purge tries to acquire MDL on the table then it should do the following:

i) Purge should release all innodb latches (including dict_operation_lock)
before acquiring metadata lock on the table.

ii) After acquiring metadata lock on the table, it should check whether the
table was dropped or renamed. If the table is dropped then purge should
ignore the undo log record. If the table is renamed then it should
release the old MDL and acquire MDL on the new name.

iii) Once purge acquires MDL, it should use the SQL table handle for all
the remaining virtual index for the purge record.

purge_node_t: Introduce new virtual column information to know whether
the MDL was acquired successfully.

This is joint work with Marko Mäkelä.

Add an explicit redo log flush. In this test
innodb_flush_log_at_trx_commit was 2 by default.
It is also possible that this failure occurs because of MDEV-15740.

1. The changed variant did not fail without the patch for MDEV-16629
while the original test case did fail.
2. In any case the test case should go to cte_recursive_not_embedded.test
that was not created yet.