Issue:
------
Prefix for externally stored columns were being stored in online_log when a
table is altered and alter causes table to be rebuilt. Space in online_log is
limited and if length of prefix of externally stored columns is very big, then
it is being written to online log without making sure if it fits. This leads to
memory corruption.

Fix:
----
After fix for Bug#16544143, there is no need to store prefixes of externally
stored columnd in online_log. Thus remove the code which stores column prefixes
for externally stored columns. Also, before writing anything on online_log,
make sure it fits to available memory to avoid memory corruption.

Read RB page for more details.
Reviewed-by: Annamalai Gurusami <annamalai.gurusami@oracle.com>
RB: 18239

a better fix

fix another similar line

followup for 7828ba0d

commit c545d544378483bf68c4fb713cd4ec872d0df2c4
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Tue Mar 27 23:13:14 2018 +0200

    - Fix MDEV-15577 CONNECT engine JDBC remote index prevents UPDATE
      Fixed in TDBJDBC::OpenDB because query can be null for updates
      modified:   storage/connect/tabjdbc.cpp

commit c4f1fe2e8e74566e9c3296f435448d9f5c718b23
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Wed Mar 14 12:21:17 2018 +0100

    - Fix MDEV-15429 CONNECT engine JDBC handling Postgresql UUID type
      Also handle Postgresql sending type VARCHAR for TEXT column and
      setting length to b x7FFFFFF when the length is unknown.
      modified:   storage/connect/Client.java
      modified:   storage/connect/JavaWrappers.jar
      modified:   storage/connect/JdbcInterface.java
      modified:   storage/connect/PostgresqlInterface.java
      modified:   storage/connect/global.h
      modified:   storage/connect/ha_connect.cc
      modified:   storage/connect/jdbconn.cpp
      modified:   storage/connect/jdbconn.h
      modified:   storage/connect/mysql-test/connect/r/jdbc_postgresql.result
      modified:   storage/connect/mysql-test/connect/t/jdbc_postgresql.test
      modified:   storage/connect/mysql-test/connect/t/jdbconn.inc
      modified:   storage/connect/plgdbsem.h
      modified:   storage/connect/tabjdbc.cpp
      modified:   storage/connect/tabjdbc.h
      added:      storage/connect/mysql-test/connect/std_data/JavaWrappers.jar

commit 3068bb4882a316c6c32c624609db2bf24ae3ad40
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Mon Feb 26 19:40:27 2018 +0100

    Revert to old cmake file to avoid compile error

commit da8c1c0bd6a327fd03bd321be3170f468122ef67
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Mon Feb 26 16:13:39 2018 +0100

    - Remove warning on not used tabtyp variable in connect_assisted_discovery
      modified:   storage/connect/ha_connect.cc

    - Fix a bug causing CONNECT to loop when expanding a JSON column
      when the expanded column value is null or void array.
    - Adding the FullArray option to JSON tables.
    - Skipping expanded JSON lines when the expanded column value is null.
      modified:   storage/connect/connect.cc
      modified:   storage/connect/tabdos.h
      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/tabjson.h

    - Fix MDEV-13353 No file privilege for ODBC, JDBC, MONGO, MAC, WMI file types.
      modified:   storage/connect/ha_connect.cc

    - Make some Json UDFs  to accept a non JSON item as 1st parameter.
      modified:   storage/connect/json.cpp
      modified:   storage/connect/json.h
      modified:   storage/connect/jsonudf.cpp
      modified:   storage/connect/jsonudf.h

    - Update Json UDF tests to cope with above changes.
      modified:   storage/connect/mysql-test/connect/r/json_udf.result
      modified:   storage/connect/mysql-test/connect/r/json_udf_bin.result
      modified:   storage/connect/mysql-test/connect/r/vcol.result
      modified:   storage/connect/mysql-test/connect/t/json_udf.test
      modified:   storage/connect/mysql-test/connect/t/vcol.test

    - Fix some compiler warning treated as error
      PlugSubAlloc no more exported because it does throw.
      modified:   storage/connect/global.h
      modified:   storage/connect/ha_connect.cc
      modified:   storage/connect/jsonudf.cpp
      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/tabjson.h

    - Other files modified (?) when going back to wrong merge
      modified:   storage/connect/CMakeLists.txt
      modified:   storage/connect/array.cpp
      modified:   storage/connect/colblk.cpp
      modified:   storage/connect/connect.cc
      modified:   storage/connect/csort.cpp
      modified:   storage/connect/domdoc.cpp
      modified:   storage/connect/filamap.cpp
      modified:   storage/connect/filamgz.cpp
      modified:   storage/connect/filamtxt.cpp
      modified:   storage/connect/filamzip.cpp
      modified:   storage/connect/filter.cpp
      modified:   storage/connect/fmdlex.c
      modified:   storage/connect/jdbconn.cpp
      modified:   storage/connect/macutil.cpp
      modified:   storage/connect/myconn.cpp
      modified:   storage/connect/odbconn.cpp
      modified:   storage/connect/plgdbutl.cpp
      modified:   storage/connect/plugutil.cpp
      modified:   storage/connect/preparse.h
      modified:   storage/connect/rcmsg.c
      modified:   storage/connect/rcmsg.h
      modified:   storage/connect/reldef.cpp
      modified:   storage/connect/tabdos.cpp
      modified:   storage/connect/tabfmt.cpp
      modified:   storage/connect/tabmac.cpp
      modified:   storage/connect/tabmul.cpp
      modified:   storage/connect/tabmysql.cpp
      modified:   storage/connect/tabmysql.h
      modified:   storage/connect/tabodbc.cpp
      modified:   storage/connect/tabtbl.cpp
      modified:   storage/connect/tabxml.cpp
      modified:   storage/connect/value.cpp
      modified:   storage/connect/xobject.cpp

commit 5095628b31ed8c94bd7c794c6b0162894a9040b1
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Tue Feb 13 14:29:00 2018 +0100

    - Fix a bug causing CONNECT to loop when expanding a JSON column
      when the expanded column value is null or void array.
    - Adding the FullArray option to JSON tables.
    - Skipping expanded JSON lines when the expanded column value is null.
      modified:   storage/connect/connect.cc
      modified:   storage/connect/tabdos.h
      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/tabjson.h

commit 9896174328da42016735630c893fd09eb128b48e
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Fri Feb 2 15:48:57 2018 +0100

    - Remove warning on not used tabtyp variable in connect_assisted_discovery
      modified:   storage/connect/ha_connect.cc

commit d7e254c05f88173eb04d8dafc813013eb42ccdbe
Author: Olivier Bertrand <bertrandop@gmail.com>
Date:   Wed Jan 31 14:42:45 2018 +0100

    - Use delayed load for the MongoDB C Drive on Windows
      modified:   storage/connect/CMakeLists.txt
      modified:   storage/connect/cmgoconn.cpp
      modified:   storage/connect/ha_connect.cc

    - Add FORCE to the connect_type_conv enum values
      This will translate binary values to TYPE_STRING
      modified:   storage/connect/checklvl.h
      modified:   storage/connect/ha_connect.cc
      modified:   storage/connect/odbconn.cpp

    - Change the connect_xtrace variable to from int to set
      modified:   storage/connect/array.cpp
      modified:   storage/connect/blkfil.cpp
      modified:   storage/connect/block.h
      modified:   storage/connect/cmgoconn.cpp
      modified:   storage/connect/colblk.cpp
      modified:   storage/connect/connect.cc
      modified:   storage/connect/filamap.cpp
      modified:   storage/connect/filamdbf.cpp
      modified:   storage/connect/filamfix.cpp
      modified:   storage/connect/filamgz.cpp
      modified:   storage/connect/filamtxt.cpp
      modified:   storage/connect/filamvct.cpp
      modified:   storage/connect/filamzip.cpp
      modified:   storage/connect/filter.cpp
      modified:   storage/connect/global.h
      modified:   storage/connect/ha_connect.cc
      modified:   storage/connect/inihandl.cpp
      modified:   storage/connect/javaconn.cpp
      modified:   storage/connect/jdbconn.cpp
      modified:   storage/connect/jmgfam.cpp
      modified:   storage/connect/jmgoconn.cpp
      modified:   storage/connect/json.cpp
      modified:   storage/connect/jsonudf.cpp
      modified:   storage/connect/libdoc.cpp
      modified:   storage/connect/mongo.cpp
      modified:   storage/connect/mycat.cc
      modified:   storage/connect/myconn.cpp
      modified:   storage/connect/odbconn.cpp
      modified:   storage/connect/plgdbutl.cpp
      modified:   storage/connect/plugutil.cpp
      modified:   storage/connect/reldef.cpp
      modified:   storage/connect/tabcol.cpp
      modified:   storage/connect/tabdos.cpp
      modified:   storage/connect/tabext.cpp
      modified:   storage/connect/tabfix.cpp
      modified:   storage/connect/tabfmt.cpp
      modified:   storage/connect/tabjdbc.cpp
      modified:   storage/connect/tabjson.cpp
      modified:   storage/connect/table.cpp
      modified:   storage/connect/tabmul.cpp
      modified:   storage/connect/tabmysql.cpp
      modified:   storage/connect/tabodbc.cpp
      modified:   storage/connect/tabpivot.cpp
      modified:   storage/connect/tabsys.cpp
      modified:   storage/connect/tabtbl.cpp
      modified:   storage/connect/tabutil.cpp
      modified:   storage/connect/tabvct.cpp
      modified:   storage/connect/tabwmi.cpp
      modified:   storage/connect/tabxml.cpp
      modified:   storage/connect/user_connect.cc
      modified:   storage/connect/valblk.cpp
      modified:   storage/connect/value.cpp
      modified:   storage/connect/xindex.cpp

    - Restore connect_enable_mongo variable (but undocumented)
      modified:   storage/connect/ha_connect.cc
      modified:   storage/connect/mycat.cc
      modified:   storage/connect/mysql-test/connect/r/json_java_2.result
      modified:   storage/connect/mysql-test/connect/r/json_java_3.result
      modified:   storage/connect/mysql-test/connect/r/json_mongo_c.result
      modified:   storage/connect/mysql-test/connect/r/mongo_c.result
      modified:   storage/connect/mysql-test/connect/r/mongo_java_2.result
      modified:   storage/connect/mysql-test/connect/r/mongo_java_3.result
      modified:   storage/connect/mysql-test/connect/r/tbl_thread.result
      modified:   storage/connect/mysql-test/connect/t/mongo.inc
      modified:   storage/connect/mysql-test/connect/t/mongo_test.inc
      modified:   storage/connect/mysql-test/connect/t/tbl_thread.test

MDEV-15456 Server crashes upon adding or dropping a partition in ALTER under LOCK TABLE after ER_SAME_NAME_PARTITION

followup for bcb36ee2

move reopening before trans_commit_stmt(thd);

This is a backport of MDEV-11429 from 10.1

multiple times with different arguments.

If the ON expression of an outer join is an OR formula with one
of the disjunct being a constant formula then the expression
cannot be null-rejected if the constant formula is true. Otherwise
it can be null-rejected and if so the outer join can be converted
into inner join. This optimization was added in the patch for
mdev-4817. Yet the code had a defect: if the query was used in
a stored procedure with parameters and the constant item contained
some of them then the value of this constant item depended on the
values of the parameters. With some parameters it may be true,
for others not. The validity of conversion to inner join is checked
only once and it happens only for the first call of procedure.
So if the  parameters in the first call allowed the conversion it
was done and next calls used the transformed query though there
could be calls whose parameters made the conversion invalid.

Fixed by cheking whether the constant disjunct in the ON expression
originally contained an SP parameter. If so the expression is not
considered as null-rejected. For this check a new item's attribute
was intruduced: Item::with_param. It is calculated for each item
by fix fields() functions.
Also moved the call of optimize_constant_subqueries() in
JOIN::optimize after the call of simplify_joins(). The reason
for this is that after the optimization introduced by the patch
for mdev-4817 simplify_joins() can use the results of execution
of non-expensive constant subqueries and this is not valid.

MDEV-13699: Assertion `!new_field->field_name.str || strlen(new_field->field_name.str) == new_field->field_name.length' failed in create_tmp_table on 2nd execution of PS with semijoin

The problem was that SJ (semi-join) used secondary list (array) of subquery select list. The items there was prepared once then cleaned up (but not really freed from memory because it was made in statement memory).
Original list was not prepared after first execution because select was removed by conversion to SJ.
The solution is to use original list but prepare it first.

dict_load_table_low(): When flagging an error, assign *table = NULL.
Failure to do so could cause a crash if an error was flagged when
accessing INFORMATION_SCHEMA.INNODB_SYS_TABLES.

While the test case crashes a MariaDB 10.2 debug build only,
let us apply the fix to the earliest applicable MariaDB series (10.0)
to avoid any data corruption on a table-rebuilding ALTER TABLE
using ALGORITHM=INPLACE.

innobase_create_key_defs(): Use altered_table->s->primary_key
when a new primary key is being created.

followup for a3c980b3

same change in Locked_tables_list::unlink_from_list(), otherwise
thd->locked_tables_list will keep pointers to a free'd TABLE if
prelocked under lock tables.

This fixes a crash in main.create_or_replace on debug Win builds
after bcb36ee2

do_stmt_close() is embedded-aware.

this fixes the failure of innodb.innodb_bug48024 --ps --embed

MDEV-15456 Server crashes upon adding or dropping a partition in ALTER under LOCK TABLE after ER_SAME_NAME_PARTITION

ALTER TABLE ... ADD PARTITION modifies the open TABLE structure,
and sets table->need_reopen=1 to reset these modifications
in case of an error.

But under LOCK TABLES the table isn't get reopened, despite need_reopen.

Fixed by reopening need_reopen tables under LOCK TABLE.

bad merge

in a specially crafted invalid packet, one can get end_pos < pos here

disable online alter add primary key for innodb, if the
table is opened/locked more than once in the current connection

(see assert in ha_innobase::add_index())

test case

one cannot do keyread on spatial indexes.