1. 29 Dec, 2016 3 commits
    • Alexander Barkov's avatar
      An improvement for the MDEV-11514 patch: adding data type name into debug output · b3706c82
      Alexander Barkov authored
      Now after the patch for MDEV-11478 added a new method Type_handler::name(),
      using the new method in debug output to make test results for MDEV-11514
      more readable (func_debug.result).
      b3706c82
    • Alexander Barkov's avatar
    • Alexander Barkov's avatar
      MDEV-11672 mysql_list_field() returns wrong default values for VIEW · f6138883
      Alexander Barkov authored
      The problem happened because Item_ident_for_show::field_type() always
      returned MYSQL_TYPE_DOUBLE and ignored the actual data type of the
      referenced Field. As a result, the execution always used
      Item_ident_for_show::val_real() to send the default value of the field,
      so most default values for non-numeric types were displayed as '0'.
      
      This patch:
      1. Cleanup:
         a. Removes Send_field::charsetnr, as it's been unused since
            introduction of Item::charset_for_protocol() in MySQL-5.5.
         b. Adds the "const" qualifier to Field::char_length().
            This is needed for (5.a), see below.
      
      2. Introduces a new virtual method Type_handler::charset_for_protocol(),
         returning item->collation.collation for string data types, or
         &my_charset_bin for non-string data types.
      
      3. Changes Item::charset_for_protocol() from virtual to non-virtual.
         It now calls type_handler()->charset_for_protocol().
         As a good side effect, duplicate code in Item::charset_for_protocol() and
         Item_temporal_hybrid_func::charset_for_protocol() is now gone.
      
      4. Fixes Item_ident_for_show::field_type() to correctly return
         its data type according to the data type of the referenced field.
         This actually fixes the problem reported in MDEV-11672.
         Now the default value is sent using a correct method, e.g.
         val_str() for VARCHAR/TEXT, or val_int() for INT/BIGINT.
         This required additional changes:
         a. in DBUG_ASSERT in Protocol::store(const char *,size_t,CHARSET_INFO),
            This method is now used by mysqld_list_fields(), which
            (unlike normal SELECT queries) does not set
            field_types/field_pos/field_count.
         b. Item_ident_for_show::Item_ident_for_show() now set standard attributes
            (collation, decimals, max_length, unsigned_flag) according to the
            referenced field, to make charset_for_protocol() return the correct
            value and to make mysqld_list_fields() correctly send default
            values.
      
      5. In order to share the code between Item_field::set_field() and
         Item_ident_for_show::Item_ident_for_show():
         a. Introduces a new method Type_std_attributes::set(const Field*)
         b. To make (a) possible, moves Item::fix_char_length() from Item
            to Type_std_attributes, also moves char_to_byte_length_safe()
            from item.h to sql_type.h
         c. Additionally, moves Item::fix_length_and_charset() and
            Item::max_char_length() from Item to Type_std_attributes.
            This is not directly needed for the fix and is done just for symmetry
            with fix_char_length(), as these three methods are directly related
            to each other.
      f6138883
  2. 27 Dec, 2016 2 commits
  3. 24 Dec, 2016 4 commits
  4. 23 Dec, 2016 1 commit
    • Marko Mäkelä's avatar
      Follow-up for MDEV-11630 Call mutex_free() before freeing the mutex list · bbb3fb31
      Marko Mäkelä authored
      fil_tablespace_iterate(): Call fil_space_destroy_crypt_data() to
      invoke mutex_free() for the mutex_create() that was done in
      fil_space_read_crypt_data(). Also, remember to free
      iter.crypt_io_buffer.
      
      The failure to call mutex_free() would cause sync_latch_meta_destroy()
      to access freed memory on shutdown. This affected the IMPORT of
      encrypted tablespaces.
      bbb3fb31
  5. 22 Dec, 2016 4 commits
    • Marko Mäkelä's avatar
      MDEV-11635 innodb.innodb_mysql test hangs · 08f79bde
      Marko Mäkelä authored
      Copy and adapt the test from MySQL 5.7.17.
      08f79bde
    • Marko Mäkelä's avatar
      MDEV-11630 Call mutex_free() before freeing the mutex list · d6a1f9f1
      Marko Mäkelä authored
      fil_space_crypt_cleanup(): Call mutex_free() to pair with
      fil_space_crypt_init().
      
      fil_space_destroy_crypt_data(): Call mutex_free() to pair with
      fil_space_create_crypt_data() and fil_space_read_crypt_data().
      
      fil_crypt_threads_cleanup(): Call mutex_free() to pair with
      fil_crypt_threads_init().
      
      fil_space_free_low(): Invoke fil_space_destroy_crypt_data().
      
      fil_close(): Invoke fil_space_crypt_cleanup(), just like
      fil_init() invoked fil_space_crypt_init().
      
      Datafile::shutdown(): Set m_crypt_info=NULL without dereferencing
      the pointer. The object will be freed along with the fil_space_t
      in fil_space_free_low().
      Remove some unnecessary conditions (ut_free(NULL) is OK).
      
      srv_shutdown_all_bg_threads(): Shut down the encryption threads
      by calling fil_crypt_threads_end().
      
      srv_shutdown_bg_undo_sources(): Do not prematurely call
      fil_crypt_threads_end(). Many pages can still be written by
      change buffer merge, rollback of incomplete transactions, and
      purge, especially in slow shutdown (innodb_fast_shutdown=0).
      
      innobase_shutdown_for_mysql(): Call fil_crypt_threads_cleanup()
      also when innodb_read_only=1, because the threads will have been
      created also in that case.
      
      sync_check_close(): Re-enable the invocation of sync_latch_meta_destroy()
      to free the mutex list.
      d6a1f9f1
    • Sergey Vojtovich's avatar
      MDEV-11227 - mysqlimport -l doesn't issue UNLOCK TABLES · 0c3791ca
      Sergey Vojtovich authored
      Merged fix for innodb_mysql from 5.7.
      0c3791ca
    • Vladislav Vaintroub's avatar
  6. 21 Dec, 2016 3 commits
    • Alexander Barkov's avatar
      99c2493c
    • Sergey Vojtovich's avatar
      Updated test results · 27f20d19
      Sergey Vojtovich authored
      27f20d19
    • Sergey Vojtovich's avatar
      MDEV-11227 - mysqlimport -l doesn't issue UNLOCK TABLES · 8774a023
      Sergey Vojtovich authored
      Implementation of MDEV-7660 introduced unwanted incompatible change:
      modifications under LOCK TABLES with autocommit enabled are rolled back on
      disconnect. Previously everything was committed, because LOCK TABLES didn't
      adjust autocommit setting.
      
      This patch restores original behavior by reverting some changes done in
      MDEV-7660:
      - sql/sql_parse.cc: do not reset autocommit on LOCK TABLES
      - sql/sql_base.cc: do not set autocommit on UNLOCK TABLES
      - test cases: main.lock_tables_lost_commit, main.partition_explicit_prune,
        rpl.rpl_switch_stm_row_mixed, tokudb.nested_txn_implicit_commit,
        tokudb_bugs.db806
      
      But it makes InnoDB tables under LOCK TABLES ... READ [LOCAL] not protected
      against DML. To restore protection some changes from WL#6671 were merged,
      specifically MDL_SHARED_READ_ONLY and test cases.
      
      WL#6671 merge highlights:
      - Not all tests merged.
      - In MySQL LOCK TABLES ... READ acquires MDL_SHARED_READ_ONLY for all engines,
        in MariaDB MDL_SHARED_READ is always acquired first and then upgraded to
        MDL_SHARED_READ_ONLY for InnoDB only.
      - The above allows us to omit MDL_SHARED_WRITE_LOW_PRIO implementation in
        MariaDB, which is rather useless with InnoDB. In MySQL it is needed to
        preserve locking behavior between low priority writes and LOCK TABLES ... READ
        for non-InnoDB engines (covered by sys_vars.sql_low_priority_updates_func).
      - Omitted HA_NO_READ_LOCAL_LOCK, we rely on lock_count() instead.
      - Omitted "piglets": in MariaDB stream of DML against InnoDB table may lead to
        concurrent LOCK TABLES ... READ starvation.
      - HANDLER ... OPEN acquires MDL_SHARED_READ instead of MDL_SHARED in MariaDB.
      - Omitted SNRW->X MDL lock upgrade for IMPORT/DISCARD TABLESPAECE under LOCK
        TABLES.
      - Omitted strong locks for views, triggers and SP under LOCK TABLES.
      - Omitted IX schema lock for LOCK TABLES READ.
      - Omitted deadlock weight juggling for LOCK TABLES.
      
      Full WL#6671 merge status:
      - innodb.innodb-lock: fully merged
      - main.alter_table: not merged due to different HANDLER solution
      - main.debug_sync: fully merged
      - main.handler_innodb: not merged due to different HANDLER solution
      - main.handler_myisam: not merged due to different HANDLER solution
      - main.innodb_mysql_lock: fully merged
      - main.insert_notembedded: fully merged
      - main.lock: not merged (due to no strong locks for views)
      - main.lock_multi: not merged
      - main.lock_sync: fully merged (partially in MDEV-7660)
      - main.mdl_sync: not merged
      - main.partition_debug_sync: not merged due to different HANDLER solution
      - main.status: fully merged
      - main.view: fully merged
      - perfschema.mdl_func: not merged (no such test in MariaDB)
      - perfschema.table_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_global_2u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_global_2u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_global_4u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_global_4u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_hist_2u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_hist_2u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_hist_4u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_hist_4u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_thread_2u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_thread_2u_3t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_thread_4u_2t: not merged (didn't fail in MariaDB)
      - perfschema.table_lock_aggregate_thread_4u_3t: not merged (didn't fail in MariaDB)
      - sys_vars.sql_low_priority_updates_func: not merged
      - include/thr_rwlock.h: not merged, rw_pr_lock_assert_write_owner and
        rw_pr_lock_assert_not_write_owner are macros in MariaDB
      - sql/handler.h: not merged (HA_NO_READ_LOCAL_LOCK)
      - sql/mdl.cc: partially merged (MDL_SHARED_READ_ONLY only)
      - sql/mdl.h: partially merged (MDL_SHARED_READ_ONLY only)
      - sql/lock.cc: fully merged
      - sql/sp_head.cc: not merged
      - sql/sp_head.h: not merged
      - sql/sql_base.cc: partially merged (MDL_SHARED_READ_ONLY only)
      - sql/sql_base.h: not merged
      - sql/sql_class.cc: fully merged
      - sql/sql_class.h: fully merged
      - sql/sql_handler.cc: merged partially (different solution in MariaDB)
      - sql/sql_parse.cc: partially merged, mostly omitted low priority write part
      - sql/sql_reload.cc: not merged comment change
      - sql/sql_table.cc: not merged SNRW->X upgrade for IMPORT/DISCARD TABLESPACE
      - sql/sql_view.cc: not merged
      - sql/sql_yacc.yy: not merged (MDL_SHARED_WRITE_LOW_PRIO, MDL_SHARED_READ_ONLY)
      - sql/table.cc: not merged (MDL_SHARED_WRITE_LOW_PRIO)
      - sql/table.h: not merged (MDL_SHARED_WRITE_LOW_PRIO)
      - sql/trigger.cc: not merged
      - storage/innobase/handler/ha_innodb.cc: merged store_lock()/lock_count()
        changes (in MDEV-7660), didn't merge HA_NO_READ_LOCAL_LOCK
      - storage/innobase/handler/ha_innodb.h: fully merged in MDEV-7660
      - storage/myisammrg/ha_myisammrg.cc: not merged comment change
      - storage/perfschema/table_helper.cc: not merged (no MDL support in MariaDB PFS)
      - unittest/gunit/mdl-t.cc: not merged
      - unittest/gunit/mdl_sync-t.cc: not merged
      
      MariaDB specific changes:
      - handler.heap: different HANDLER solution, MDEV-7660
      - handler.innodb: different HANDLER solution, MDEV-7660
      - handler.interface: different HANDLER solution, MDEV-7660
      - handler.myisam: different HANDLER solution, MDEV-7660
      - main.mdl_sync: MDEV-7660 specific changes
      - main.partition_debug_sync: removed test due to different HANDLER solution,
        MDEV-7660
      - main.truncate_coverage: removed test due to different HANDLER solution,
        MDEV-7660
      - mysql-test/include/mtr_warnings.sql: additional cleanup, MDEV-7660
      - mysql-test/lib/v1/mtr_report.pl: additional cleanup, MDEV-7660
      - plugin/metadata_lock_info/metadata_lock_info.cc: not in MySQL
      - sql/sql_handler.cc: MariaDB specific fix for mysql_ha_read(), MDEV-7660
      8774a023
  7. 20 Dec, 2016 7 commits
  8. 19 Dec, 2016 7 commits
  9. 17 Dec, 2016 4 commits
    • Alexander Barkov's avatar
      MDEV-11514, MDEV-11497, MDEV-11554, MDEV-11555 - IN and CASE type aggregation problems · 74891ed2
      Alexander Barkov authored
      This patch fixes a number of data type aggregation problems in IN and CASE:
      - MDEV-11497 Wrong result for (int_expr IN (mixture of signed and unsigned expressions))
      - MDEV-11514 IN with a mixture of TIME and DATETIME returns a wrong result
      - MDEV-11554 Wrong result for CASE on a mixture of signed and unsigned expressions
      - MDEV-11555 CASE with a mixture of TIME and DATETIME returns a wrong result
      
      1. The problem reported in MDEV-11514 and MDEV-11555 was in the wrong assumption
      that items having the same cmp_type() can reuse the same cmp_item instance.
      So Item_func_case and Item_func_in used a static array of cmp_item*,
      one element per one XXX_RESULT.
      
      TIME and DATETIME cannot reuse the same cmp_item, because arguments of
      these types are compared very differently. TIME and DATETIME must have
      different instances in the cmp_item array. Reusing the same cmp_item
      for TIME and DATETIME leads to unexpected result and unexpected warnings.
      
      Note, after adding more data types soon (e.g. INET6), the problem would
      become more serious, as INET6 will most likely have STRING_RESULT, but
      it won't be able to reuse the same cmp_item with VARCHAR/TEXT.
      
      This patch introduces a new class Predicant_to_list_comparator,
      which maintains an array of cmp_items, one element per distinct
      Type_handler rather than one element per XXX_RESULT.
      
      2. The problem reported in MDEV-11497 and MDEV-11554 happened because
      Item_func_in and Item_func_case did not take into account the fact
      that UNSIGNED and SIGNED values must be compared as DECIMAL rather than INT,
      because they used item_cmp_type() to aggregate the arguments.
      The relevant code now resides in Predicant_to_list_comparator::add_value()
      and uses Type_handler_hybrid_field_type::aggregate_for_comparison(),
      like Item_func_between does.
      74891ed2
    • Alexander Barkov's avatar
      Adding "DBUG" prefix into the debug messages in Item_func_in::fix_length_and_dec() · 191c3f49
      Alexander Barkov authored
      As agreed with Sanja, debug messages printed in
      Item_func_in::fix_length_and_dec() now have the "DBUG:" prefix,
      to make the *.result files more readable.
      Also changing level from WARN_LEVEL_WARN to WARN_LEVEL_NOTE.
      191c3f49
    • Alexander Barkov's avatar
    • Daniel Black's avatar
      Travis: parallel_jobs=3 · c35b8c46
      Daniel Black authored
      c35b8c46
  10. 16 Dec, 2016 5 commits
    • Alexander Barkov's avatar
      MDEV-11503 Introduce Type_handler::make_in_vector() and... · 93bc7283
      Alexander Barkov authored
      MDEV-11503 Introduce Type_handler::make_in_vector() and Item_func_in_fix_comparator_compatible_types()
      
      This patch implements the task according to the description:
      
      1. The old code from Item_func_in::fix_length_and_dec() was decomposed
      into smaller methods:
      - all_items_are_consts()
      - compatible_types_scalar_bisection_possible()
      - compatible_types_row_bisection_possible()
      - fix_in_vector()
      - fix_for_scalar_comparison_using_bisection()
      - fix_for_scalar_comparison_using_cmp_items()
      - fix_for_row_comparison_using_bisection()
      - fix_for_row_comparison_using_cmp_items()
      
      The data type dependend pieces where moved as methods to Type_handler.
      
      2. Splits in_datetime into separate:
         - in_datetime, for DATETIME and DATE,
         - in_time, for TIME
         to make the code more symmetric across data types.
      
      Additionally:
      - Adds a test func_debug.test to see which calculation strategy
        (bisect or no bisect) is chosen to handle IN with various arguments.
      - Adds a new helper method (to avoid duplicate code):
        cmp_item_rows::prepare_comparators()
      - Changes the propotype for cmp_item_row::alloc_comparators(),
        to avoid duplicate code, and to use less current_thd.
      - Changes "friend" sections in cmp_item_row and in_row from
        an exact Item_func_in method to the entire class Item_func_in,
        as their internals are now needed in multiple Item_func_in methods.
      - Added more comments (e.g. on bisection, on the problem reported in MDEV-11511)
      93bc7283
    • Alexander Barkov's avatar
    • Alexander Barkov's avatar
      MDEV-11485 Split Item_func_between::val_int() into virtual methods in Type_handler · cfda0a71
      Alexander Barkov authored
      - Removes "Item_result Item_func_opt_neg::m_compare_type" and introduces
        "Type_handler_hybrid_field_type Item_func_opt_neg::m_comparator" instead.
      
      - Removes Item_func_between::compare_as_dates, because
        the new member m_comparator now contains the precise information
        about the data type that is used for comparison, which is important
        for TIME vs DATETIME.
      
      - Adds a new method:
        Type_handler_hybrid_field_type::aggregate_for_comparison(const Type_handler*),
        as a better replacement for item_cmp_type(), which additionally can handle
        TIME vs DATE/DATETIME/TIMESTAMP correctly. Additionally, it correctly
        handles TIMESTAMP which fixes the problem reported in MDEV-11482.
        The old compare_as_dates/find_date_time_item() based code didn't handle
        comparison between TIME and TIMESTAMP correctly and erroneously used TIME
        comparison instead of DATETIME comparison.
      
      - Adds a new method:
        Type_handler_hybrid_field_type::aggregate_for_comparison(Item **, uint nitems),
        as a better replacement for agg_cmp_type(), which can handle TIME.
      - Splits Item_func_between::val_int() into pieces val_int_cmp_xxx(),
        one new method per XXX_RESULT.
      - Adds a new virtual method Type_handler::Item_func_between_val_int()
        whose implementations use Item_func_between::val_int_cmp_xxx().
      - Makes type_handler_longlong and type_handler_newdecimal public,
        as they are now needed in item_cmpfunc.cc.
      
      Note:
      This patch does not change Item_func_in to use the new aggregation methods,
      so it still uses collect_cmp_type()/item_cmp_type() based aggregation.
      Item_func_in will be changed in a separate patch and item_cmp_type() will be
      removed.
      cfda0a71
    • Alexander Barkov's avatar
      Moving LEX::set_last_field_type() to Column_definition::set_attributes() · 1c1d8fe9
      Alexander Barkov authored
      This is an extraction from the patch for MDEV-10577, which is not
      directly related to %TYPE implementation. This patch does the following:
      - Moves LEX::set_last_field_type() to Column_definition::set_attributes()
      - Adds initialization of Column_definition members length, decimals,
        charset, on_update into the constructor.
      - Column_definition::set_attributes() now does not set length and decimal
        to 0 any more, as they are not initialized in the constructor.
      - Move Column_definition::prepare_interval_field() from field.h to field.cc,
        as it's too huge.
      1c1d8fe9
    • Alexander Barkov's avatar