1. 27 Apr, 2017 2 commits
    • Alexander Barkov's avatar
      MDEV-12592 Illegal mix of collations with the HEX function · cfb4d9f9
      Alexander Barkov authored
      Fixing repertoire for HEX() from MY_REPERTOIRE_EXTENDED to MY_REPERTOIRE_ASCII.
      cfb4d9f9
    • Alexander Barkov's avatar
      MDEV-12588 Add Type_handler::type_handler_for_tmp_table() and... · 441349aa
      Alexander Barkov authored
      MDEV-12588 Add Type_handler::type_handler_for_tmp_table() and Type_handler::type_handler_for_union()
      
      1. Implementing the task according to the description:
      
      a. Adding Type_handler::type_handler_for_tmp_table().
      b. Adding Type_handler::type_handler_for_union_table.
      c. Adding helper methods Type_handler::varstring_type_handler(const Item*),
         Type_handler::blob_type_handler(const Item*)
      d. Removing Item::make_string_field() and
         Item_func_group_concat::make_string_field().
         They are not needed any more.
      e. Simplifying Item::tmp_table_field_from_field_type() to just two lines.
      f. Renaming Item_type_holder::make_field_by_type() and implementing
         virtual Item_type_holder::create_tmp_field() instead.
         The new implementation is also as simple as two lines.
      g. Adding a new virtual method Type_all_attributes::get_typelib(),
         to access to TYPELIB definitions for ENUM and SET columns.
      h. Simplifying the code branch for TIME_RESULT, DECIMAL_RESULT, STRING_RESULT
         in Item::create_tmp_field(). It's now just one line.
      i. Implementing Type_handler_enum::make_table_field() and
         Type_handler_set::make_table_field().
      
      2. Code simplification in Field_str constructor calls.
      
      a. Changing the "CHARSET_INFO *cs" argument in constuctors for Field_str
         and its descendants to "const DTCollation &collation". This is to
         avoid two step initialization:
         - setting Field_str::derivation and Field_str::repertoire to the
           default values first
         - then resetting them using:
           set_derivation(item->derivation, item->repertoire).
      
      b. Removing Field::set_derivation()
      
      c. Adding a new constructor DTCollation(CHARSET_INFO *cs),
         for the old code compatibility.
      
      3. Changes in test results
      
      As a side effect some test results have changed, because
      in the old version Item::make_string_field() converted
      TINYBLOB to VARCHAR(255). Now TINYBLOB is preserved.
      
      a. sp-row.result
         This query:
           CREATE TABLE t1 AS SELECT tinyblob_sp_variable;
         Now preserves TINYBLOB as the data type.
         Before the patch a VARCHAR(255) was created.
      
      b. gis-debug.result
         This is a debug test, to make sure that + and - operators
         are commutative and non-commutative correspondingly.
         The exact data type is not really important.
         (But anyway, it now chooses a better data type that fits the result)
      441349aa
  2. 26 Apr, 2017 3 commits
  3. 25 Apr, 2017 3 commits
    • Alexander Barkov's avatar
      MDEV-12426 Add Field::type_handler() + MDEV-12432 · 2fd63540
      Alexander Barkov authored
      This is a joint patch for:
      
      - MDEV-12426 Add Field::type_handler()
      - MDEV-12432 Range optimizer for ENUM and SET does not return "Impossible WHERE" in some case
      
      With the new type handler approach being added to Field, it was easier to fix
      MDEV-12432 rather than to reproduce the old ENUM/SET behavior.
      
      The patch does the following:
      
      1. Adds Field::type_handler(), according to the task description.
      
      2. Fixes the asymmetry between Fields and Items of ENUM and SET field types.
         Field_enum::cmp_type() returned INT_RESULT
         Item*::cmp_type() returned STRING_RESULT for ENUM and SET expressions
      
      This asymmetry was originally done for easier coding in the optimizer sources.
      However, in 10.1 we moved a lot of code to methods of the class Field:
      - test_if_equality_guarantees_uniqueness()
      - can_be_substituted_to_equal_item()
      - get_equal_const_item()
      - can_optimize_keypart_ref()
      - can_optimize_hash_join()
      - can_optimize_group_min_max()
      - can_optimize_range()
      - can_optimize_outer_join_table_elimination()
      
      As of 10.2 only a few lines of the code in opt_range.cc, field.cc and field.h
      still relayed on the fact that Field_enum::cmp_type() returns INT_RESULT:
      - Some asserts in field.cc
      - Field_year::get_copy_func()
      - Item_func_like::get_mm_leaf()
      - Item_bool_func::get_mm_leaf()
      These lines have been fixed.
      
      3. Item_bool_func::get_mm_leaf() did not work well for ENUM/SET,
      see MDEV-12432. So the ENUM/SET code was rewritten, and the relevant
      code in Field_enum::store() and Field_set::store() was fixed to
      properly return errors to the caller.
      
      4. The result of Field_decimal::result_type() was changed from REAL_RESULT
      to DECIMAL_RESULT. Data type aggregation (e.g. in COALESCE()) is now more
      precise for old DECIMAL, because Item::decimal_precision() now goes through
      the DECIMAL_RESULT branch. Earlier it went through the REAL_RESULT branch.
      2fd63540
    • Alexander Barkov's avatar
    • Alexander Barkov's avatar
      A safety patch for MDEV-9217 Split Item::tmp_table_field_from_field_type()... · 6cc40856
      Alexander Barkov authored
      A safety patch for MDEV-9217 Split Item::tmp_table_field_from_field_type() into virtual methods in Type_handler
      
      During refactoring in Type_handler, it's safer to use new_Field_xxx
      in Type_handler_xxx2::make_table_field(), to avoid creations of wrong
      fields for fields of the TIME(0), DATETIME(0) and TIMESTAMP(0) types.
      This will be changed back to "new" when we finish refactoring and
      reuse Type_handler::make_table_field() in make_field() in field.cc.
      6cc40856
  4. 24 Apr, 2017 3 commits
    • Alexander Barkov's avatar
      A cleanup for MDEV-9217 Split Item::tmp_table_field_from_field_type() into... · 5f1544fe
      Alexander Barkov authored
      A cleanup for MDEV-9217 Split Item::tmp_table_field_from_field_type() into virtual methods in Type_handler
      
      Fixing that the server tried to create the old decimal for:
      
        CREATE TABLE t2 AS SELECT old_decimal_field FROM t1
                     UNION SELECT bigint_field FROM t1;
      
        CREATE TABLE t2 AS SELECT old_decimal_field FROM t1
                     UNION SELECT mediumint_field FROM t1;
      
        CREATE TABLE t2 AS SELECT year FROM t1
                     UNION SELECT old_decimal_field FROM t1;
      
        CREATE TABLE t2 AS SELCT COALESCE(old_decimal_field) FROM t1;
      
      Solution:
      
      1. field_types_merge_rules[][] had three MYSQL_TYPE_DECIMAL remainders.
         Fixing to MYSQL_TYPE_NEWDECIMAL, like it is done for all other type pairs.
         This fixes the above queries with UNION.
      
      2. Convert MYSQL_TYPE_DECIMAL to MYSQL_TYPE_NEWDECIMAL in
         Item::tmp_table_field_from_field_type().
         This fixed the above query with COALESCE.
      
      3. Adding "new Field_decimal" into Type_handler_olddecimal::make_table_field().
         In case if something goes wrong it will crash on assert only in debug builds,
         while create the old decimal in release.
         Note, this "new Field_decimal" will be needed later anyway,
         when we reuse Type_handler::make_table_field() in make_field() in field.cc.
      5f1544fe
    • Alexander Barkov's avatar
      MDEV-9217 Split Item::tmp_table_field_from_field_type() into virtual methods in Type_handler · 79137435
      Alexander Barkov authored
      - Adding Type_handler::make_table_field() and moving pieces of the code
        from Item::tmp_table_field_from_field_type() to virtual implementations
        for various type handlers.
      
      - Adding a new Type_all_attributes, to access to Item's extended
        attributes, such as decimal_precision() and geometry_type().
      
      - Adding a new class Record_addr, to pass record related information
        to Type_handler methods (ptr, null_ptr and null_bit) as a single structure.
        Note, later it will possibly be extended for BIT-alike field purposes,
        by adding new members (bit_ptr_arg, bit_ofs_arg).
      
      - Moving the code from Field_new_decimal::create_from_item()
        to Type_handler_newdecimal::make_table_field().
      
      - Removing Field_new_decimal() and Field_geom() helper constructor
        variants that were used for temporary field creation.
      
      - Adding Item_field::type_handler(), Field::type_handler() and
        Field_blob::type_handler() to return correct type handlers for
        blob variants, according to Field_blob::packlength.
      
      - Adding Type_handler_blob_common, as a common parent for
        Type_handler_tiny_blob, Type_handler_blob, Type_handler_medium_blob
        and Type_handler_long_blob.
      
      - Implementing Type_handler_blob_common::Item_hybrid_func_fix_attributes().
      
        It's needed for cases when TEXT variants of different character sets are mixed
        in LEAST, GREATEST, CASE and its abreviations (IF, IFNULL, COALESCE), e.g.:
            CREATE TABLE t1 (
              a TINYTEXT CHARACTER SET latin1,
              b TINYTEXT CHARACTER SET utf8
            );
            CREATE TABLE t2 AS SELECT COALESCE(a,b) FROM t1;
        Type handler aggregation returns TINYTEXT as a common data type
        for the two columns. But as conversion from latin1 to utf8
        happens for "a", the maximum possible length of "a" grows from 255 to 255*3.
        Type_handler_blob_common::Item_hybrid_func_fix_attributes() makes sure
        to update the blob type handler according to max_length.
      
      - Adding Type_handler::blob_type_handler(uint max_octet_length).
      
      - Adding a few m_type_aggregator_for_result.add() pairs, because
        now Item_xxx::type_handler() can return pointers to type_handler_tiny_blob,
        type_handler_blob, type_handler_medium_blob, type_handler_long_blob.
        Before the patch only type_handler_blob was possible result of type_handler().
      
      - Making type_handler_tiny_blob, type_handler_blob, type_handler_medium_blob,
        type_handler_long_blob public.
      
      - Removing the condition in Item_sum_avg::create_tmp_field()
        checking Item_sum_avg::result_type() against DECIMAL_RESULT.
        Now both REAL_RESULT and DECIMAL_RESULT are symmetrically handled
        by tmp_table_field_from_field_type().
      
      - Removing Item_geometry_func::create_field_for_create_select(),
        as the inherited version perfectly works.
      
      - Fixing Item_func_as_wkb::field_type() to return MYSQL_TYPE_LONG_BLOB
        rather than MYSQL_TYPE_BLOB. It's needed to make sure that
        tmp_table_field_from_field_type() creates a LONGBLOB field for AsWKB().
      
      - Fixing Item_func_as_wkt::fix_length_and_dec() to set max_length to
        UINT32_MAX rather than MAX_BLOB_WIDTH, to make sure that
        tmp_table_field_from_field_type() creates a LONGTEXT field for AsWKT().
      
      - Removing Item_func_set_user_var::create_field_for_create_select(),
        as the inherited version works fine.
      
      - Adding Item_func_get_user_var::create_field_for_create_select() to
        make sure that "CREATE TABLE t1 AS SELECT @string_user variable"
        always creates a field of LONGTEXT/LONGBLOB type.
      
      - Item_func_ifnull::create_field_for_create_select()
        behavior has changed. Before the patch it passed set_blob_packflag=false,
        which meant to create LONGBLOB for all blob variants.
        Now it takes into account max_length, which gives better column
        data types for:
          CREATE TABLE t2 AS SELECT IFNULL(blob_column1, blob_column2) FROM t1;
      
      - Fixing Item_func_nullif::fix_length_and_dec() to use
        set_handler(args[2]->type_handler()) instead of
        set_handler_by_field_type(args[2]->field_type()).
        This is needed to distinguish between BLOB variants.
      
      - Implementing Item_blob::type_handler(), to make sure to create
        proper BLOB field variant, according to max_length, for queries like:
          CREATE TABLE t1 AS
            SELECT some_blob_field FROM INFORMATION_SCHEMA.SOME_TABLE;
      
      - Fixing Item_field::real_type_handler() to make sure that
        the code aggregating fields for UNION gets a proper BLOB
        variant type handler from fields.
      
      - Adding a special code into Item_type_holder::make_field_by_type(),
        to make sure that after aggregating field types it also properly
        takes into account max_length when mixing TEXT variants of different
        character sets and chooses a proper TEXT variant:
            CREATE TABLE t1 (
              a TINYTEXT CHARACTER SET latin1,
              b TINYTEXT CHARACTER SET utf8
            );
            CREATE TABLE t2 AS SELECT a FROM t1 UNION SELECT b FROM t1;
      
      - Adding tests, for better coverage of IFNULL, NULLIF, UNION.
      
      - The fact that tmp_table_field_from_field_type() now takes
        into account BLOB variants (instead of always creating LONGBLOB),
        tests results for WEIGHT_STRING() and NULLIF() and UNION
        have become more precise.
      79137435
    • Alexander Barkov's avatar
  5. 23 Apr, 2017 3 commits
    • Monty's avatar
      Changing field::field_name and Item::name to LEX_CSTRING · 5a759d31
      Monty authored
      Benefits of this patch:
      - Removed a lot of calls to strlen(), especially for field_string
      - Strings generated by parser are now const strings, less chance of
        accidently changing a string
      - Removed a lot of calls with LEX_STRING as parameter (changed to pointer)
      - More uniform code
      - Item::name_length was not kept up to date. Now fixed
      - Several bugs found and fixed (Access to null pointers,
        access of freed memory, wrong arguments to printf like functions)
      - Removed a lot of casts from (const char*) to (char*)
      
      Changes:
      - This caused some ABI changes
        - lex_string_set now uses LEX_CSTRING
        - Some fucntions are now taking const char* instead of char*
      - Create_field::change and after changed to LEX_CSTRING
      - handler::connect_string, comment and engine_name() changed to LEX_CSTRING
      - Checked printf() related calls to find bugs. Found and fixed several
        errors in old code.
      - A lot of changes from LEX_STRING to LEX_CSTRING, especially related to
        parsing and events.
      - Some changes from LEX_STRING and LEX_STRING & to LEX_CSTRING*
      - Some changes for char* to const char*
      - Added printf argument checking for my_snprintf()
      - Introduced null_clex_str, star_clex_string, temp_lex_str to simplify
        code
      - Added item_empty_name and item_used_name to be able to distingush between
        items that was given an empty name and items that was not given a name
        This is used in sql_yacc.yy to know when to give an item a name.
      - select table_name."*' is not anymore same as table_name.*
      - removed not used function Item::rename()
      - Added comparision of item->name_length before some calls to
        my_strcasecmp() to speed up comparison
      - Moved Item_sp_variable::make_field() from item.h to item.cc
      - Some minimal code changes to avoid copying to const char *
      - Fixed wrong error message in wsrep_mysql_parse()
      - Fixed wrong code in find_field_in_natural_join() where real_item() was
        set when it shouldn't
      - ER_ERROR_ON_RENAME was used with extra arguments.
      - Removed some (wrong) ER_OUTOFMEMORY, as alloc_root will already
        give the error.
      
      TODO:
      - Check possible unsafe casts in plugin/auth_examples/qa_auth_interface.c
      - Change code to not modify LEX_CSTRING for database name
        (as part of lower_case_table_names)
      5a759d31
    • Monty's avatar
      Fixed compiler warnings and wrong test results · cba84469
      Monty authored
      cba84469
    • Alexander Barkov's avatar
      Fixing a bug in the recent commit that added trigger names into error messages · f8221294
      Alexander Barkov authored
      Trigger names were erroneously cut to 8 or 4 bytes (on 64 and 32 bit machines).
      Recording correct tests results.
      f8221294
  6. 22 Apr, 2017 3 commits
    • Alexander Barkov's avatar
      MDEV-12506 Split Item_func_min_max::fix_length_and_dec() into methods in... · 65808255
      Alexander Barkov authored
      MDEV-12506 Split Item_func_min_max::fix_length_and_dec() into methods in Type_handler + MDEV-12497 + MDEV-12504
      
      This patch does the following:
      
      1. Adds a new method Type_handler_hybrid_field_type::aggregate_for_min_max()
      
         - For non-traditional data types it uses
           type_handler_data->m_type_aggregator_for_result.find_handler()
           This allows pluggable data types to define in the future their
           own behavior of the result data type detection for LEAST/GREATEST.
           Also, this disallows expressions of the GEOMETRY data type
           (and its variants such as POINT) to be mixed in with
           numeric and temporal data types in LEAST/GREATEST.
      
         - For traditional data types it reproduces the old behavior of
           the result data type detection (but not attributes, see below).
      
      2. Adds a new virtual method Type_handler::Item_func_min_max_fix_attributes()
         and reuses as much as possible the code that calculates data type attributes
         for CASE-alike functions (e.g. CASE..THEN, COALESCE, IF).
      
         As the old code responsible for attributes calculation in the old
         implementation of Item_func_min_max::fix_length_and_dec()
         was not fully correct, this automatically fixes the following bugs:
      
         - MDEV-12497 Wrong data type for LEAST(latin1_expr, utf8_expr)
           The old fix_length_and_dec() calculated max_length before
           character set aggregation. Now max_length is calculated after, in
           Item_func::count_string_length() called from
           Item_func::aggregate_attributes_string() called from
           Type_handler_string_result::Item_hybrid_func_fix_attributes() called from
           Type_handler::Item_func_min_max_fix_attributes() called from
           Item_func_min_max::fix_length_and_dec().
      
         - MDEV-12504 Wrong data type for LEAST(date_expr,time_expr)
           The old fix_length_and_dec() simply used the maximum of max_length
           among all arguments to set its own max_length and did not take
           into account that a mixture of DATE and TIME becomes DATETIME.
           Now this is correctly handled by:
           Type_handler_datetime_common::Item_hybrid_func_fix_attributes() called from
           Type_handler::Item_func_min_max_fix_attributes() called from
           Item_func_min_max::fix_length_and_dec().
      
      3. Removes the old implementation of Item_func_min_max::fix_length_and_dec()
         and replaces it to calls of the new methods.
      
      4. Cleanup: moves the code related to unsigned_flag processing
         from Type_handler_hybrid_field_type::aggregate_for_result()
         to   Type_handler_int_result::Item_hybrid_func_fix_attributes().
         This is done:
         - to avoid code duplication in
           Type_handler_hybrid_field_type::aggregate_for_min_max()
         - to get rid of one more call for field_type(), which is unfriendly
           to the conceipt of pluggable data types.
      65808255
    • Alexander Barkov's avatar
      MDEV-12559 Split Item::temporal_precision() into virtual methods in Type_handler · ba670edf
      Alexander Barkov authored
      - Adding a new virtual method Type_handler::Item_time_precision()
      - Adding a new virtual method Type_handler::Item_datetime_precision()
      - Removing Item::temporal_precision() and adding Item::time_precision()
        and Item::datetime_precision() instead.
      - Moving Item_func_convert_tz::fix_length_and_dec() from item_timefunc.cc
        to item_timefunc.h. It's only two lines, and we're changing it anyway.
      - Removing Item_temporal_typecast::fix_length_and_dec_generic(),
        moving this code to
        Type_handler::Item_{date|time|datetime}_typecast_fix_length_and_dec().
        This allows to get rid of one more field_type() call.
        Also, in the old reduction, Item_date_typecast::fix_length_and_dec()
        unnecessarily called args[0]->temporal_precision(). The new reduction
        does not call args[0]->datetime_precision(), as DATE does not
        have fractional digits.
      ba670edf
    • Alexander Barkov's avatar
      b7814086
  7. 20 Apr, 2017 2 commits
  8. 19 Apr, 2017 4 commits
    • Monty's avatar
      e6c11717
    • Monty's avatar
      Fixed some wrong/inconsistent error message · 847eb24b
      Monty authored
      - Added trigger name to "Trigger already exists" error message
      - Added also missing query name to ER_DUP_QUERY_NAME
      - Fixed wrong use of MASTER_DELAY_VALUE_OUT_OF_RANGE
      847eb24b
    • Monty's avatar
      Removed complex and wrong set_name_for_rollback() · b478276b
      Monty authored
      This was wrong because:
      - There was no reason to rollback name for item that will be deleted
        after query.
      - name_length was not rolled back
      - Changing real_item() doesn't work as it may be used many times in the
        same query
      
      After removing all the old code and extending the test case, all the
      related test cases passes.
      
      Sanja and I concluded that the old code isn't needed anymore.  If it
      still needed for some scenario not covered by our test system, it needs
      to be coded in some other way, so better to remove the wrong code.
      b478276b
    • Alexander Barkov's avatar
      MDEV-12514 Split Item_temporal_func::fix_length_and_dec() + MDEV-12515 · e2b03cd3
      Alexander Barkov authored
      This patch implements MDEV-12514 according to the task descriptions.
      It automatically fixes:
      MDEV-12515 Wrong value when storing DATE_ADD() and ADDTIME() to a numeric field
      
      Additionally:
      
      a. Moves Item_func::set_attributes_temporal() to
         Type_str_attributes::fix_attributes_temporal(),
        which is a more proper place and name for it.
      
      b. Continues replacing calls for:
           set_handler_by_field_type(MYSQL_TYPE_XXX)
         to corresponding:
           set_handler(&type_handler_xxx)
         which is faster.
         Note, we should eventually get rid of almost all set_handler_by_field_type().
      
      c. Makes type_handler_string, type_handler_time2, type_handler_newdate,
         type_handler_datetime2 public.
         (all built-in handlers will become public eventually)
      
      d. Removing Item_temporal_func::sql_mode, as it was not used.
      e2b03cd3
  9. 18 Apr, 2017 5 commits
  10. 15 Apr, 2017 1 commit
  11. 14 Apr, 2017 1 commit
  12. 13 Apr, 2017 2 commits
  13. 11 Apr, 2017 3 commits
    • Alexander Barkov's avatar
    • Alexander Barkov's avatar
    • Daniel Black's avatar
      Travis: Test more suites, latest OSX · eecce3d7
      Daniel Black authored
      Remove clang-3.8 which doesn't have a repository on apt.llvm.org any
      more.
      
      For OSX, xcode8.3 is explicitly specified.
      
      /usr/local/Cellar is used as a cache repository to save brew install
      time on OSX (and /usr/local was too big).
      
      Debian autobake.sh is moved to a matrix include.
      
      Other branches of the matrix build test other test suites.
      An Ubuntu galera is downloaded and used in the test suite.
      
      TYPE=RelWithDebInfo used with the test to provide backtraces with line
      numbers when crashes occur.
      
      Build of PLUGIN_AWS_KEY_MANAGEMENT enabled in build.
      
      Code supporting TYPE=Debug and -DWITH_ASAN=ON included by not enabled
      due to large numbers of errors.
      
      Running more tests in parallel (6) as container based builds seem to
      support them. The test case timeout has been set to 2 minutes as large
      stalls will put test cases over 50 minute interval.
      
      ccache enabled where possible. Linux clang builds don't use them as the
      minimum CMake version isn't there.
      eecce3d7
  14. 09 Apr, 2017 2 commits
  15. 08 Apr, 2017 3 commits