1. 01 Aug, 2020 4 commits
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 52edc374
      Marko Mäkelä authored
      52edc374
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 50a11f39
      Marko Mäkelä authored
      50a11f39
    • Otto Kekäläinen's avatar
      Unify config syntax in default files · 842da858
      Otto Kekäläinen authored
      - Include a link to the relevant KB article for more info
      - Use spaced around the equal sign for better readability and so that
        the examples are more aligned with the general style in the KB
      - Load plugins with just the base name, the .so is optional and excess
      842da858
    • Otto Kekäläinen's avatar
      MDEV-22980: Allow plugin-maturity=alpha so S3 plugin loads · afbd6181
      Otto Kekäläinen authored
      Verified by running before and after:
      
        mariadb --skip-column-names -e "select plugin_name, plugin_status,
        plugin_type, plugin_library, plugin_license from
        information_schema.all_plugins order by plugin_name, plugin_library"
      
      Nothing else but exactly this line changed so there are no side effects:
      
        -S3 NOT INSTALLED	STORAGE ENGINE	ha_s3.so	GPL
        +S3 ACTIVE	       STORAGE ENGINE	ha_s3.so	GPL
      
      Also enrich config file with link to KB and unify option syntax and
      standard comments.
      afbd6181
  2. 31 Jul, 2020 14 commits
    • Sergei Golubchik's avatar
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 70d4500c
      Marko Mäkelä authored
      70d4500c
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · 9216114c
      Marko Mäkelä authored
      9216114c
    • Alexander Barkov's avatar
      MDEV-23351 Rounding functions return wrong data types for DATE input · dc513dff
      Alexander Barkov authored
      Fixing ROUND(date,0), TRUNCATE(date,x), FLOOR(date), CEILING(date)
      to return the `int(8) unsigned` data type.
      
      Details:
      1. Cleanup: moving virtual implementations
         - Type_handler_temporal_result::Item_func_int_val_fix_length_and_dec()
         - Type_handler_temporal_result::Item_func_round_fix_length_and_dec()
         to Type_handler_date_common. Other temporal data type handlers
         override these methods anyway. So they were only DATE specific.
         This change makes the code clearer.
      2. Backporting DTCollation_numeric from 10.5, to reuse the code easier.
      3. Adding the `preferred_attrs` argument to Item_func_round::fix_arg_int(). Now
         Type_handler_xxx::Item_func_round_val_fix_length_and_dec() work as follows:
         - The INT-alike and YEAR handlers copy preferred_attrs from args[0].
         - The DATE handler passes explicit attributes, to get `int(8) unsigned`.
         - The hex hybrid handler passes NULL, so fix_arg_int() calculates attributes.
      4. Type_handler_date_common::Item_func_int_val_fix_length_and_dec()
         now sets the type handler and attributes to get `int(8) unsigned`.
      dc513dff
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 8bb2170d
      Marko Mäkelä authored
      8bb2170d
    • Marko Mäkelä's avatar
      Merge 10.2 into 10.3 · 66ec3a77
      Marko Mäkelä authored
      66ec3a77
    • Sergei Golubchik's avatar
    • Marko Mäkelä's avatar
      MDEV-11799 Doublewrite recovery can corrupt data pages · 879ba197
      Marko Mäkelä authored
      The purpose of the InnoDB doublewrite buffer is to make InnoDB
      tolerant against cases where the server was killed in the middle
      of a page write. (In Linux, killing a process may interrupt a
      write system call, typically on a 4096-byte boundary.)
      
      There may exist multiple copies of a page number in the doublewrite
      buffer. Recovery should choose the latest valid copy of the page.
      By design, the FIL_PAGE_LSN must not precede the latest checkpoint LSN
      nor be later than the end of the recovered log.
      
      For page_compressed and encrypted pages, we were missing proper
      consistency checks. In the 10.4 data set generated for in MDEV-23231,
      the data file contained a valid page_compressed page, and an
      identical copy of that page was also present in the doublewrite
      buffer. But, recovery would incorrectly consider the page invalid
      and restore an uncompressed copy of the same page that had been
      written before the log checkpoint. (In fact, no redo log was to
      be applied to that page.)
      
      buf_dblwr_process(): Validate the FIL_PAGE_LSN in the doublewrite
      buffer pages, and always skip page 0, because those pages should
      have been recovered by Datafile::restore_from_doublewrite() if
      necessary.
      
      Datafile::restore_from_doublewrite(): Choose the latest applicable
      page from the doublewrite buffer.
      
      recv_dblwr_t::find_page(): Also validate encrypted or
      page_compressed pages.
      
      recv_dblwr_t::validate_page(): New function to validate a page,
      either a copy in a data file or in the doublewrite buffer.
      Also validate encrypted or page_compressed pages.
      
      This is joint work with Thirunarayanan Balathandayuthapani.
      879ba197
    • Marko Mäkelä's avatar
      MDEV-23198 Crash in REPLACE · f35d1721
      Marko Mäkelä authored
      row_vers_impl_x_locked_low(): clust_offsets may point to memory
      that is allocated by mem_heap_alloc() and may have been freed.
      For initializing clust_offsets, try to use the stack-allocated
      buffer instead of a pointer that may point to freed memory.
      
      This fixes a regression that was introduced in
      commit f0aa073f (MDEV-20950).
      f35d1721
    • Nikita Malyavin's avatar
      MDEV-18042 Server crashes upon adding a non-null date column under... · fd0abc89
      Nikita Malyavin authored
      MDEV-18042 Server crashes upon adding a non-null date column under NO_ZERO_DATE with ALGORITHM=INPLACE
      
      accept table_name and db_name instead of table_share in make_truncated_value_warning
      fd0abc89
    • Nikita Malyavin's avatar
      MDEV-19338 InnoDB: Failing assertion: !cursor->index->is_committed() · 91ebf184
      Nikita Malyavin authored
      Call mark_columns_per_binlog_row_image before find_row() to set up table->vcol_set early,
      so the virtual column value will be updated after record read (ha_rnd_pos/ha_index_next/etc)
      by table->update_virtual_fields() call
      91ebf184
    • Sergei Golubchik's avatar
      MDEV-23237 mariadb.sys has too many privileges · 153cd6a2
      Sergei Golubchik authored
      don't GRANT UPDATE ON mysql.global_priv TO mariadb.sys@localhost;
      153cd6a2
    • Sergei Golubchik's avatar
      MDEV-23009 SIGSEGV in get_field from acl_load (on optimized builds) · 78f09b12
      Sergei Golubchik authored
      Classes that handle privilege tables (like Tables_priv_table)
      could read some columns conditionally but they expect a certain
      minimal number of colunms always to exist.
      
      Add a check for a minimal required number of columns in privilege tables,
      don't use a table that has fewer columns than required.
      78f09b12
    • Alexander Barkov's avatar
      MDEV-23337 Rounding functions create a wrong data type for integer input · a874b6c4
      Alexander Barkov authored
      1. Fixing ROUND(x) and TRUNCATE(x,0) with TINYINT, SMALLINT, MEDIUMINT, BIGINT
         input to preserve the exact data type of the argument when it's possible.
      
      2. Fixing FLOOR(x) and CEILING(x) with TINYINT, SMALLINT, MEDIUMINT, BIGINT
        to preserve the exact data type of the argument.
      
      3. Adding dedicated Type_handler_year::Item_func_round_fix_length_and_dec()
        to easier handle ROUND(x) and TRUNCATE(x,y) for the YEAR(2) and YEAR(4)
        input. They still return INT(2) UNSIGNED and INT(4) UNSIGNED correspondingly,
        as before.
      a874b6c4
  3. 30 Jul, 2020 16 commits
    • Marko Mäkelä's avatar
      MDEV-23334 Crash in rec_get_nth_cfield()/rec_offs_validate() · 6053eb1c
      Marko Mäkelä authored
      rec_get_nth_cfield(): Remove a bogus debug assertion.
      The function may be invoked by innobase_rec_to_mysql()
      for reporting a duplicate key error during CREATE UNIQUE INDEX
      or ALTER TABLE...ADD UNIQUE KEY, and in that case the record
      will be missing the 5-byte or 6-byte fixed header.
      
      It turns out that in every other code path leading to
      rec_get_nth_cfield() we either invoked rec_get_offsets()
      ourselves or asserted rec_offs_validate(). So, we can
      safely remove the assertion and make debug builds
      smaller and faster.
      6053eb1c
    • Krunal Bauskar's avatar
      MDEV-23030: ARM crash on Raspberry Pi 4 · c69520c9
      Krunal Bauskar authored
      MariaDB adopted a hardware optimized crc32c approach on ARM64 starting 10.5.
      Said implementation of crc32c needs support from target hardware for crc32
      and pmull instructions. Existing logic is checking only for crc32 support
      from target hardware through a runtime check and so if target hardware
      doesn't support pmull it would cause things to fail/crash.
      
      Expanded runtime check to ensure pmull support is also checked on the target
      hardware along with existing crc32.
      
      Thanks to Marko and Daniel for review.
      c69520c9
    • Jan Lindström's avatar
      Enable test for testing. · c3958ae4
      Jan Lindström authored
      c3958ae4
    • Vladislav Vaintroub's avatar
      MDEV-21101 skip test for embedded · 0435fcf9
      Vladislav Vaintroub authored
      0435fcf9
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23332 Index online status assert failure in btr_search_drop_page_hash_index · 8a612314
      Thirunarayanan Balathandayuthapani authored
      Problem:
      ========
      In row_merge_drop_indexes(), InnoDB drops only the index from
      dictionary and frees the index pages but it maintains the index
      object if the table is being used by other DML threads. It sets
      the online status of the index to ONLINE_INDEX_ABORTED_DROPPED.
      Removing the index from dictionary doesn't remove the
      corressponding ahi entries of the index. When block is being
      reused, InnoDB tries to remove ahi entries for the block and
      it fails if index online status is ONLINE_INDEX_ABORTED_DROPPED.
      
      Fix:
      ====
      MDEV-22456 allows the index ahi entries to be dropped lazily.
      so checking online status in btr_search_drop_page_hash_index()
      is meaningless and should be removed.
      8a612314
    • Vladislav Vaintroub's avatar
      MDEV-21101 unexpected wait_timeout with pool-of-threads · 71015d84
      Vladislav Vaintroub authored
      Due to restricted size of the threadpool, execution of client queries can
      be delayed (queued) for a while. This delay was interpreted as client
      inactivity, and connection is closed, if client idle time + queue time
      exceeds wait_timeout.
      
      But users did not expect queue time to be included into wait_timeout.
      
      This patch changes the behavior. We don't close connection anymore,
      if there is some unread data present on connection,
      even if wait_timeout is exceeded. Unread data means that client
      was not idle, it sent a query, which we did not have time to process yet.
      71015d84
    • Marko Mäkelä's avatar
      MDEV-23339 innodb_force_recovery=2 may still abort the rollback of recovered transactions · c5d4dd25
      Marko Mäkelä authored
      trx_rollback_active(), trx_rollback_resurrected(): Replace
      an incorrect condition that we failed to replace in
      commit b68f1d84 (MDEV-21217).
      c5d4dd25
    • Alexander Barkov's avatar
      MDEV-23323 Rounding functions return a wrong data type for a BIT, ENUM, SET argument · 6d3186e3
      Alexander Barkov authored
      Implementing dedicated fixing methods:
      - Type_handler_bit::Item_func_round_fix_length_and_dec()
      - Type_handler_bit::Item_func_int_val_fix_length_and_dec()
      - Type_handler_typelib::Item_func_round_fix_length_and_dec()
      
      because the inherited methods did not work well.
      
      Fixing:
      - Type_handler_typelib::Item_func_int_val_fix_length_and_dec
        It did not work well, because it used args[0]->max_length to
        calculate the result data type. In case of ENUM and SET it was
        not correct, because in FLOOR() and CEILING() context
        ENUM and SET return not more than 5 digits (65535 is the biggest
        possible value).
      
      Misc:
      - Changing the API of
          Type_handler_bit::Bit_decimal_notation_int_digits(const Item *item)
        to a more generic form:
          Type_handler_bit::Bit_decimal_notation_int_digits_by_nbits(uint nbits)
      
      - Fixing Type_handler_bit::Bit_decimal_notation_int_digits_by_nbits() to
        return the exact number of decimal digits for all nbits 1..64.
        The old implementation was approximate.
        This change gives better (more precise) data types.
      6d3186e3
    • Otto Kekäläinen's avatar
      Deb: Make RocksDB plugin depend on python3 as myrocks_hotbackup needs it · f99de891
      Otto Kekäläinen authored
      This fixes the Lintian error and thus the Lintian override can also be
      removed.
      f99de891
    • Otto Kekäläinen's avatar
      Deb: Proper DH_ and DEB_ flag use in debian/rules · a10f72aa
      Otto Kekäläinen authored
      - DEB_BUILD_HARDENING is only used with hardening-wrapper which is
        deprecated in Debian, so remove it
      - The word 'terse' should be checked in DEB_BUILD_OPTIONS and verbosity
        controlled by it
      a10f72aa
    • Otto Kekäläinen's avatar
      Deb: Stop suggesting tinyca, upstream project does not exist anymore · 497e7eda
      Otto Kekäläinen authored
      TinyCA has not been updated since 2016 and was removed from Debian in 2019
      and the dev site https://tinyca.alioth.debian.org/ is offline.
      
      See https://tracker.debian.org/pkg/tinyca
      497e7eda
    • Otto Kekäläinen's avatar
      Deb: Simplify and unify autobake-deb.sh · bdf82683
      Otto Kekäläinen authored
      - Remove excess build flag from debian/rules that does nothing, introduced
        wrongly in commit 7cbde2d0. Instead
        implement the embedded server build skipping on Travis-CI correctly.
      - Simplify structure by doing all Travis-CI slimdown in one step.
      - Remove unnecessary -e from sed, as it does nothing. When regex is needed,
        use -r. Move -i to last so it is close to the file name it has an
        argument.
      - Remove backwards compat checks that are no longer relevant as neither
        Debian Jessie (was before Stretch) nor Ubuntu Trusty (before Xenial)
        are supported nor built for anymore. For example the GCC 4.8 check
        if not relevant anymore, since Debian Jessie already has 4.9 and
        Ubuntu Xenial has 5.3 and there is no GCC < 4.8 around anymore.
      - Skip building ColumnStore on both Travis-CI and Gitlab-CI as it is
        way too slow (time) and big (disk space) to pass.
      bdf82683
    • Otto Kekäläinen's avatar
      Deb: Cleanup and document · 7a0fa9da
      Otto Kekäläinen authored
      - Remove unnecessary unused files
      - Remove duplicate encryption configuration sample from sources and
        re-use the identical file in RPM directory instead
      - Clean away harmful "default-character-set = utf8mb4" from client config
        as it is unnecassary (server enforces utf8mb4 anyway by default) and
        could cause issues with mysqlbinlog and other tools (MDEV-22981).
      - Update S3 plugin description to be long enough
      - Remove trailing whitespace from support-files and Debian packaging.
      - Clean away fixed Lintian issues
      - Clean away temporary Salsa-CI fixes now that 10.5.4 is out and is fixed
      - Apply wrap-and-sort -a -v
      7a0fa9da
    • Otto Kekäläinen's avatar
    • Otto Kekäläinen's avatar
      Use mktemp instead of deprecated tempfile · b8031e36
      Otto Kekäläinen authored
      This fixes the warning emitted during `/etc/init.d/mariadb restart`:
        WARNING: tempfile is deprecated; consider using mktemp instead.
      b8031e36
    • Otto Kekäläinen's avatar
      Fix spelling errors · ab48901d
      Otto Kekäläinen authored
      ab48901d
  4. 29 Jul, 2020 5 commits
    • Alexander Barkov's avatar
      MDEV-23320 Hex hybrid constants 0xHHHH work badly in rounding functions · 92499ae9
      Alexander Barkov authored
      - Type_handler_hex_hybrid did not override
        Type_handler_string_result::Item_func_round_fix_length_and_dec(),
        so the result type of ROUND(0xFFFFFFFFFFFFFFFF) was erroneously
        calculated ad DOUBLE with a wrong length.
        Overriding Item_func_round_fix_length_and_dec(), to calculated
        the result type as INT/BIGINT.
      
        Also, fixing Item_func_round::fix_arg_int() to use
        args[0]->decimal_precision() instead of args[0]->max_length
        when calculating this->max_length, to get a correct result
        for hex hybrids.
      
      - Type_handler_hex_hybrid::Item_func_int_val_fix_length_and_dec()
        called item->fix_length_and_dec_int_or_decimal(), which did not
        produce a correct result data type for hex hybrid.
        Implementing a dedicated code instead, to return INT UNSIGNED or
        BIGINT UNSIGNED depending in the number of digits in the arguments.
      92499ae9
    • Oleksandr Byelkin's avatar
      MDEV-21258: Can't uninstall plugin if the library file doesn't exist · 2107e3bb
      Oleksandr Byelkin authored
      Removing plugin from the mysql.plugin even if the plugin is not loaded
      2107e3bb
    • Eugene Kosov's avatar
      speed up my_timer_init() · 8ec877f4
      Eugene Kosov authored
      I run perf top during ./mtr testing and constantly see times()
      function there. It's so slow, that it has no sense to run it
      in a loop too many times.
      
      This patch speeds up -suite=innodb for me from 218s to 208s.
      9s of times() function!
      8ec877f4
    • Nikita Malyavin's avatar
      MDEV-16023 Unfortunate error message WARN_VERS_PART_FULL (partition <name> is... · 34f2be3b
      Nikita Malyavin authored
      MDEV-16023 Unfortunate error message WARN_VERS_PART_FULL (partition <name> is full) when rotation time for the last interval passed
      
      * remove one case of WARN_VERS_PART_FULL
      34f2be3b
    • Eugene Kosov's avatar
      MDEV-23245 MDEV-22898 Still getting assertion failure in file data0type.cc line 67 · 423de1e5
      Eugene Kosov authored
      Doesn't allow instant alter of a field which is a prefix part of some key
      
      is_part_of_a_key_prefix(): I hope the name of the function is clear
      
      ha_innobase::can_convert_string()
      ha_innobase::can_convert_varstring()
      ha_innobase::can_convert_blob(): it can't when field is_part_of_a_key_prefix()
      423de1e5
  5. 28 Jul, 2020 1 commit