1. 30 Jul, 2020 7 commits
    • Otto Kekäläinen's avatar
      Deb: Proper DH_ and DEB_ flag use in debian/rules · a10f72aa
      Otto Kekäläinen authored
      - DEB_BUILD_HARDENING is only used with hardening-wrapper which is
        deprecated in Debian, so remove it
      - The word 'terse' should be checked in DEB_BUILD_OPTIONS and verbosity
        controlled by it
      a10f72aa
    • Otto Kekäläinen's avatar
      Deb: Stop suggesting tinyca, upstream project does not exist anymore · 497e7eda
      Otto Kekäläinen authored
      TinyCA has not been updated since 2016 and was removed from Debian in 2019
      and the dev site https://tinyca.alioth.debian.org/ is offline.
      
      See https://tracker.debian.org/pkg/tinyca
      497e7eda
    • Otto Kekäläinen's avatar
      Deb: Simplify and unify autobake-deb.sh · bdf82683
      Otto Kekäläinen authored
      - Remove excess build flag from debian/rules that does nothing, introduced
        wrongly in commit 7cbde2d0. Instead
        implement the embedded server build skipping on Travis-CI correctly.
      - Simplify structure by doing all Travis-CI slimdown in one step.
      - Remove unnecessary -e from sed, as it does nothing. When regex is needed,
        use -r. Move -i to last so it is close to the file name it has an
        argument.
      - Remove backwards compat checks that are no longer relevant as neither
        Debian Jessie (was before Stretch) nor Ubuntu Trusty (before Xenial)
        are supported nor built for anymore. For example the GCC 4.8 check
        if not relevant anymore, since Debian Jessie already has 4.9 and
        Ubuntu Xenial has 5.3 and there is no GCC < 4.8 around anymore.
      - Skip building ColumnStore on both Travis-CI and Gitlab-CI as it is
        way too slow (time) and big (disk space) to pass.
      bdf82683
    • Otto Kekäläinen's avatar
      Deb: Cleanup and document · 7a0fa9da
      Otto Kekäläinen authored
      - Remove unnecessary unused files
      - Remove duplicate encryption configuration sample from sources and
        re-use the identical file in RPM directory instead
      - Clean away harmful "default-character-set = utf8mb4" from client config
        as it is unnecassary (server enforces utf8mb4 anyway by default) and
        could cause issues with mysqlbinlog and other tools (MDEV-22981).
      - Update S3 plugin description to be long enough
      - Remove trailing whitespace from support-files and Debian packaging.
      - Clean away fixed Lintian issues
      - Clean away temporary Salsa-CI fixes now that 10.5.4 is out and is fixed
      - Apply wrap-and-sort -a -v
      7a0fa9da
    • Otto Kekäläinen's avatar
    • Otto Kekäläinen's avatar
      Use mktemp instead of deprecated tempfile · b8031e36
      Otto Kekäläinen authored
      This fixes the warning emitted during `/etc/init.d/mariadb restart`:
        WARNING: tempfile is deprecated; consider using mktemp instead.
      b8031e36
    • Otto Kekäläinen's avatar
      Fix spelling errors · ab48901d
      Otto Kekäläinen authored
      ab48901d
  2. 28 Jul, 2020 7 commits
    • Marko Mäkelä's avatar
      MDEV-22110 InnoDB unnecessarily writes unmodified pages · 05fa4558
      Marko Mäkelä authored
      At least since commit 6a7be48b
      InnoDB appears to be invoking buf_flush_note_modification() on pages
      that were exclusively latched but not modified in a mini-transaction.
      
      MTR_MEMO_MODIFY, mtr_t::modify(): Define not only in debug code,
      but also in release code. We will set the MTR_MEMO_MODIFY flag
      on the earliest mtr_t::m_memo entry that we find.
      
      MTR_LOG_NONE: Only use this mode in cases where the previous
      mode will be restored before anything is modified in the mini-transaction.
      
      MTR_MEMO_PAGE_X_MODIFY, MTR_MEMO_PAGE_SX_MODIFY: The allowed flag
      combinations that include MTR_MEMO_MODIFY.
      
      ReleaseBlocks: Only invoke buf_flush_note_modification()
      on those buffer pool blocks on which mtr_t::set_modified()
      and mtr_t::modify() were invoked.
      05fa4558
    • Marko Mäkelä's avatar
      MDEV-23304 Insert into TEMPORARY TABLE fails to invoke mtr_t::set_modified() · cf3c3cce
      Marko Mäkelä authored
      In MDEV-21724 or possibly already in MDEV-12353 a bug was introduced to
      the handling of temporary tables.
      
      Whether or not redo log will be written, mtr_t::set_modified()
      must be invoked to register any changes to a page.
      
      page_cur_insert_rec_low(): Invoke mtr_t::set_modified() also
      when skipping the redo log write.
      cf3c3cce
    • Roman Nozdrin's avatar
      MCOL-4166 The solution to the minor upgrade issue · 2c5831b2
      Roman Nozdrin authored
      DEB package doesn't need expect and libreadline as dependencies
      
      RPM packages now depends on jemalloc again
      2c5831b2
    • Daniel Black's avatar
      MDEV-23176: s3 curl library path fix · 6a70398c
      Daniel Black authored
      On FreeBSD (and possibly elsewhere) the FindPackage(curl)
      may have found the curl library in a non-standard directory.
      
      We use the exposed path as a library path to correct linking.
      6a70398c
    • Daniel Black's avatar
      MDEV-23175: my_timer_milliseconds clock_gettime for multiple platfomrs · 3cb9131a
      Daniel Black authored
      Small postfix to MDEV-23175 to ensure faster option on FreeBSD
      and compatibility to Solaris that isn't high resolution.
      
      ftime is left as a backup in case an implementation doesn't
      contain any of these clocks.
      
      FreeBSD
          $ ./unittest/mysys/my_rdtsc-t
          1..11
          # ----- Routine ---------------
          # myt.cycles.routine          :             5
          # myt.nanoseconds.routine     :            11
          # myt.microseconds.routine    :            13
          # myt.milliseconds.routine    :            11
          # myt.ticks.routine           :            17
          # ----- Frequency -------------
          # myt.cycles.frequency        :    3610295566
          # myt.nanoseconds.frequency   :    1000000000
          # myt.microseconds.frequency  :       1000000
          # myt.milliseconds.frequency  :           899
          # myt.ticks.frequency         :           136
          # ----- Resolution ------------
          # myt.cycles.resolution       :             1
          # myt.nanoseconds.resolution  :             1
          # myt.microseconds.resolution :             1
          # myt.milliseconds.resolution :             7
          # myt.ticks.resolution        :             1
          # ----- Overhead --------------
          # myt.cycles.overhead         :            26
          # myt.nanoseconds.overhead    :         19140
          # myt.microseconds.overhead   :         19036
          # myt.milliseconds.overhead   :           578
          # myt.ticks.overhead          :         21544
          ok 1 - my_timer_init() did not crash
          ok 2 - The cycle timer is strictly increasing
          ok 3 - The cycle timer is implemented
          ok 4 - The nanosecond timer is increasing
          ok 5 - The nanosecond timer is implemented
          ok 6 - The microsecond timer is increasing
          ok 7 - The microsecond timer is implemented
          ok 8 - The millisecond timer is increasing
          ok 9 - The millisecond timer is implemented
          ok 10 - The tick timer is increasing
          ok 11 - The tick timer is implemented
      3cb9131a
    • Daniel Black's avatar
      mtr: mysqld--help-aria test on OSX fail · ba191f7e
      Daniel Black authored
      Previous test failure:
      
      --- /Users/travis/build/grooverdan/mariadb-server/mysql-test/main/mysqld--help-aria.result	2020-04-29 03:44:40.000000000 +0000
      +++ /Users/travis/build/grooverdan/mariadb-server/mysql-test/main/mysqld--help-aria.reject	2020-04-29 04:19:13.000000000 +0000
      @@ -1,3 +1,4 @@
      +[Warning] Setting lower_case_table_names=2 because file system for /Users/travis/build/grooverdan/mariadb-server/mysql-test/var/4/mariadbd.1/data/ is case insensitive
       [ERROR] mariadbd: Can't lock aria aria_log_control for exclusive use, error: #. Will retry for 0 seconds
       [ERROR] Plugin 'Aria' init function returned error.
       [ERROR] Plugin 'Aria' registration as a STORAGE ENGINE failed.
      @@ -9,4 +10,5 @@
       #
       # Check with existing directory
       #
      +[Warning] Setting lower_case_table_names=2 because file system for /Users/travis/build/grooverdan/mariadb-server/mysql-test/var/tmp/4/help/ is case insensitive
       [Warning] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some options may be missing from the help text
      ba191f7e
    • Daniel Black's avatar
  3. 24 Jul, 2020 1 commit
  4. 23 Jul, 2020 12 commits
    • Varun Gupta's avatar
      MDEV-23216: LONGTEXT column with collation doesn't sort · a18639f1
      Varun Gupta authored
      An overflow was happening with LONGTEXT columns, when the length was converted to the length
      in the strxfrm form (mem-comparable keys).
      Introduced a function to truncate the length to the max_sort_length before calculating
      the length of the strxfrm form.
      a18639f1
    • Jan Lindström's avatar
      Fix test cases · a8d5f57e
      Jan Lindström authored
      Changes to be committed:
      	modified:   mysql-test/suite/galera_3nodes/t/GCF-354.test
      	modified:   mysql-test/suite/galera_3nodes/t/inconsistency_shutdown.test
      a8d5f57e
    • Monty's avatar
      thd->m_transaction_psi was not properly cleared for new connections · ce699df9
      Monty authored
      This happend when using XA transactions. I also added some extra asserts
      to ensure that m_transactions are properly cleared.
      
      Other things:
      - Removed set_time() from THD::init_for_queries() as dispatch_command()
        is already doing that.
      - Removed duplicate init_for_queries() from prepare_new_connection_state().
        The init_for_queries() functions should only be called once per
        connection.
      ce699df9
    • Monty's avatar
      MDEV-7947 strcmp() takes 0.37% in OLTP RO · dbcd3384
      Monty authored
      This patch ensures that all identical character sets shares the same
      cs->csname.
      This allows us to replace strcmp() in my_charset_same() with comparisons
      of pointers. This fixes a long standing performance issue that could cause
      as strcmp() for every item sent trough the protocol class to the end user.
      
      One consequence of this patch is that we don't allow one to add a character
      definition in the Index.xml file that changes the csname of an existing
      character set. This is by design as changing character set names of existing
      ones is extremely dangerous, especially as some storage engines just records
      character set numbers.
      
      As we now have a hash over character set's csname, we can in the future
      use that for faster access to a specific character set. This could be done
      by changing the hash to non unique and use the hash to find the next
      character set with same csname.
      dbcd3384
    • Monty's avatar
      Fixed wrong free in comp_err · 46ffd47f
      Monty authored
      46ffd47f
    • Monty's avatar
      d55f8a24
    • Monty's avatar
      Fixed removed warning from valgrind in Protocol::store_str · 747479ab
      Monty authored
      The problem was that field_count is not initialized for the Protocol
      variable used when printing metadata.
      747479ab
    • Monty's avatar
      Remove String::lex_string() and String::lex_cstring() · 61c15ebe
      Monty authored
      - Better to use 'String *' directly.
      - Added String::get_value(LEX_STRING*) for the few cases where we want to
        convert a String to LEX_CSTRING.
      
      Other things:
      - Use StringBuffer for some functions to avoid mallocs
      61c15ebe
    • Monty's avatar
      Use larger buffer when reading binary and relay logs · 26824581
      Monty authored
      - Should speed up replication
      26824581
    • Monty's avatar
      Clean up Item_uint() & Item_int() · c89e927a
      Monty authored
      - Removed val_str() and print() as these are handled by Item_int()
      - Use local StringBuffer for Item_int::print() to avoid mallocs
      c89e927a
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 5e76e234
      Marko Mäkelä authored
      5e76e234
    • Marko Mäkelä's avatar
      MDEV-22778 Slow InnoDB shutdown on large instance · 5f2628d1
      Marko Mäkelä authored
      Starting with MDEV-17441 we would no longer have os_once,
      and we would always initialize zip_pad_info_t::mutex and
      dict_table_t::autoinc_mutex, even for tables are not in
      ROW_FORMAT=COMPRESSED nor include any AUTO_INCREMENT column.
      
      mutex_free() on those unnecessary objects would make shutdown very slow
      compared to older versions.
      
      Let us use std::mutex for those two mutexes, to reduce the overhead.
      The critical sections protected by these mutexes is very small, and
      therefore contention or the need for any instrumentation should
      be unlikely.
      5f2628d1
  5. 22 Jul, 2020 7 commits
    • Oleksandr Byelkin's avatar
      MDEV-21997 Server crashes in LEX::create_item_ident_sp upon use of unknown identifier · ddb8309e
      Oleksandr Byelkin authored
      If there is no current_select and variable is not found among SP variables it can be only an error.
      ddb8309e
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23252 Assertion failure 'req_type.is_dblwr_recover() || err ==... · 92014bd1
      Thirunarayanan Balathandayuthapani authored
      MDEV-23252 Assertion failure 'req_type.is_dblwr_recover() || err == DB_SUCCESS' for page_compressed tables
      
      - This issue is caused by a5584b13
      (MDEV-15528). os_file_punch_hole() is added to fil_io() in MDEV-15528.
      But it fails to handle failure of os_file_punch_hole(). InnoDB should
      handle the DB_IO_NO_PUNCH_HOLE error and silently transform to
      DB_SUCCESS. InnoDB should set the punch hole flag correctly when
      tablespace is loaded
      
      fil_node_t::read_page0(): Set the punch hole flag when tablespace is loaded
      
      fil_io(): Handle the DB_IO_NO_PUNCH_HOLE error
      
      buf_flush_free_pages(): Checks the punch hole condition earlier using
      tablespace punch hole flag
      92014bd1
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23254 Replace FSP_FLAGS_HAS_PAGE_COMPRESSION with fil_space_t::is_compressed · d96027c8
      Thirunarayanan Balathandayuthapani authored
      InnoDB should replace FSP_FLAGS_HAS_PAGE_COMPRESSION check with
      fil_space_t::is_compressed(). fil_space_t::is_compressed() checks
      for both non full crc32 and crc32 format.
      d96027c8
    • Jan Lindström's avatar
      Fix regex on test. · 3d01576a
      Jan Lindström authored
      3d01576a
    • Thirunarayanan Balathandayuthapani's avatar
      MDEV-23254 Replace FSP_FLAGS_HAS_PAGE_COMPRESSION with fil_space_t::is_compressed · 1ca52b96
      Thirunarayanan Balathandayuthapani authored
      InnoDB should replace FSP_FLAGS_HAS_PAGE_COMPRESSION check with
      fil_space_t::is_compressed(). fil_space_t::is_compressed() checks
      for both non full crc32 and crc32 format.
      1ca52b96
    • Jan Lindström's avatar
      Fix regex on test. · 8c7f7bae
      Jan Lindström authored
      8c7f7bae
    • sjaakola's avatar
      MDEV-21910 Deadlock between BF abort and manual KILL command · 7bffe468
      sjaakola authored
      When high priority replication slave applier encounters lock conflict in innodb,
      it will force the conflicting lock holder transaction (victim) to rollback.
      This is a must in multi-master sychronous replication model to avoid cluster lock-up.
      This high priority victim abort (aka "brute force" (BF) abort), is started
      from innodb lock manager while holding the victim's transaction's (trx) mutex.
      Depending on the execution state of the victim transaction, it may happen that the
      BF abort will call for THD::awake() to wake up the victim transaction for the rollback.
      Now, if BF abort requires THD::awake() to be called, then the applier thread executed
      locking protocol of: victim trx mutex -> victim THD::LOCK_thd_data
      
      If, at the same time another DBMS super user issues KILL command to abort the same victim,
      it will execute locking protocol of: victim THD::LOCK_thd_data  -> victim trx mutex.
      These two locking protocol acquire mutexes in opposite order, hence unresolvable mutex locking
      deadlock may occur.
      
      The fix in this commit adds THD::wsrep_aborter flag to synchronize who can kill the victim
      This flag is set both when BF is called for from innodb and by KILL command.
      Either path of victim killing will bail out if victim's wsrep_killed is already
      set to avoid mutex conflicts with the other aborter execution. THD::wsrep_aborter
      records the aborter THD's ID. This is needed to preserve the right to kill
      the victim from different locations for the same aborter thread.
      It is also good error logging, to see who is reponsible for the abort.
      
      A new test case was added in galera.galera_bf_kill_debug.test for scenario where
      wsrep applier thread and manual KILL command try to kill same idle victim
      7bffe468
  6. 21 Jul, 2020 5 commits
    • Marko Mäkelä's avatar
      Merge 10.4 into 10.5 · 4ec032b4
      Marko Mäkelä authored
      4ec032b4
    • Marko Mäkelä's avatar
      Merge 10.3 into 10.4 · b1538f4d
      Marko Mäkelä authored
      b1538f4d
    • Marko Mäkelä's avatar
      MDEV-15880: ASAN heap-use-after-free with innodb_evict_tables_on_commit_debug · b75563cd
      Marko Mäkelä authored
      trx_update_mod_tables_timestamp(): When implementing
      innodb_evict_tables_on_commit_debug, do not evict tables
      on which transactional locks exist.
      
      This debug variable was broken since its introduction in
      commit 947b0b57.
      b75563cd
    • Monty's avatar
      MDEV-16929 Assertion ... in close_thread_tables upon killing connection · e26c822a
      Monty authored
      Problem was that the code didn't handle a transaction created in innodb
      as part of a failed mysql_lock_tables()
      e26c822a
    • Monty's avatar
      MDEV-21953 deadlock between BACKUP STAGE BLOCK_COMMIT and parallel repl. · fc48c8ff
      Monty authored
      The issue was:
      T1, a parallel slave worker thread, is waiting for another worker thread to
      commit. While waiting, it has the MDL_BACKUP_COMMIT lock.
      T2, working for mariabackup, is doing BACKUP STAGE BLOCK_COMMIT and blocks
      all commits.
      This causes a deadlock as the thread T1 is waiting for can't commit.
      
      Fixed by moving locking of MDL_BACKUP_COMMIT from ha_commit_trans() to
      commit_one_phase_2()
      
      Other things:
      - Added a new argument to ha_comit_one_phase() to signal if the
        transaction was a write transaction.
      - Ensured that ha_maria::implicit_commit() is always called under
        MDL_BACKUP_COMMIT. This code is not needed in 10.5
      - Ensure that MDL_Request values 'type' and 'ticket' are always
        initialized. This makes it easier to check the state of the MDL_Request.
      - Moved thd->store_globals() earlier in handle_rpl_parallel_thread() as
        thd->init_for_queries() could use a MDL that could crash if store_globals
        where not called.
      - Don't call ha_enable_transactions() in THD::init_for_queries() as this
        is both slow (uses MDL locks) and not needed.
      fc48c8ff
  7. 20 Jul, 2020 1 commit
    • Eugene Kosov's avatar
      MDEV-22899 Assertion `field->col->is_binary() || field->prefix_len %... · c4d5b6b1
      Eugene Kosov authored
      MDEV-22899 Assertion `field->col->is_binary() || field->prefix_len % field->col->mbmaxlen == 0' failed in dict_index_add_to_cache
      
      is_part_of_a_key(): detect is TEXT field is a part of some key
      
      ha_innobase::can_convert_blob(): now correctly detect whether our blob
      is a part of some key. Previously the check didn't work in some cases.
      c4d5b6b1