1. 09 Jan, 2017 3 commits
  2. 07 Jan, 2017 3 commits
  3. 06 Jan, 2017 2 commits
  4. 05 Jan, 2017 10 commits
    • Marko Mäkelä's avatar
      Merge 10.1 into 10.2 · 4ce579d2
      Marko Mäkelä authored
      4ce579d2
    • Marko Mäkelä's avatar
      Merge 10.0 into 10.1 · 8049d2e9
      Marko Mäkelä authored
      8049d2e9
    • Marko Mäkelä's avatar
      MDEV-11730 Memory leak in innodb.innodb_corrupt_bit · f0c19b6a
      Marko Mäkelä authored
      Memory was leaked when ALTER TABLE is attempted on a table
      that contains corrupted indexes.
      The memory leak was reported by AddressSanitizer for the test
      innodb.innodb_corrupt_bit. The leak was introduced into
      MariaDB Server 10.0.26, 10.1.15, 10.2.1 by the following:
      
      commit c081c978
      Merge: 1d21b221 a482e76e
      Author: Sergei Golubchik <serg@mariadb.org>
      Date:   Tue Jun 21 14:11:02 2016 +0200
      
         Merge branch '5.5' into bb-10.0
      f0c19b6a
    • Marko Mäkelä's avatar
      Merge 10.1 to 10.2. · 11b7dff5
      Marko Mäkelä authored
      This makes no functional change to MariaDB Server 10.2.
      XtraDB is not being built, and in InnoDB, there was no
      memory leak in buf_dblwr_process() in MariaDB Server 10.2.
      11b7dff5
    • Marko Mäkelä's avatar
      fb5ee7d6
    • Marko Mäkelä's avatar
      Post-merge fix for MDEV-11638. · 30f27b0d
      Marko Mäkelä authored
      logs_empty_and_mark_files_at_shutdown(): Wait for the log_scrub_thread
      to exit.
      30f27b0d
    • Marko Mäkelä's avatar
      Fix InnoDB compilation warnings. · a8ac6dc5
      Marko Mäkelä authored
      Most of them are trivial, except for the thread_sync_t refactoring.
      We must not invoke memset() on non-POD objects.
      
      mtflush_work_initialized: Remove. Refer to mtflush_ctx != NULL instead.
      
      thread_sync_t::thread_sync_t(): Refactored from
      buf_mtflu_handler_init().
      
      thread_sync_t::~thread_sync_t(): Refactored from
      buf_mtflu_io_thread_exit().
      a8ac6dc5
    • Marko Mäkelä's avatar
      MDEV-11713 Optimize DBUG_PRINT and introduce DBUG_LOG · bf35deda
      Marko Mäkelä authored
      MariaDB Server is unnecessarily evaluating the arguments of
      DBUG_PRINT() macros when the label is not defined.
      
      The macro DBUG_LOG() for C++ operator<< output which was added for
      InnoDB diagnostics in MySQL 5.7 is missing from MariaDB. Unlike the
      MySQL 5.7 implementation, MariaDB will avoid allocating and
      initializing the output string when the label is not defined.
      
      Introduce DBUG_OUT("crypt") and DBUG_OUT("checksum") for some InnoDB
      diagnostics, replacing some use of ib::info().
      bf35deda
    • Marko Mäkelä's avatar
      Merge 10.1 to 10.2 · 4e7b382d
      Marko Mäkelä authored
      Most conflicts are related to the MDEV-11638 InnoDB shutdown refactoring.
      4e7b382d
    • Marko Mäkelä's avatar
      Post-push fix for Part 1 of MDEV-8139 Fix scrubbing tests · 758af98f
      Marko Mäkelä authored
      In the backport of Bug#24450908 UNDO LOG EXISTS AFTER SLOW SHUTDOWN
      from MySQL 5.7 to the MySQL 5.6 based MariaDB Server 10.1, we must
      use a mutex when HAVE_ATOMIC_BUILTINS is not defined.
      
      Also, correct a function comment. In MySQL 5.6 and MariaDB Server 10.1,
      also temporary InnoDB tables are redo-logged.
      758af98f
  5. 04 Jan, 2017 7 commits
    • Igor Babaev's avatar
      Fixed bug mdev-11674. · 348ccb6f
      Igor Babaev authored
      1. The rows of a recursive CTE at some point may overflow
      the HEAP temporary table containing them. At this point
      the table is converted to a MyISAM temporary table and the
      new added rows are placed into this MyISAM table.
      A bug in the of select_union_recursive::send_data prevented
      the server from writing the row that caused the overflow
      into the temporary table used for the result of the iteration
      steps. This could lead, in particular,to a premature end
      of the iterations.
      2. The method TABLE::insert_all_rows_into() that was used
      to copy all rows of one temporary table into another
      did not take into account that the destination temporary
      table must be converted to a MyISAM table at some point.
      This patch fixed this problem. It also renamed the method
      into TABLE::insert_all_rows_into_tmp_table() and added
      an extra parameter needed for the conversion.
      348ccb6f
    • Marko Mäkelä's avatar
      MDEV-8139 Fix scrubbing tests · ffb38c97
      Marko Mäkelä authored
      encryption.innodb_scrub: Clean up. Make it also cover ROW_FORMAT=COMPRESSED,
      removing the need for encryption.innodb_scrub_compressed.
      Add a FIXME comment saying that we should create a secondary index, to
      demonstrate that also undo log pages get scrubbed. Currently that is
      not working!
      
      Also clean up encryption.innodb_scrub_background, but keep it disabled,
      because the background scrubbing does not work reliably.
      
      Fix both tests so that if something is not scrubbed, the test will be
      aborted, so that the data files will be preserved. Allow the tests to
      run on Windows as well.
      ffb38c97
    • Marko Mäkelä's avatar
      MDEV-11638 Encryption causes race conditions in InnoDB shutdown · 719321e7
      Marko Mäkelä authored
      InnoDB shutdown failed to properly take fil_crypt_thread() into account.
      The encryption threads were signalled to shut down together with other
      non-critical tasks. This could be much too early in case of slow shutdown,
      which could need minutes to complete the purge. Furthermore, InnoDB
      failed to wait for the fil_crypt_thread() to actually exit before
      proceeding to the final steps of shutdown, causing the race conditions.
      
      Furthermore, the log_scrub_thread() was shut down way too early.
      Also it should remain until the SRV_SHUTDOWN_FLUSH_PHASE.
      
      fil_crypt_threads_end(): Remove. This would cause the threads to
      be terminated way too early.
      
      srv_buf_dump_thread_active, srv_dict_stats_thread_active,
      lock_sys->timeout_thread_active, log_scrub_thread_active,
      srv_monitor_active, srv_error_monitor_active: Remove a race condition
      between startup and shutdown, by setting these in the startup thread
      that creates threads, not in each created thread. In this way, once the
      flag is cleared, it will remain cleared during shutdown.
      
      srv_n_fil_crypt_threads_started, fil_crypt_threads_event: Declare in
      global rather than static scope.
      
      log_scrub_event, srv_log_scrub_thread_active, log_scrub_thread():
      Declare in static rather than global scope. Let these be created by
      log_init() and freed by log_shutdown().
      
      rotate_thread_t::should_shutdown(): Do not shut down before the
      SRV_SHUTDOWN_FLUSH_PHASE.
      
      srv_any_background_threads_are_active(): Remove. These checks now
      exist in logs_empty_and_mark_files_at_shutdown().
      
      logs_empty_and_mark_files_at_shutdown(): Shut down the threads in
      the proper order. Keep fil_crypt_thread() and log_scrub_thread() alive
      until SRV_SHUTDOWN_FLUSH_PHASE, and check that they actually terminate.
      719321e7
    • Marko Mäkelä's avatar
      Part 1 of MDEV-8139 Fix scrubbing tests · 0f8e17af
      Marko Mäkelä authored
      Port a bug fix from MySQL 5.7, so that all undo log pages will be freed
      during a slow shutdown. We cannot scrub pages that are left allocated.
      
      commit 173e171c6fb55f064eea278c76fbb28e2b1c757b
      Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com>
      Date:   Fri Sep 9 18:01:27 2016 +0530
      
          Bug #24450908   UNDO LOG EXISTS AFTER SLOW SHUTDOWN
      
          Problem:
          ========
      
          1) cached undo segment is not removed from rollback segment history
          (RSEG_HISTORY) during slow shutdown. In other words, If the segment is
          not completely free, we are failing to remove an entry from the history
          list. While starting the server, we traverse all rollback segment slots
          history list and make it as list of undo logs to be purged in purge
          queue.
          In that case, purge queue will never be empty after slow shutdown.
      
          2) Freeing of undo log segment is linked with removing undo log header
          from history.
      
          Fix:
          ====
          1) Have separate logic of removing the undo log header from
          history list from rollback segment slots and remove it from
          rollback segment history even though it is not completely free.
      Reviewed-by: default avatarDebarun Banerjee <debarun.banerjee@oracle.com>
      Reviewed-by: default avatarMarko Mäkelä <marko.makela@oracle.com>
          RB:13672
      0f8e17af
    • Elena Stepanova's avatar
      MDEV-8518 rpl.sec_behind_master-5114 fails sporadically in buildbot · 9bf92706
      Elena Stepanova authored
      - fix the test to avoid false-negatives before MDEV-5114 patch;
      - fix the race condition which made the test fail on slow builders
      9bf92706
    • Oleksandr Byelkin's avatar
      MDEV-10035: DBUG_ASSERT on CREATE VIEW v1 AS SELECT * FROM t1 FOR UPDATE · bc4cac35
      Oleksandr Byelkin authored
      Ability to print lock type added.
      Restoring correct lock type for CREATE VIEW added.
      bc4cac35
    • Marko Mäkelä's avatar
      Merge 10.0 into 10.1 · 0c1de94d
      Marko Mäkelä authored
      0c1de94d
  6. 03 Jan, 2017 11 commits
    • Marko Mäkelä's avatar
      MDEV-11694 InnoDB tries to create unused table SYS_ZIP_DICT · 80d5d145
      Marko Mäkelä authored
      MariaDB Server 10.0.28 and 10.1.19 merged code from Percona XtraDB
      that introduced support for compressed columns. Much but not all
      of this code was disabled by placing #ifdef HAVE_PERCONA_COMPRESSED_COLUMNS
      around it.
      
      Among the unused but not disabled code is code to access
      some new system tables related to compressed columns.
      
      The creation of these system tables SYS_ZIP_DICT and SYS_ZIP_DICT_COLS
      would cause a crash in --innodb-read-only mode when upgrading
      from an earlier version to 10.0.28 or 10.1.19.
      
      Let us remove all the dead code related to compressed columns.
      Users who already upgraded to 10.0.28 and 10.1.19 will have the two
      above mentioned empty tables in their InnoDB system tablespace.
      Subsequent versions of MariaDB Server will completely ignore those tables.
      80d5d145
    • Marko Mäkelä's avatar
      Post-fix for MDEV-11688 fil_crypt_threads_end() tries to create threads · a758479c
      Marko Mäkelä authored
      fil_crypt_threads_cleanup(): Do nothing if nothing was initialized.
      a758479c
    • Marko Mäkelä's avatar
      Post-fix for MDEV-11688 fil_crypt_threads_end() tries to create threads · ba8198a3
      Marko Mäkelä authored
      fil_crypt_threads_cleanup(): Do nothing if nothing was initialized.
      ba8198a3
    • Jan Lindström's avatar
      MDEV-11684: post-10.1-merge fixes · a0d396fd
      Jan Lindström authored
      10.1 is merged into 10.2 now. Two issues are left to fix:
      (1) encryption.innochecksum test
      (2) read_page0 vs page_0_crypt_read
      
      (1) innochecksum tool did not compile after merge because
      buf_page_is_corrupted uses fil_crypt_t that has been changed.
      
      extra/CMakeLists.txt: Added fil/fil0crypt.cc as dependency
      as we need to use fil_crypt_verify_checksum for encrypted pages.
      
      innochecksum.cc: If we think page is encrypted i.e.
      FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION != 0 we call
      fil_crypt_verify_checksum() function to compare calculated
      checksum to stored checksum calculated after encryption
      (this is stored on different offset i.e.
      FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION + 4).
      If checksum does not match we call normal buf_page_is_corrupted
      to compare calculated checksum to stored checksum.
      
      fil0crypt.cc: add #ifdef UNIV_INNOCHECKSUM to be able to compile
      this file for innochecksum tool.
      
      (2) read_page0 is not needed and thus removed.
      a0d396fd
    • Marko Mäkelä's avatar
      MDEV-11688 fil_crypt_threads_end() tries to create threads · 8a04b8ca
      Marko Mäkelä authored
      after aborted InnoDB startup
      
      This bug was repeatable by starting MariaDB 10.2 with an
      invalid option, such as --innodb-flush-method=foo.
      It is not repeatable in MariaDB 10.1 in the same way, but the
      problem exists already there.
      8a04b8ca
    • Marko Mäkelä's avatar
      MDEV-11688 fil_crypt_threads_end() tries to create threads · fc779252
      Marko Mäkelä authored
      after aborted InnoDB startup
      
      This bug was repeatable by starting MariaDB 10.2 with an
      invalid option, such as --innodb-flush-method=foo.
      It is not repeatable in MariaDB 10.1 in the same way, but the
      problem exists already there.
      fc779252
    • Marko Mäkelä's avatar
      MDEV-11695 Define a reasonable upper limit for innodb_spin_wait_delay · 509e7773
      Marko Mäkelä authored
      The upper limit of innodb_spin_wait_delay was ~0UL. It does not make
      any sense to wait more than a few dozens of microseconds between
      attempts to acquire a busy mutex.
      
      Make the new upper limit 6000. ut_delay(6000) could correspond to
      several milliseconds even today.
      509e7773
    • Jan Lindström's avatar
      MDEV-11705: InnoDB: Failing assertion: (&log_sys->mutex)->is_owned() if server... · 403f6e96
      Jan Lindström authored
      MDEV-11705: InnoDB: Failing assertion: (&log_sys->mutex)->is_owned() if server started with innodb-scrub-log
      
      Problem was that log_scrub function did not take required log_sys mutex.
      
      Background: Unused space in log blocks are padded with MLOG_DUMMY_RECORD if innodb-scrub-log
      is enabled. As log files are written on circular fashion old log blocks can be reused
      later for new redo-log entries. Scrubbing pads unused space in log blocks to avoid visibility
      of the possible old redo-log contents.
      
      log_scrub(): Take log_sys mutex
      
      log_pad_current_log_block(): Increase srv_stats.n_log_scrubs if padding is done.
      
      srv0srv.cc: Set srv_stats.n_log_scrubs to export vars innodb_scrub_log
      
      ha_innodb.cc: Export innodb_scrub_log to global status.
      403f6e96
    • Marko Mäkelä's avatar
      Post-fix for MDEV-11195 NUMA does not get enabled even when checks are passed · 4c610d10
      Marko Mäkelä authored
      The C preprocessor symbol WITH_NUMA is never defined. Instead, the symbol
      HAVE_LIBNUMA is used for checking if the feature is to be used.
      
      If cmake -DWITH_NUMA=OFF is specified, HAVE_LIBNUMA will not be defined
      at compilation time even if the library is available.
      
      If cmake -DWITH_NUMA=ON is specified but the library is not available
      at configuration time, the compilation will be aborted.
      4c610d10
    • Sachin Setiya's avatar
      MDEV-7955 WSREP() appears on radar in OLTP RO · b4616c40
      Sachin Setiya authored
      This commit is for optimizing WSREP(thd) macro.
      
      #define WSREP(thd) \
        (WSREP_ON && wsrep && (thd && thd->variables.wsrep_on))
      
      In this we can safely remove wsrep and thd. We are not removing WSREP_ON
      because this will change WSREP(thd) behaviour.
      
      Patch Credit:- Nirbhay Choubay, Sergey Vojtovich
      b4616c40
    • Sachin Setiya's avatar
      MDEV-11016 wsrep_node_is_ready() check is too strict · d9a1a201
      Sachin Setiya authored
      Problem:-
        The condition that checks for node readiness is too strict as it does
        not allow SELECTs even if these selects do not access any tables.
          For example,if we run
             SELECT 1;
          OR
             SELECT @@max_allowed_packet;
      Solution:-
        We need not to report this error when all_tables(lex->query_tables)
        is NULL:
      d9a1a201
  7. 01 Jan, 2017 4 commits
    • Elena Stepanova's avatar
      MDEV-10100 main.pool_of_threads fails sporadically in buildbot · 3871477c
      Elena Stepanova authored
      The patch fixes two test failures:
      - on slow builders, sometimes a connection attempt which should
        fail due to the exceeded number of thread_pool_max_threads
        actually succeeds;
      - on even slow builders, MTR sometimes cannot establish the
        initial connection, and check-testcase fails prior to the
        test start
      
      The problem with check-testcase was caused by connect-timeout=2
      which was set for all clients in the test config file. On slow
      builders it might be not enough.
      There is no way to override it for the pre-test check, so it needed
      to be substantially increased or removed.
      
      The other problem was caused by a race condition between sleeps
      that the test performs in existing connections and the connect
      timeout for the connection attempt which was expected to fail.
      If sleeps finished before the connect-timeout was exceeded, it
      would allow the connection to succeed.
      
      To solve each problem without making the other one worse,
      connect-timeout should be configured dynamically during the test.
      Due to the nature of the test (all connections must be busy
      at the moment when we need to change the timeout, and cannot execute
      SET GLOBAL ...), it needs to be done independently from the server.
      
      The solution:
      - recognize 'connect_timeout' as a connection option in mysqltest's
        "connect" command;
      - remove connect-timeout from the test configuration file;
      - use the new connect_timeout option for those connections which
        are expected to fail;
      - re-arrange the test flow to allow running a huge SLEEP
        without affecting the test execution time (because it would be
        interrupted after the main test flow is finished).
      
      The test is still subject to false negatives, e.g. if the connection
      fails due to timeout rather than due to the exceeded number of
      allowed threads, or if the connection on extra port succeeds due
      to a race condition and not because the special logic for the extra
      port. But those false negatives have always been possible there
      on slow builders, they should not be critical because faster builders
      should catch such failures if they appear.
      3871477c
    • Sachin Setiya's avatar
      MDEV-11636 Extra persistent columns on slave always gets NULL in RBR · efcd0935
      Sachin Setiya authored
      Problem:- In replication if slave has extra persistent column then these
      column are not computed while applying write-set from master.
      
      Solution:- While applying row events from server, we will generate values
      for extra persistent columns.
      efcd0935
    • Sachin Setiya's avatar
      MDEV-11636 Extra persistent columns on slave always gets NULL in RBR · d02a77bc
      Sachin Setiya authored
      Problem:- In replication if slave has extra persistent column then these
      column are not computed while applying write-set from master.
      
      Solution:- While applying row events from server, we will generate values
      for extra persistent columns.
      d02a77bc
    • Sachin Setiya's avatar
      MDEV-11636 Extra persistent columns on slave always gets NULL in RBR · 2f5670dc
      Sachin Setiya authored
      Problem:- In replication if slave has extra persistent column then these
      column are not computed while applying write-set from master.
      
      Solution:- While applying row events from server, we will generate values
      for extra persistent columns.
      2f5670dc