1. 26 Feb, 2014 1 commit
    • unknown's avatar
      MDEV-5657: Parallel replication. · e90f68c0
      unknown authored
      Clean up and improve the parallel implementation code, mainly related to
      scheduling of work to threads and handling of stop and errors.
      
      Fix a lot of bugs in various corner cases that could lead to crashes or
      corruption.
      
      Fix that a single replication domain could easily grab all worker threads and
      stall all other domains; now a configuration variable
      --slave-domain-parallel-threads allows to limit the number of
      workers.
      
      Allow next event group to start as soon as previous group begins the commit
      phase (as opposed to when it ends it); this allows multiple event groups on
      the slave to participate in group commit, even when no other opportunities for
      parallelism are available.
      
      Various fixes:
      
       - Fix some races in the rpl.rpl_parallel test case.
      
       - Fix an old incorrect assertion in Log_event iocache read.
      
       - Fix repeated malloc/free of wait_for_commit and rpl_group_info objects.
      
       - Simplify wait_for_commit wakeup logic.
      
       - Fix one case in queue_for_group_commit() where killing one thread would
         fail to correctly signal the error to the next, causing loss of the
         transaction after slave restart.
      
       - Fix leaking of pthreads (and their allocated stack) due to missing
         PTHREAD_CREATE_DETACHED attribute.
      
       - Fix how one batch of group-committed transactions wait for the previous
         batch before starting to execute themselves. The old code had a very
         complex scheduling where the first transaction was handled differently,
         with subtle bugs in corner cases. Now each event group is always scheduled
         for a new worker (in a round-robin fashion amongst available workers).
         Keep a count of how many transactions have started to commit, and wait for
         that counter to reach the appropriate value.
      
       - Fix slave stop to wait for all workers to actually complete processing;
         before, the wait was for update of last_committed_sub_id, which happens a
         bit earlier, and could leave worker threads potentially accessing bits of
         the replication state that is no longer valid after slave stop.
      
       - Fix a couple of places where the test suite would kill a thread waiting
         inside enter_cond() in connection with debug_sync; debug_sync + kill can
         crash in rare cases due to a race with mysys_var_current_mutex in this
         case.
      
       - Fix some corner cases where we had enter_cond() but no exit_cond().
      
       - Fix that we could get failure in wait_for_prior_commit() but forget to flag
         the error with my_error().
      
       - Fix slave stop (both for normal stop and stop due to error). Now, at stop
         we pick a specific safe point (in terms of event groups executed) and make
         sure that all event groups before that point are executed to completion,
         and that no event group after start executing; this ensures a safe place to
         restart replication, even for non-transactional stuff/DDL. In error stop,
         make sure that all prior event groups are allowed to execute to completion,
         and that any later event groups that have started are rolled back, if
         possible. The old code could leave eg. T1 and T3 committed but T2 not, or
         it could even leave half a transaction not rolled back in some random
         worker, which would cause big problems when that worker was later reused
         after slave restart.
      
       - Fix the accounting of amount of events queued for one worker. Before, the
         amount was reduced immediately as soon as the events were dequeued (which
         happens all at once); this allowed twice the amount of events to be queued
         in memory for each single worker, which is not what users would expect.
      
       - Fix that an error set during execution of one event was sometimes not
         cleared before executing the next, causing problems with the error
         reporting.
      
       - Fix incorrect handling of thd->killed in worker threads.
      e90f68c0
  2. 11 Feb, 2014 3 commits
  3. 10 Feb, 2014 2 commits
  4. 09 Feb, 2014 1 commit
  5. 08 Feb, 2014 3 commits
    • unknown's avatar
      MDEV-5636: Deadlock in RESET MASTER · 07eaf6ea
      unknown authored
      The problem is a deadlock between MYSQL_BIN_LOG::reset_logs() and
      MYSQL_BIN_LOG::mark_xid_done(). The former takes LOCK_log and waits for the
      latter to complete. But the latter also tries to take LOCK_log; this can lead
      to a deadlock.
      
      There was already code that tries to deal with this, with the flag
      reset_master_pending. However, there was still a small opportunity for
      deadlock, when an previous mark_xid_done() is still running when reset_logs()
      is called and is at the precise point where it first releases LOCK_xid_list
      and then re-aquires both LOCK_log and LOCK_xid_list.
      
      Solve by setting reset_master_pending in reset_logs() before taking
      LOCK_log. And also count how many invocations of LOCK_xid_list are in the
      progress of releasing and re-aquiring locks, and in reset_logs() wait for that
      number to drop to zero after setting reset_master_pending and before taking
      LOCK_log.
      07eaf6ea
    • unknown's avatar
      MDEV-4984: Implement MASTER_GTID_WAIT() and @@LAST_GTID. · 76e929a9
      unknown authored
      Rewrite the gtid_waiting::wait_for_gtid() function.
      The code was rubbish (and buggy). Now the logic is
      much clearer.
      
      Also fix a missing slave sync that could cause test failure.
      76e929a9
    • unknown's avatar
      MDEV-4984: Implement MASTER_GTID_WAIT() and @@LAST_GTID. · 3c97d24f
      unknown authored
      Couple of small fixes following buildbot testing.
      3c97d24f
  6. 07 Feb, 2014 2 commits
    • unknown's avatar
      MDEV-4726: Race in mysql-test/suite/rpl/t/rpl_gtid_stop_start.test · 7bb022f3
      unknown authored
      Some GTID test cases were using include/wait_condition.inc with a
      condition like SELECT COUNT(*)=4 FROM t1 to wait for the slave to
      catch up with the master. This causes races and test failures, as the
      changes to the tables become visible at the COMMIT of the SQL thread
      (or even before in case of MyISAM), but the changes to
      @@gtid_slave_pos only become visible a little bit after the COMMIT.
      
      Now that we have MASTER_GTID_WAIT(), just use that to sync up in a
      GTID-friendly way, wrapped in nice include/save_master_gtid.inc and
      include/sync_with_master_gtid.inc scripts.
      7bb022f3
    • unknown's avatar
      MDEV-4984: Implement MASTER_GTID_WAIT() and @@LAST_GTID. · 4e6606ac
      unknown authored
      MASTER_GTID_WAIT() is similar to MASTER_POS_WAIT(), but works with a
      GTID position rather than an old-style filename/offset.
      
      @@LAST_GTID gives the GTID assigned to the last transaction written
      into the binlog.
      
      Together, the two can be used by applications to obtain the GTID of
      an update on the master, and then do a MASTER_GTID_WAIT() for that
      position on any read slave where it is important to get results that
      are caught up with the master at least to the point of the update.
      
      The implementation of MASTER_GTID_WAIT() is implemented in a way
      that tries to minimise the performance impact on the SQL threads,
      even in the presense of many waiters on single GTID positions (as
      from @@LAST_GTID).
      4e6606ac
  7. 31 Jan, 2014 1 commit
  8. 08 Jan, 2014 1 commit
    • unknown's avatar
      MDEV-5509: Seconds_behind_master incorrect in parallel replication · 8cc6e90d
      unknown authored
      The problem was a race between the SQL driver thread and the worker threads.
      The SQL driver thread would set rli->last_master_timestamp to zero to
      mark that it has caught up with the master, while the worker threads would
      set it to the timestamp of the executed event. This can happen out-of-order
      in parallel replication, causing the "caught up" status to be overwritten
      and Seconds_Behind_Master to wrongly grow when the slave is idle.
      
      To fix, introduce a separate flag rli->sql_thread_caught_up to mark that the
      SQL driver thread is caught up. This avoids issues with worker threads
      overwriting the SQL driver thread status. In parallel replication, we then
      make SHOW SLAVE STATUS check in addition that all worker threads are idle
      before showing Seconds_Behind_Master as 0 due to slave idle.
      8cc6e90d
  9. 05 Feb, 2014 2 commits
  10. 03 Feb, 2014 1 commit
  11. 31 Jan, 2014 3 commits
  12. 28 Jan, 2014 9 commits
  13. 27 Jan, 2014 5 commits
  14. 26 Jan, 2014 6 commits
    • Sergei Golubchik's avatar
    • Sergei Golubchik's avatar
      workaround test failures in buildbot: · 8ece9de8
      Sergei Golubchik authored
      in some VMs readline thinks that the window size is zero. ignore it.
      8ece9de8
    • Sergei Golubchik's avatar
      MDEV-5461 Assertion `length <= column->length' fails in write_block_record... · 90e22408
      Sergei Golubchik authored
      MDEV-5461 Assertion `length <= column->length' fails in write_block_record with functions in select list, GROUP BY, ORDER BY
      
      Old code in create_tmp_table(), that created an extra one-byte field (recinfo)
      before every NULL-able grouping field (Field) in the tmp table, did not actually work.
      Because the matching code in end_update(), that was supposed to update this byte,
      was using a wrong offset, updating the first byte of the Field, not a byte before it.
      Normally this wasn't an issue, because the Field value (written later in end_update)
      was overwriting this byte anyway. But in this bug the Field was Field_null, with zero
      length, so end_update() was overwriting the first byte of the following field.
      And the following field was not-nullable constant, which was stored only once in
      create_tmp_table and never updated later.
      
      Fixed by removing the code that didn't do any useful work anyway.
      90e22408
    • Sergei Golubchik's avatar
    • Sergei Golubchik's avatar
      0df3c203
    • Michael Widenius's avatar
      Fix for MDEV-5168: MariaDB returns warnings for INSERT IGNORE · 68028887
      Michael Widenius authored
      Added variable "OLD_MODE" that can be used to turn off the new behavior
      
      mysql-test/r/insert.result:
        Added test case
      mysql-test/r/mysqld--help.result:
        Added old_mode
      mysql-test/suite/sys_vars/r/old_mode_basic.result:
        Added testing of new variable
      mysql-test/suite/sys_vars/t/old_mode_basic.test:
        Added testing of new variable
      mysql-test/t/insert.test:
        Added test case
      sql/sql_class.h:
        Added bit flags for OLD_MODE
      sql/sql_insert.cc:
        Disable duplicate key warnings for INSERT IGNORE of OLD_MODE NO_DUP_KEY_WARNINGS_WITH_IGNORE is used
      sql/sql_show.cc:
        Don't show progress reporting on SHOW PROCESSLIST if OLD_MODE NO_PROGRESS_INFO is used
      sql/sys_vars.cc:
        Added OLD_MODE
      68028887