1. 05 Feb, 2010 4 commits
    • Konstantin Osipov's avatar
      next-mr -> next-4284 merge. · 75ec27ff
      Konstantin Osipov authored
      After merge fixes.
      Adjust replication test cases.
      
      mysql-test/suite/rpl/r/rpl_mixed_row_innodb.result:
        Update results with a new test.
      mysql-test/suite/rpl/r/rpl_stm_loadfile.result:
        Add a warning, which I believe is an expected one.
      mysql-test/suite/rpl/t/rpl_killed_ddl.test:
        Sort results to avoid test failurs under load.
      mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
        Update results (next-4284 merge).
      mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
        Adjust test output to the new table opening scheme: decide_logging_format() is now called in CREATE/DROP trigger.
      75ec27ff
    • Konstantin Osipov's avatar
      Add a suppression for use of functions unsafe in statement · 2ed3784e
      Konstantin Osipov authored
      format.
      
      mysql-test/suite/rpl/t/rpl_view_multi.test:
        Add a suppression (fix a failing test).
      2ed3784e
    • Konstantin Osipov's avatar
      next-mr -> next-4284 merge: · 54be2ba7
      Konstantin Osipov authored
      Change the error code for ER_WARN_I_S_SKIPPED_TABLE, to not
      upset the tests that rely on ER_SLAVE_CONVERSION_ERROR error
      code = 1667.
      Fix a merge bug with binlogging of CREATE TABLE (temporary tables).
      54be2ba7
    • Konstantin Osipov's avatar
      next-mr -> next-4284 merge: · 7366ac4f
      Konstantin Osipov authored
      fix a merge bug when write_bin_log called from mysql_routine_grant()
      would chew up the error.
      rpl_do_grant test would fail on assert that the diagnostics area is
      empty.
      7366ac4f
  2. 04 Feb, 2010 7 commits
    • Konstantin Osipov's avatar
      A post-merge fix for next-mr -> next-4284 merge: · 3ce2dbc6
      Konstantin Osipov authored
      Make all mutexes and conditions of type mysql_mutex_t, mysql_cond_t,
      since it's now the expectation of THD::awake().
      3ce2dbc6
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · ee619f51
      Konstantin Osipov authored
      ee619f51
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · caa966cf
      Konstantin Osipov authored
      Cherry-pick a fix Bug#37148 from next-mr, to preserve
      file ids of the added files, and ensure that all the necessary
      changes have been pulled.
      
      Since initially Bug#37148 was null-merged into 6.0,
      the changeset that is now being cherry-picked was likewise
      null merged into next-4284.
      
      Now that Bug#37148 has been reapplied to 6.0, try to make
      it work with next-4284. This is also necessary to be able
      to pull other changes from 5.1-rep into next-4284.
      
      To resolve the merge issues use this changeset applied
      to 6.0:
      revid:jperkin@sun.com-20091216103628-ylhqf7s6yegui2t9
      revno: 3776.1.1
      committer: He Zhenxing <zhenxing.he@sun.com>
      branch nick: 6.0-codebase-bugfixing
      timestamp: Thu 2009-12-17 17:02:50 +0800
      message:
        Fix merge problem with Bug#37148
      
      
      
      caa966cf
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284-merge. · 9cd23662
      Konstantin Osipov authored
      9cd23662
    • Dmitry Lenev's avatar
      Small clean-up in CREATE TABLE LIKE implementation. · 8d6a192d
      Dmitry Lenev authored
      Removed local variables which became unused when we
      have switched to new approach for CREATE TABLE LIKE
      (i.e. abondoned .FRM file copying) and were causing
      warnings during compilation.
      8d6a192d
    • Jon Olav Hauglid's avatar
      Bug #50821 Deadlock between LOCK TABLES and ALTER TABLE · df56339a
      Jon Olav Hauglid authored
      This was a deadlock between ALTER TABLE and another DML statement
      (or LOCK TABLES ... READ). ALTER TABLE would wait trying to upgrade
      its lock to MDL_EXCLUSIVE and the DML statement would wait trying
      to acquire a TL_READ_NO_INSERT table level lock.
      
      This could happen if one connection first acquired a MDL_SHARED_READ
      lock on a table. In another connection ALTER TABLE is then started.
      ALTER TABLE eventually blocks trying to upgrade to MDL_EXCLUSIVE,
      but while holding a TL_WRITE_ALLOW_READ table level lock.
      
      If the first connection then tries to acquire TL_READ_NO_INSERT,
      it will block and we have a deadlock since neither connection can
      proceed.
      
      This patch fixes the problem by allowing TL_READ_NO_INSERT 
      locks to be granted if another connection holds TL_WRITE_ALLOW_READ
      on the same table. This will allow the DML statement to proceed
      such that it eventually can release its MDL lock which in turn
      makes ALTER TABLE able to proceed.
      
      Note that TL_READ_NO_INSERT was already partially compatible with
      TL_WRITE_ALLOW_READ as the latter would be granted if the former
      lock was held. This patch just makes the opposite true as well.
      
      Also note that since ALTER TABLE takes an upgradable MDL lock,
      there will be no starvation of ALTER TABLE statements by
      statements acquiring TL_READ or TL_READ_NO_INSERT.
      
      Test case added to lock_sync.test.
      df56339a
    • Dmitry Lenev's avatar
      Improve concurrency in metadata locking subsystem by · 5565c82f
      Dmitry Lenev authored
      moving calculation of hash value when looking up
      MDL_lock objects in MDL_map out of critical section.
      5565c82f
  3. 03 Feb, 2010 12 commits
    • Konstantin Osipov's avatar
      Merge next-4284 -> next-4284-merge · 7821295f
      Konstantin Osipov authored
      7821295f
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · 3e1a8973
      Konstantin Osipov authored
      3e1a8973
    • Dmitry Lenev's avatar
      A follow-up for the patch which implemented new · 8113adc5
      Dmitry Lenev authored
      type-of-operation-aware metadata locks and added a
      wait-for graph based deadlock detector to the MDL
      subsystem (this patch fixed bug #46272 "MySQL 5.4.4,
      new MDL: unnecessary deadlock" and bug #37346
      "innodb does not detect deadlock between update and
      alter table").
      
      Removed unused and redundant method.
      8113adc5
    • Konstantin Osipov's avatar
      Merge next-4248 -> next-4284-merge · 5a362b46
      Konstantin Osipov authored
      5a362b46
    • Konstantin Osipov's avatar
      Fix a failure of rpl_cross_version, caused by hand-initalization · eb2edae8
      Konstantin Osipov authored
      of the TABLE_LIST instance, which did not initialize the 
      MDL request properly.
      Use init_one_table() to initialize the MDL request.
      eb2edae8
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · af83bf12
      Konstantin Osipov authored
      af83bf12
    • Jon Olav Hauglid's avatar
      Bug #50786 Assertion `thd->mdl_context.trans_sentinel() == __null' · 98037884
      Jon Olav Hauglid authored
                 failed in open_ltable()
      
      The problem was too restrictive asserts that enforced that 
      open_ltable() was called without any active HANDLERs, LOCK TABLES
      or global read locks. 
      
      However, this can happen in several cases when opening system
      tables. The assert would, for example, be triggered when drop
      function was called from a connection with active HANDLERs as
      this would cause open_ltable() to be called for mysql.proc.
      The assert could also be triggered when using table-based
      general log (mysql.general_log).
      
      This patch removes the asserts since they will be triggered in
      several legitimate cases and because the asserts are no longer
      relevant due to changes in how locks are released.
      
      The patch also fixes set_needs_thr_lock_abort() that before 
      ignored its parameter and always set the member variable to TRUE.
      
      Test case added to mdl_sync.test.
      Thanks to Dmitry Lenev for help with this bug!
      98037884
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284 · ef06b62e
      Konstantin Osipov authored
      ef06b62e
    • Konstantin Osipov's avatar
      Merge next-4284 -> next-4284-merge. · 1d783bfd
      Konstantin Osipov authored
      1d783bfd
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · f9a9c10a
      Konstantin Osipov authored
      Post merge fixes. In sys_vars tests suite, whenever we use LOCK TABLE WRITE,
      adjust to lock the subject table indirecty via a view, to ensure that
      there the desired lock conflict in thr_lock.c is still taking place.
      If locked directly in LOCK TABLES list, thr_lock.c does not get
      involved.
      Adjust the thread wait state names in information_schema.processlist.
      f9a9c10a
    • Dmitry Lenev's avatar
      Fix for bug #50784 "MDL: Assertion `m_tickets.is_empty() || · 9ebc7af0
      Dmitry Lenev authored
      m_tickets.front() == m_trans_sentinel'".
      
      Debug build of server crashed due to assert failure in MDL
      subsystem when one tried to execute multi-table REPAIR or
      OPTIMIZE in autocommit=0 mode.
      
      The assert failure occured when multi-table REPAIR or OPTIMIZE
      started processing of second table from its table list and
      tried to acquire upgradable metadata lock on this table.
      The cause of the assert failure were MDL locks left over from
      processing of previous table. It turned out that in autocommit=0
      mode close_thread_tables() which happens at the end of table
      processing doesn't release metadata locks.
      
      This fix solves problem by releasing locks explicitly using
      MDL_context::release_trans_locks() call.
      
      mysql-test/r/repair.result:
        Added test for bug #50784 "MDL: Assertion `m_tickets.is_empty() ||
        m_tickets.front() == m_trans_sentinel'".
      mysql-test/t/repair.test:
        Added test for bug #50784 "MDL: Assertion `m_tickets.is_empty() ||
        m_tickets.front() == m_trans_sentinel'".
      sql/sql_table.cc:
        Ensure that metadata locks are released after multi-table REPAIR/
        OPTIMIZE/... processes each table. close_thread_tables() which is
        called after processing each table doesn't release metadata locks
        if we are in autocommit=0 mode. So this have to be done
        explicitly by calling MDL_context::release_trans_locks().
      9ebc7af0
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · 514a2e75
      Konstantin Osipov authored
      514a2e75
  4. 02 Feb, 2010 2 commits
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · d2208aaf
      Konstantin Osipov authored
      Fix Bug#50555 "handler commands crash server in my_hash_first()"
      as a post-merge fix (the new handler tests are not passing 
      otherwise).
      - in hash.c, don't call calc_hash if ! my_hash_inited().
      - add tests and results for the test case for Bug#50555
      
      
      mysys/hash.c:
        Assert that the hash is initialized when it's used.
      sql/set_var.cc:
        Check that the hash is initalized before using it (Bug#50555)
      d2208aaf
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · 0ceecaa5
      Konstantin Osipov authored
      0ceecaa5
  5. 01 Feb, 2010 6 commits
    • Konstantin Osipov's avatar
      Merge next-mr -> next-4284. · 70225f0f
      Konstantin Osipov authored
      70225f0f
    • Konstantin Osipov's avatar
    • Dmitry Lenev's avatar
      Fix for sporadical hangs of mdl_sync.test caused by patch · 2aca77a5
      Dmitry Lenev authored
      which implemented new type-of-operation-aware metadata
      locks and added a wait-for graph based deadlock detector
      to the MDL subsystem (this patch fixed bug #46272 "MySQL
      5.4.4, new MDL: unnecessary deadlock" and bug #37346
      "innodb does not detect deadlock between update and alter
      table").
      
      These hangs were caused by missing include of
      wait_condition.inc. This fix simply adds them.
      
      mysql-test/t/mdl_sync.test:
        Added missing include of wait_condition.inc.
      2aca77a5
    • Dmitry Lenev's avatar
      Fix for sporadical crashes of lock_multi_bug38499.test · 575c7b37
      Dmitry Lenev authored
      caused by patch which implemented new type-of-operation-aware
      metadata locks and added a wait-for graph based deadlock
      detector to the MDL subsystem (this patch fixed bug #46272
      "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346
      "innodb does not detect deadlock between update and alter
      table").
      
      Crashes were caused by a race in MDL_context::try_acquire_lock().
      This method added MDL_ticket to the list of granted tickets and
      released lock protecting list before setting MDL_ticket::m_lock.
      Thus some other thread was able to see ticket without properly
      set m_lock member for some short period of time. If this thread
      called method involving this member during this period crash
      happened.
      
      This fix ensures that MDL_ticket::m_lock is set in all cases
      when ticket is added to granted/pending lists in MDL_lock.
      
      sql/mdl.cc:
        We must set MDL_ticket::m_lock member before adding ticket
        to the list of granted tickets, since such tickets can be
        accessed by other threads which might call methods using
        this member.
        Added assert which ensures that all MDL_tickets which are
        added to the granted/pending lists have properly set
        MDL_ticket::m_lock member.
      sql/mdl.h:
        Adjusted comment describing MDL_ticket::m_lock member to
        reflect current reality.
        Added accessor method for this member.
      575c7b37
    • Konstantin Osipov's avatar
      Fix a Windows compilation warning (req_count is later used · 6d9a7ada
      Konstantin Osipov authored
      in a pointer arithmetics expression).
      6d9a7ada
    • Dmitry Lenev's avatar
      Implement new type-of-operation-aware metadata locks. · ff6fd58d
      Dmitry Lenev authored
      Add a wait-for graph based deadlock detector to the
      MDL subsystem.
      
      Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and
      bug #37346 "innodb does not detect deadlock between update and
      alter table".
      
      The first bug manifested itself as an unwarranted abort of a
      transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER
      statement, when this transaction tried to repeat use of a
      table, which it has already used in a similar fashion before
      ALTER started.
      
      The second bug showed up as a deadlock between table-level
      locks and InnoDB row locks, which was "detected" only after
      innodb_lock_wait_timeout timeout.
      
      A transaction would start using the table and modify a few
      rows.
      Then ALTER TABLE would come in, and start copying rows
      into a temporary table. Eventually it would stumble on
      the modified records and get blocked on a row lock.
      The first transaction would try to do more updates, and get
      blocked on thr_lock.c lock.
      This situation of circular wait would only get resolved
      by a timeout.
      
      Both these bugs stemmed from inadequate solutions to the
      problem of deadlocks occurring between different
      locking subsystems.
      
      In the first case we tried to avoid deadlocks between metadata
      locking and table-level locking subsystems, when upgrading shared
      metadata lock to exclusive one.
      Transactions holding the shared lock on the table and waiting for
      some table-level lock used to be aborted too aggressively.
      
      We also allowed ALTER TABLE to start in presence of transactions
      that modify the subject table. ALTER TABLE acquires
      TL_WRITE_ALLOW_READ lock at start, and that block all writes
      against the table (naturally, we don't want any writes to be lost
      when switching the old and the new table). TL_WRITE_ALLOW_READ
      lock, in turn, would block the started transaction on thr_lock.c
      lock, should they do more updates. This, again, lead to the need
      to abort such transactions.
      
      The second bug occurred simply because we didn't have any
      mechanism to detect deadlocks between the table-level locks
      in thr_lock.c and row-level locks in InnoDB, other than
      innodb_lock_wait_timeout.
      
      This patch solves both these problems by moving lock conflicts
      which are causing these deadlocks into the metadata locking
      subsystem, thus making it possible to avoid or detect such
      deadlocks inside MDL.
      
      To do this we introduce new type-of-operation-aware metadata
      locks, which allow MDL subsystem to know not only the fact that
      transaction has used or is going to use some object but also what
      kind of operation it has carried out or going to carry out on the
      object.
      
      This, along with the addition of a special kind of upgradable
      metadata lock, allows ALTER TABLE to wait until all
      transactions which has updated the table to go away.
      This solves the second issue.
      Another special type of upgradable metadata lock is acquired
      by LOCK TABLE WRITE. This second lock type allows to solve the
      first issue, since abortion of table-level locks in event of
      DDL under LOCK TABLES becomes also unnecessary.
      
      Below follows the list of incompatible changes introduced by
      this patch:
      
      - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those
        statements that acquire TL_WRITE_ALLOW_READ lock)
        wait for all transactions which has *updated* the table to
        complete.
      
      - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE
        (i.e. all statements which acquire TL_WRITE table-level lock) wait
        for all transaction which *updated or read* from the table
        to complete.
        As a consequence, innodb_table_locks=0 option no longer applies
        to LOCK TABLES ... WRITE.
      
      - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort
        statements or transactions which use tables being dropped or
        renamed, and instead wait for these transactions to complete.
      
      - Since LOCK TABLES WRITE now takes a special metadata lock,
        not compatible with with reads or writes against the subject table
        and transaction-wide, thr_lock.c deadlock avoidance algorithm
        that used to ensure absence of deadlocks between LOCK TABLES
        WRITE and other statements is no longer sufficient, even for
        MyISAM. The wait-for graph based deadlock detector of MDL
        subsystem may sometimes be necessary and is involved. This may
        lead to ER_LOCK_DEADLOCK error produced for multi-statement
        transactions even if these only use MyISAM:
      
        session 1:         session 2:
        begin;
      
        update t1 ...      lock table t2 write, t1 write;
                           -- gets a lock on t2, blocks on t1
      
        update t2 ...
        (ER_LOCK_DEADLOCK)
      
      - Finally,  support of LOW_PRIORITY option for LOCK TABLES ... WRITE
        was abandoned.
        LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same
        priority as the usual LOCK TABLE ... WRITE.
        SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE  in
        the wait queue.
      
      - We do not take upgradable metadata locks on implicitly
        locked tables. So if one has, say, a view v1 that uses
        table t1, and issues:
        LOCK TABLE v1 WRITE;
        FLUSH TABLE t1; -- (or just 'FLUSH TABLES'),
        an error is produced.
        In order to be able to perform DDL on a table under LOCK TABLES,
        the table must be locked explicitly in the LOCK TABLES list.
      
      mysql-test/include/handler.inc:
        Adjusted test case to trigger an execution path on which bug 41110
        "crash with handler command when used concurrently with alter
        table" and bug 41112 "crash in mysql_ha_close_table/get_lock_data
        with alter table" were originally discovered. Left old test case
        which no longer triggers this execution path for the sake of
        coverage.
        Added test coverage for HANDLER SQL statements and type-aware
        metadata locks.
        Added a test for the global shared lock and HANDLER SQL.
        Updated tests to take into account that the old simple deadlock
        detection heuristics was replaced with a graph-based deadlock
        detector.
      mysql-test/r/debug_sync.result:
        Updated results (see debug_sync.test).
      mysql-test/r/handler_innodb.result:
        Updated results (see handler.inc test).
      mysql-test/r/handler_myisam.result:
        Updated results (see handler.inc test).
      mysql-test/r/innodb-lock.result:
        Updated results (see innodb-lock.test).
      mysql-test/r/innodb_mysql_lock.result:
        Updated results (see innodb_mysql_lock.test).
      mysql-test/r/lock.result:
        Updated results (see lock.test).
      mysql-test/r/lock_multi.result:
        Updated results (see lock_multi.test).
      mysql-test/r/lock_sync.result:
        Updated results (see lock_sync.test).
      mysql-test/r/mdl_sync.result:
        Updated results (see mdl_sync.test).
      mysql-test/r/sp-threads.result:
        SHOW PROCESSLIST output has changed due to the fact that waiting
        for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/r/truncate_coverage.result:
        Updated results (see truncate_coverage.test).
      mysql-test/suite/funcs_1/datadict/processlist_val.inc:
        SELECT FROM I_S.PROCESSLIST output has changed due to fact that
        waiting for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
        SELECT FROM I_S.PROCESSLIST output has changed due to fact that
        waiting for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/suite/rpl/t/rpl_sp.test:
        Updated to a new SHOW PROCESSLIST state name.
      mysql-test/t/debug_sync.test:
        Use LOCK TABLES READ instead of LOCK TABLES WRITE as the latter
        no longer allows to trigger execution path involving waiting on
        thr_lock.c lock and therefore reaching debug sync-point covered
        by this test.
      mysql-test/t/innodb-lock.test:
        Adjusted test case to the fact that innodb_table_locks=0 option is
        no longer supported, since LOCK TABLES WRITE handles all its
        conflicts within MDL subsystem.
      mysql-test/t/innodb_mysql_lock.test:
        Added test for bug #37346 "innodb does not detect deadlock between
        update and alter table".
      mysql-test/t/lock.test:
        Added test coverage which checks the fact that we no longer support
        DDL under LOCK TABLES on tables which were locked implicitly.
        Adjusted existing test cases accordingly.
      mysql-test/t/lock_multi.test:
        Added test for bug #46272 "MySQL 5.4.4, new MDL: unnecessary
        deadlock".  Adjusted other test cases to take into account the
        fact that waiting for LOCK TABLES ... WRITE now happens within MDL
        subsystem.
      mysql-test/t/lock_sync.test:
        Since LOCK TABLES ... WRITE now takes SNRW metadata lock for
        tables locked explicitly we have to implicitly lock InnoDB tables
        (through view) to trigger the table-level lock conflict between
        TL_WRITE and TL_WRITE_ALLOW_WRITE.
      mysql-test/t/mdl_sync.test:
        Added basic test coverage for type-of-operation-aware metadata
        locks. Also covered with tests some use cases involving HANDLER
        statements in which a deadlock could arise.
        Adjusted existing tests to take type-of-operation-aware MDL into
        account.
      mysql-test/t/multi_update.test:
        Update to a new SHOW PROCESSLIST state name.
      mysql-test/t/truncate_coverage.test:
        Adjusted test case after making LOCK TABLES WRITE to wait until
        transactions that use the table to be locked are completed.
        Updated to the changed name of DEBUG_SYNC point.
      sql/handler.cc:
        Global read lock functionality has been
        moved into a class.
      sql/lock.cc:
        Global read lock functionality has been
        moved into a class.
        Updated code to use the new MDL API.
      sql/mdl.cc:
        Introduced new type-of-operation aware metadata locks.
        To do this:
        - Changed MDL_lock to use one list for waiting requests and one
          list for granted requests. For each list, added a bitmap
          that holds information what lock types a list contains.
          Added a helper class MDL_lock::List to manipulate with granted
          and waited lists while keeping the bitmaps in sync
          with list contents.
        - Changed lock-compatibility functions to use bitmaps that
          define compatibility.
        - Introduced a graph based deadlock detector inspired by
          waiting_threads.c from Maria implementation.
        - Now that we have a deadlock detector, and no longer have
          a global lock to protect individual lock objects, but rather
          use an rw lock per object, removed redundant code for upgrade,
          and the global read lock. Changed the MDL API to
          no longer require the caller to acquire the global
          intention exclusive lock by means of a separate method.
          Removed a few more methods that became redundant.
        - Removed deadlock detection heuristic, it has been made
          obsolete by the deadlock detector.
        - With operation-type-aware metadata locks, MDL subsystem has
          become aware of potential conflicts between DDL and open
          transactions. This made it possible to remove calls to
          mysql_abort_transactions_with_shared_lock() from acquisition
          paths for exclusive lock and lock upgrade. Now we can simply
          wait for these transactions to complete without fear of
          deadlock. Function mysql_lock_abort() has also become
          unnecessary for all conflicting cases except when a DDL
          conflicts with a connection that has an open HANDLER.
      sql/mdl.h:
        Introduced new type-of-operation aware metadata locks.
        Introduced a graph based deadlock detector and supporting
        methods.
        Added comments.
        God rid of redundant API calls.
        Renamed m_lt_or_ha_sentinel to m_trans_sentinel,
        since now it guards the global read lock as well as
        LOCK TABLES and HANDLER locks.
      sql/mysql_priv.h:
        Moved the global read lock functionality into a
        class.
        Added MYSQL_OPEN_FORCE_SHARED_MDL flag which forces
        open_tables() to take MDL_SHARED on tables instead of
        metadata locks specified in the parser. We use this to
        allow PREPARE run concurrently in presence of
        LOCK TABLES ... WRITE.
        Added signature for find_table_for_mdl_ugprade().
      sql/set_var.cc:
        Global read lock functionality has been
        moved into a class.
      sql/sp_head.cc:
        When creating TABLE_LIST elements for prelocking or
        system tables set the type of request for metadata
        lock according to the operation that will be performed
        on the table.
      sql/sql_base.cc:
        - Updated code to use the new MDL API.
        - In order to avoid locks starvation we take upgradable
          locks all at once. As result implicitly locked tables no
          longer get an upgradable lock. Consequently DDL and FLUSH
          TABLES for such tables is prohibited.
          find_write_locked_table() was replaced by
          find_table_for_mdl_upgrade() function.
          open_table() was adjusted to return TABLE instance with
          upgradable ticket when necessary.
        - We no longer wait for all locks on OT_WAIT back off
          action -- only on the lock that caused the wait
          conflict. Moreover, now we distinguish cases when we
          have to wait due to conflict in MDL and old version
          of table in TDC.
        - Upate mysql_notify_threads_having_share_locks()
          to only abort thr_lock.c waits of threads that
          have open HANDLERs, since lock conflicts with only
          these threads now can lead to deadlocks not detectable
          by the MDL deadlock detector.
        - Remove mysql_abort_transactions_with_shared_locks()
          which is no longer needed.
      sql/sql_class.cc:
        Global read lock functionality has been moved into a class.
        Re-arranged code in THD::cleanup() to simplify assert.
      sql/sql_class.h:
        Introduced class to incapsulate global read lock
        functionality.
        Now sentinel in MDL subsystem guards the global read lock
        as well as LOCK TABLES and HANDLER locks. Adjusted code
        accordingly.
      sql/sql_db.cc:
        Global read lock functionality has been moved into a class.
      sql/sql_delete.cc:
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result
        TRUNCATE TABLE is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_handler.cc:
        Inform MDL_context about presence of open HANDLERs.
        Since HANLDERs break MDL protocol by acquiring table-level
        lock while holding only S metadata lock on a table MDL
        subsystem should take special care about such contexts (Now
        this is the only case when mysql_lock_abort() is used).
      sql/sql_parse.cc:
        Global read lock functionality has been moved into a class.
        Do not take upgradable metadata locks when opening tables
        for CREATE TABLE SELECT as it is not necessary and limits
        concurrency.
        When initializing TABLE_LIST objects before adding them
        to the table list set the type of request for metadata lock
        according to the operation that will be performed on the
        table.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result FLUSH
        TABLES is no longer allowed for such tables.
      sql/sql_prepare.cc:
        Use MYSQL_OPEN_FORCE_SHARED_MDL flag when opening
        tables during PREPARE. This allows PREPARE to run
        concurrently in presence of LOCK TABLES ... WRITE.
      sql/sql_rename.cc:
        Global read lock functionality has been moved into a class.
      sql/sql_show.cc:
        Updated code to use the new MDL API.
      sql/sql_table.cc:
        Global read lock functionality has been moved into a class.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result DROP
        TABLE is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_trigger.cc:
        Global read lock functionality has been moved into a class.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result
        CREATE/DROP TRIGGER is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_view.cc:
        Global read lock functionality has been moved into a class.
        Fixed results of wrong merge that led to misuse of GLR API.
        CREATE VIEW statement is not a commit statement.
      sql/table.cc:
        When resetting TABLE_LIST objects for PS or SP re-execution
        set the type of request for metadata lock according to the
        operation that will be performed on the table. Do the same
        in auxiliary function initializing metadata lock requests
        in a table list.
      sql/table.h:
        When initializing TABLE_LIST objects set the type of request
        for metadata lock according to the operation that will be
        performed on the table.
      sql/transaction.cc:
        Global read lock functionality has been moved into a class.
      ff6fd58d
  6. 21 Jan, 2010 2 commits
    • Dmitry Lenev's avatar
      Patch that changes metadata locking subsystem to use mutex per lock and · b3f63454
      Dmitry Lenev authored
      condition variable per context instead of one mutex and one conditional
      variable for the whole subsystem.
      
      This should increase concurrency in this subsystem.
      
      It also opens the way for further changes which are necessary to solve
      such bugs as bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock"
      and bug #37346 "innodb does not detect deadlock between update and alter
      table".
      
      Two other notable changes done by this patch:
      
      - MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object metadata lock is acquired. Now this has
        to be done by explicit calls outside of MDL subsystem.
      - Instead of using separate MDL_context for opening system tables/tables
        for purposes of I_S we now create MDL savepoint in the main context
        before opening tables and rollback to this savepoint after closing
        them. This means that it is now possible to get ER_LOCK_DEADLOCK error
        even not inside a transaction. This might happen in unlikely case when
        one runs DDL on one of system tables while also running DDL on some
        other tables. Cases when this ER_LOCK_DEADLOCK error is not justified
        will be addressed by advanced deadlock detector for MDL subsystem which
        we plan to implement.
      
      mysql-test/include/handler.inc:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/handler_innodb.result:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/handler_myisam.result:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/mdl_sync.result:
        Adjusted mdl_sync.test to the fact that exclusive metadata locks on
        tables are now acquired according to alphabetical order of fully
        qualified table names instead of order in which tables are mentioned
        in statement.
      mysql-test/t/mdl_sync.test:
        Adjusted mdl_sync.test to the fact that exclusive metadata locks on
        tables are now acquired according to alphabetical order of fully
        qualified table names instead of order in which tables are mentioned
        in statement.
      sql/events.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/ha_ndbcluster.cc:
        Since manipulations with open table state no longer install proxy
        MDL_context it does not make sense to perform them in order to
        satisfy assert in mysql_rm_tables_part2(). Removed them per agreement
        with Cluster team. This has not broken test suite since scenario in
        which deadlock can occur and assertion fails is not covered by tests.
      sql/lock.cc:
        MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object exclusive metadata lock is acquired.
        Now this has to be done by explicit calls outside of MDL subsystem.
      sql/log.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/mdl.cc:
        Changed metadata locking subsystem to use mutex per lock and condition
        variable per context instead of one mutex and one conditional variable
        for the whole subsystem.
        Changed approach to handling of global metadata locks. Instead of
        implicitly acquiring intention exclusive locks when user requests
        per-object upgradeable or exclusive locks now we require them to be
        acquired explicitly in the same way as ordinary metadata locks.
        In fact global lock are now ordinary metadata locks in new GLOBAL
        namespace.
        
        To implement these changes:
        - Removed LOCK_mdl mutex and COND_mdl condition variable.
        - Introduced MDL_lock::m_mutex mutexes which protect individual lock
          objects.
        - Replaced mdl_locks hash with MDL_map class, which has hash for
          MDL_lock objects as a member and separate mutex which protects this
          hash. Methods of this class allow to find(), find_or_create() or
          remove() MDL_lock objects in concurrency-friendly fashion (i.e.
          for most common operation, find_or_create(), we don't acquire
          MDL_lock::m_mutex while holding MDL_map::m_mutex. Thanks to MikaelR
          for this idea and benchmarks!). Added three auxiliary members to
          MDL_lock class (m_is_destroyed, m_ref_usage, m_ref_release) to
          support this concurrency-friendly behavior.
        - Introduced MDL_context::m_ctx_wakeup_cond condition variable to be
          used for waiting until this context's pending request can be
          satisfied or its thread has to perform actions to resolve potential
          deadlock. Context which want to wait add ticket corresponding to the
          request to an appropriate queue of waiters in MDL_lock object so
          they can be noticed when other contexts change state of lock and be
          awaken by them by signalling on MDL_context::m_ctx_wakeup_cond.
          As consequence MDL_ticket objects has to be used for any waiting
          in metadata locking subsystem including one which happens in
          MDL_context::wait_for_locks() method.
          Another consequence is that MDL_context is no longer copyable and
          can't be saved/restored when working with system tables.
        - Made MDL_lock an abstract class, which delegates specifying exact
          compatibility matrix to its descendants. Added MDL_global_lock child
          class for global lock (The old is_lock_type_compatible() method
          became can_grant_lock() method of this class). Added MDL_object_lock
          class to represent per-object lock (The old MDL_lock::can_grant_lock()
          became its method). Choice between two classes happens based on MDL
          namespace in MDL_lock::create() method.
        - Got rid of MDL_lock::type member as its meaning became ambigous for
          global locks.
        - To simplify waking up of contexts waiting for lock split waiting queue
          in MDL_lock class in two queues. One for pending requests for exclusive
          (including intention exclusive) locks and another for requests for
          shared locks.
        - Added virtual wake_up_waiters() method to MDL_lock, MDL_global_lock and
          MDL_object_lock classes which allows to wake up waiting contexts after
          state of lock changes. Replaced old duplicated code with calls to this
          method.
        - Adjusted MDL_context::try_acquire_shared_lock()/exclusive_lock()/
          global_shared_lock(), MDL_ticket::upgrade_shared_lock_to_exclusive_lock()
          and MDL_context::release_ticket() methods to use MDL_map and
          MDL_lock::m_mutex instead of single LOCK_mdl mutex and wake up
          waiters according to the approach described above. The latter method
          also was renamed to MDL_context::release_lock().
        - Changed MDL_context::try_acquire_shared_lock()/exclusive_lock() and
          release_lock() not to handle global locks. They are now supposed to
          be taken explicitly like ordinary metadata locks.
        - Added helper MDL_context::try_acquire_global_intention_exclusive_lock()
          and acquire_global_intention_exclusive_lock() methods.
        - Moved common code from MDL_context::acquire_global_shared_lock() and
          acquire_global_intention_exclusive_lock() to new method -
          MDL_context::acquire_lock_impl().
        - Moved common code from MDL_context::try_acquire_shared_lock(),
          try_acquire_global_intention_exclusive_lock()/exclusive_lock()
          to MDL_context::try_acquire_lock_impl().
        - Since acquiring of several exclusive locks can no longer happen under
          single LOCK_mdl mutex the approach to it had to be changed. Now we do
          it in one by one fashion. This is done in alphabetical order to avoid
          deadlocks. Changed MDL_context::acquire_exclusive_locks() accordingly
          (as part of this change moved code responsible for acquiring single
          exclusive lock to new MDL_context::acquire_exclusive_lock_impl()
          method).
        - Since we no longer have single LOCK_mdl mutex which protects all
          MDL_context::m_is_waiting_in_mdl members using these members to
          determine if we have really awaken context holding conflicting
          shared lock became inconvinient. Got rid of this member and changed
          notify_shared_lock() helper function and process of acquiring
          of/upgrading to exclusive lock not to rely on such information.
          Now in MDL_context::acquire_exclusive_lock_impl() and
          MDL_ticket::upgrade_shared_lock_to_exclusive_lock() we simply
          re-try to wake up threads holding conflicting shared locks after
          small time out.
        - Adjusted MDL_context::can_wait_lead_to_deadlock() and
          MDL_ticket::has_pending_conflicting_lock() to use per-lock
          mutexes instead of LOCK_mdl. To do this introduced
          MDL_lock::has_pending_exclusive_lock() method.
      sql/mdl.h:
        Changed metadata locking subsystem to use mutex per lock and condition
        variable per context instead of one mutex and one conditional variable
        for the whole subsystem. In order to implement this change:
        
        - Added MDL_key::cmp() method to be able to sort MDL_key objects
          alphabetically. Changed length fields in MDL_key class to uint16
          as 16-bit is enough for length of any key.
        - Changed MDL_ticket::get_ctx() to return pointer to non-const
          object in order to be able to use MDL_context::awake() method
          for such contexts.
        - Got rid of unlocked versions of can_wait_lead_to_deadlock()/
          has_pending_conflicting_lock() methods in MDL_context and
          MDL_ticket. We no longer has single mutex which protects all
          locks. Thus one always has to use versions of these methods
          which acquire per-lock mutexes.
        - MDL_request_list type of list now counts its elements.
        - Added MDL_context::m_ctx_wakeup_cond condition variable to be used
          for waiting until this context's pending request can be satisfied
          or its thread has to perform actions to resolve potential deadlock.
          Added awake() method to wake up context from such wait.
          Addition of condition variable made MDL_context uncopyable.
          As result we no longer can save/restore MDL_context when working
          with system tables. Instead we create MDL savepoint before opening
          those tables and rollback to it once they are closed.
        - MDL_context::release_ticket() became release_lock() method.
        - Added auxiliary MDL_context::acquire_exclusive_lock_impl() method
          which does all necessary work to acquire exclusive lock on one object
          but should not be used directly as it does not enforce any asserts
          ensuring that no deadlocks are possible.
        - Since we no longer need to know if thread trying to acquire exclusive
          lock managed to wake up any threads having conflicting shared locks
          (as, anyway, we will try to wake up such threads again shortly)
        - MDL_context::m_is_waiting_in_mdl member became unnecessary and
          notify_shared_lock() no longer needs to be friend of MDL_context.
        
        Changed approach to handling of global metadata locks. Instead of
        implicitly acquiring intention exclusive locks when user requests
        per-object upgradeable or exclusive locks now we require them to be
        acquired explicitly in the same way as ordinary metadata locks.
        
        - Added new GLOBAL namespace for such locks.
        - Added new type of lock to be requested MDL_INTENTION_EXCLISIVE.
        - Added MDL_context::try_acquire_global_intention_exclusive_lock()
          and acquire_global_intention_exclusive_lock() methods.
        - Moved common code from MDL_context::acquire_global_shared_lock()
          and acquire_global_intention_exclusive_lock() to new method -
          MDL_context::acquire_lock_impl().
        - Moved common code from MDL_context::try_acquire_shared_lock(),
          try_acquire_global_intention_exclusive_lock()/exclusive_lock()
          to MDL_context::try_acquire_lock_impl().
        - Added helper MDL_context::is_global_lock_owner() method to be
          able easily to find what kind of global lock this context holds.
        - MDL_context::m_has_global_shared_lock became unnecessary as
          global read lock is now represented by ordinary ticket.
        - Removed assert in MDL_context::set_lt_or_ha_sentinel() which became
          false for cases when we execute LOCK TABLES under global read lock
          mode.
      sql/mysql_priv.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result calls
        opening/closing system tables were changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sp.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sp.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sql_base.cc:
        close_thread_tables():
          Since we no longer use separate MDL_context for opening system
          tables we need to avoid releasing all transaction locks when
          closing system table. Releasing metadata lock on system table
          is now responsibility of THD::restore_backup_open_tables_state().
        open_table_get_mdl_lock(),
        Open_table_context::recover_from_failed_open():
          MDL subsystem no longer implicitly acquires global intention exclusive
          metadata lock when per-object upgradable or exclusive metadata lock is
          acquired. So this have to be done explicitly from these calls.
          Changed Open_table_context class to store MDL_request object for
          global intention exclusive lock acquired when opening tables.
        open_table():
          Do not release metadata lock if we have failed to open table as
          this lock might have been acquired by one of previous statements
          in transaction, and therefore should not be released.
        open_system_tables_for_read()/close_system_tables()/
        open_performance_schema_table():
          Instead of using separate MDL_context for opening system tables we now
          create MDL savepoint in the main context before opening such tables
          and rollback to this savepoint after closing them. To support this
          change methods of THD responsible for saving/restoring open table
          state were changed to use Open_tables_backup class which in addition
          to Open_table_state has a member for this savepoint. As result code
          opening/closing system tables was changed to use Open_tables_backup
          instead of Open_table_state class as well.
        close_performance_schema_table():
          Got rid of duplicated code.
      sql/sql_class.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. Also releasing
        metadata lock on system table is now responsibility of
        THD::restore_backup_open_tables_state().
        Adjusted assert in THD::cleanup() to take into account fact that now
        we also use MDL sentinel for global read lock.
      sql/sql_class.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. As result:
        - 'mdl_context' member was moved out of Open_tables_state to THD class.
          enter_locked_tables_mode()/leave_locked_tables_mode() had to follow.
        - Methods of THD responsible for saving/restoring open table state were
          changed to use Open_tables_backup class which in addition to
          Open_table_state has a member for this savepoint.
        Changed Open_table_context class to store MDL_request object for
        global intention exclusive lock acquired when opening tables.
      sql/sql_delete.cc:
        MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object exclusive metadata lock is acquired.
        Now this has to be done by explicit calls outside of MDL subsystem.
      sql/sql_help.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sql_parse.cc:
        Adjusted assert reload_acl_and_cache() to the fact that global read
        lock now takes full-blown metadata lock.
      sql/sql_plist.h:
        Added support for element counting to I_P_List list template.
        One can use policy classes to specify if such counting is needed
        or not needed for particular list.
      sql/sql_show.cc:
        Instead of using separate MDL_context for opening tables for I_S
        purposes we now create MDL savepoint in the main context before
        opening tables and rollback to this savepoint after closing them.
        To support this and similar change for system tables methods of
        THD responsible for saving/restoring open table state were changed
        to use Open_tables_backup class which in addition to Open_table_state
        has a member for this savepoint. As result code opening/closing tables
        for I_S purposes was changed to use Open_tables_backup instead of
        Open_table_state class as well.
      sql/sql_table.cc:
        mysql_rm_tables_part2():
          Since now global intention exclusive metadata lock is ordinary
          metadata lock we no longer can rely that by releasing MDL locks
          on all tables we will release all locks acquired by this routine.
          So in non-LOCK-TABLES mode we have to release all locks acquired
          explicitly.
        prepare_for_repair(), mysql_alter_table():
          MDL subsystem no longer implicitly acquires global intention
          exclusive metadata lock when per-object exclusive metadata lock
          is acquired. Now this has to be done by explicit calls outside of
          MDL subsystem.
      sql/tztime.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
        Also changed code not to use special mechanism for open system tables
        when it is not really necessary.
      b3f63454
    • Alexander Nozdrin's avatar
      Auto-merge from mysql-next-mr. · d1be11fa
      Alexander Nozdrin authored
      d1be11fa
  7. 20 Jan, 2010 4 commits
    • Luis Soares's avatar
      Removes duplicate entry for ER_SLAVE_IGNORE_SERVER_IDS · ecc67501
      Luis Soares authored
      in errmsg.txt. Kept the one in that matched the location
      in errmsg-utf8.txt, ie after ER_UNKNOWN_LOCALE.
      ecc67501
    • Alexander Nozdrin's avatar
      Add client/dtoa.c to .bzrignore. · e8287e20
      Alexander Nozdrin authored
      e8287e20
    • Alexander Nozdrin's avatar
      99974a7c
    • Jon Olav Hauglid's avatar
      Bug #50412 Assertion `! is_set()' failed in · e31f20e5
      Jon Olav Hauglid authored
                 Diagnostics_area::set_ok_status at PREPARE
      
      The problem occured during processing of stored routines. 
      Routines are loaded from mysql.proc, parsed and put into the sp cache by
      sp_cache_routine().  The assert occured because the return value from
      sp_cache_routine() was not checked for top level CALLs. This meant that any
      errors during sp_cache_routine() went unoticed and triggered the assert when
      my_ok() was later called.
      
      This is a regression introduced by the patch for Bug#30977, only visible in
      source trees with MDL and using debug builds of the server.
      
      This patch fixes the problem by checking the return value from sp_cache_routine() 
      for top level CALLs and propagating any errors similar to what is done for other 
      calls to sp_cache_routine().
      
      No test case added.
      e31f20e5
  8. 19 Jan, 2010 1 commit
  9. 18 Jan, 2010 2 commits