An error occurred fetching the project authors.
  1. 08 May, 2012 1 commit
  2. 26 Mar, 2012 1 commit
    • Michael Widenius's avatar
      Fixed lp:963603 "Assertion `lock_type != TL_UNLOCK && (lock_type == TL_IGNORE... · 9ef537ac
      Michael Widenius authored
      Fixed lp:963603 "Assertion `lock_type != TL_UNLOCK && (lock_type == TL_IGNORE || file->lock.type == TL_UNLOCK)' failed in ha_maria::store_lock with DML, triggers, views"
      
      
      mysys/thr_lock.c:
        Ensure that all locks are marked as TL_UNLOCK in case of error (Safety fix)
      sql/lock.cc:
        Ensure that all locks are marked as TL_UNLOCK in case of error
        (This is the real fix for lp:963603)
      9ef537ac
  3. 20 Mar, 2012 1 commit
    • Michael Widenius's avatar
      Cleanups: · a579adea
      Michael Widenius authored
      - Don't use SAFEMALLOC on valgrind builds (slows things down)
      - Added back lost option from 5.3: debug-mutex-deadlock-detector
      - Flush pages before taking lock mutex (speeds up closing of Aria tables).
      
      BUILD/SETUP.sh:
        - Don't use SAFEMALLOC on valgrind builds (slows things down)
      sql/lock.cc:
        Make default argument explicit (improves readability)
      sql/mysqld.cc:
        Removed compiler warnings
        Sorted debug options alphabetically
        Added back lost option from 5.3: debug-mutex-deadlock-detector
      storage/maria/ma_close.c:
        Flush pages before taking lock mutex (speeds up closing of Aria tables).
      storage/maria/ma_open.c:
        More DBUG_PRINT
      storage/maria/maria_def.h:
        Better DBUG_PRINT
      storage/maria/trnman.c:
        Better DBUG_PRINT
      a579adea
  4. 23 Feb, 2012 1 commit
    • Michael Widenius's avatar
      Fixed lp:933719, "Assertion open_tables == 0 ... " in THD::restore_backup_open_tables_state. · 2b625ac3
      Michael Widenius authored
      This also fixes a (not likely) crashing bug when forcing a thread that was doing a table lock to re-open it's files, for example by creating a trigger.
      
      
      mysys/thr_lock.c:
        Added more checking to find wrong locks.
        Removed one, not needed, parameter to thr_lock
      sql/lock.cc:
        Fixed mysql_lock_tables() to retry with new sql_lock if lock fails. This was needed as table may be closed and reopened between retry's and then the old sql_lock will point to stale data.
      sql/mysql_priv.h:
        Updated prototype
      sql/sql_base.cc:
        Ensure that all tables are closed if opening of system table failes; This fixes the assert in THD::restore_backup_open_tables_state
      sql/sql_handler.cc:
        Updated variable type
      2b625ac3
  5. 06 Dec, 2011 1 commit
  6. 05 Dec, 2011 1 commit
  7. 30 Jun, 2011 1 commit
  8. 28 Apr, 2011 1 commit
    • Michael Widenius's avatar
      Added option "AND DISABLE CHECKPOINT" to "FLUSH TABLES WITH READ LOCK" · 6da8ac5f
      Michael Widenius authored
      This makes it possible to do safe multi volume snapshots as long as one snapshots the volume with the transaction logs last.
      
      
      include/mysql_com.h:
        Added REFRESH_CHECKPOINT
      mysql-test/r/flush.result:
        Added test of new FLUSH TABLES syntax + calls to checkpoint_status handler calls
      mysql-test/t/flush.test:
        Added test of new FLUSH TABLES syntax + calls to checkpoint_status handler calls
      sql/handler.cc:
        Added code to call checkpoint_state for all handlertons that supports it
      sql/handler.h:
        Added new checkpoint_state() handlerton call to temporarly disable checkpoints.
      sql/lex.h:
        Added CHECKPOINT keyword
      sql/sql_yacc.yy:
        Added support for FLUSH TABLES WITH READ LOCK AND DISABLE CHECKPOINT
      storage/maria/ha_maria.cc:
        Added handlerton call to disable checkpoints.
      storage/maria/ma_checkpoint.c:
        Don't do checkpoint if checkpoints are disabled.
      storage/maria/ma_static.c:
        Added maria_checkpoint_disabled
      storage/maria/maria_def.h:
        Added maria_checkpoint_disabled
      storage/xtradb/handler/ha_innodb.cc:
        Added handlerton call to disable checkpoints.
      storage/xtradb/include/log0log.h:
        Added option to log_checkpoint() to allow one to ignore not critical checkpoints during the time checkpoints are disabled.
      storage/xtradb/log/log0log.c:
        Added code to allow one to disable checkpoints during a FLUSH TABLES ... DISABLE CHECKPOINT
        This was done by adding a new argument to log_checkpoint() which tells us when the checkpoint is called by srv_master_thread (which are safe to ignore)
      storage/xtradb/srv/srv0srv.c:
        Tell log_checkpoint() that checkpoints from srv_master_thread() are safe to ignore (will just delay recovery time a bit).
      6da8ac5f
  9. 25 Apr, 2011 1 commit
  10. 04 Mar, 2011 1 commit
  11. 28 Feb, 2011 1 commit
    • Michael Widenius's avatar
      Change TABLE->alias to String for less memory reallocation · ff3da0f9
      Michael Widenius authored
      Changed some String.ptr() -> String.c_ptr() for String that are not guaranteed to end with \0
      Removed some c_ptr() usage from parameters to functions that takes ptr & length
      Use preallocate buffers to avoid calling malloc() for most operations. 
      
      
      sql/event_db_repository.cc:
        alias is now a String
      sql/event_scheduler.cc:
        c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
      sql/events.cc:
        c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
        c_ptr -> ptr() as function takes ptr & length
      sql/field.cc:
        alias is now a String
      sql/field.h:
        alias is now a String
      sql/ha_partition.cc:
        alias is now a String
      sql/handler.cc:
        alias is now a String
        ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
      sql/item.cc:
        Store error parameter in separarte buffer to ensure correct error message
      sql/item_func.cc:
        ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
      sql/item_sum.h:
        Use my_strtod() instead of my_atof() to not have to make string \0 terminated
      sql/lock.cc:
        alias is now a String
      sql/log.cc:
        c_ptr() -> ptr() as function takes ptr & length
      sql/log_event.cc:
        c_ptr_quick() -> ptr() as we only want to get the pointer to String buffer
      sql/opt_range.cc:
        ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
      sql/opt_table_elimination.cc:
        alias is now a String
      sql/set_var.cc:
        ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
        c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
        c_ptr() -> ptr() as function takes ptr & length
        Simplify some code.
      sql/sp.cc:
        c_ptr() -> ptr() as function takes ptr & length
      sql/sp_rcontext.cc:
        alias is now a String
      sql/sql_base.cc:
        alias is now a String.
        Here we win a realloc() for most alias usage.
      sql/sql_class.cc:
        Use size descriptor for printf() to avoid accessing bytes outside of buffer
      sql/sql_insert.cc:
        Change allocation of TABLE as it's now contains a String
        _ptr() -> ptr() as function takes ptr & length
      sql/sql_load.cc:
        Use preallocate buffers to avoid calling malloc() for most operations.
      sql/sql_parse.cc:
        Use c_ptr_safe() to ensure string is \0 terminated.
      sql/sql_plugin.cc:
        c_ptr_quick() -> ptr() as function takes ptr & length
      sql/sql_select.cc:
        alias is now a String
      sql/sql_show.cc:
        alias is now a String
      sql/sql_string.h:
        Added move() function to change who owns the string (owner does the free)
      sql/sql_table.cc:
        alias is now a String
        c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
      sql/sql_test.cc:
        c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
        alias is now a String
      sql/sql_trigger.cc:
        c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
        Use field->init() to setup pointers to alias.
      sql/sql_update.cc:
        alias is now a String
      sql/sql_view.cc:
        ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
      sql/sql_yacc.yy:
        r() -> c_ptr() as string is not guaranteed to be \0 terminated
      sql/table.cc:
        alias is now a String
      sql/table.h:
        alias is now a String
      storage/federatedx/ha_federatedx.cc:
        Remove extra 1 byte alloc that is automaticly done by strmake()
        Ensure that error message ends with \0
      storage/maria/ha_maria.cc:
        alias is now a String
      storage/myisam/ha_myisam.cc:
        alias is now a String
      ff3da0f9
  12. 26 Jan, 2011 1 commit
    • Michael Widenius's avatar
      Fixed bugs found by buildbot: · 69fe020f
      Michael Widenius authored
      - Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables.
      - Fixed compiler warnings
      - Added a name for each thr_lock to get better error messages (This is needed to find out why 'archive.test' sometimes fails)
      
      
      BUILD/SETUP.sh:
        Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables.
      BUILD/build_mccge.sh:
        Use -Wno-uninitialized if -DFORCE_INIT_OF_VARS is not used, to avoid warnings about not initialized variables.
      client/mysqltest.cc:
        Fixed bug in remove_files_wildcards (the orignal code never removed anything)
      extra/libevent/devpoll.c:
        Fixed compiler warning
      include/thr_lock.h:
        Added a name for each thr_lock to get better error messages.
      mysql-test/suite/maria/t/maria3.test:
        Speed up test.
      mysys/thr_lock.c:
        Added a name for each thr_lock to get better error messages.
        Added a second 'check_locks' to find if something goes wrong in 'wake_up_waiters'.
      sql/lock.cc:
        Added a name for each thr_lock to get better error messages.
      storage/xtradb/fil/fil0fil.c:
        Fixed compiler warning
      69fe020f
  13. 03 Jan, 2011 1 commit
    • Michael Widenius's avatar
      Implementation of MWL#172: Add support for prepared statements to HANDLER READ · e63b5546
      Michael Widenius authored
      It includes speed optimizations for HANDLER READ by caching as much as possible in HANDLER OPEN
      Other things:
      - Added mysqld option --disable-thr-alarm to be able to benchmark things without thr_alarm
      - Changed 'Locked' state to 'System lock' and 'Table lock' (these where used in the code but never shown to end user)
      - Better error message if mysql_install_db.sh fails
      - Moved handler function prototypes to sql_handler.h
      - Remove not anymore used 'thd->locked' member
      
      
      include/thr_alarm.h:
        Added my_disable_thr_alarm
      include/thr_lock.h:
        Add new member to THR_LOCK_DATA to remember original lock type state. This is needed as thr_unlock() resets type to TL_UNLOCK.
      mysql-test/include/check_no_concurrent_insert.inc:
        Locked -> Table lock
      mysql-test/include/handler.inc:
        Locked -> Table lock
      mysql-test/r/handler_innodb.result:
        Updated results for new tests
      mysql-test/r/handler_myisam.result:
        Updated results for new tests
      mysql-test/r/sp-threads.result:
        Locked -> Table lock
      mysql-test/suite/binlog/t/binlog_stm_row.test:
        Locked -> Table lock
      mysql-test/suite/funcs_1/datadict/processlist_val.inc:
        Locked -> Table lock
      mysql-test/suite/pbxt/t/lock_multi.test:
        Locked -> Table lock
      mysql-test/suite/sys_vars/r/concurrent_insert_func.result:
        Locked -> Table lock
      mysql-test/suite/sys_vars/t/concurrent_insert_func.test:
        Locked -> Table lock
      mysql-test/suite/sys_vars/t/delayed_insert_limit_func.test:
        Locked -> Table lock
      mysql-test/suite/sys_vars/t/query_cache_wlock_invalidate_func.test:
        Locked -> Table lock
      mysql-test/suite/sys_vars/t/sql_low_priority_updates_func.test:
        Locked -> Table lock
      mysql-test/t/insert_notembedded.test:
        Locked -> Table lock
      mysql-test/t/lock_multi.test:
        Locked -> Table lock
      mysql-test/t/merge-big.test:
        Locked -> Table lock
      mysql-test/t/multi_update.test:
        Locked -> Table lock
      mysql-test/t/query_cache_28249.test:
        Locked -> Table lock
      mysql-test/t/sp_notembedded.test:
        Locked -> Table lock
      mysql-test/t/sp_sync.test:
        Locked -> Table lock
      mysql-test/t/status.test:
        Locked -> Table lock
      mysql-test/t/trigger_notembedded.test:
        Locked -> Table lock
      mysys/thr_alarm.c:
        Added option to disable thr_alarm
      mysys/thr_lock.c:
        Detect loops
      scripts/mysql_install_db.sh:
        Give better error message if something goes wrong
      sql/Makefile.am:
        Added sql_handler.h
      sql/lock.cc:
        Split functions to allow one to cache value if store_lock() (for HANDLER functions).
        - Split mysql_lock_tables() into two functions, where first one allocates MYSQL_LOCK and other other one uses it.
        - Made get_lock_data() an external function.
        - Added argument to mysql_unlock_tables() to not free sql_lock.
        - Added argument to reset_lock_data() to reset lock structure to initial state (as after get_lock_data())
      sql/mysql_priv.h:
        Moved handler function prototypes to sql_handler.h
        Added new lock functions.
      sql/mysqld.cc:
        Added --thread-alarm startup option
      sql/net_serv.cc:
        Don't call vio_blocking() if not needed
      sql/sql_base.cc:
        include sql_handler.h
      sql/sql_class.cc:
        include sql_handler.h
        Remove not anymore used 'thd->locked' member
      sql/sql_class.h:
        Remove not anymore used 'thd->locked' member
      sql/sql_db.cc:
        include sql_handler.h
      sql/sql_delete.cc:
        include sql_handler.h
      sql/sql_handler.cc:
        Rewrote all code to use SQL_HANDLER instead of TABLE_LIST (original interface)
        Rewrote mysql_ha_open() to cache all things from TABLE_LIST and items for field list, where etc.
        In mysql_ha_open() also cache MYSQL_LOCK structure from get_lock_data().
        Split functions into smaller sub functions (needed to be able to implement mysql_ha_read_prepare())
        Added mysql_ha_read_prepare() to allow one to prepare HANDLER READ.
      sql/sql_handler.h:
        Interface to sql_handler.cc
      sql/sql_parse.cc:
        include sql_handler.h
      sql/sql_prepare.cc:
        Added mysql_test_handler_read(), prepare for HANDLER READ
      sql/sql_rename.cc:
        include sql_handler.h
      sql/sql_show.cc:
        Removed usage of thd->locked
      sql/sql_table.cc:
        include sql_handler.h
      sql/sql_trigger.cc:
        include sql_handler.h
      e63b5546
  14. 23 Nov, 2010 1 commit
    • Michael Widenius's avatar
      Code cleanup to get fewer reallocs() during execution. · 52090a44
      Michael Widenius authored
      - Changed TABLE->alias to String to get fewer reallocs when alias are used.
      - Preallocate some buffers
      
      Changed some String->c_ptr() -> String->ptr() when \0 is not needed.
      Fixed wrong usage of String->ptr() when we need a \0 terminated string.
      Use my_strtod() instead of my_atof() to avoid having to add \0 to string.
      c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
      zr 
      
      sql/event_db_repository.cc:
        Update usage of TABLE->alias
      sql/event_scheduler.cc:
        c_ptr() -> c_ptr_safe()
      sql/events.cc:
        c_ptr() -> ptr() as \0 was not needed
      sql/field.cc:
        Update usage of TABLE->alias
      sql/field.h:
        Update usage of TABLE->alias
      sql/ha_partition.cc:
        Update usage of TABLE->alias
      sql/handler.cc:
        Update usage of TABLE->alias
        Fixed wrong usage of str.ptr()
      sql/item.cc:
        Fixed error where code wrongly assumed string was \0 terminated.
      sql/item_func.cc:
        c_ptr() -> c_ptr_safe()
        Update usage of TABLE->alias
      sql/item_sum.h:
        Use my_strtod() instead of my_atof() to avoid having to add \0 to string
      sql/lock.cc:
        Update usage of TABLE->alias
      sql/log.cc:
        c_ptr() -> ptr() as \0 was not needed
      sql/log_event.cc:
        c_ptr_quick() -> ptr() as \0 was not needed
      sql/opt_range.cc:
        ptr() -> c_ptr() as \0 is needed
      sql/opt_subselect.cc:
        Update usage of TABLE->alias
      sql/opt_table_elimination.cc:
        Update usage of TABLE->alias
      sql/set_var.cc:
        ptr() -> c_ptr() as \0 is needed
        c_ptr() -> c_ptr_safe()
      sql/sp.cc:
        c_ptr() -> ptr() as \0 was not needed
      sql/sp_rcontext.cc:
        
        Update usage of TABLE->alias
      sql/sql_base.cc:
        Preallocate buffers
        Update usage of TABLE->alias
      sql/sql_class.cc:
        Fix arguments to sprintf() to work even if string is not \0 terminated
      sql/sql_insert.cc:
        Update usage of TABLE->alias
        c_ptr() -> ptr() as \0 was not needed
      sql/sql_load.cc:
        Preallocate buffers
        Trivial optimizations
      sql/sql_parse.cc:
        Trivial optimization
      sql/sql_plugin.cc:
        c_ptr() -> ptr() as \0 was not needed
      sql/sql_select.cc:
        Update usage of TABLE->alias
      sql/sql_show.cc:
        Update usage of TABLE->alias
      sql/sql_string.h:
        Added move() function to move allocated memory from one object to another.
      sql/sql_table.cc:
        Update usage of TABLE->alias
        c_ptr() -> c_ptr_safe()
      sql/sql_test.cc:
        ptr() -> c_ptr_safe()
      sql/sql_trigger.cc:
        Update usage of TABLE->alias
        c_ptr() -> c_ptr_safe()
      sql/sql_update.cc:
        Update usage of TABLE->alias
      sql/sql_view.cc:
        ptr() -> c_ptr_safe()
      sql/sql_yacc.yy:
        ptr() -> c_ptr()
      sql/table.cc:
        
        Update usage of TABLE->alias
      sql/table.h:
        Changed TABLE->alias to String to get fewer reallocs when alias are used.
      storage/federatedx/ha_federatedx.cc:
        Use c_ptr_safe() to ensure strings are \0 terminated.
      storage/maria/ha_maria.cc:
        Update usage of TABLE->alias
      storage/myisam/ha_myisam.cc:
        Update usage of TABLE->alias
      storage/xtradb/row/row0sel.c:
        Ensure that null bits in record are properly reset.
        (Old code didn't work as row_search_for_mysql() can be called twice while reading fields from one row.
      52090a44
  15. 11 Nov, 2010 1 commit
    • Dmitry Lenev's avatar
      Patch that refactors global read lock implementation and fixes · 6bf6272f
      Dmitry Lenev authored
      bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ
      LOCK" and bug #54673 "It takes too long to get readlock for
      'FLUSH TABLES WITH READ LOCK'".
      
      The first bug manifested itself as a deadlock which occurred
      when a connection, which had some table open through HANDLER
      statement, tried to update some data through DML statement
      while another connection tried to execute FLUSH TABLES WITH
      READ LOCK concurrently.
      
      What happened was that FTWRL in the second connection managed
      to perform first step of GRL acquisition and thus blocked all
      upcoming DML. After that it started to wait for table open
      through HANDLER statement to be flushed. When the first connection
      tried to execute DML it has started to wait for GRL/the second
      connection creating deadlock.
      
      The second bug manifested itself as starvation of FLUSH TABLES
      WITH READ LOCK statements in cases when there was a constant
      stream of concurrent DML statements (in two or more
      connections).
      
      This has happened because requests for protection against GRL
      which were acquired by DML statements were ignoring presence of
      pending GRL and thus the latter was starved.
      
      This patch solves both these problems by re-implementing GRL
      using metadata locks.
      
      Similar to the old implementation acquisition of GRL in new
      implementation is two-step. During the first step we block
      all concurrent DML and DDL statements by acquiring global S
      metadata lock (each DML and DDL statement acquires global IX
      lock for its duration). During the second step we block commits
      by acquiring global S lock in COMMIT namespace (commit code
      acquires global IX lock in this namespace).
      
      Note that unlike in old implementation acquisition of
      protection against GRL in DML and DDL is semi-automatic.
      We assume that any statement which should be blocked by GRL
      will either open and acquires write-lock on tables or acquires
      metadata locks on objects it is going to modify. For any such
      statement global IX metadata lock is automatically acquired
      for its duration.
      
      The first problem is solved because waits for GRL become
      visible to deadlock detector in metadata locking subsystem
      and thus deadlocks like one in the first bug become impossible.
      
      The second problem is solved because global S locks which
      are used for GRL implementation are given preference over
      IX locks which are acquired by concurrent DML (and we can
      switch to fair scheduling in future if needed).
      
      Important change:
      FTWRL/GRL no longer blocks DML and DDL on temporary tables.
      Before this patch behavior was not consistent in this respect:
      in some cases DML/DDL statements on temporary tables were
      blocked while in others they were not. Since the main use cases
      for FTWRL are various forms of backups and temporary tables are
      not preserved during backups we have opted for consistently
      allowing DML/DDL on temporary tables during FTWRL/GRL.
      
      Important change:
      This patch changes thread state names which are used when
      DML/DDL of FTWRL is waiting for global read lock. It is now
      either "Waiting for global read lock" or "Waiting for commit
      lock" depending on the stage on which FTWRL is.
      
      Incompatible change:
      To solve deadlock in events code which was exposed by this
      patch we have to replace LOCK_event_metadata mutex with
      metadata locks on events. As result we have to prohibit
      DDL on events under LOCK TABLES.
      
      This patch also adds extensive test coverage for interaction
      of DML/DDL and FTWRL.
      
      Performance of new and old global read lock implementations
      in sysbench tests were compared. There were no significant
      difference between new and old implementations.
      
      mysql-test/include/check_ftwrl_compatible.inc:
        Added helper script which allows to check that a statement is
        compatible with FLUSH TABLES WITH READ LOCK.
      mysql-test/include/check_ftwrl_incompatible.inc:
        Added helper script which allows to check that a statement is
        incompatible with FLUSH TABLES WITH READ LOCK.
      mysql-test/include/handler.inc:
        Adjusted test case to the fact that now DROP TABLE closes
        open HANDLERs for the table to be dropped before checking
        if there active FTWRL in this connection.
      mysql-test/include/wait_show_condition.inc:
        Fixed small error in the timeout message. The correct name
        of variable used as parameter for this script is "$condition"
        and not "$wait_condition".
      mysql-test/r/delayed.result:
        Added test coverage for scenario which triggered assert in
        metadata locking subsystem.
      mysql-test/r/events_2.result:
        Updated test results after prohibiting event DDL operations
        under LOCK TABLES.
      mysql-test/r/flush.result:
        Added test coverage for bug #57006 "Deadlock between HANDLER
        and FLUSH TABLES WITH READ LOCK".
      mysql-test/r/flush_read_lock.result:
        Added test coverage for various aspects of FLUSH TABLES WITH
        READ LOCK functionality.
      mysql-test/r/flush_read_lock_kill.result:
        Adjusted test case after replacing custom global read lock
        implementation with one based on metadata locks. Use new
        debug_sync point. Do not disable concurrent inserts as now
        InnoDB we always use InnoDB table.
      mysql-test/r/handler_innodb.result:
        Adjusted test case to the fact that now DROP TABLE closes
        open HANDLERs for the table to be dropped before checking
        if there active FTWRL in this connection.
      mysql-test/r/handler_myisam.result:
        Adjusted test case to the fact that now DROP TABLE closes
        open HANDLERs for the table to be dropped before checking
        if there active FTWRL in this connection.
      mysql-test/r/mdl_sync.result:
        Adjusted test case after replacing custom global read lock
        implementation with one based on metadata locks. Replaced
        usage of GRL-specific debug_sync's with appropriate sync
        points in MDL subsystem.
      mysql-test/suite/perfschema/r/dml_setup_instruments.result:
        Updated test results after removing global
        COND_global_read_lock condition variable.
      mysql-test/suite/perfschema/r/func_file_io.result:
        Ensure that this test doesn't affect subsequent tests.
        At the end of its execution enable back P_S instrumentation
        which this test disables at some point.
      mysql-test/suite/perfschema/r/func_mutex.result:
        Ensure that this test doesn't affect subsequent tests.
        At the end of its execution enable back P_S instrumentation
        which this test disables at some point.
      mysql-test/suite/perfschema/r/global_read_lock.result:
        Adjusted test case to take into account that new GRL
        implementation is based on MDL.
      mysql-test/suite/perfschema/r/server_init.result:
        Adjusted test case after replacing custom global read
        lock implementation with one based on MDL and replacing
        LOCK_event_metadata mutex with metadata lock.
      mysql-test/suite/perfschema/t/func_file_io.test:
        Ensure that this test doesn't affect subsequent tests.
        At the end of its execution enable back P_S instrumentation
        which this test disables at some point.
      mysql-test/suite/perfschema/t/func_mutex.test:
        Ensure that this test doesn't affect subsequent tests.
        At the end of its execution enable back P_S instrumentation
        which this test disables at some point.
      mysql-test/suite/perfschema/t/global_read_lock.test:
        Adjusted test case to take into account that new GRL
        implementation is based on MDL.
      mysql-test/suite/perfschema/t/server_init.test:
        Adjusted test case after replacing custom global read
        lock implementation with one based on MDL and replacing
        LOCK_event_metadata mutex with metadata lock.
      mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result:
        Updated test results after prohibiting event DDL under
        LOCK TABLES.
      mysql-test/t/delayed.test:
        Added test coverage for scenario which triggered assert in
        metadata locking subsystem.
      mysql-test/t/events_2.test:
        Updated test case after prohibiting event DDL operations
        under LOCK TABLES.
      mysql-test/t/flush.test:
        Added test coverage for bug #57006 "Deadlock between HANDLER
        and FLUSH TABLES WITH READ LOCK".
      mysql-test/t/flush_block_commit.test:
        Adjusted test case after changing thread state name which
        is used when COMMIT waits for FLUSH TABLES WITH READ LOCK
        from "Waiting for release of readlock" to "Waiting for commit
        lock".
      mysql-test/t/flush_block_commit_notembedded.test:
        Adjusted test case after changing thread state name which is
        used when DML waits for FLUSH TABLES WITH READ LOCK. Now we
        use "Waiting for global read lock" in this case.
      mysql-test/t/flush_read_lock.test:
        Added test coverage for various aspects of FLUSH TABLES WITH
        READ LOCK functionality.
      mysql-test/t/flush_read_lock_kill-master.opt:
        We no longer need to use make_global_read_lock_block_commit_loop
        debug tag in this test. Instead we rely on an appropriate
        debug_sync point in MDL code.
      mysql-test/t/flush_read_lock_kill.test:
        Adjusted test case after replacing custom global read lock
        implementation with one based on metadata locks. Use new
        debug_sync point. Do not disable concurrent inserts as now
        InnoDB we always use InnoDB table.
      mysql-test/t/lock_multi.test:
        Adjusted test case after changing thread state names which
        are used when DML or DDL waits for FLUSH TABLES WITH READ
        LOCK to "Waiting for global read lock".
      mysql-test/t/mdl_sync.test:
        Adjusted test case after replacing custom global read lock
        implementation with one based on metadata locks. Replaced
        usage of GRL-specific debug_sync's with appropriate sync
        points in MDL subsystem. Updated thread state names which
        are used when DDL waits for FTWRL.
      mysql-test/t/trigger_notembedded.test:
        Adjusted test case after changing thread state names which
        are used when DML or DDL waits for FLUSH TABLES WITH READ
        LOCK to "Waiting for global read lock".
      sql/event_data_objects.cc:
        Removed Event_queue_element::status/last_executed_changed
        members and Event_queue_element::update_timing_fields()
        method. We no longer use this class for updating mysql.events
        once event is chosen for execution. Accesses to instances of
        this class in scheduler thread require protection by
        Event_queue::LOCK_event_queue mutex and we try to avoid
        updating table while holding this lock.
      sql/event_data_objects.h:
        Removed Event_queue_element::status/last_executed_changed
        members and Event_queue_element::update_timing_fields()
        method. We no longer use this class for updating mysql.events
        once event is chosen for execution. Accesses to instances of
        this class in scheduler thread require protection by
        Event_queue::LOCK_event_queue mutex and we try to avoid
        updating table while holding this lock.
      sql/event_db_repository.cc:
        - Changed Event_db_repository methods to not release all
          metadata locks once they are done updating mysql.events
          table. This allows to keep metadata lock protecting
          against GRL and lock protecting particular event around
          until corresponding DDL statement is written to the binary
          log.
        - Removed logic for conditional update of "status" and
          "last_executed" fields from update_timing_fields_for_event()
          method. In the only case when this method is called now
          "last_executed" is always modified and tracking change
          of "status" is too much hassle.
      sql/event_db_repository.h:
        Removed logic for conditional update of "status" and
        "last_executed" fields from Event_db_repository::
        update_timing_fields_for_event() method.
        In the only case when this method is called now "last_executed"
        is always modified and tracking change of "status" field is
        too much hassle.
      sql/event_queue.cc:
        Changed event scheduler code not to update mysql.events
        table while holding Event_queue::LOCK_event_queue mutex.
        Doing so led to a deadlock with a new GRL implementation.
        This deadlock didn't occur with old implementation due to
        fact that code acquiring protection against GRL ignored
        pending GRL requests (which lead to GRL starvation).
        One of goals of new implementation is to disallow GRL
        starvation and so we have to solve problem with this
        deadlock in a different way.
      sql/events.cc:
        Changed methods of Events class to acquire protection
        against GRL while perfoming DDL statement and keep it
        until statement is written to the binary log.
        Unfortunately this step together with new GRL implementation
        exposed deadlock involving Events::LOCK_event_metadata
        and GRL. To solve it Events::LOCK_event_metadata mutex was
        replaced with a metadata lock on event. As a side-effect
        events DDL has to be prohibited under LOCK TABLES even in
        cases when mysql.events table was explicitly locked for
        write.
      sql/events.h:
        Replaced Events::LOCK_event_metadata mutex with a metadata
        lock on event.
      sql/ha_ndbcluster.cc:
        Updated code after replacing custom global read lock
        implementation with one based on MDL. Since MDL subsystem
        should now be able to detect deadlocks involving metadata
        locks and GRL there is no need for special handling of
        active GRL.
      sql/handler.cc:
        Replaced custom implementation of global read lock with
        one based on metadata locks. Consequently when doing
        commit instead of calling method of Global_read_lock
        class to acquire protection against GRL we simply acquire
        IX in COMMIT namespace.
      sql/lock.cc:
        Replaced custom implementation of global read lock with
        one based on metadata locks. This step allows to expose
        wait for GRL to deadlock detector of MDL subsystem and
        thus succesfully resolve deadlocks similar to one behind
        bug #57006 "Deadlock between HANDLER and FLUSH TABLES
        WITH READ LOCK". It also solves problem with GRL starvation
        described in bug #54673 "It takes too long to get readlock
        for 'FLUSH TABLES WITH READ LOCK'" since metadata locks used
        by GRL give preference to FTWRL statement instead of DML
        statements (if needed in future this can be changed to
        fair scheduling).
        
        Similar to old implementation of acquisition of GRL is
        two-step. During the first step we block all concurrent
        DML and DDL statements by acquiring global S metadata lock
        (each DML and DDL statement acquires global IX lock for
        its duration). During the second step we block commits by
        acquiring global S lock in COMMIT namespace (commit code
        acquires global IX lock in this namespace).
        
        Note that unlike in old implementation acquisition of
        protection against GRL in DML and DDL is semi-automatic.
        We assume that any statement which should be blocked by GRL
        will either open and acquires write-lock on tables or acquires
        metadata locks on objects it is going to modify. For any such
        statement global IX metadata lock is automatically acquired
        for its duration.
        
        To support this change:
        - Global_read_lock::lock/unlock_global_read_lock and
          make_global_read_lock_block_commit methods were changed
          accordingly.
        - Global_read_lock::wait_if_global_read_lock() and
          start_waiting_global_read_lock() methods were dropped.
          It is now responsibility of code acquiring metadata locks
          opening tables to acquire protection against GRL by
          explicitly taking global IX lock with statement duration.
        - Global variables, mutex and condition variable used by
          old implementation was removed.
        - lock_routine_name() was changed to use statement duration for
          its global IX lock. It was also renamed to lock_object_name()
          as it now also used to take metadata locks on events.
        - Global_read_lock::set_explicit_lock_duration() was added which
          allows not to release locks used for GRL when leaving prelocked
          mode.
      sql/lock.h:
        - Renamed lock_routine_name() to lock_object_name() and changed
          its signature to allow its usage for events.
        - Removed broadcast_refresh() function. It is no longer needed
          with new GRL implementation.
      sql/log_event.cc:
        Release metadata locks with statement duration at the end
        of processing legacy event for LOAD DATA. This ensures that
        replication thread processing such event properly releases
        its protection against global read lock.
      sql/mdl.cc:
        Changed MDL subsystem to support new MDL-based implementation
        of global read lock.
        
        Added COMMIT and EVENTS namespaces for metadata locks. Changed
        thread state name for GLOBAL namespace to "Waiting for global
        read lock".
        
        Optimized MDL_map::find_or_insert() method to avoid taking
        m_mutex mutex when looking up MDL_lock objects for GLOBAL
        or COMMIT namespaces. We keep pre-created MDL_lock objects
        for these namespaces around and simply return pointers to
        these global objects when needed.
        
        Changed MDL_lock/MDL_scoped_lock to properly handle
        notification of insert delayed handler threads when FTWRL
        takes global S lock.
        
        Introduced concept of lock duration. In addition to locks with
        transaction duration which work in the way which is similar to
        how locks worked before (i.e. they are released at the end of
        transaction), locks with statement and explicit duration were
        introduced.
        Locks with statement duration are automatically released at the
        end of statement. Locks with explicit duration require explicit
        release and obsolete concept of transactional sentinel.
        
        * Changed MDL_request and MDL_ticket classes to support notion
          of duration.
        * Changed MDL_context to keep locks with different duration in
          different lists. Changed code handling ticket list to take
          this into account.
        * Changed methods responsible for releasing locks to take into
          account duration of tickets. Particularly public
          MDL_context::release_lock() method now only can release
          tickets with explicit duration (there is still internal
          method which allows to specify duration). To release locks
          with statement or transaction duration one have to use
          release_statement/transactional_locks() methods.
        * Concept of savepoint for MDL subsystem now has to take into
          account locks with statement duration. Consequently
          MDL_savepoint class was introduced and methods working with
          savepoints were updated accordingly.
        * Added methods which allow to set duration for one or all
          locks in the context.
      sql/mdl.h:
        Changed MDL subsystem to support new MDL-based implementation
        of global read lock.
        
        Added COMMIT and EVENTS namespaces for metadata locks.
        
        Introduced concept of lock duration. In addition to locks with
        transaction duration which work in the way which is similar to
        how locks worked before (i.e. they are released at the end of
        transaction), locks with statement and explicit duration were
        introduced.
        Locks with statement duration are automatically released at the
        end of statement. Locks with explicit duration require explicit
        release and obsolete concept of transactional sentinel.
        
        * Changed MDL_request and MDL_ticket classes to support notion
          of duration.
        * Changed MDL_context to keep locks with different duration in
          different lists. Changed code handling ticket list to take
          this into account.
        * Changed methods responsible for releasing locks to take into
          account duration of tickets. Particularly public
          MDL_context::release_lock() method now only can release
          tickets with explicit duration (there is still internal
          method which allows to specify duration). To release locks
          with statement or transaction duration one have to use
          release_statement/transactional_locks() methods.
        * Concept of savepoint for MDL subsystem now has to take into
          account locks with statement duration. Consequently
          MDL_savepoint class was introduced and methods working with
          savepoints were updated accordingly.
        * Added methods which allow to set duration for one or all
          locks in the context.
      sql/mysqld.cc:
        Removed global mutex and condition variables which were used
        by old implementation of GRL.
        Also we no longer need to initialize Events::LOCK_event_metadata
        mutex as it was replaced with metadata locks on events.
      sql/mysqld.h:
        Removed global variable, mutex and condition variables which
        were used by old implementation of GRL.
      sql/rpl_rli.cc:
        When slave thread closes tables which were open for handling
        of RBR events ensure that it releases global IX lock which
        was acquired as protection against GRL.
      sql/sp.cc:
        Adjusted code to the new signature of lock_object/routine_name(),
        to the fact that one now needs specify duration of lock when
        initializing MDL_request and to the fact that savepoints for MDL
        subsystem are now represented by MDL_savepoint class.
      sql/sp_head.cc:
        Ensure that statements in stored procedures release statement
        metadata locks and thus release their protectiong against GRL
        in proper moment in time.
        Adjusted code to the fact that one now needs specify duration
        of lock when initializing MDL_request.
      sql/sql_admin.cc:
        Adjusted code to the fact that one now needs specify duration
        of lock when initializing MDL_request.
      sql/sql_base.cc:
        - Implemented support for new approach to acquiring protection
          against global read lock. We no longer acquire such protection
          explicitly on the basis of statement flags. Instead we always
          rely on code which is responsible for acquiring metadata locks
          on object to be changed acquiring this protection. This is
          achieved by acquiring global IX metadata lock with statement
          duration. Code doing this also responsible for checking that
          current connection has no active GRL by calling an
          Global_read_lock::can_acquire_protection() method.
          Changed code in open_table() and lock_table_names()
          accordingly.
          Note that as result of this change DDL and DML on temporary
          tables is always compatible with GRL (before it was
          incompatible in some cases and compatible in other cases).
        - To speed-up code acquiring protection against GRL introduced
          m_has_protection_against_grl member in Open_table_context
          class. It indicates that protection was already acquired
          sometime during open_tables() execution and new attempts
          can be skipped.
        - Thanks to new GRL implementation calls to broadcast_refresh()
          became unnecessary and were removed.
        - Adjusted code to the fact that one now needs specify duration
          of lock when initializing MDL_request and to the fact that
          savepoints for MDL subsystem are now represented by
          MDL_savepoint class.
      sql/sql_base.h:
        Adjusted code to the fact that savepoints for MDL subsystem are
        now represented by MDL_savepoint class.
        Also introduced Open_table_context::m_has_protection_against_grl
        member which allows to avoid acquiring protection against GRL
        while opening tables if such protection was already acquired.
      sql/sql_class.cc:
        Changed THD::leave_locked_tables_mode() after transactional
        sentinel for metadata locks was obsoleted by introduction of
        locks with explicit duration.
      sql/sql_class.h:
        - Adjusted code to the fact that savepoints for MDL subsystem
          are now represented by MDL_savepoint class.
        - Changed Global_read_lock class according to changes in
          global read lock implementation:
          * wait_if_global_read_lock and start_waiting_global_read_lock
            are now gone. Instead code needing protection against GRL
            has to acquire global IX metadata lock with statement
            duration itself. To help it new can_acquire_protection()
            was introduced. Also as result of the above change
            m_protection_count member is gone too.
          * Added m_mdl_blocks_commits_lock member to store metadata
            lock blocking commits.
          * Adjusted code to the fact that concept of transactional
            sentinel was obsoleted by concept of lock duration.
        - Removed CF_PROTECT_AGAINST_GRL flag as it is no longer
          necessary. New GRL implementation acquires protection
          against global read lock automagically when statement
          acquires metadata locks on tables or other objects it
          is going to change.
      sql/sql_db.cc:
        Adjusted code to the fact that one now needs specify duration
        of lock when initializing MDL_request.
      sql/sql_handler.cc:
        Removed call to broadcast_refresh() function. It is no longer
        needed with new GRL implementation.
        Adjusted code after introducing duration concept for metadata
        locks. Particularly to the fact transactional sentinel was
        replaced with explicit duration.
      sql/sql_handler.h:
        Renamed mysql_ha_move_tickets_after_trans_sentinel() to
        mysql_ha_set_explicit_lock_duration() after transactional
        sentinel was obsoleted by locks with explicit duration.
      sql/sql_insert.cc:
        Adjusted code handling delaying inserts after switching to
        new GRL implementation. Now connection thread initiating
        delayed insert has to acquire global IX lock in addition
        to metadata lock on table being inserted into. This IX lock
        protects against GRL and similarly to SW lock on table being
        inserted into has to be passed to handler thread in order to
        avoid deadlocks.
      sql/sql_lex.cc:
        LEX::protect_against_global_read_lock member is no longer
        necessary since protection against GRL is automatically
        taken by code acquiring metadata locks/opening tables.
      sql/sql_lex.h:
        LEX::protect_against_global_read_lock member is no longer
        necessary since protection against GRL is automatically
        taken by code acquiring metadata locks/opening tables.
      sql/sql_parse.cc:
        - Implemented support for new approach to acquiring protection
          against global read lock. We no longer acquire such protection
          explicitly on the basis of statement flags. Instead we always
          rely on code which is responsible for acquiring metadata locks
          on object to be changed acquiring this protection. This is
          achieved by acquiring global IX metadata lock with statement
          duration. This lock is automatically released at the end of
          statement execution.
        - Changed implementation of CREATE/DROP PROCEDURE/FUNCTION not
          to release metadata locks and thus protection against of GRL
          in the middle of statement execution.
        - Adjusted code to the fact that one now needs specify duration
          of lock when initializing MDL_request and to the fact that
          savepoints for MDL subsystem are now represented by
          MDL_savepoint class.
      sql/sql_prepare.cc:
        Adjusted code to the to the fact that savepoints for MDL
        subsystem are now represented by MDL_savepoint class.
      sql/sql_rename.cc:
        With new GRL implementation there is no need to explicitly
        acquire protection against GRL before renaming tables.
        This happens automatically in code which acquires metadata
        locks on tables being renamed.
      sql/sql_show.cc:
        Adjusted code to the fact that one now needs specify duration
        of lock when initializing MDL_request and to the fact that
        savepoints for MDL subsystem are now represented by
        MDL_savepoint class.
      sql/sql_table.cc:
        - With new GRL implementation there is no need to explicitly
          acquire protection against GRL before dropping tables.
          This happens automatically in code which acquires metadata
          locks on tables being dropped.
        - Changed mysql_alter_table() not to release lock on new table
          name explicitly and to rely on automatic release of locks
          at the end of statement instead. This was necessary since
          now MDL_context::release_lock() is supported only for locks
          for explicit duration.
      sql/sql_trigger.cc:
        With new GRL implementation there is no need to explicitly
        acquire protection against GRL before changing table triggers.
        This happens automatically in code which acquires metadata
        locks on tables which triggers are to be changed.
      sql/sql_update.cc:
        Fix bug exposed by GRL testing. During prepare phase acquire
        only S metadata locks instead of SW locks to keep prepare of
        multi-UPDATE compatible with concurrent LOCK TABLES WRITE
        and global read lock.
      sql/sql_view.cc:
        With new GRL implementation there is no need to explicitly
        acquire protection against GRL before creating view.
        This happens automatically in code which acquires metadata
        lock on view to be created.
      sql/sql_yacc.yy:
        LEX::protect_against_global_read_lock member is no longer
        necessary since protection against GRL is automatically
        taken by code acquiring metadata locks/opening tables.
      sql/table.cc:
        Adjusted code to the fact that one now needs specify duration
        of lock when initializing MDL_request.
      sql/table.h:
        Adjusted code to the fact that one now needs specify duration
        of lock when initializing MDL_request.
      sql/transaction.cc:
        Replaced custom implementation of global read lock with
        one based on metadata locks. Consequently when doing
        commit instead of calling method of Global_read_lock
        class to acquire protection against GRL we simply acquire
        IX in COMMIT namespace.
        Also adjusted code to the fact that MDL savepoint is now
        represented by MDL_savepoint class.
      6bf6272f
  16. 02 Nov, 2010 1 commit
    • Michael Widenius's avatar
      Fix for: LP #634955: Assert in _ma_update_at_original_place() · 20acfbf3
      Michael Widenius authored
      Added locking of lock mutex when updating status in external_unlock() for Aria and MyISAM tables.
      Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
      DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards. (Allows one to run --debug with mysql-test-run scripts that uses @debug)
      Fixed several core dump, deadlock and crashed table bugs in handling of LOCK TABLE with MERGE tables:
      - Added priority of locks to avoid crashes with MERGE tables.
      - Added thr_lock_merge() to allow one to merge two results of thr_lock().
      Fixed 'not found row' bug in REPLACE with Aria tables.
      Mark MyISAM tables that are part of MERGE with HA_OPEN_MERGE_TABLE and set the locks to have priority THR_LOCK_MERGE_PRIV.
      - By sorting MERGE tables last in thr_multi_unlock() it's safer to release and relock them many times (can happen when TRIGGERS are created)
      Avoid printing (null) in debug file (to easier find out wrong NULL pointer usage with %s).
      
      
      
      client/mysql.cc:
        Fixed that 'source' command doesn't cause mysql command line tool to exit on error.
      client/mysqltest.cc:
        Don't send NULL to fn_format(). (Can cause crash on Solaris when using --debug)
      dbug/dbug.c:
        DEBUG_EXECUTE() and DEBUG_EVALUATE_IF() should not execute things based on wildcards.
      include/my_base.h:
        Added flag to signal if one opens a MERGE table.
        Added extra() command to signal that one is not part of a MERGE table anymore.
      include/thr_lock.h:
        Added priority for locks (needed to fix bug in thr_lock when using MERGE tables)
        Added option to thr_unlock() if get_status() should be called.
        Added prototype for thr_merge_locks().
      mysql-test/mysql-test-run.pl:
        Ignore crashed table warnings for tables named 'crashed'.
      mysql-test/r/merge.result:
        Renamed triggers to make debugging easier.
        Added some CHECK TABLES to catch errors earlier.
        Additional tests.
      mysql-test/r/merge_debug.result:
        Test of error handling when reopening MERGE tables.
      mysql-test/r/udf_query_cache.result:
        Added missing flush status
      mysql-test/suite/parts/r/partition_repair_myisam.result:
        Update results
      mysql-test/t/merge.test:
        Renamed triggers to make debugging easier.
        Added some CHECK TABLES to catch errors earlier.
        Additional tests.
      mysql-test/t/merge_debug.test:
        Test of error handling when reopening MERGE tables.
      mysql-test/t/udf_query_cache.test:
        Added missing flush status
      mysys/my_getopt.c:
        Removed not used variable
      mysys/my_symlink2.c:
        Changed (null) to (NULL) to make it easier to find NULL arguments to DBUG_PRINT() functions.
        (On linux, NULL to sprintf is printed 'null')
      mysys/thr_lock.c:
        Added priority of locks to avoid crashes with MERGE tables.
        Added thr_lock_merge() to allow one to merge two results of thr_lock().
        - This is needed for MyISAM as all locked table must share the same status. If not, you will not see newly inserted rows in other instances of the table.
        If calling thr_unlock() with THR_UNLOCK_UPDATE_STATUS, call update_status() and restore_status() for the locks. This is needed in some rare cases where we call thr_unlock() followed by thr_lock() without calling external_unlock/external_lock in between.
        Simplify loop in thr_multi_lock().
        Added 'start_trans', which is called at end of thr_multi_lock() when all locks are taken.
        - This was needed by Aria to ensure that transaction is started when we got all locks, not at get_status(). Without this, some rows could not be visible when we lock two tables at the same time, causing REPLACE using two tables to fail unexpectedly.
      sql/handler.cc:
        Add an assert() in handler::print_error() for "impossible errors" (like table is crashed) when --debug-assert-if-crashed-table is used.
      sql/lock.cc:
        Simplify mysql_lock_tables() code if get_lock_data() returns 0 locks.
        Added new parameter to thr_multi_unlock()
        In mysql_unlock_read_tables(), call first externa_unlock(), then thr_multi_unlock();  This is same order as we do in mysql_unlock_tables().
        Don't abort locks in mysql_lock_abort() for merged tables when a MERGE table is deleted; Would cause a spin lock.
        Added call to thr_merge_locks() in mysql_lock_merge() to ensure consistency in thr_locks().
        - New locks of same type and table is stored after the old lock to ensure that we get the status from the original lock.
      sql/mysql_priv.h:
        Added debug_assert_if_crashed_table
      sql/mysqld.cc:
        Added --debug-assert-if-crashed-table
      sql/parse_file.cc:
        Don't print '(null)' in DBUG_PRINT of no dir given
      sql/set_var.cc:
        Increase default size of buffer for @debug variable.
      sql/sql_base.cc:
        In case of error from reopen_table() in reopen_tables(), call unlock_open_table() and restart loop.
        - This fixed bug when we twice deleted same table from open_cache.
        Don't take name lock for already name locked table in open_unireg_entry().
        - Fixed bug when doing repair in reopen_table().
        - In detach_merge_children(), always detach if 'clear_refs' is given. We can't trust parent->children_attached as this function can be called twice, first time with clear_refs set to 0.
      sql/sql_class.cc:
        Changed printing of (null) to "" in set_thd_proc_info()
      sql/sql_parse.cc:
        Added DBUG
      sql/sql_trigger.cc:
        Don't call unlink_open_table() if reopen_table() fails as the table may already be freed.
      storage/maria/ma_bitmap.c:
        Fixed DBUG_ASSERT() in allocate_tail()
      storage/maria/ma_blockrec.c:
        Fixed wrong calculation of row length for very small rows in undo_row_update().
        - Fixes ASSERT() when doing undo.
      storage/maria/ma_blockrec.h:
        Added _ma_block_start_trans() and _ma_block_start_trans_no_versioning()
      storage/maria/ma_locking.c:
        Call _ma_update_status_with_lock() when releasing write locks.
        - Fixes potential problem with updating status without the proper lock.
      storage/maria/ma_open.c:
        Changed to use start_trans() instead of get_status() to ensure that we see all rows in all locked tables when we got the locks.
        - Fixed 'not found row' bug in REPLACE with Aria tables.
      storage/maria/ma_state.c:
        Added _ma_update_status_with_lock() and _ma_block_start_trans().
        This is to ensure that we see all rows in all locked tables when we got the locks.
      storage/maria/ma_state.h:
        Added _ma_update_status_with_lock()
      storage/maria/ma_write.c:
        More DBUG_PRINT
      storage/myisam/mi_check.c:
        Fixed error message
      storage/myisam/mi_extra.c:
        Added HA_EXTRA_DETACH_CHILD:
        - Detach MyISAM table to not be part of MERGE table (remove flag & lock priority).
      storage/myisam/mi_locking.c:
        Call mi_update_status_with_lock() when releasing write locks.
        - Fixes potential problem with updating status without the proper lock.
        Change to use new HA_OPEN_MERGE_TABLE flag to test if MERGE table.
        Added mi_fix_status(), called by thr_merge().
      storage/myisam/mi_open.c:
        Added marker if part of MERGE table.
        Call mi_fix_status() in thr_lock() for transactional tables.
      storage/myisam/myisamdef.h:
        Change my_once_flag to uint, as it stored different values than just 0/1
        Added 'open_flag' to store state given to mi_open()
      storage/myisammrg/ha_myisammrg.cc:
        Add THR_LOCK_MERGE_PRIV to THR_LOCK_DATA to get MERGE locks sorted after other types of locks.
      storage/myisammrg/myrg_locking.c:
        Remove windows specific code.
      storage/myisammrg/myrg_open.c:
        Use HA_OPEN_MERGE_TABLE to mi_open().
        Set HA_OPEN_MERGE_TABLE for linked MyISAM tables.
      storage/xtradb/buf/buf0buf.c:
        Fixed compiler warning
      storage/xtradb/buf/buf0lru.c:
        Initialize variable that could be used not initialized.
      20acfbf3
  17. 27 Oct, 2010 1 commit
  18. 22 Oct, 2010 1 commit
    • Davi Arnaut's avatar
      Bug#37780: Make KILL reliable (main.kill fails randomly) · 2881b801
      Davi Arnaut authored
      - A prerequisite cleanup patch for making KILL reliable.
      
      The test case main.kill did not work reliably.
      
      The following problems have been identified:
      
      1. A kill signal could go lost if it came in, short before a
      thread went reading on the client connection.
      
      2. A kill signal could go lost if it came in, short before a
      thread went waiting on a condition variable.
      
      These problems have been solved as follows. Please see also
      added code comments for more details.
      
      1. There is no safe way to detect, when a thread enters the
      blocking state of a read(2) or recv(2) system call, where it
      can be interrupted by a signal. Hence it is not possible to
      wait for the right moment to send a kill signal. It has been
      decided, not to fix it in the code.  Instead, the test case
      repeats the KILL statement until the connection terminates.
      
      2. Before waiting on a condition variable, we register it
      together with a synchronizating mutex in THD::mysys_var. After
      this, we need to test THD::killed again. At some places we did
      only test it in a loop condition before the registration. When
      THD::killed had been set between this test and the registration,
      we entered waiting without noticing the killed flag. Additional
      checks ahve been introduced where required.
      
      In addition to the above, a re-write of the main.kill test
      case has been done. All sleeps have been replaced by Debug
      Sync Facility synchronization. A couple of sync points have
      been added to the server code.
      
      To avoid further problems, if the test case fails in spite of
      the fixes, the test case has been added to the "experimental"
      list for now.
      
      - Most of the work on this patch is authored by Ingo Struewing
      
      
      mysql-test/t/kill.test:
        Re-wrote test case to use Debug Sync points instead of sleeps
      sql/event_queue.cc:
        Fixed kill detection in Event_queue::cond_wait() by adding a check
        after enter_cond().
      sql/lock.cc:
        Moved Debug Sync points behind enter_cond().
        Fixed comments.
      sql/slave.cc:
        Fixed kill detection in start_slave_thread() by adding a check
        after enter_cond().
      sql/sql_class.cc:
        Swapped order of kill and close in THD::awake().
        Added comments.
      sql/sql_class.h:
        Added a comment to THD::killed.
      sql/sql_parse.cc:
        Added a sync point in do_command().
      sql/sql_select.cc:
        Added a sync point in JOIN::optimize().
      2881b801
  19. 20 Oct, 2010 1 commit
    • Davi Arnaut's avatar
      Bug#45288: pb2 returns a lot of compilation warnings · 3e9c5225
      Davi Arnaut authored
      Fix assorted warnings that are generated in optimized builds.
      Most of it is silencing variables that are set but unused.
      
      This patch also introduces the MY_ASSERT_UNREACHABLE macro
      which helps the compiler to deduce that a certain piece of
      code is unreachable.
      
      include/my_compiler.h:
        Use GCC's __builtin_unreachable if available. It allows
        GCC to deduce the unreachability of certain code paths,
        thus avoiding warnings that, for example, accused that a
        variable could be used without being initialized (due to
        unreachable code paths).
      3e9c5225
  20. 27 Jul, 2010 2 commits
    • Dmitry Lenev's avatar
      Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH · 00496b7a
      Dmitry Lenev authored
      TABLES <list> WITH READ LOCK are incompatible".
      
      The problem was that FLUSH TABLES <list> WITH READ LOCK
      which was issued when other connection has acquired global
      read lock using FLUSH TABLES WITH READ LOCK was blocked
      and has to wait until global read lock is released.
      
      This issue stemmed from the fact that FLUSH TABLES <list>
      WITH READ LOCK implementation has acquired X metadata locks
      on tables to be flushed. Since these locks required acquiring
      of global IX lock this statement was incompatible with global
      read lock.
      
      This patch addresses problem by using SNW metadata type of
      lock for tables to be flushed by FLUSH TABLES <list> WITH
      READ LOCK. It is OK to acquire them without global IX lock
      as long as we won't try to upgrade those locks. Since SNW
      locks allow concurrent statements using same table FLUSH
      TABLE <list> WITH READ LOCK now has to wait until old
      versions of tables to be flushed go away after acquiring
      metadata locks. Since such waiting can lead to deadlock
      MDL deadlock detector was extended to take into account
      waits for flush and resolve such deadlocks.
      
      As a bonus code in open_tables() which was responsible for
      waiting old versions of tables to go away was refactored.
      Now when we encounter old version of table in open_table()
      we don't back-off and wait for all old version to go away,
      but instead wait for this particular table to be flushed.
      Such approach supported by deadlock detection should reduce
      number of scenarios in which FLUSH TABLES aborts concurrent
      multi-statement transactions.
      
      Note that active FLUSH TABLES <list> WITH READ LOCK still
      blocks concurrent FLUSH TABLES WITH READ LOCK statement
      as the former keeps tables open and thus prevents the
      latter statement from doing flush.
      
      mysql-test/include/handler.inc:
        Adjusted test case after changing status which is set
        when FLUSH TABLES waits for tables to be flushed from
        "Flushing tables" to "Waiting for table".
      mysql-test/r/flush.result:
        Added test which checks that "flush tables <list> with
        read lock" is compatible with active "flush tables with
        read lock" but not vice-versa. This test also covers
        bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES
        <list> WITH READ LOCK are incompatible".
      mysql-test/r/mdl_sync.result:
        Added scenarios in which wait for table to be flushed
        causes deadlocks to the coverage of MDL deadlock detector.
      mysql-test/suite/perfschema/r/dml_setup_instruments.result:
        Adjusted test results after removal of COND_refresh
        condition variable.
      mysql-test/suite/perfschema/r/server_init.result:
        Adjusted test and its results after removal of COND_refresh
        condition variable.
      mysql-test/suite/perfschema/t/server_init.test:
        Adjusted test and its results after removal of COND_refresh
        condition variable.
      mysql-test/t/flush.test:
        Added test which checks that "flush tables <list> with
        read lock" is compatible with active "flush tables with
        read lock" but not vice-versa. This test also covers
        bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES
        <list> WITH READ LOCK are incompatible".
      mysql-test/t/kill.test:
        Adjusted test case after changing status which is set
        when FLUSH TABLES waits for tables to be flushed from
        "Flushing tables" to "Waiting for table".
      mysql-test/t/lock_multi.test:
        Adjusted test case after changing status which is set
        when FLUSH TABLES waits for tables to be flushed from
        "Flushing tables" to "Waiting for table".
      mysql-test/t/mdl_sync.test:
        Added scenarios in which wait for table to be flushed
        causes deadlocks to the coverage of MDL deadlock detector.
      sql/ha_ndbcluster.cc:
        Adjusted code after adding one more parameter for
        close_cached_tables() call - timeout for waiting for
        table to be flushed.
      sql/ha_ndbcluster_binlog.cc:
        Adjusted code after adding one more parameter for
        close_cached_tables() call - timeout for waiting for
        table to be flushed.
      sql/lock.cc:
        Removed COND_refresh condition variable. See comment
        for sql_base.cc for details.
      sql/mdl.cc:
        Now MDL deadlock detector takes into account information
        about waits for table flushes when searching for deadlock.
        To implement this change:
        - Declaration of enum_deadlock_weight and
          Deadlock_detection_visitor were moved to mdl.h header
          to make them available to the code in table.cc which
          implements deadlock detector traversal through edges
          of waiters graph representing waiting for flush.
        - Since now MDL_context may wait not only for metadata
          lock but also for table to be flushed an abstract
          Wait_for_edge class was introduced. Its descendants
          MDL_ticket and Flush_ticket incapsulate specifics
          of inspecting waiters graph when following through
          edge representing wait of particular type.
        
        We no longer require global IX metadata lock when acquiring
        SNW or SNRW locks. Such locks are needed only when metadata
        locks of these types are upgraded to X locks. This allows
        to use SNW locks in FLUSH TABLES <list> WITH READ LOCK
        implementation and keep the latter compatible with global
        read lock.
      sql/mdl.h:
        Now MDL deadlock detector takes into account information
        about waits for table flushes when searching for deadlock.
        To implement this change:
        - Declaration of enum_deadlock_weight and
          Deadlock_detection_visitor were moved to mdl.h header
          to make them available to the code in table.cc which
          implements deadlock detector traversal through edges
          of waiters graph representing waiting for flush.
        - Since now MDL_context may wait not only for metadata
          lock but also for table to be flushed an abstract
          Wait_for_edge class was introduced. Its descendants
          MDL_ticket and Flush_ticket incapsulate specifics
          of inspecting waiters graph when following through
          edge representing wait of particular type.
        - Deadlock_detection_visitor now has m_table_shares_visited
          member which allows to support recursive locking for
          LOCK_open. This is required when deadlock detector
          inspects waiters graph which contains several edges
          representing waits for flushes or needs to come through
          the such edge more than once.
      sql/mysqld.cc:
        Removed COND_refresh condition variable. See comment
        for sql_base.cc for details.
      sql/mysqld.h:
        Removed COND_refresh condition variable. See comment
        for sql_base.cc for details.
      sql/sql_base.cc:
        Changed approach to how threads are waiting for table
        to be flushed. Now thread that wants to wait for old
        table to go away subscribes for notification by adding
        Flush_ticket to table's share and waits using
        MDL_context::m_wait object. Once table gets flushed
        (i.e. all tables are closed and table share is ready
        to be destroyed) all such waiters are notified
        individually.
        Thanks to this change MDL deadlock detector can take
        such waits into account.
        
        To implement this/as result of this change:
        - tdc_wait_for_old_versions() was replaced with
          tdc_wait_for_old_version() which waits for individual
          old share to go away and which is called by open_table()
          after finding out that share is outdated. We don't
          need to perform back-off before such waiting thanks
          to the fact that deadlock detector now sees such waits.
        - As result Open_table_ctx::m_mdl_requests became
          unnecessary and was removed. We no longer allocate
          copies of MDL_request objects on MEM_ROOT when
          MYSQL_OPEN_FORCE_SHARED/SHARED_HIGH_PRIO flags are
          in effect.
        - close_cached_tables() and tdc_wait_for_old_version()
          share code which implements waiting for share to be
          flushed - the both use TABLE_SHARE::wait_until_flush()
          method. Thanks to this close_cached_tables() supports
          timeouts and has extra parameter for this.
        - Open_table_context::OT_MDL_CONFLICT enum element was
          renamed to OT_CONFLICT as it is now also used in cases
          when back-off is required to resolve deadlock caused
          by waiting for flush and not metadata lock.
        - In cases when we discover that current connection tries
          to open tables from different generation we now simply
          back-off and restart process of opening tables. To
          support this Open_table_context::OT_REOPEN_TABLES enum
          element was added.
        - COND_refresh condition variable became unnecessary and
          was removed.
        - mysql_notify_thread_having_shared_lock() no longer wakes
          up connections waiting for flush as all such connections
          can be waken up by deadlock detector if necessary.
      sql/sql_base.h:
        - close_cached_tables() now has one more parameter -
          timeout for waiting for table to be flushed.
        - Open_table_context::OT_MDL_CONFLICT enum element was
          renamed to OT_CONFLICT as it is now also used in cases
          when back-off is required to resolve deadlock caused
          by waiting for flush and not metadata lock.
          Added new OT_REOPEN_TABLES enum element to be used in
          cases when we need to restart open tables process even
          in the middle of transaction.
        - Open_table_ctx::m_mdl_requests became unnecessary and
          was removed.
      sql/sql_class.h:
        Added assert ensuring that we won't use LOCK_open mutex
        with THD::enter_cond(). Otherwise deadlocks can arise in
        MDL deadlock detector.
      sql/sql_parse.cc:
        Changed FLUSH TABLES <list> WITH READ LOCK to take SNW
        metadata locks instead of X locks on tables to be flushed.
        Since we no longer require global IX lock to be taken
        when SNW locks are taken this makes this statement
        compatible with FLUSH TABLES WITH READ LOCK statement.
        Since SNW locks allow other connections to have table
        opened FLUSH TABLES <list> WITH READ LOCK now has to
        wait during open_tables() for old version to go away.
        Such waits can lead to deadlocks which will be detected
        by MDL deadlock detector which now takes waits for table
        to be flushed into account.
        
        Also adjusted code after adding one more parameter for
        close_cached_tables() call - timeout for waiting for
        table to be flushed.
      sql/sql_yacc.yy:
        FLUSH TABLES <list> WITH READ LOCK now needs only SNW
        metadata locks on tables.
      sql/sys_vars.cc:
        Adjusted code after adding one more parameter for
        close_cached_tables() call - timeout for waiting for
        table to be flushed.
      sql/table.cc:
        Implemented new approach to how threads are waiting for
        table to be flushed. Now thread that wants to wait for
        old table to go away subscribes for notification by
        adding Flush_ticket to table's share and waits using
        MDL_context::m_wait object. Once table gets flushed
        (i.e. all tables are closed and table share is ready
        to be destroyed) all such waiters are notified
        individually. This change allows to make such waits
        visible inside of MDL deadlock detector.
        To do it:
        
        - Added list of waiters/Flush_tickets to TABLE_SHARE
          class.
        - Changed free_table_share() to postpone freeing of
          share memory until last waiter goes away and to
          wake up subscribed waiters.
        - Added TABLE_SHARE::wait_until_flushed() method which
          implements subscription to the list of waiters for
          table to be flushed and waiting for this event.
        
        Implemented interface which allows to expose waits for
        flushes to MDL deadlock detector:
        
        - Introduced Flush_ticket class a descendant of
          Wait_for_edge class.
        - Added TABLE_SHARE::find_deadlock() method which allows
          deadlock detector to find out what contexts are still
          using old version of table in question (i.e. to find
          out what contexts are waited for by owner of
          Flush_ticket).
      sql/table.h:
        In order to support new strategy of waiting for table flush
        (see comment for table.cc for details) added list of
        waiters/Flush_tickets to TABLE_SHARE class.
        
        Implemented interface which allows to expose waits for
        flushes to MDL deadlock detector:
        - Introduced Flush_ticket class a descendant of
          Wait_for_edge class.
        - Added TABLE_SHARE::find_deadlock() method which allows
          deadlock detector to find out what contexts are still
          using old version of table in question (i.e. to find
          out what contexts are waited for by owner of
          Flush_ticket).
      00496b7a
    • Konstantin Osipov's avatar
      Implement WL#5502 Remove dead 5.0 class Sensitive_cursor. · 740c0d3a
      Konstantin Osipov authored
      Remove dead and unused code.
      Update to reflect the code review requests.
      
      include/thr_lock.h:
        Remove declarations for THR_LOCK_OWNER,
        added along with the patch for sensitive cursors.
      mysys/thr_lock.c:
        Remove support for multiple thr_lock requestors
        per THD.
      sql/lock.cc:
        Revert the patch that added support for sensitive cursors.
      sql/sp_rcontext.cc:
        Updated the use of mysql_open_cursor().
      sql/sql_class.cc:
        Move the instance of Server_side_cursor
        from class Prepared_statement to class Statement.
      sql/sql_class.h:
        Move the isntance of Server_side_cursor
        from class Prepared_statement to class
        Statement.
        Remove multiple lock_ids of thr_lock.
      sql/sql_cursor.cc:
        Remove Sensitive_cursor implementation.
      sql/sql_cursor.h:
        Remove declarations for sensitive cursors.
      sql/sql_prepare.cc:
        Move the declaration of instance of Server_side_cursor
        from class Statement to class Prepared_statement, 
        where it's used.
      sql/sql_select.cc:
        Remove sensitive cursor support.
      sql/sql_select.h:
        Remove sensitive cursor support.
      sql/sql_union.cc:
        Remove sensitive cursor support.
      740c0d3a
  21. 08 Jul, 2010 1 commit
    • Davi Arnaut's avatar
      Bug#34043: Server loops excessively in _checkchunk() when safemalloc is enabled · f56dd32b
      Davi Arnaut authored
      Essentially, the problem is that safemalloc is excruciatingly
      slow as it checks all allocated blocks for overrun at each
      memory management primitive, yielding a almost exponential
      slowdown for the memory management functions (malloc, realloc,
      free). The overrun check basically consists of verifying some
      bytes of a block for certain magic keys, which catches some
      simple forms of overrun. Another minor problem is violation
      of aliasing rules and that its own internal list of blocks
      is prone to corruption.
      
      Another issue with safemalloc is rather the maintenance cost
      as the tool has a significant impact on the server code.
      Given the magnitude of memory debuggers available nowadays,
      especially those that are provided with the platform malloc
      implementation, maintenance of a in-house and largely obsolete
      memory debugger becomes a burden that is not worth the effort
      due to its slowness and lack of support for detecting more
      common forms of heap corruption.
      
      Since there are third-party tools that can provide the same
      functionality at a lower or comparable performance cost, the
      solution is to simply remove safemalloc. Third-party tools
      can provide the same functionality at a lower or comparable
      performance cost. 
      
      The removal of safemalloc also allows a simplification of the
      malloc wrappers, removing quite a bit of kludge: redefinition
      of my_malloc, my_free and the removal of the unused second
      argument of my_free. Since free() always check whether the
      supplied pointer is null, redudant checks are also removed.
      
      Also, this patch adds unit testing for my_malloc and moves
      my_realloc implementation into the same file as the other
      memory allocation primitives.
      
      client/mysqldump.c:
        Pass my_free directly as its signature is compatible with the
        callback type -- which wasn't the case for free_table_ent.
      f56dd32b
  22. 01 Jul, 2010 1 commit
    • Jon Olav Hauglid's avatar
      A 5.5 version of the fix for Bug #54360 "Deadlock DROP/ALTER/CREATE · 9ff272fb
      Jon Olav Hauglid authored
      DATABASE with open HANDLER"
      
      Remove LOCK_create_db, database name locks, and use metadata locks instead.
      This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based
      deadlock detector in MDL, and paves the way for a safe, deadlock-free
      implementation of RENAME DATABASE.
      
      Database DDL statements will now take exclusive metadata locks on
      the database name, while table/view/routine DDL statements take
      intention exclusive locks on the database name. This prevents race
      conditions between database DDL and table/view/routine DDL.
      (e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE)
      
      By adding database name locks, this patch implements
      WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and
      WL#4985 "DDL locking: namespace/hierarchical locks".
      
      The patch also changes code to use init_one_table() where appropriate.
      The new lock_table_names() function requires TABLE_LIST::db_length to
      be set correctly, and this is taken care of by init_one_table().
      
      This patch also adds a simple template to help work with 
      the mysys HASH data structure.
      
      Most of the patch was written by Konstantin Osipov.
      9ff272fb
  23. 29 Jun, 2010 1 commit
    • Konstantin Osipov's avatar
      A fix for Bug#54811 "Assert in mysql_lock_have_duplicate()". · 52f361ec
      Konstantin Osipov authored
      Remove mysql_lock_have_duplicate(), since now we always
      have TABLE_LIST objects for MyISAMMRG children
      in lex->query_tables and keep it till the end of the
      statement (sub-statement).
      
      
      mysql-test/r/merge.result:
        Update results (Bug#54811).
      mysql-test/t/merge-big.test:
        Update to new wait state.
      mysql-test/t/merge.test:
        Add a test case for Bug#54811.
      sql/lock.cc:
        Remove a function that is now unused.
      sql/lock.h:
        Remove a function that is now unused.
      sql/sql_base.cc:
        Don't try to search for duplicate table among THR_LOCK objects, TABLE_LIST list contains all used tables.
      52f361ec
  24. 11 Jun, 2010 1 commit
    • Konstantin Osipov's avatar
      WL#5419 "LOCK_open scalability: make tdc_refresh_version · 33c57e2f
      Konstantin Osipov authored
      an atomic counter"
      
      Split the large LOCK_open section in open_table(). 
      Do not call open_table_from_share() under LOCK_open.
      Remove thd->version.
      
      This fixes
      Bug#50589 "Server hang on a query evaluated using a temporary 
      table"
      Bug#51557 "LOCK_open and kernel_mutex are not happy together"
      Bug#49463 "LOCK_table and innodb are not nice when handler 
      instances are created".
      
      This patch has effect on storage engines that rely on
      ha_open() PSEA method being called under LOCK_open.
      In particular:
      
      1) NDB is broken and left unfixed. NDB relies on LOCK_open
      being kept as part of ha_open(), since it uses auto-discovery.
      While previously the NDB open code was race-prone, now
      it simply fails on asserts.
      
      2) HEAP engine had a race in ha_heap::open() when
      a share for the same table could be added twice
      to the list of shares, or a dangling reference to a share
      stored in HEAP handler. This patch aims to address this
      problem by 'pinning' the newly created share in the 
      internal HEAP engine share list until at least one
      handler instance is created using that share.
      
      
      include/heap.h:
        Add members to HP_CREATE_INFO.
        Declare heap_release_share().
      sql/lock.cc:
        Remove thd->version, use thd->open_tables->s->version instead.
      sql/repl_failsafe.cc:
        Remove thd->version.
      sql/sql_base.cc:
        - close_thread_table(): move handler cleanup code outside the critical section protected by LOCK_open.
        - remove thd->version
        - split the large critical section in open_table() that
        opens a new table from share and is protected by LOCK_open
        into 2 critical sections, thus reducing the critical path.
        - make check_if_table_exists() acquire LOCK_open internally.
        - use thd->open_tables->s->version instead of thd->refresh_version to make sure that all tables in
        thd->open_tables are in the same refresh series.
      sql/sql_base.h:
        Add declaration for check_if_table_exists().
      sql/sql_class.cc:
        Remove init_open_tables_state(), it's now equal to
        reset_open_tables_state().
      sql/sql_class.h:
        Remove thd->version, THD::init_open_tables_state().
      sql/sql_plugin.cc:
        Use table->m_needs_reopen to mark the table as stale
        rather than manipulate with thd->version, which is no more.
      sql/sql_udf.cc:
        Use table->m_needs_reopen to mark the table as stale
        rather than manipulate with thd->version, which is no more.
      sql/table.h:
        Remove an unused variable.
      sql/tztime.cc:
        Use table->m_needs_reopen to mark the table as stale
        rather than manipulate with thd->version, which is no more.
      storage/heap/CMakeLists.txt:
        Add heap tests to cmake build files.
      storage/heap/ha_heap.cc:
        Fix a race when ha_heap::ha_open() could insert two 
        HP_SHARE objects into the internal share list or store
        a dangling reference to a share in ha_heap instance,
        or wrongly set implicit_emptied.
      storage/heap/hp_create.c:
        Optionally pin a newly created share in the list of shares
        by increasing its open_count. This is necessary to 
        make sure that a newly created share doesn't disappear while
        a HP_INFO object is being created to reference it.
      storage/heap/hp_open.c:
        When adding a new HP_INFO object to the list of objects
        in the heap share, make sure the open_count is not increased
        twice.
      storage/heap/hp_test1.c:
        Adjust the test to new function signatures.
      storage/heap/hp_test2.c:
        Adjust the test to new function signatures.
      33c57e2f
  25. 28 May, 2010 1 commit
    • Davi Arnaut's avatar
      Backport: Remove unused and ancient files, functions, and facilities. · 11648411
      Davi Arnaut authored
      client/mysql.cc:
        Remove unused functions.
      client/sql_string.cc:
        Remove unused functions.
      include/my_pthread.h:
        Remove unused prototype.
      mysys/my_pthread.c:
        Remove unused function.
      sql/lock.cc:
        Remove unused function.
      sql/lock.h:
        Remove unused and duplicated prototypes.
      sql/sql_class.h:
        Removed unused variables.
      sql/sql_const.h:
        Remove unused defines.
      sql/sql_priv.h:
        Remove unused defines.
      sql/sql_string.cc:
        Remove unused prototype.
      sql/thr_malloc.cc:
        Remove unused function.
      sql/thr_malloc.h:
        Remove unused prototype.
      storage/myisam/CMakeLists.txt:
        Remove obsolete fulltext file.
      storage/myisam/Makefile.am:
        Remove obsolete files (were already commented out).
      storage/myisam/ft_eval.c:
        Remove obsolete fulltext file.
      storage/myisam/ft_eval.h:
        Remove obsolete fulltext file.
      storage/myisam/ft_stem.c:
        Remove obsolete fulltext file.
      storage/myisam/ft_test1.c:
        Remove obsolete fulltext file.
      storage/myisam/ft_test1.h:
        Remove obsolete fulltext file.
      11648411
  26. 26 May, 2010 1 commit
    • Dmitry Lenev's avatar
      Fix for bug #51263 "Deadlock between transactional · ae1ae4bd
      Dmitry Lenev authored
      SELECT and ALTER TABLE ...  REBUILD PARTITION".
      
      ALTER TABLE on InnoDB table (including partitioned tables)
      acquired exclusive locks on rows of table being altered.
      In cases when there was concurrent transaction which did
      locking reads from this table this sometimes led to a
      deadlock which was not detected by MDL subsystem nor by
      InnoDB engine (and was reported only after exceeding
      innodb_lock_wait_timeout).
      
      This problem stemmed from the fact that ALTER TABLE acquired
      TL_WRITE_ALLOW_READ lock on table being altered. This lock
      was interpreted as a write lock and thus for table being
      altered handler::external_lock() method was called with
      F_WRLCK as an argument. As result InnoDB engine treated
      ALTER TABLE as an operation which is going to change data
      and acquired LOCK_X locks on rows being read from old
      version of table.
      
      In case when there was a transaction which already acquired
      SR metadata lock on table and some LOCK_S locks on its rows
      (e.g. by using it in subquery of DML statement) concurrent
      ALTER TABLE was blocked at the moment when it tried to
      acquire LOCK_X lock before reading one of these rows.
      The transaction's attempt to acquire SW metadata lock on
      table being altered led to deadlock, since it had to wait
      for ALTER TABLE to release SNW lock. This deadlock was not
      detected and got resolved only after timeout expiring
      because waiting were happening in two different subsystems.
      
      Similar deadlocks could have occured in other situations.
      This patch tries to solve the problem by changing ALTER TABLE
      implementation to use TL_READ_NO_INSERT lock instead of
      TL_WRITE_ALLOW_READ. After this step handler::external_lock()
      is called with F_RDLCK as an argument and InnoDB engine
      correctly interprets ALTER TABLE as operation which only
      reads data from original version of table. Thanks to this
      ALTER TABLE acquires only LOCK_S locks on rows it reads.
      This, in its turn, causes inter-subsystem deadlocks to go
      away, as all potential lock conflicts and thus deadlocks will
      be limited to metadata locking subsystem:
      
      - When ALTER TABLE reads rows from table being altered it
        can't encounter any locks which conflict with LOCK_S row
        locks. There should be no concurrent transactions holding
        LOCK_X row locks. Such a transaction should have been
        acquired SW metadata lock on table first which would have
        conflicted with ALTER's SNW lock.
      - Vice versa, when DML which runs concurrently with ALTER
        TABLE tries to lock row it should be requesting only LOCK_S
        lock which is compatible with locks acquired by ALTER,
        as otherwise such DML must own an SW metadata lock on table
        which would be incompatible with ALTER's SNW lock.
      
      mysql-test/r/innodb_mysql_lock2.result:
        Added test for bug #51263 "Deadlock between transactional
        SELECT and ALTER TABLE ... REBUILD PARTITION".
      mysql-test/suite/rpl_ndb/r/rpl_ndb_binlog_format_errors.result:
        Since CREATE TRIGGER no longer acquires write lock on table
        it is no longer interpreted as an operation which modifies
        table data and therefore no longer fails if invoked for
        SBR-only engine in ROW mode.
      mysql-test/suite/rpl_ndb/t/rpl_ndb_binlog_format_errors.test:
        Since CREATE TRIGGER no longer acquires write lock on table
        it is no longer interpreted as an operation which modifies
        table data and therefore no longer fails if invoked for
        SBR-only engine in ROW mode.
      mysql-test/t/innodb_mysql_lock2.test:
        Added test for bug #51263 "Deadlock between transactional
        SELECT and ALTER TABLE ... REBUILD PARTITION".
      sql/ha_partition.cc:
        When ALTER TABLE creates a new partition to be filled from
        other partition lock it in F_WRLCK mode instead of using
        mode which was used for locking the whole table (it is
        F_RDLCK now).
      sql/lock.cc:
        Replaced conditions which used TL_WRITE_ALLOW_READ
        lock type with equivalent conditions using
        TL_WRITE_ALLOW_WRITE. This should allow to get rid
        of TL_WRITE_ALLOW_READ lock type eventually.
      sql/mdl.cc:
        Updated outdated comment to reflect current situation.
      sql/sql_base.cc:
        Replaced conditions which used TL_WRITE_ALLOW_READ
        lock type with equivalent conditions using
        TL_WRITE_ALLOW_WRITE. This should allow to get rid
        of TL_WRITE_ALLOW_READ lock type eventually.
      sql/sql_table.cc:
        mysql_admin_table():
          Use TL_WRITE_ALLOW_WRITE lock type instead of
          TL_WRITE_ALLOW_READ to determine that we need to acquire
          upgradable metadata lock. This should allow to completely
          get rid of TL_WRITE_ALLOW_READ in long term.
        mysql_recreate_table():
          ALTER TABLE now requires TL_READ_NO_INSERT thr_lock.c lock
          instead of TL_WRITE_ALLOW_READ.
      sql/sql_trigger.cc:
        Changed CREATE/DROP TRIGGER implementation to use
        TL_READ_NO_INSERT lock instead of TL_WRITE_ALLOW_READ lock.
        The latter is no longer necessary since:
        a) We now can rely on metadata locks to achieve proper
           isolation between two DDL statements or DDL and DML
           statements.
        b) This statement does not change any data in table so there
           is no need to inform storage engine about it.
      sql/sql_yacc.yy:
        Changed implementation of ALTER TABLE (and CREATE/DROP INDEX
        as a consequence) to use TL_READ_NO_INSERT lock instead of
        TL_WRITE_ALLOW_READ lock. This is possible since:
        a) We now can rely on metadata locks to achieve proper
           isolation between two DDL statements or DDL and DML
           statements.
        b) This statement only reads data in table being open.
           We write data only to the new version of table and
           then replace with it old version of table under
           X metadata lock.
        
        Thanks to this change InnoDB will no longer acquire LOCK_X
        locks on rows being read by ALTER TABLE (instead LOCK_S
        locks will be acquired) and thus cause of bug #51263
        "Deadlock between transactional SELECT and ALTER TABLE ...
        REBUILD PARTITION" is removed.
        
        Did the similar change for CREATE TRIGGER (see comments
        for sql_trigger.cc for details).
      ae1ae4bd
  27. 25 May, 2010 1 commit
    • Dmitry Lenev's avatar
      Pre-requisite patch for bug #51263 "Deadlock between · bee0f214
      Dmitry Lenev authored
      transactional SELECT and ALTER TABLE ... REBUILD PARTITION".
      
      The goal of this patch is to decouple type of metadata
      lock acquired for table by open_tables() from type of
      table-level lock to be acquired on it.
      
      To achieve this we change approach to how we determine what
      type of metadata lock should be acquired on table to be open.
      Now instead of inferring it at open_tables() time from flags
      and type of table-level lock we rely on that type of metadata
      lock is properly set at parsing time and is not changed
      further.
      
      sql/ha_ndbcluster.cc:
        Now one needs to properly initialize table list element's
        MDL_request object before calling mysql_rm_table_part2().
      sql/lock.cc:
        lock_table_names() no longer initializes table list elements'
        MDL_request objects. Now proper initialization of these
        requests is a responsibility of the caller.
      sql/lock.h:
        Removed MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag which became
        unnecessary. Thanks to the fact that we don't reset type of
        requests for metadata locks between re-executions we now can
        figure out that upgradable locks are requested by simply
        looking at their type which were set in the parser. As result
        this flag became redundant.
      sql/mdl.h:
        Added version of new operator which simplifies allocation of
        MDL_request objects on a MEM_ROOT.
      sql/sp_head.cc:
        Added comment explaining why it is OK to infer type of
        metadata lock to request from type of table-level lock
        for prelocking.
        Added enum_mdl_type argument to sp_add_to_query_tables()
        to simplify its usage in trigger implementation.
      sql/sp_head.h:
        Added enum_mdl_type argument to sp_add_to_query_tables()
        to simplify its usage in trigger implementation.
      sql/sql_base.cc:
        - open_table_get_mdl_lock():
          Preserve type of MDL_request for table list element which
          was set in the parser by creating MDL_request objects on
          memory root if MYSQL_OPEN_FORCE_SHARED_MDL or
          MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag were specified.
          Thanks to this and to the fact that we no longer reset
          type of requests for metadata locks between re-executions
          we no longer need to acquire exclusive metadata lock on
          table to be created in a special way. This lock is acquired
          by code handling acquiring of upgradable locks.
          Also changed signature/calling convention for this function
          to simplify its usage.
        - Accordingly special lock strategy for table list elements
          which was used for such locks became unnecessary and was
          removed. Other strategies were renamed.
        - Since we no longer have guarantee that MDL_request object
          which were not satisfied due to lock conflict belongs to
          table list element Open_table_context class and its methods
          were extended to remember pointer to MDL_request which has
          caused problem at request_backoff_action() time and use it
          in recover_from_failed_open(). Similar approach is used
          for cases when problem from which we need to recover is
          not related to MDL but to the table itself. In this case
          we store pointer to the element of table list.
        - Changed open_tables()/open_tables_check_upgradable_mdl()/
          open_tables_acquire_upgradable_mdl() not to rely on
          MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to understand when
          upgradable metadata locks should be acquired and not to
          infer type of MDL lock from type of table-level lock.
          Instead we assume that type of MDL to be acquired was set
          in the parser (we can do this as type of MDL_request is
          no longer reset between re-executions).
      sql/sql_class.h:
        Since we no longer have guarantee that MDL_request object
        which were not satisfied due to lock conflict belongs to
        table list element Open_table_context class and its methods
        were extended to remember pointer to MDL_request which has
        caused problem at request_backoff_action() time and use it
        in recover_from_failed_open(). Similar approach is used
        for cases when problem from which we need to recover is
        not related to MDL but to the table itself. In this case
        we store pointer to the element of table list.
      sql/sql_db.cc:
        Now one needs to properly initialize table list element's
        MDL_request object before calling mysql_rm_table_part2()
        or mysql_rename_tables().
      sql/sql_lex.cc:
        st_select_lex/st_select_lex_node::add_table_to_list() method
        now has argument which allows specify type of metadata lock
        to be requested for table list element being added.
      sql/sql_lex.h:
        - st_select_lex/st_select_lex_node::add_table_to_list()
          method now has argument which specifies type of metadata
          lock to be requested for table list element being added.
          This allows to explicitly set type of MDL lock to be
          acquired for a DDL statement in parser. It is also more
          future-proof than inferring type of MDL request from type
          of table-level lock.
        - Added Yacc_state::m_mdl_type member which specifies which
          type of metadata lock should be requested for tables to be
          added to table list by a grammar rule in cases when the same
          rule is used in several statements requiring different kinds
          of metadata locks.
      sql/sql_parse.cc:
        - st_select_lex::add_table_to_list() method now has argument
          which specifies type of metadata lock to be requested for
          table list element being added. This allows to explicitly
          set type of MDL lock to be acquired for a DDL statement in
          parser. It is also more future-proof than inferring type of
          MDL request from type of table-level lock.
        - EXCLUSIVE_DOWNGRADABLE_MDL lock strategy has a new name -
          OTLS_DOWNGRADE_IF_EXISTS.
        - Adjusted LOCK TABLES implementation to the fact that we no
          longer infer type of metadata lock to be acquired from table
          level lock and that type of MDL request is set at parsing.
          And thus MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag became
          unnecessary.
      sql/sql_prepare.cc:
        TABLE_LIST's lock strategy SHARED_MDL was renamed to OTLS_NONE
        as now it means that metadata lock should not be changed during
        call to open_table() (if it has been already acquired) and is
        also used for exclusive metadata lock.
      sql/sql_show.cc:
        st_select_lex::add_table_to_list() method now has argument
        which specifies type of metadata lock to be requested for
        table list element being added.
      sql/sql_table.cc:
        - Adjusted mysql_admin_table()'s code to the fact that
          open_tables() no longer determines what kind of metadata
          lock should be obtained basing on type of table-level
          lock and flags. Instead type of metadata lock for table
          to be open should be set before calling open_tables().
        - Changed mysql_alter_table() code to the facts:
          a) that now it is responsibility of caller to properly
          initalize MDL_request in table list elements before calling
          lock_table_names()
          b) and that MYSQL_OPEN_TAKE_UPGRADABLE_MDL is no longer
          necessary since type of metadata lock to be obtained
          at open_tables() time is set during parsing.
        - Changed code of mysql_recreate_table() to properly set
          type of metadata and table-level lock to be obtained
          by mysql_alter_table() which it calls.
      sql/sql_trigger.cc:
        Instead of relying on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to
        force open_tables() to take an upgradable lock we now specify
        exact type of lock to be taken when constructing table list
        element for table to be open for CREATE/DROP TRIGGER.
      sql/sql_view.cc:
        We no longer use TABLE_LIST::EXCLUSIVE_MDL strategy to force
        open_tables() to take an exclusive metadata lock on view to
        be created. Instead we rely on parser setting proper type of
        metadata lock to request and open_tables() acquiring it.
        This became possible thanks to the fact that we no longer
        reset type of MDL_request between statement re-executions.
      sql/sql_yacc.yy:
        Instead of inferring type of MDL_request for table to be
        open from type of table-level lock and flags passed to
        open_tables() we now explicitly specify them at parsing.
        This became possible thanks to the fact that we no longer
        reset type of MDL_request between statement re-executions.
        In future this should allow to decouple type of metadata
        lock from type of table-level lock.
        The only exception to this approach is statements implemented
        through mysql_admin_table() which re-uses same table list
        element several times with different types of table-level
        and metadata locks.
        We now also properly initialize MDL_request objects for table
        list elements which are later passed to lock_table_names()
        function.
      sql/table.cc:
        Do not reset type of MDL_request between statement
        re-executions. This became unnecessesary as we no longer
        change type of MDL_request residing in table list element.
        In its turn this change allows to set type of MDL_request
        only once - at parsing time.
      sql/table.h:
        Got rid of TABLE_LIST::EXCLUSIVE_MDL lock strategy.
        Now we can specify that we need to acquire exclusive lock
        on table to be processed by open_tables() through setting
        an appropriate type of MDL_request at parsing time (this
        became possible thanks to the fact that we no longer reset
        types of MDL_request's belonging to table list elements
        between statement re-execution).
        Strategy SHARED_MDL was renamed to OTLS_NONE as now it
        means that metadata lock should not be changed during call
        to open_table() (if it has been already acquired) and is
        also used for exclusive metadata lock.
        Strategy EXCLUSIVE_DOWNGRADABLE_MDL was renamed to
        OTLS_DOWNGRADE_IF_EXISTS.
      bee0f214
  28. 31 Mar, 2010 1 commit
    • Mats Kindahl's avatar
      WL#5030: Split and remove mysql_priv.h · 23d8586d
      Mats Kindahl authored
      This patch:
      
      - Moves all definitions from the mysql_priv.h file into
        header files for the component where the variable is
        defined
      - Creates header files if the component lacks one
      - Eliminates all include directives from mysql_priv.h
      - Eliminates all circular include cycles
      - Rename time.cc to sql_time.cc
      - Rename mysql_priv.h to sql_priv.h
      23d8586d
  29. 13 Mar, 2010 1 commit
  30. 10 Mar, 2010 1 commit
    • Davi Arnaut's avatar
      Bug#33669: Transactional temporary tables do not work under --read-only · b1174089
      Davi Arnaut authored
      The problem was that in read only mode (read_only enabled),
      the server would mistakenly deny data modification attempts
      for temporary tables which belong to a transactional storage
      engine (eg. InnoDB).
      
      The solution is to allow transactional temporary tables to be
      modified under read only mode. As a whole, the read only mode
      does not apply to any kind of temporary table.
      
      
      mysql-test/r/read_only_innodb.result:
        Add test case result for Bug#33669
      mysql-test/t/read_only_innodb.test:
        Add test case for Bug#33669
      sql/lock.cc:
        Rename mysql_lock_tables_check to lock_tables_check and make
        it static. Move locking related checks from get_lock_data to
        lock_tables_check. Allow write locks to temporary tables even
        under read-only.
      b1174089
  31. 24 Feb, 2010 1 commit
    • Jon Olav Hauglid's avatar
      Followup to Bug#45225 Locking: hang if drop table with no timeout · e60ef893
      Jon Olav Hauglid authored
      This patch prevents system threads and system table accesses from
      using user-specified values for "lock_wait_timeout". Instead all
      such accesses are done using the default value (1 year).
      
      This prevents background tasks (such as replication, events, 
      accessing stored function definitions, logging, reading time-zone
      information, etc.) from failing in cases where the global value
      of "lock_wait_timeout" is set very low.
      
      The patch also simplifies the open tables API. Rather than adding
      another convenience function for opening and locking system tables,
      this patch removes most of the existing convenience functions for
      open_and_lock_tables_derived(). Before, open_and_lock_tables() was
      a convenience function that enforced derived tables handling, while
      open_and_lock_tables_derived() was the main function where derived
      tables handling was optional. Now, this convencience function is
      gone and the main function is renamed to open_and_lock_tables(). 
      
      No test case added as it would have required the use of --sleep to
      check that system threads and system tables have a different timeout
      value from the user-specified "lock_wait_timeout" system variable.
      e60ef893
  32. 11 Feb, 2010 1 commit
    • Jon Olav Hauglid's avatar
      Bug #45225 Locking: hang if drop table with no timeout · 3d6a89e7
      Jon Olav Hauglid authored
      This patch introduces timeouts for metadata locks. 
      
      The timeout is specified in seconds using the new dynamic system 
      variable  "lock_wait_timeout" which has both GLOBAL and SESSION
      scopes. Allowed values range from 1 to 31536000 seconds (= 1 year). 
      The default value is 1 year.
      
      The new server parameter "lock-wait-timeout" can be used to set
      the default value parameter upon server startup.
      
      "lock_wait_timeout" applies to all statements that use metadata locks.
      These include DML and DDL operations on tables, views, stored procedures
      and stored functions. They also include LOCK TABLES, FLUSH TABLES WITH
      READ LOCK and HANDLER statements.
      
      The patch also changes thr_lock.c code (table data locks used by MyISAM
      and other simplistic engines) to use the same system variable.
      InnoDB row locks are unaffected.
      
      One exception to the handling of the "lock_wait_timeout" variable
      is delayed inserts. All delayed inserts are executed with a timeout
      of 1 year regardless of the setting for the global variable. As the
      connection issuing the delayed insert gets no notification of 
      delayed insert timeouts, we want to avoid unnecessary timeouts.
      
      It's important to note that the timeout value is used for each lock
      acquired and that one statement can take more than one lock.
      A statement can therefore block for longer than the lock_wait_timeout 
      value before reporting a timeout error. When lock timeout occurs, 
      ER_LOCK_WAIT_TIMEOUT is reported.
      
      Test case added to lock_multi.test.
      
      
      include/my_pthread.h:
        Added macros for comparing two timespec structs.
      include/thr_lock.h:
        Introduced timeouts for thr_lock.c locks.
      mysql-test/r/mysqld--help-notwin.result:
        Updated result file with the new server variable.
      mysql-test/r/mysqld--help-win.result:
        Updated result file with the new server variable.
      mysql-test/suite/sys_vars/r/lock_wait_timeout_basic.result:
        Added basic test for the new server variable.
      mysql-test/suite/sys_vars/t/lock_wait_timeout_basic.test:
        Added basic test for the new server variable.
      mysys/thr_lock.c:
        Introduced timeouts for thr_lock.c locks.
      sql/mdl.cc:
        Introduced timeouts for metadata locks.
      sql/mdl.h:
        Introduced timeouts for metadata locks.
      sql/sql_base.cc:
        Introduced timeouts in tdc_wait_for_old_versions().
      sql/sql_class.h:
        Added new server variable lock_wait_timeout.
      sql/sys_vars.cc:
        Added new server variable lock_wait_timeout.
      3d6a89e7
  33. 10 Feb, 2010 1 commit
    • Michael Widenius's avatar
      When one does a drop table, the indexes are not flushed to disk before drop... · d77e3cde
      Michael Widenius authored
      When one does a drop table, the indexes are not flushed to disk before drop anymore (with MyISAM/Maria)
      myisam-recover options changed from OFF to 'DEFAULT' to get less change of data loss when using MyISAM.
      (The disadvantage is that changed MyISAM tables will be checked at access time; Use --myisam-recover=OFF for old behavior)
      Don't call extra(HA_EXTRA_FORCE_REOPEN) in ALTER TABLE if table is locked as this will mark table as crashed!
      Added assert to detect if we accidently would use MyISAM versioning in MySQL
      
      include/my_base.h:
        Mark NOT_USED as USED, as we now use this as a flag to not call extra()
      mysql-test/mysql-test-run.pl:
        Don't write all options when there is something wrong with the arguments
      mysql-test/r/sp-destruct.result:
        Add missing flush of mysql.proc (as the test copied live tables)
      mysql-test/r/variables.result:
        myisam-recover options changed to 'default'
      mysql-test/r/view.result:
        Don't show create time in result
      mysql-test/suite/maria/t/maria-recovery2-master.opt:
        Don't run test with myisam-recover (as this produces extra warnings during simulated death)
      mysql-test/t/sp-destruct.test:
        Add missing flush of mysql.proc (as the test copied live tables)
      mysql-test/t/view.test:
        Don't show create time in result
      sql/lock.cc:
        Added marker if table was deleted to argument list
      sql/mysql_priv.h:
        Added marker if table was deleted to argument list
      sql/mysqld.cc:
        myisam-recover options changed from OFF to 'DEFAULT' to get less change of data loss when using MyISAM
        Allow one to specify OFF as argument to myisam-recover (was default before but one couldn't specify it)
      sql/sql_base.cc:
        Mark if table is going to be deleted
      sql/sql_delete.cc:
        Mark if table is going to be deleted
      sql/sql_table.cc:
        Mark if table is going to be deleted
        Don't call extra(HA_EXTRA_FORCE_REOPEN) in ALTER TABLE if table is locked as this will mark table as crashed!
      sql/table.cc:
        Signal to handler if table is getting deleted as part of getting droped from table cache.
      sql/table.h:
        Added marker if table is going to be deleted.
      storage/maria/ha_maria.cc:
        Don't search for transaction handler if file is not transactional or outside of transaction
        (Fixed possible core dump)
      storage/maria/ma_blockrec.c:
        Don't write changed information if table is going to be deleted.
      storage/maria/ma_close.c:
        Don't write changed information if table is going to be deleted.
      storage/maria/ma_extra.c:
        Mark tables that are deleted as crased, to ensure good behavior on restart if we suddenly crash.
      storage/maria/ma_locking.c:
        Cleanup
      storage/maria/ma_recovery.c:
        We need trnman to be inited during redo phase (to be able to open tables checked with maria_chk)
      storage/maria/maria_def.h:
        Added marker if table is going to be deleted.
      storage/myisam/mi_close.c:
        Don't write changed information if table is going to be deleted.
      storage/myisam/mi_extra.c:
        Mark tables that are deleted as crased, to ensure good behavior on restart if we suddenly crash.
      storage/myisam/mi_open.c:
        Added assert to detect if we accidently would use MyISAM versioning in MySQL
      storage/myisam/myisamdef.h:
        Added marker if table is going to be deleted.
      d77e3cde
  34. 08 Feb, 2010 1 commit
    • Dmitry Lenev's avatar
      Fix for bug #50913 "Deadlock between open_and_lock_tables_derived · 8018ec5a
      Dmitry Lenev authored
      and MDL".
      
      Concurrent execution of a multi-DELETE statement and ALTER
      TABLE statement which affected one of the tables used in
      the multi-DELETE sometimes led to deadlock.
      Similar deadlocks might have occured when one performed
      INSERT/UPDATE/DELETE on a view and concurrently executed
      ALTER TABLE for the view's underlying table, or when one
      concurrently executed TRUNCATE TABLE for InnoDB table and
      ALTER TABLE for the same table.
      
      These deadlocks were caused by a discrepancy between types of
      metadata and thr_lock.cc locks acquired by those statements.
      
      What happened was that multi-DELETE/TRUNCATE/DML-through-the-
      view statement in the first connection acquired SR lock on a
      table, then ALTER TABLE would come in in the second connection
      and acquire SNW metadata lock and TL_WRITE_ALLOW_READ
      thr_lock.c lock and then would start waiting for the first
      connection during lock upgrade. After that the statement in
      the first connection would try to acquire TL_WRITE lock on
      table and would start waiting for the second connection,
      creating a deadlock.
      
      This patch solves this problem by ensuring that we acquire
      SW metadata lock in all cases in which we acquiring write
      thr_lock.c lock. This guarantees that deadlocks like the
      one described above won't occur since all lock conflicts
      in such situation are resolved within MDL subsystem.
      
      This patch also adds assert which should guarantee that
      such situations won't arise in future.
      
      mysql-test/r/lock_multi.result:
        Added main test for bug #50913 "Deadlock between
        open_and_lock_tables_derived and MDL".
      mysql-test/r/mdl_sync.result:
        Added additional coverage for bug #50913 "Deadlock
        between open_and_lock_tables_derived and MDL".
      mysql-test/t/lock_multi.test:
        Added main test for bug #50913 "Deadlock between
        open_and_lock_tables_derived and MDL".
      mysql-test/t/mdl_sync.test:
        Added additional coverage for bug #50913 "Deadlock
        between open_and_lock_tables_derived and MDL".
      sql/lock.cc:
        Added assert that enforces that when we are locking
        a non-temporary table we have an appropriate type of
        metadata lock on this table.
      sql/mysql_priv.h:
        Added separate flag for open_tables() to be able specify that
        SH metadata locks on table to be open should be acquired.
        We can no longer use MYSQL_LOCK_IGNORE_FLUSH flag for this
        as in addition to use in I_S implementation it is also used
        for opening system tables. Since in the latter case we also
        acquire thr_lock.c locks using SH metadata lock in it instead
        of SR or SW locks may lead to deadlock.
      sql/sql_base.cc:
        When opening tables don't interpret MYSQL_LOCK_IGNORE_FLUSH
        flag as request to acquire SH metadata locks. This flag is
        also used for opening system tables for which we also take
        thr_lock.c locks and thus proper metadata lock to take in
        this case is SR or SW lock (otherwise deadlocks can occur).
        In cases when SH lock is really required (e.g. when tables
        are open by I_S implementation) we rely on that newly
        introduced MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag is
        used.
      sql/sql_delete.cc:
        mysql_truncate_by_delete():
          Adjust type of metadata lock to be requested after changing
          type of thr_lock.c lock for table list element from one
          which was set in parser to TL_WRITE.
          This removes discrepancy between types of these locks which
          allowed deadlocks to creep in.
      sql/sql_handler.cc:
        When closing table which was open by HANDLER statement clear
        TABLE::open_by_handler flag. This allows to use this flag as
        a reliable indication that TABLE instance was open by HANDLER
        statement in assert which was added to mysql_lock_tables().
      sql/sql_parse.cc:
        multi_delete_set_locks_and_link_aux_tables():
          Adjust type of metadata lock to be requested after changing
          type of thr_lock.c lock for table list element from one
          which was set in parser to TL_WRITE.
          This removes discrepancy between types of these locks which
          allowed deadlocks to creep in.
      sql/sql_show.cc:
        Use newly introduced MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL
        flag in order to acquire SH metadata locks when opening tables
        in I_S implementation.
      sql/sql_update.cc:
        Added comment explaining why in multi-update after deciding
        that we need weaker thr_lock.c lock on a table we don't
        downgrade metadata lock on it.
      sql/sql_view.cc:
        When merging view into main statement adjust type of metadata
        lock to be requested after changing type of thr_lock.c lock
        for table. This removes discrepancy between types of these
        locks which allowed deadlocks to creep in.
      8018ec5a
  35. 01 Feb, 2010 1 commit
    • Dmitry Lenev's avatar
      Implement new type-of-operation-aware metadata locks. · eba5d30e
      Dmitry Lenev authored
      Add a wait-for graph based deadlock detector to the
      MDL subsystem.
      
      Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and
      bug #37346 "innodb does not detect deadlock between update and
      alter table".
      
      The first bug manifested itself as an unwarranted abort of a
      transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER
      statement, when this transaction tried to repeat use of a
      table, which it has already used in a similar fashion before
      ALTER started.
      
      The second bug showed up as a deadlock between table-level
      locks and InnoDB row locks, which was "detected" only after
      innodb_lock_wait_timeout timeout.
      
      A transaction would start using the table and modify a few
      rows.
      Then ALTER TABLE would come in, and start copying rows
      into a temporary table. Eventually it would stumble on
      the modified records and get blocked on a row lock.
      The first transaction would try to do more updates, and get
      blocked on thr_lock.c lock.
      This situation of circular wait would only get resolved
      by a timeout.
      
      Both these bugs stemmed from inadequate solutions to the
      problem of deadlocks occurring between different
      locking subsystems.
      
      In the first case we tried to avoid deadlocks between metadata
      locking and table-level locking subsystems, when upgrading shared
      metadata lock to exclusive one.
      Transactions holding the shared lock on the table and waiting for
      some table-level lock used to be aborted too aggressively.
      
      We also allowed ALTER TABLE to start in presence of transactions
      that modify the subject table. ALTER TABLE acquires
      TL_WRITE_ALLOW_READ lock at start, and that block all writes
      against the table (naturally, we don't want any writes to be lost
      when switching the old and the new table). TL_WRITE_ALLOW_READ
      lock, in turn, would block the started transaction on thr_lock.c
      lock, should they do more updates. This, again, lead to the need
      to abort such transactions.
      
      The second bug occurred simply because we didn't have any
      mechanism to detect deadlocks between the table-level locks
      in thr_lock.c and row-level locks in InnoDB, other than
      innodb_lock_wait_timeout.
      
      This patch solves both these problems by moving lock conflicts
      which are causing these deadlocks into the metadata locking
      subsystem, thus making it possible to avoid or detect such
      deadlocks inside MDL.
      
      To do this we introduce new type-of-operation-aware metadata
      locks, which allow MDL subsystem to know not only the fact that
      transaction has used or is going to use some object but also what
      kind of operation it has carried out or going to carry out on the
      object.
      
      This, along with the addition of a special kind of upgradable
      metadata lock, allows ALTER TABLE to wait until all
      transactions which has updated the table to go away.
      This solves the second issue.
      Another special type of upgradable metadata lock is acquired
      by LOCK TABLE WRITE. This second lock type allows to solve the
      first issue, since abortion of table-level locks in event of
      DDL under LOCK TABLES becomes also unnecessary.
      
      Below follows the list of incompatible changes introduced by
      this patch:
      
      - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those
        statements that acquire TL_WRITE_ALLOW_READ lock)
        wait for all transactions which has *updated* the table to
        complete.
      
      - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE
        (i.e. all statements which acquire TL_WRITE table-level lock) wait
        for all transaction which *updated or read* from the table
        to complete.
        As a consequence, innodb_table_locks=0 option no longer applies
        to LOCK TABLES ... WRITE.
      
      - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort
        statements or transactions which use tables being dropped or
        renamed, and instead wait for these transactions to complete.
      
      - Since LOCK TABLES WRITE now takes a special metadata lock,
        not compatible with with reads or writes against the subject table
        and transaction-wide, thr_lock.c deadlock avoidance algorithm
        that used to ensure absence of deadlocks between LOCK TABLES
        WRITE and other statements is no longer sufficient, even for
        MyISAM. The wait-for graph based deadlock detector of MDL
        subsystem may sometimes be necessary and is involved. This may
        lead to ER_LOCK_DEADLOCK error produced for multi-statement
        transactions even if these only use MyISAM:
      
        session 1:         session 2:
        begin;
      
        update t1 ...      lock table t2 write, t1 write;
                           -- gets a lock on t2, blocks on t1
      
        update t2 ...
        (ER_LOCK_DEADLOCK)
      
      - Finally,  support of LOW_PRIORITY option for LOCK TABLES ... WRITE
        was abandoned.
        LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same
        priority as the usual LOCK TABLE ... WRITE.
        SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE  in
        the wait queue.
      
      - We do not take upgradable metadata locks on implicitly
        locked tables. So if one has, say, a view v1 that uses
        table t1, and issues:
        LOCK TABLE v1 WRITE;
        FLUSH TABLE t1; -- (or just 'FLUSH TABLES'),
        an error is produced.
        In order to be able to perform DDL on a table under LOCK TABLES,
        the table must be locked explicitly in the LOCK TABLES list.
      
      mysql-test/include/handler.inc:
        Adjusted test case to trigger an execution path on which bug 41110
        "crash with handler command when used concurrently with alter
        table" and bug 41112 "crash in mysql_ha_close_table/get_lock_data
        with alter table" were originally discovered. Left old test case
        which no longer triggers this execution path for the sake of
        coverage.
        Added test coverage for HANDLER SQL statements and type-aware
        metadata locks.
        Added a test for the global shared lock and HANDLER SQL.
        Updated tests to take into account that the old simple deadlock
        detection heuristics was replaced with a graph-based deadlock
        detector.
      mysql-test/r/debug_sync.result:
        Updated results (see debug_sync.test).
      mysql-test/r/handler_innodb.result:
        Updated results (see handler.inc test).
      mysql-test/r/handler_myisam.result:
        Updated results (see handler.inc test).
      mysql-test/r/innodb-lock.result:
        Updated results (see innodb-lock.test).
      mysql-test/r/innodb_mysql_lock.result:
        Updated results (see innodb_mysql_lock.test).
      mysql-test/r/lock.result:
        Updated results (see lock.test).
      mysql-test/r/lock_multi.result:
        Updated results (see lock_multi.test).
      mysql-test/r/lock_sync.result:
        Updated results (see lock_sync.test).
      mysql-test/r/mdl_sync.result:
        Updated results (see mdl_sync.test).
      mysql-test/r/sp-threads.result:
        SHOW PROCESSLIST output has changed due to the fact that waiting
        for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/r/truncate_coverage.result:
        Updated results (see truncate_coverage.test).
      mysql-test/suite/funcs_1/datadict/processlist_val.inc:
        SELECT FROM I_S.PROCESSLIST output has changed due to fact that
        waiting for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
        SELECT FROM I_S.PROCESSLIST output has changed due to fact that
        waiting for LOCK TABLES WRITE now happens within metadata locking
        subsystem.
      mysql-test/suite/rpl/t/rpl_sp.test:
        Updated to a new SHOW PROCESSLIST state name.
      mysql-test/t/debug_sync.test:
        Use LOCK TABLES READ instead of LOCK TABLES WRITE as the latter
        no longer allows to trigger execution path involving waiting on
        thr_lock.c lock and therefore reaching debug sync-point covered
        by this test.
      mysql-test/t/innodb-lock.test:
        Adjusted test case to the fact that innodb_table_locks=0 option is
        no longer supported, since LOCK TABLES WRITE handles all its
        conflicts within MDL subsystem.
      mysql-test/t/innodb_mysql_lock.test:
        Added test for bug #37346 "innodb does not detect deadlock between
        update and alter table".
      mysql-test/t/lock.test:
        Added test coverage which checks the fact that we no longer support
        DDL under LOCK TABLES on tables which were locked implicitly.
        Adjusted existing test cases accordingly.
      mysql-test/t/lock_multi.test:
        Added test for bug #46272 "MySQL 5.4.4, new MDL: unnecessary
        deadlock".  Adjusted other test cases to take into account the
        fact that waiting for LOCK TABLES ... WRITE now happens within MDL
        subsystem.
      mysql-test/t/lock_sync.test:
        Since LOCK TABLES ... WRITE now takes SNRW metadata lock for
        tables locked explicitly we have to implicitly lock InnoDB tables
        (through view) to trigger the table-level lock conflict between
        TL_WRITE and TL_WRITE_ALLOW_WRITE.
      mysql-test/t/mdl_sync.test:
        Added basic test coverage for type-of-operation-aware metadata
        locks. Also covered with tests some use cases involving HANDLER
        statements in which a deadlock could arise.
        Adjusted existing tests to take type-of-operation-aware MDL into
        account.
      mysql-test/t/multi_update.test:
        Update to a new SHOW PROCESSLIST state name.
      mysql-test/t/truncate_coverage.test:
        Adjusted test case after making LOCK TABLES WRITE to wait until
        transactions that use the table to be locked are completed.
        Updated to the changed name of DEBUG_SYNC point.
      sql/handler.cc:
        Global read lock functionality has been
        moved into a class.
      sql/lock.cc:
        Global read lock functionality has been
        moved into a class.
        Updated code to use the new MDL API.
      sql/mdl.cc:
        Introduced new type-of-operation aware metadata locks.
        To do this:
        - Changed MDL_lock to use one list for waiting requests and one
          list for granted requests. For each list, added a bitmap
          that holds information what lock types a list contains.
          Added a helper class MDL_lock::List to manipulate with granted
          and waited lists while keeping the bitmaps in sync
          with list contents.
        - Changed lock-compatibility functions to use bitmaps that
          define compatibility.
        - Introduced a graph based deadlock detector inspired by
          waiting_threads.c from Maria implementation.
        - Now that we have a deadlock detector, and no longer have
          a global lock to protect individual lock objects, but rather
          use an rw lock per object, removed redundant code for upgrade,
          and the global read lock. Changed the MDL API to
          no longer require the caller to acquire the global
          intention exclusive lock by means of a separate method.
          Removed a few more methods that became redundant.
        - Removed deadlock detection heuristic, it has been made
          obsolete by the deadlock detector.
        - With operation-type-aware metadata locks, MDL subsystem has
          become aware of potential conflicts between DDL and open
          transactions. This made it possible to remove calls to
          mysql_abort_transactions_with_shared_lock() from acquisition
          paths for exclusive lock and lock upgrade. Now we can simply
          wait for these transactions to complete without fear of
          deadlock. Function mysql_lock_abort() has also become
          unnecessary for all conflicting cases except when a DDL
          conflicts with a connection that has an open HANDLER.
      sql/mdl.h:
        Introduced new type-of-operation aware metadata locks.
        Introduced a graph based deadlock detector and supporting
        methods.
        Added comments.
        God rid of redundant API calls.
        Renamed m_lt_or_ha_sentinel to m_trans_sentinel,
        since now it guards the global read lock as well as
        LOCK TABLES and HANDLER locks.
      sql/mysql_priv.h:
        Moved the global read lock functionality into a
        class.
        Added MYSQL_OPEN_FORCE_SHARED_MDL flag which forces
        open_tables() to take MDL_SHARED on tables instead of
        metadata locks specified in the parser. We use this to
        allow PREPARE run concurrently in presence of
        LOCK TABLES ... WRITE.
        Added signature for find_table_for_mdl_ugprade().
      sql/set_var.cc:
        Global read lock functionality has been
        moved into a class.
      sql/sp_head.cc:
        When creating TABLE_LIST elements for prelocking or
        system tables set the type of request for metadata
        lock according to the operation that will be performed
        on the table.
      sql/sql_base.cc:
        - Updated code to use the new MDL API.
        - In order to avoid locks starvation we take upgradable
          locks all at once. As result implicitly locked tables no
          longer get an upgradable lock. Consequently DDL and FLUSH
          TABLES for such tables is prohibited.
          find_write_locked_table() was replaced by
          find_table_for_mdl_upgrade() function.
          open_table() was adjusted to return TABLE instance with
          upgradable ticket when necessary.
        - We no longer wait for all locks on OT_WAIT back off
          action -- only on the lock that caused the wait
          conflict. Moreover, now we distinguish cases when we
          have to wait due to conflict in MDL and old version
          of table in TDC.
        - Upate mysql_notify_threads_having_share_locks()
          to only abort thr_lock.c waits of threads that
          have open HANDLERs, since lock conflicts with only
          these threads now can lead to deadlocks not detectable
          by the MDL deadlock detector.
        - Remove mysql_abort_transactions_with_shared_locks()
          which is no longer needed.
      sql/sql_class.cc:
        Global read lock functionality has been moved into a class.
        Re-arranged code in THD::cleanup() to simplify assert.
      sql/sql_class.h:
        Introduced class to incapsulate global read lock
        functionality.
        Now sentinel in MDL subsystem guards the global read lock
        as well as LOCK TABLES and HANDLER locks. Adjusted code
        accordingly.
      sql/sql_db.cc:
        Global read lock functionality has been moved into a class.
      sql/sql_delete.cc:
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result
        TRUNCATE TABLE is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_handler.cc:
        Inform MDL_context about presence of open HANDLERs.
        Since HANLDERs break MDL protocol by acquiring table-level
        lock while holding only S metadata lock on a table MDL
        subsystem should take special care about such contexts (Now
        this is the only case when mysql_lock_abort() is used).
      sql/sql_parse.cc:
        Global read lock functionality has been moved into a class.
        Do not take upgradable metadata locks when opening tables
        for CREATE TABLE SELECT as it is not necessary and limits
        concurrency.
        When initializing TABLE_LIST objects before adding them
        to the table list set the type of request for metadata lock
        according to the operation that will be performed on the
        table.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result FLUSH
        TABLES is no longer allowed for such tables.
      sql/sql_prepare.cc:
        Use MYSQL_OPEN_FORCE_SHARED_MDL flag when opening
        tables during PREPARE. This allows PREPARE to run
        concurrently in presence of LOCK TABLES ... WRITE.
      sql/sql_rename.cc:
        Global read lock functionality has been moved into a class.
      sql/sql_show.cc:
        Updated code to use the new MDL API.
      sql/sql_table.cc:
        Global read lock functionality has been moved into a class.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result DROP
        TABLE is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_trigger.cc:
        Global read lock functionality has been moved into a class.
        We no longer acquire upgradable metadata locks on tables
        which are locked by LOCK TABLES implicitly. As result
        CREATE/DROP TRIGGER is no longer allowed for such tables.
        Updated code to use the new MDL API.
      sql/sql_view.cc:
        Global read lock functionality has been moved into a class.
        Fixed results of wrong merge that led to misuse of GLR API.
        CREATE VIEW statement is not a commit statement.
      sql/table.cc:
        When resetting TABLE_LIST objects for PS or SP re-execution
        set the type of request for metadata lock according to the
        operation that will be performed on the table. Do the same
        in auxiliary function initializing metadata lock requests
        in a table list.
      sql/table.h:
        When initializing TABLE_LIST objects set the type of request
        for metadata lock according to the operation that will be
        performed on the table.
      sql/transaction.cc:
        Global read lock functionality has been moved into a class.
      eba5d30e
  36. 21 Jan, 2010 1 commit
    • Dmitry Lenev's avatar
      Patch that changes metadata locking subsystem to use mutex per lock and · 6ddd01c2
      Dmitry Lenev authored
      condition variable per context instead of one mutex and one conditional
      variable for the whole subsystem.
      
      This should increase concurrency in this subsystem.
      
      It also opens the way for further changes which are necessary to solve
      such bugs as bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock"
      and bug #37346 "innodb does not detect deadlock between update and alter
      table".
      
      Two other notable changes done by this patch:
      
      - MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object metadata lock is acquired. Now this has
        to be done by explicit calls outside of MDL subsystem.
      - Instead of using separate MDL_context for opening system tables/tables
        for purposes of I_S we now create MDL savepoint in the main context
        before opening tables and rollback to this savepoint after closing
        them. This means that it is now possible to get ER_LOCK_DEADLOCK error
        even not inside a transaction. This might happen in unlikely case when
        one runs DDL on one of system tables while also running DDL on some
        other tables. Cases when this ER_LOCK_DEADLOCK error is not justified
        will be addressed by advanced deadlock detector for MDL subsystem which
        we plan to implement.
      
      mysql-test/include/handler.inc:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/handler_innodb.result:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/handler_myisam.result:
        Adjusted handler_myisam.test and handler_innodb.test to the fact that
        exclusive metadata locks on tables are now acquired according to
        alphabetical order of fully qualified table names instead of order
        in which tables are mentioned in statement.
      mysql-test/r/mdl_sync.result:
        Adjusted mdl_sync.test to the fact that exclusive metadata locks on
        tables are now acquired according to alphabetical order of fully
        qualified table names instead of order in which tables are mentioned
        in statement.
      mysql-test/t/mdl_sync.test:
        Adjusted mdl_sync.test to the fact that exclusive metadata locks on
        tables are now acquired according to alphabetical order of fully
        qualified table names instead of order in which tables are mentioned
        in statement.
      sql/events.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/ha_ndbcluster.cc:
        Since manipulations with open table state no longer install proxy
        MDL_context it does not make sense to perform them in order to
        satisfy assert in mysql_rm_tables_part2(). Removed them per agreement
        with Cluster team. This has not broken test suite since scenario in
        which deadlock can occur and assertion fails is not covered by tests.
      sql/lock.cc:
        MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object exclusive metadata lock is acquired.
        Now this has to be done by explicit calls outside of MDL subsystem.
      sql/log.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/mdl.cc:
        Changed metadata locking subsystem to use mutex per lock and condition
        variable per context instead of one mutex and one conditional variable
        for the whole subsystem.
        Changed approach to handling of global metadata locks. Instead of
        implicitly acquiring intention exclusive locks when user requests
        per-object upgradeable or exclusive locks now we require them to be
        acquired explicitly in the same way as ordinary metadata locks.
        In fact global lock are now ordinary metadata locks in new GLOBAL
        namespace.
        
        To implement these changes:
        - Removed LOCK_mdl mutex and COND_mdl condition variable.
        - Introduced MDL_lock::m_mutex mutexes which protect individual lock
          objects.
        - Replaced mdl_locks hash with MDL_map class, which has hash for
          MDL_lock objects as a member and separate mutex which protects this
          hash. Methods of this class allow to find(), find_or_create() or
          remove() MDL_lock objects in concurrency-friendly fashion (i.e.
          for most common operation, find_or_create(), we don't acquire
          MDL_lock::m_mutex while holding MDL_map::m_mutex. Thanks to MikaelR
          for this idea and benchmarks!). Added three auxiliary members to
          MDL_lock class (m_is_destroyed, m_ref_usage, m_ref_release) to
          support this concurrency-friendly behavior.
        - Introduced MDL_context::m_ctx_wakeup_cond condition variable to be
          used for waiting until this context's pending request can be
          satisfied or its thread has to perform actions to resolve potential
          deadlock. Context which want to wait add ticket corresponding to the
          request to an appropriate queue of waiters in MDL_lock object so
          they can be noticed when other contexts change state of lock and be
          awaken by them by signalling on MDL_context::m_ctx_wakeup_cond.
          As consequence MDL_ticket objects has to be used for any waiting
          in metadata locking subsystem including one which happens in
          MDL_context::wait_for_locks() method.
          Another consequence is that MDL_context is no longer copyable and
          can't be saved/restored when working with system tables.
        - Made MDL_lock an abstract class, which delegates specifying exact
          compatibility matrix to its descendants. Added MDL_global_lock child
          class for global lock (The old is_lock_type_compatible() method
          became can_grant_lock() method of this class). Added MDL_object_lock
          class to represent per-object lock (The old MDL_lock::can_grant_lock()
          became its method). Choice between two classes happens based on MDL
          namespace in MDL_lock::create() method.
        - Got rid of MDL_lock::type member as its meaning became ambigous for
          global locks.
        - To simplify waking up of contexts waiting for lock split waiting queue
          in MDL_lock class in two queues. One for pending requests for exclusive
          (including intention exclusive) locks and another for requests for
          shared locks.
        - Added virtual wake_up_waiters() method to MDL_lock, MDL_global_lock and
          MDL_object_lock classes which allows to wake up waiting contexts after
          state of lock changes. Replaced old duplicated code with calls to this
          method.
        - Adjusted MDL_context::try_acquire_shared_lock()/exclusive_lock()/
          global_shared_lock(), MDL_ticket::upgrade_shared_lock_to_exclusive_lock()
          and MDL_context::release_ticket() methods to use MDL_map and
          MDL_lock::m_mutex instead of single LOCK_mdl mutex and wake up
          waiters according to the approach described above. The latter method
          also was renamed to MDL_context::release_lock().
        - Changed MDL_context::try_acquire_shared_lock()/exclusive_lock() and
          release_lock() not to handle global locks. They are now supposed to
          be taken explicitly like ordinary metadata locks.
        - Added helper MDL_context::try_acquire_global_intention_exclusive_lock()
          and acquire_global_intention_exclusive_lock() methods.
        - Moved common code from MDL_context::acquire_global_shared_lock() and
          acquire_global_intention_exclusive_lock() to new method -
          MDL_context::acquire_lock_impl().
        - Moved common code from MDL_context::try_acquire_shared_lock(),
          try_acquire_global_intention_exclusive_lock()/exclusive_lock()
          to MDL_context::try_acquire_lock_impl().
        - Since acquiring of several exclusive locks can no longer happen under
          single LOCK_mdl mutex the approach to it had to be changed. Now we do
          it in one by one fashion. This is done in alphabetical order to avoid
          deadlocks. Changed MDL_context::acquire_exclusive_locks() accordingly
          (as part of this change moved code responsible for acquiring single
          exclusive lock to new MDL_context::acquire_exclusive_lock_impl()
          method).
        - Since we no longer have single LOCK_mdl mutex which protects all
          MDL_context::m_is_waiting_in_mdl members using these members to
          determine if we have really awaken context holding conflicting
          shared lock became inconvinient. Got rid of this member and changed
          notify_shared_lock() helper function and process of acquiring
          of/upgrading to exclusive lock not to rely on such information.
          Now in MDL_context::acquire_exclusive_lock_impl() and
          MDL_ticket::upgrade_shared_lock_to_exclusive_lock() we simply
          re-try to wake up threads holding conflicting shared locks after
          small time out.
        - Adjusted MDL_context::can_wait_lead_to_deadlock() and
          MDL_ticket::has_pending_conflicting_lock() to use per-lock
          mutexes instead of LOCK_mdl. To do this introduced
          MDL_lock::has_pending_exclusive_lock() method.
      sql/mdl.h:
        Changed metadata locking subsystem to use mutex per lock and condition
        variable per context instead of one mutex and one conditional variable
        for the whole subsystem. In order to implement this change:
        
        - Added MDL_key::cmp() method to be able to sort MDL_key objects
          alphabetically. Changed length fields in MDL_key class to uint16
          as 16-bit is enough for length of any key.
        - Changed MDL_ticket::get_ctx() to return pointer to non-const
          object in order to be able to use MDL_context::awake() method
          for such contexts.
        - Got rid of unlocked versions of can_wait_lead_to_deadlock()/
          has_pending_conflicting_lock() methods in MDL_context and
          MDL_ticket. We no longer has single mutex which protects all
          locks. Thus one always has to use versions of these methods
          which acquire per-lock mutexes.
        - MDL_request_list type of list now counts its elements.
        - Added MDL_context::m_ctx_wakeup_cond condition variable to be used
          for waiting until this context's pending request can be satisfied
          or its thread has to perform actions to resolve potential deadlock.
          Added awake() method to wake up context from such wait.
          Addition of condition variable made MDL_context uncopyable.
          As result we no longer can save/restore MDL_context when working
          with system tables. Instead we create MDL savepoint before opening
          those tables and rollback to it once they are closed.
        - MDL_context::release_ticket() became release_lock() method.
        - Added auxiliary MDL_context::acquire_exclusive_lock_impl() method
          which does all necessary work to acquire exclusive lock on one object
          but should not be used directly as it does not enforce any asserts
          ensuring that no deadlocks are possible.
        - Since we no longer need to know if thread trying to acquire exclusive
          lock managed to wake up any threads having conflicting shared locks
          (as, anyway, we will try to wake up such threads again shortly)
        - MDL_context::m_is_waiting_in_mdl member became unnecessary and
          notify_shared_lock() no longer needs to be friend of MDL_context.
        
        Changed approach to handling of global metadata locks. Instead of
        implicitly acquiring intention exclusive locks when user requests
        per-object upgradeable or exclusive locks now we require them to be
        acquired explicitly in the same way as ordinary metadata locks.
        
        - Added new GLOBAL namespace for such locks.
        - Added new type of lock to be requested MDL_INTENTION_EXCLISIVE.
        - Added MDL_context::try_acquire_global_intention_exclusive_lock()
          and acquire_global_intention_exclusive_lock() methods.
        - Moved common code from MDL_context::acquire_global_shared_lock()
          and acquire_global_intention_exclusive_lock() to new method -
          MDL_context::acquire_lock_impl().
        - Moved common code from MDL_context::try_acquire_shared_lock(),
          try_acquire_global_intention_exclusive_lock()/exclusive_lock()
          to MDL_context::try_acquire_lock_impl().
        - Added helper MDL_context::is_global_lock_owner() method to be
          able easily to find what kind of global lock this context holds.
        - MDL_context::m_has_global_shared_lock became unnecessary as
          global read lock is now represented by ordinary ticket.
        - Removed assert in MDL_context::set_lt_or_ha_sentinel() which became
          false for cases when we execute LOCK TABLES under global read lock
          mode.
      sql/mysql_priv.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result calls
        opening/closing system tables were changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sp.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sp.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sql_base.cc:
        close_thread_tables():
          Since we no longer use separate MDL_context for opening system
          tables we need to avoid releasing all transaction locks when
          closing system table. Releasing metadata lock on system table
          is now responsibility of THD::restore_backup_open_tables_state().
        open_table_get_mdl_lock(),
        Open_table_context::recover_from_failed_open():
          MDL subsystem no longer implicitly acquires global intention exclusive
          metadata lock when per-object upgradable or exclusive metadata lock is
          acquired. So this have to be done explicitly from these calls.
          Changed Open_table_context class to store MDL_request object for
          global intention exclusive lock acquired when opening tables.
        open_table():
          Do not release metadata lock if we have failed to open table as
          this lock might have been acquired by one of previous statements
          in transaction, and therefore should not be released.
        open_system_tables_for_read()/close_system_tables()/
        open_performance_schema_table():
          Instead of using separate MDL_context for opening system tables we now
          create MDL savepoint in the main context before opening such tables
          and rollback to this savepoint after closing them. To support this
          change methods of THD responsible for saving/restoring open table
          state were changed to use Open_tables_backup class which in addition
          to Open_table_state has a member for this savepoint. As result code
          opening/closing system tables was changed to use Open_tables_backup
          instead of Open_table_state class as well.
        close_performance_schema_table():
          Got rid of duplicated code.
      sql/sql_class.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. Also releasing
        metadata lock on system table is now responsibility of
        THD::restore_backup_open_tables_state().
        Adjusted assert in THD::cleanup() to take into account fact that now
        we also use MDL sentinel for global read lock.
      sql/sql_class.h:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. As result:
        - 'mdl_context' member was moved out of Open_tables_state to THD class.
          enter_locked_tables_mode()/leave_locked_tables_mode() had to follow.
        - Methods of THD responsible for saving/restoring open table state were
          changed to use Open_tables_backup class which in addition to
          Open_table_state has a member for this savepoint.
        Changed Open_table_context class to store MDL_request object for
        global intention exclusive lock acquired when opening tables.
      sql/sql_delete.cc:
        MDL subsystem no longer implicitly acquires global intention exclusive
        metadata lock when per-object exclusive metadata lock is acquired.
        Now this has to be done by explicit calls outside of MDL subsystem.
      sql/sql_help.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
      sql/sql_parse.cc:
        Adjusted assert reload_acl_and_cache() to the fact that global read
        lock now takes full-blown metadata lock.
      sql/sql_plist.h:
        Added support for element counting to I_P_List list template.
        One can use policy classes to specify if such counting is needed
        or not needed for particular list.
      sql/sql_show.cc:
        Instead of using separate MDL_context for opening tables for I_S
        purposes we now create MDL savepoint in the main context before
        opening tables and rollback to this savepoint after closing them.
        To support this and similar change for system tables methods of
        THD responsible for saving/restoring open table state were changed
        to use Open_tables_backup class which in addition to Open_table_state
        has a member for this savepoint. As result code opening/closing tables
        for I_S purposes was changed to use Open_tables_backup instead of
        Open_table_state class as well.
      sql/sql_table.cc:
        mysql_rm_tables_part2():
          Since now global intention exclusive metadata lock is ordinary
          metadata lock we no longer can rely that by releasing MDL locks
          on all tables we will release all locks acquired by this routine.
          So in non-LOCK-TABLES mode we have to release all locks acquired
          explicitly.
        prepare_for_repair(), mysql_alter_table():
          MDL subsystem no longer implicitly acquires global intention
          exclusive metadata lock when per-object exclusive metadata lock
          is acquired. Now this has to be done by explicit calls outside of
          MDL subsystem.
      sql/tztime.cc:
        Instead of using separate MDL_context for opening system tables we now
        create MDL savepoint in the main context before opening such tables
        and rollback to this savepoint after closing them. To support this
        change methods of THD responsible for saving/restoring open table
        state were changed to use Open_tables_backup class which in addition
        to Open_table_state has a member for this savepoint. As result code
        opening/closing system tables was changed to use Open_tables_backup
        instead of Open_table_state class as well.
        Also changed code not to use special mechanism for open system tables
        when it is not really necessary.
      6ddd01c2
  37. 12 Jan, 2010 1 commit
  38. 07 Jan, 2010 1 commit
  39. 29 Dec, 2009 1 commit
    • Konstantin Osipov's avatar
      Apply and review: · bf9c1b73
      Konstantin Osipov authored
      3655 Jon Olav Hauglid   2009-10-19
      Bug #30977 Concurrent statement using stored function and DROP FUNCTION 
                 breaks SBR
      Bug #48246 assert in close_thread_table
      
      Implement a fix for:
      Bug #41804 purge stored procedure cache causes mysterious hang for many
                 minutes
      Bug #49972 Crash in prepared statements
      
      The problem was that concurrent execution of DML statements that
      use stored functions and DDL statements that drop/modify the same
      function might result in incorrect binary log in statement (and
      mixed) mode and therefore break replication.
      
      This patch fixes the problem by introducing metadata locking for
      stored procedures and functions. This is similar to what is done
      in Bug#25144 for views. Procedures and functions now are
      locked using metadata locks until the transaction is either
      committed or rolled back. This prevents other statements from
      modifying the procedure/function while it is being executed. This
      provides commit ordering - guaranteeing serializability across
      multiple transactions and thus fixes the reported binlog problem.
      
      Note that we do not take locks for top-level CALLs. This means
      that procedures called directly are not protected from changes by
      simultaneous DDL operations so they are executed at the state they
      had at the time of the CALL. By not taking locks for top-level
      CALLs, we still allow transactions to be started inside
      procedures.
      
      This patch also changes stored procedure cache invalidation.
      Upon a change of cache version, we no longer invalidate the entire
      cache, but only those routines which we use, only when a statement
      is executed that uses them.
      
      This patch also changes the logic of prepared statement validation.
      A stored procedure used by a prepared statement is now validated
      only once a metadata lock has been acquired. A version mismatch
      causes a flush of the obsolete routine from the cache and
      statement reprepare.
      Incompatible changes:
      1) ER_LOCK_DEADLOCK is reported for a transaction trying to access
         a procedure/function that is locked by a DDL operation in
         another connection.
      
      2) Procedure/function DDL operations are now prohibited in LOCK
         TABLES mode as exclusive locks must be taken all at once and
         LOCK TABLES provides no way to specifiy procedures/functions to
         be locked.
      
      Test cases have been added to sp-lock.test and rpl_sp.test.
      
      Work on this bug has very much been a team effort and this patch
      includes and is based on contributions from Davi Arnaut, Dmitry
      Lenev, Magne Mæhre and Konstantin Osipov.
      
      
      mysql-test/r/ps_ddl.result:
        Update results (Bug#30977).
      mysql-test/r/ps_ddl1.result:
        Update results (Bug#30977).
      mysql-test/r/sp-error.result:
        Update results (Bug#30977).
      mysql-test/r/sp-lock.result:
        Update results (Bug#30977).
      mysql-test/suite/rpl/r/rpl_sp.result:
        Update results (Bug#30977).
      mysql-test/suite/rpl/t/rpl_sp.test:
        Add a test case for Bug#30977.
      mysql-test/t/ps_ddl.test:
        Update comments. We no longer re-prepare a prepared statement
        when a stored procedure used in top-level CALL is changed.
      mysql-test/t/ps_ddl1.test:
        Modifying stored procedure p1 no longer invalidates prepared
        statement "call p1" -- we can re-use the prepared statement
        without invalidation.
      mysql-test/t/sp-error.test:
        Use a constant for an error value.
      mysql-test/t/sp-lock.test:
        Add test coverage for Bug#30977.
      sql/lock.cc:
        Implement lock_routine_name() - a way to acquire an 
        exclusive metadata lock (ex- name-lock) on 
        stored procedure/function.
      sql/sp.cc:
        Change semantics of sp_cache_routine() -- now it has an option
        to make sure that the routine that is cached is up to date (has
        the latest sp cache version).
        
        Add sp_cache_invalidate() to sp_drop_routine(), where it was
        missing (a bug!).
        
        Acquire metadata locks for SP DDL (ALTER/CREATE/DROP). This is
        the core of the fix for Bug#30977.
        
        Since caching and cache invalidation scheme was changed, make 
        sure we don't invalidate the SP cache in the middle of a stored
        routine execution. At the same time, make sure we don't access
        stale data due to lack of invalidation. 
        For that, change ALTER FUNCTION/PROCEDURE to not use the cache,
        and SHOW PROCEDURE CODE/SHOW CREATE PROCEDURE/FUNCTION to always
        read an up to date version of the routine from the cache.
      sql/sp.h:
        Add a helper wrapper around sp_cache_routine().
      sql/sp_cache.cc:
        Implement new sp_cache_version() and sp_cache_flush_obsolete().
        Now we flush stale routines individually, rather than all at once.
      sql/sp_cache.h:
        Update signatures of sp_cache_version() and sp_cache_flush_obsolete().
      sql/sp_head.cc:
        Add a default initialization of sp_head::m_sp_cache_version.
        Remove a redundant sp_head::create().
      sql/sp_head.h:
        Add m_sp_cache_version to sp_head class - we now 
        keep track of every routine in the stored procedure cache, rather than
        of the entire cache.
      sql/sql_base.cc:
        Implement prelocking for stored routines. Validate stored
        routines after they were locked.
        Flush obsolete routines upon next access, one by one, not all at once
        (Bug#41804).
        Style fixes.
      sql/sql_class.h:
        Rename a Open_table_context method.
      sql/sql_parse.cc:
        Make sure stored procedures DDL commits the active transaction 
        (issues an implicit commit before and after).
        Remove sp_head::create(), a pure redundancy.
        Move the semantical check during alter routine inside sp_update_routine() code in order to:
        - avoid using SP cache during update, it may be obsolete.
        - speed up and simplify the update procedure.
        
        Remove sp_cache_flush_obsolete() calls, we no longer flush the entire
        cache, ever, stale routines are flushed before next use, one at a time.
      sql/sql_prepare.cc:
        Move routine metadata validation to open_and_process_routine().
        Fix Bug#49972 (don't swap flags at reprepare).
        Reset Sroutine_hash_entries in reinit_stmt_before_use().
        Remove SP cache invalidation, it's now done by open_tables().
      sql/sql_show.cc:
        Fix a warning: remove an unused label.
      sql/sql_table.cc:
        Reset mdl_request.ticket for tickets acquired for routines inlined
        through a view, in CHECK TABLE statement, to satisfy an MDL assert.
      sql/sql_update.cc:
        Move the cleanup of "translation items" to close_tables_for_reopen(),
        since it's needed in all cases when we back off, not just
        the back-off in multi-update. This fixes a bug when the server
        would crash on attempt to back off when opening tables
        for a statement that uses information_schema tables.
      bf9c1b73