From revision r5733 to r5747

Detailed revision comments:

r5733 | sunny | 2009-09-02 02:05:15 -0500 (Wed, 02 Sep 2009) | 6 lines
branches/zip: Fix a regression introduced by the fix for bug#26316. We check
whether a transaction holds any AUTOINC locks before we acquire the kernel
mutex and release those locks.

Fix for rb://153. Approved by Marko.

r5734 | sunny | 2009-09-02 02:08:45 -0500 (Wed, 02 Sep 2009) | 2 lines
branches/zip: Update ChangeLog with r5733 changes.

r5735 | marko | 2009-09-02 02:43:09 -0500 (Wed, 02 Sep 2009) | 2 lines
branches/zip: univ.i: Do not undefine PACKAGE or VERSION.
InnoDB source code does not refer to these macros.
r5736 | marko | 2009-09-02 02:53:19 -0500 (Wed, 02 Sep 2009) | 1 line
branches/zip: Enclose some timestamp functions in #ifndef UNIV_HOTBACKUP.
r5743 | marko | 2009-09-03 01:36:12 -0500 (Thu, 03 Sep 2009) | 3 lines
branches/zip: log_reserve_and_write_fast(): Remove the redundant
output parameter "success".
Success is also indicated by a nonzero return value.
r5744 | marko | 2009-09-03 03:28:35 -0500 (Thu, 03 Sep 2009) | 1 line
branches/zip: ut_align(): Make ptr const, like in ut_align_down().
r5745 | marko | 2009-09-03 03:38:22 -0500 (Thu, 03 Sep 2009) | 2 lines
branches/zip: log_check_log_recs(): Enclose in #ifdef UNIV_LOG_DEBUG.
Add const qualifiers.
r5746 | marko | 2009-09-03 03:55:36 -0500 (Thu, 03 Sep 2009) | 2 lines
branches/zip: log_reserve_and_write_fast(): Do not cache the log_sys pointer
in a local variable.
r5747 | marko | 2009-09-03 05:46:38 -0500 (Thu, 03 Sep 2009) | 2 lines
branches/zip: recv_scan_log_recs(): Replace while with do...while,
because the termination condition will always hold on the first iteration.

From revision r5703 to r5716

Detailed revision comments:

r5703 | marko | 2009-08-27 02:25:00 -0500 (Thu, 27 Aug 2009) | 41 lines
branches/zip: Replace the constant 3/8 ratio that controls the LRU_old
size with the settable global variable innodb_old_blocks_pct. The
minimum and maximum values are 5 and 95 per cent, respectively. The
default is 100*3/8, in line with the old behavior.

ut_time_ms(): New utility function, to return the current time in
milliseconds. TODO: Is there a more efficient timestamp function, such
as rdtsc divided by a power of two?

buf_LRU_old_threshold_ms: New variable, corresponding to
innodb_old_blocks_time. The value 0 is the default behaviour: no
timeout before making blocks 'new'.

bpage->accessed, bpage->LRU_position, buf_pool->ulint_clock: Remove.

bpage->access_time: New field, replacing bpage->accessed. Protected by
buf_pool_mutex instead of bpage->mutex. Updated when a page is created
or accessed the first time in the buffer pool.

buf_LRU_old_ratio, innobase_old_blocks_pct: New variables,
corresponding to innodb_old_blocks_pct

buf_LRU_old_ratio_update(), innobase_old_blocks_pct_update(): Update
functions for buf_LRU_old_ratio, innobase_old_blocks_pct.

buf_page_peek_if_too_old(): Compare ut_time_ms() to bpage->access_time
if buf_LRU_old_threshold_ms && bpage->old.  Else observe
buf_LRU_old_ratio and bpage->freed_page_clock.

buf_pool_t: Add n_pages_made_young, n_pages_not_made_young,
n_pages_made_young_old, n_pages_not_made_young, for statistics.

buf_print(): Display buf_pool->n_pages_made_young,
buf_pool->n_pages_not_made_young.  This function is only for crash
diagnostics.

buf_print_io(): Display buf_pool->LRU_old_len and quantities derived
from buf_pool->n_pages_made_young, buf_pool->n_pages_not_made_young.
This function is invoked by SHOW ENGINE INNODB STATUS.

rb://129 approved by Heikki Tuuri.  This addresses Bug #45015.
r5704 | marko | 2009-08-27 03:31:17 -0500 (Thu, 27 Aug 2009) | 32 lines
branches/zip: Fix a critical bug in fast index creation that could
corrupt the created indexes.

row_merge(): Make "half" an in/out parameter. Determine the offset of
half the output file. Copy the last blocks record-by-record instead of
block-by-block, so that the records can be counted. Check that the
input and output have matching n_rec.

row_merge_sort(): Do not assume that two blocks of size N are merged
into a block of size 2*N. The output block can be shorter than the
input if the last page of each input block is almost empty. Use an
accurate termination condition, based on the "half" computed by
row_merge().

row_merge_read(), row_merge_write(), row_merge_blocks(): Add debug output.

merge_file_t, row_merge_file_create(): Add n_rec, the number of records
in the merge file.

row_merge_read_clustered_index(): Update n_rec.

row_merge_blocks(): Update and check n_rec.

row_merge_blocks_copy(): New function, for copying the last blocks in
row_merge().  Update and check n_rec.

This bug was discovered with a user-supplied test case that creates an
index where the initial temporary file is 249 one-megabyte blocks and
the merged files become smaller. In the test, possible merge record
sizes are 10, 18, and 26 bytes.

rb://150 approved by Sunny Bains.  This addresses Issue #320.
r5705 | marko | 2009-08-27 06:56:24 -0500 (Thu, 27 Aug 2009) | 11 lines
branches/zip: dict_index_find_cols(): On column name lookup failure,
return DB_CORRUPTION (HA_ERR_CRASHED) instead of abnormally
terminating the server.  Also, disable the previously added diagnostic
output to the error log, because mysql-test-run does not like extra
output in the error log.  (Bug #44571)

dict_index_add_to_cache(): Handle errors from dict_index_find_cols().

mysql-test/innodb_bug44571.test: A test case for triggering the bug.

rb://135 approved by Sunny Bains.
r5706 | inaam | 2009-08-27 11:00:27 -0500 (Thu, 27 Aug 2009) | 20 lines
branches/zip rb://147

Done away with following two status variables:

innodb_buffer_pool_read_ahead_rnd
innodb_buffer_pool_read_ahead_seq

Introduced two new status variables:
innodb_buffer_pool_read_ahead = number of pages read as part of
readahead since server startup
innodb_buffer_pool_read_ahead_evicted = number of pages that are read
in as readahead but were evicted before ever being accessed since
server startup i.e.: a measure of how badly our readahead is
performing

SHOW INNODB STATUS will show two extra numbers in buffer pool section:
pages read ahead/sec and pages evicted without access/sec

Approved by: Marko

r5707 | inaam | 2009-08-27 11:20:35 -0500 (Thu, 27 Aug 2009) | 6 lines
branches/zip

Remove unused macros as we erased the random readahead code in r5703.
Also fixed some comments.


r5708 | inaam | 2009-08-27 17:43:32 -0500 (Thu, 27 Aug 2009) | 4 lines
branches/zip

Remove redundant TRUE : FALSE from the return statement

r5709 | inaam | 2009-08-28 01:22:46 -0500 (Fri, 28 Aug 2009) | 5 lines
branches/zip rb://152

Disable display of deprecated parameter innodb_file_io_threads in
'show variables'.

r5714 | marko | 2009-08-31 01:10:10 -0500 (Mon, 31 Aug 2009) | 5 lines
branches/zip: buf_chunk_not_freed(): Do not acquire block->mutex unless
block->page.state == BUF_BLOCK_FILE_PAGE.  Check that block->page.state
makes sense.

Approved by Sunny Bains over the IM.
r5716 | vasil | 2009-08-31 02:47:49 -0500 (Mon, 31 Aug 2009) | 9 lines
branches/zip:

Fix Bug#46718 InnoDB plugin incompatible with gcc 4.1 (at least: on PPC): "Undefined symbol"

by implementing our own check in plug.in instead of using the result from
the check from MySQL because it is insufficient.

Approved by:	Marko (rb://154)

From revision r5686 to r5702
r5696 is already applied

Detailed revision comments:

r5686 | vasil | 2009-08-20 04:15:05 -0500 (Thu, 20 Aug 2009) | 4 lines
branches/zip:

White-space fixup.

r5687 | vasil | 2009-08-20 04:20:22 -0500 (Thu, 20 Aug 2009) | 8 lines
branches/zip:

ChangeLog:
Follow the convention from the rest of the ChangeLog: for bugfixes from
bugs.mysql.com only the bug number and title goes in the ChangeLog. Detailed
explanation on what is the problem and how it was fixed is present in
the bugs database.

r5694 | marko | 2009-08-26 06:25:26 -0500 (Wed, 26 Aug 2009) | 2 lines
branches/zip: buf_page_t: Clarify that bpage->list may contain garbage.
This comment was provoked by Inaam.
r5695 | marko | 2009-08-26 08:14:59 -0500 (Wed, 26 Aug 2009) | 1 line
branches/zip: UNIV_DEBUG_LOCK_VALIDATE: Move the definition to univ.i.
r5697 | vasil | 2009-08-26 08:44:40 -0500 (Wed, 26 Aug 2009) | 4 lines
branches/zip:

Fix typo.

r5698 | inaam | 2009-08-26 09:34:35 -0500 (Wed, 26 Aug 2009) | 13 lines
branches/zip  bug#42885  rb://148

The call to put IO threads to sleep was most probably meant for Windows
only as the comment in buf0rea.c suggests. However it was enabled on
all platforms. This patch restricts the sleep call to windows. This
approach of not putting threads to sleep makes even more sense because
now we have multiple threads working in the background and it probably
is not a good idea to put all of them to sleep because a user thread
wants to post a batch for readahead.

Approved by: Marko


r5701 | marko | 2009-08-27 02:01:42 -0500 (Thu, 27 Aug 2009) | 1 line
branches/zip: Document r5698 in the ChangeLog.
r5702 | marko | 2009-08-27 02:03:15 -0500 (Thu, 27 Aug 2009) | 1 line
branches/zip: Document also the files affected by r5698 in the ChangeLog.

From r5639 to r5685 

Detailed revision comments:

r5639 | marko | 2009-08-06 05:39:34 -0500 (Thu, 06 Aug 2009) | 3 lines
branches/zip: mem_heap_block_free(): If innodb_use_sys_malloc is set,
do not tell Valgrind that the memory is free, to avoid
a bogus warning in Valgrind's built-in free() hook.
r5642 | calvin | 2009-08-06 18:04:03 -0500 (Thu, 06 Aug 2009) | 2 lines
branches/zip: remove duplicate "the" in comments.

r5662 | marko | 2009-08-11 04:54:16 -0500 (Tue, 11 Aug 2009) | 1 line
branches/zip: Bump the version number to 1.0.5 after releasing 1.0.4.
r5663 | marko | 2009-08-11 06:42:37 -0500 (Tue, 11 Aug 2009) | 2 lines
branches/zip: trx_general_rollback_for_mysql(): Remove the redundant
parameter partial. If savept==NULL, partial==FALSE.
r5670 | marko | 2009-08-12 08:16:37 -0500 (Wed, 12 Aug 2009) | 2 lines
branches/zip: trx_undo_rec_copy(): Add const qualifier to undo_rec.
This is a non-functional change.
r5671 | marko | 2009-08-13 03:46:33 -0500 (Thu, 13 Aug 2009) | 5 lines
branches/zip: ha_innobase::add_index(): Fix Bug #46557:
after a successful operation, read innodb_table->flags from
the newly created table object, not from the old one that was just freed.

Approved by Sunny.
r5681 | sunny | 2009-08-14 01:16:24 -0500 (Fri, 14 Aug 2009) | 3 lines
branches/zip: When building HotBackup srv_use_sys_malloc is #ifdef out. We
move access to the this variable within a !UNIV_HOTBACKUP block.

r5684 | sunny | 2009-08-20 03:05:30 -0500 (Thu, 20 Aug 2009) | 10 lines
branches/zip: Fix bug# 46650:  Innodb assertion autoinc_lock == lock in lock_table_remove_low on INSERT SELECT

We only store the autoinc locks that are granted in the transaction's autoinc
lock vector. A transacton, that has been rolled back due to a deadlock because
of an AUTOINC lock attempt, will not have added that lock to the vector. We
need to check for that when we remove that lock.

rb://145
Approved by Marko.

r5685 | sunny | 2009-08-20 03:18:29 -0500 (Thu, 20 Aug 2009) | 2 lines
branches/zip: Update the ChangeLog with r5684 change.

From revision r5537 to r5636
r5527 is already applied before

Detailed revision comments:
For HotBackup builds we don't want to hide the symbols.

r5537 | vasil | 2009-07-21 04:31:26 -0500 (Tue, 21 Jul 2009) | 5 lines
branches/zip:

Fixups in ChangeLog: sort filenames alphabetically and wrap to 78 chars per
line.

r5539 | vasil | 2009-07-21 05:28:27 -0500 (Tue, 21 Jul 2009) | 4 lines
branches/zip:

Add a test program to check whether the PAUSE instruction is available.

r5625 | vasil | 2009-08-04 00:52:48 -0500 (Tue, 04 Aug 2009) | 32 lines
branches/zip: Merge 5518:5622 from branches/5.1, resolving conflict in r5622
(after resolving the conflict Makefile.am was not changed so I have made
a dummy change so I can commit and thus record that branches/5.1 has been
merged in branches/zip up to 5622):

  ------------------------------------------------------------------------
  r5622 | vasil | 2009-08-03 15:27:00 +0300 (Mon, 03 Aug 2009) | 20 lines
  Changed paths:
     M /branches/5.1/Makefile.am
  
  branches/5.1:
  
  Merge a change from MySQL:
  
    ------------------------------------------------------------
    revno: 2988
    committer: Satya B <satya.bn@sun.com>
    branch nick: mysql-5.1-bugteam
    timestamp: Wed 2009-07-01 11:06:05 +0530
    message:
      Fix build failure after applying Innodb snapshot 5.1-ss5282
      
      After applying Innodb snapshot 5.1-ss5282, build was broken
      because of missing header file. 
      
      Adding the header file to Makefile.am after informing the 
      innodb developers.
    modified:
      storage/innobase/Makefile.am
  
  ------------------------------------------------------------------------

r5626 | vasil | 2009-08-04 00:53:31 -0500 (Tue, 04 Aug 2009) | 4 lines
branches/zip:

Revert the dummy change from c5625.

r5629 | marko | 2009-08-04 06:42:44 -0500 (Tue, 04 Aug 2009) | 1 line
branches/zip: mysql-test: Pass MTR's internal checks.
r5635 | marko | 2009-08-05 06:06:55 -0500 (Wed, 05 Aug 2009) | 2 lines
branches/zip: Replace <number> with NUMBER in some comments,
to avoid problems with Doxygen XML output.
r5636 | marko | 2009-08-05 07:27:30 -0500 (Wed, 05 Aug 2009) | 2 lines
branches/zip: lock_rec_validate_page(): Add the parameter zip_size.
This should help track down Mantis Issue #289.

The pthread_cond_wait implementations for windows might
dead lock in some rare circumstances.

1) One thread (I) enter a timed wait and at a point in
   time ends up after mutex unlock and before
   WaitForMultipleObjects(...)
2) Another thread (II) enters pthread_cond_broadcast.
   Grabs the mutex and discovers one waiter. It set
   the broadcast event and closes the broadcast gate
   then unlocks the mutex.
3) A third thread (III) issues a pthread_cond_signal.
   It grabs the mutex, discovers one waiter, sets the
   signal event then unlock the mutex.
4) The first threads (I) enters WaitForMultipleObjects
   and finds out that the signal object is in a
   signalled state and exits the wait.
5) Thread (I) grabs the mutex and checks result status.
   The number of waiters is decreased and becomes equal
   to 0. The event returned was a signal event so the
   broadcast gate isn't opened. The mutex is released.
6) Thread (II) issues a new broadcast. The mutex is
   acquired but the number of waiters are 0 hence
   the broadcast gate remains closed.
7) Thread (I) enters the wait again but is blocked by
   the broadcast gate.

      This fix resolves the above issue by always resetting
      broadcast gate when there are no more waiters in th queue.


mysys/my_wincond.c:
  * Always reset the broadcast gate if there are no more waiters left.

  Let
    - T be a transactional table and N non-transactional table.
    - B be begin, C commit and R rollback.
    - N be a statement that accesses and changes only N-tables.
    - T be a statement that accesses and changes only T-tables.

In RBR, changes to N-tables that happen early in a transaction are not immediately flushed
upon committing a statement. This behavior may, however, break consistency in the presence
of concurrency since changes done to N-tables become immediately visible to other
connections. To fix this problem, we do the following:

  . B N N T C would log - B N C B N C B T C.
  . B N N T R would log - B N C B N C B T R.

Note that we are not preserving history from the master as we are introducing a commit that
never happened. However, this seems to be more acceptable than the possibility of breaking
consistency in the presence of concurrency.

Let
  - T be a transactional table and N non-transactional table.
  - B be begin, C commit and R rollback.
  - M be a mixed statement, i.e. a statement that updates both T and N.
  - M* be a mixed statement that fails while updating either T or N.

This patch restore the behavior presented in 5.1.37 for rows either produced in
the RBR or MIXED modes, when a M* statement that happened early in a transaction
had their changes written to the binary log outside the boundaries of the
transaction and wrapped in a BEGIN/ROLLBACK. This was done to keep the slave
consistent with with the master as the rollback would keep the changes on N and
undo them on T. In particular, we do what follows:

  . B M* T C would log - B M* R B T C.

Note that, we are not preserving history from the master as we are introducing a
rollback that never happened. However, this seems to be more acceptable than
making the slave diverge. We do not fix the following case:

  . B T M* C would log B T M* C.

The slave will diverge as the changes on T tables that originated from the M
statement are rolled back on the master but not on the slave. Unfortunately, we
cannot simply rollback the transaction as this would undo any uncommitted
changes on T tables.

SBR is not considered in this patch because a failing statement is written to
the binary along with the error code and a slave executes and then rolls back
the statement when it has an associated error code, thus undoing the effects
on T. In RBR and MBR, a full-fledged fix will be pushed after the WL 2687.

Mask part of EXPLAIN output with '#' to account for varying row count estimation.

mysql-test/include/mix1.inc:
  Mask 'rows' column in EXPLAIN output (number varies sometimes between 1 and 2).
mysql-test/r/innodb_mysql.result:
  Update result file after masking of rows estimation in EXPLAIN output.

1. BUG#46256 - drop table with unknown collation crashes innodb

Note: No testcase attached and has to be verified manually

Detailed revision comments:

r5799 | calvin | 2009-09-09 20:47:31 +0300 (Wed, 09 Sep 2009) | 10 lines
branches/5.1: fix bug#46256

Allow tables to be dropped even if the collation is not found,
but issue a warning.

Could not find an easy way to add mysql-test since it requires
changes to charsets and restarting the server. Tests were
executed manually.

Approved by: Heikki (on IM)
r5805 | vasil | 2009-09-10 08:41:48 +0300 (Thu, 10 Sep 2009) | 7 lines
branches/5.1:

Fix a compilation warning caused by c5799:

handler/ha_innodb.cc: In function 'void innobase_get_cset_width(ulint, ulint*, ulint*)':
handler/ha_innodb.cc:830: warning: format '%d' expects type 'int', but argument 2 has type 'ulint'

BUG#44369 - InnoDB: Does not uniformly disallow disallowed column names

Detailed revision comments:

r5741 | jyang | 2009-09-03 07:16:01 +0300 (Thu, 03 Sep 2009) | 5 lines
branches/5.1: Block creating table with column name conflicting
with Innodb reserved key words. (Bug #44369) rb://151 approved
by Sunny Bains.


r5760 | jyang | 2009-09-04 07:07:34 +0300 (Fri, 04 Sep 2009) | 3 lines
branches/5.1: This is to revert change 5741. A return status for
create_table_def() needs to be fixed.

r5834 | jyang | 2009-09-11 00:43:05 +0300 (Fri, 11 Sep 2009) | 5 lines
branches/5.1: Block creating table with column name conflicting
with Innodb reserved key words. (Bug #44369) rb://151 approved
by Sunny Bains.

1. BUG#46000 - using index called GEN_CLUST_INDEX crashes server

Detailed revision comments:

r5895 | jyang | 2009-09-15 03:39:21 +0300 (Tue, 15 Sep 2009) | 5 lines
branches/5.1: Disallow creating index with the name of
"GEN_CLUST_INDEX" which is reserved for the default system
primary index. (Bug #46000) rb://149 approved by Marko Makela.

SELECT ... WHERE ... IN (NULL, ...) does full table scan,
even if the same query without the NULL uses efficient range scan.

The bugfix for the bug 18360 introduced an optimization:
if
  1) all right-hand arguments of the IN function are constants
  2) result types of all right argument items are compatible
     enough to use the same single comparison function to
     compare all of them to the left argument,

then

  we can convert the right-hand list of constant items to an array
  of equally-typed constant values for the further
  QUICK index access etc. (see Item_func_in::fix_length_and_dec()).

The Item_null constant item objects have STRING_RESULT
result types, so, as far as Item_func_in::fix_length_and_dec()
is aware of NULLs in the right list, this improvement efficiently
optimizes IN function calls with a mixed right list of NULLs and
string constants. However, the optimization doesn't affect mixed
lists of NULLs and integers, floats etc., because there is no
unique common comparator.


New optimization has been added to ignore the result type
of NULL constants in the static analysis of mixed right-hand lists.
This is safe, because at the execution phase we care about
presence of NULLs anyway.

1. The collect_cmp_types() function has been modified to optionally
   ignore NULL constants in the item list.
2. NULL-skipping code of the Item_func_in::fix_length_and_dec()
   function has been modified to work not only with in_string
   vectors but with in_vectors of other types.


mysql-test/r/func_in.result:
  Added test case for the bug #44139.
mysql-test/t/func_in.test:
  Added test case for the bug #44139.
sql/item_cmpfunc.cc:
  Bug #44139: Table scan when NULL appears in IN clause
  
  1. The collect_cmp_types() function has been modified to optionally
     ignore NULL constants in the item list.
  2. NULL-skipping code of the Item_func_in::fix_length_and_dec()
     function has been modified to work not only with in_string
     vectors but with in_vectors of other types.

rpl.rpl_trigger.test fails with valgrind errors with the innodb plugin

The problem is that there is only one autoinc value associated with 
the query when binlogging. If more than one autoinc values are used 
in the query, the autoinc values after the first one can be inserted 
wrongly on slave. So these autoinc values can become inconsistent on 
master and slave.

The problem is resolved by marking all the statements that invoke 
a trigger or call a function that updated autoinc fields as unsafe, 
and will switch to row-format in Mixed mode. Actually, the statement 
is safe if just one autoinc value is used in sub-statement, but it's 
impossible to check how many autoinc values are used in sub-statement.)

mysql-test/suite/rpl/r/rpl_auto_increment_update_failure.result:
  Test result for bug#45677
mysql-test/suite/rpl/t/rpl_auto_increment_update_failure.test:
  Added test to verify the following two properties:
  P1) insert/update in an autoinc column causes statement to 
  be logged in row format if binlog_format=mixed
  P2) if binlog_format=mixed, and a trigger or function contains 
      two or more inserts/updates in a table that has an autoinc 
      column, then the slave should not go out of sync, even if 
      there are concurrent transactions.
sql/sql_base.cc:
  Added function 'has_write_table_with_auto_increment' to check 
  if one (or more) write tables have auto_increment columns.
  
  Removed function 'has_two_write_locked_tables_with_auto_increment', 
  because the function is included in function 
  'has_write_table_with_auto_increment'.

                    Removes the need of a hack (the jump to label).

sql/sql_parse.cc:
  DBUG macros are wrapped inside a loop. Allow to break
  the command switch from within a DBUG macro.

On Mac OS X or Windows, sending a SIGHUP to the server or a
asynchronous flush (triggered by flush_time), would cause the
server to crash.

The problem was that a hook used to detach client API handles
wasn't prepared to handle cases where the thread does not have
a associated session.

The solution is to verify whether the thread has a associated
session before trying to detach a handle.

mysql-test/r/federated_debug.result:
  Add test case result for Bug#47525
mysql-test/t/federated_debug-master.opt:
  Debug point.
mysql-test/t/federated_debug.test:
  Add test case for Bug#47525
sql/slave.cc:
  Check whether a the thread has a associated session.
sql/sql_parse.cc:
  Add debug code to simulate a reload without thread session.

Make configure.js bail with an error if trying to build bdb from a bzr
tree.