low myisam_sort_buffer_size

Repair by sort (default) or parallel repair of a MyISAM table
(doesn't matter partitioned or not) as well as bulk inserts
and enable indexes some times didn't failover to repair with
key cache.

The problem was that after unsuccessful attempt, data file was
closed. Whereas repair with key cache requires open data file.
Fixed by reopening data file.

Also fixed a valgrind warning, which may appear during repair
by sort or parallel repair with certain myisam_sort_buffer_size
number of rows and length of an index entry (very dependent).

mysql-test/r/myisam.result:
  A test case for BUG#47073.
mysql-test/t/myisam.test:
  A test case for BUG#47073.
storage/myisam/ha_myisam.cc:
  Reverted fix for BUG25289. Not needed anymore.
storage/myisam/mi_check.c:
  Reopen data file, when repair by sort or parallel repair
  fails.
  
  When repair by sort is requested to rebuild data file, data file
  gets rebuilt while fixing first index. When rebuild is completed,
  info->dfile is pointing to temporary data file, original data file
  is closed.
  
  It may happen that repair has successfully fixed first index and
  rebuilt data file, but failed to fix second index. E.g.
  myisam_sort_buffer_size was big enough to fix first shorter index,
  but not enough to fix subsequent longer index.
  
  In this case we end up with info->dfile pointing to temporary file,
  which is removed and info->dfile is set to -1.
  
  Though repair by sort failed, the upper layer may still want to
  try repair with key cache. But it needs info->dfile pointing to
  valid data file.
storage/myisam/sort.c:
  When performing a copy of IO_CACHE structure, current_pos and
  current_end must be updated separatly to point to memory we're
  copying to (not to memory we're copying from).
  
  As t_file2 is always WRITE cache, proper members are write_pos
  and write_end accordingly.

Disable the test when it will not hit the open_files_limit

covering index
      
When two range predicates were combined under an OR
predicate, the algorithm tried to merge overlapping ranges
into one. But the case when a range overlapped several other
ranges was not handled. This lead to

1) ranges overlapping, which gave repeated results and 
2) a range that overlapped several other ranges was cut off.  

Fixed by 

1) Making sure that a range got an upper bound equal to the
next range with a greater minimum.
2) Removing a continue statement


mysql-test/r/group_min_max.result:
  Bug#42846: Changed query plans
mysql-test/r/range.result:
  Bug#42846: Test result.
mysql-test/t/range.test:
  Bug#42846: Test case.
sql/opt_range.cc:
  Bug#42846: The fix. 
  
  Part1: Previously, both endpoints from key2 were copied,
  which is not safe. Since ranges are processed in ascending
  order of minimum endpoints, it is safe to copy the minimum
  endpoint from key2 but not the maximum. The maximum may only
  be copied if there is no other range or the other range's
  minimum is greater than key2's maximum.

 - Don't build split.c or debug.c since they are not part of the actual regex library

backport for bug#44059 from mysql-pe to mysql-5.1-bugteam

Using the partition with most rows instead of first partition
to estimate the cardinality of indexes.

mysql-test/r/partition.result:
  Bug#44059: Incorrect cardinality of indexes on a partitioned table
  
  Added test result
mysql-test/t/partition.test:
  Bug#44059: Incorrect cardinality of indexes on a partitioned table
  
  Added test case
sql/ha_partition.cc:
  Bug#44059: Incorrect cardinality of indexes on a partitioned table
  
  Checking which partition that has the most rows, and using that
  partition for HA_STATUS_CONST instead of first partition

is reached

Problem was bad error handling, leaving some new temporary
partitions locked and initialized and some not yet initialized
and locked, leading to a crash when trying to unlock the not
yet initialized and locked partitions

Solution was to unlock the already locked partitions, and not
include any of the new temporary partitions in later unlocks

mysql-test/r/partition_open_files_limit.result:
  Bug#46922: crash when adding partitions and open_files_limit
  is reached
  
  New test result
mysql-test/t/partition_open_files_limit-master.opt:
  Bug#46922: crash when adding partitions and open_files_limit
  is reached
  
  New test opt-file for testing when open_files_limit is reached
mysql-test/t/partition_open_files_limit.test:
  Bug#46922: crash when adding partitions and open_files_limit
  is reached
  
  New test case testing when open_files_limit is reached
sql/ha_partition.cc:
  Bug#46922: crash when adding partitions and open_files_limit
  is reached
  
  When cleaning up the partitions already locked need to be unlocked,
  and not be unlocked/closed after cleaning up.

 - Read plug.in to fid the name of the engine to link with, does not have
   to be same as engine dir
 - Use engine dir when figuring out which libraries to build limbysqld with

 - Make it possible for the CmakeLists.txt files in an engine to use
   ${engine}_LIBS to set additional libraries to link with
 Example: NDBCLUSTER_LIBS = ndbclient

can lead to bad memory access

Problem: Field_bit is the only field which returns INT_RESULT
and doesn't have unsigned flag. As it's not a descendant of the 
Field_num, so using ((Field_num *) field_bit)->unsigned_flag may lead
to unpredictable results.

Fix: check the field type before casting.


mysql-test/r/type_bit.result:
  Fix for bug #42803: Field_bit does not have unsigned_flag field,
  can lead to bad memory access
    - test result.
mysql-test/t/type_bit.test:
  Fix for bug #42803: Field_bit does not have unsigned_flag field,
  can lead to bad memory access
    - test case.
sql/opt_range.cc:
  Fix for bug #42803: Field_bit does not have unsigned_flag field,
  can lead to bad memory access
    - don't cast to (Field_num *) Field_bit, as it's not a Field_num
  descendant and is always unsigned by nature.

buffering is used

FORCE INDEX FOR ORDER BY now prevents the optimizer from 
using join buffering. As a result the optimizer can use
indexed access on the first table and doesn't need to 
sort the complete resultset at the end of the statement.

 - Remove mf_strip.c and the declaration of 'strip_sp'

The pthread_cond_wait implementations for windows might
dead lock in some rare circumstances.

1) One thread (I) enter a timed wait and at a point in
   time ends up after mutex unlock and before
   WaitForMultipleObjects(...)
2) Another thread (II) enters pthread_cond_broadcast.
   Grabs the mutex and discovers one waiter. It set
   the broadcast event and closes the broadcast gate
   then unlocks the mutex.
3) A third thread (III) issues a pthread_cond_signal.
   It grabs the mutex, discovers one waiter, sets the
   signal event then unlock the mutex.
4) The first threads (I) enters WaitForMultipleObjects
   and finds out that the signal object is in a
   signalled state and exits the wait.
5) Thread (I) grabs the mutex and checks result status.
   The number of waiters is decreased and becomes equal
   to 0. The event returned was a signal event so the
   broadcast gate isn't opened. The mutex is released.
6) Thread (II) issues a new broadcast. The mutex is
   acquired but the number of waiters are 0 hence
   the broadcast gate remains closed.
7) Thread (I) enters the wait again but is blocked by
   the broadcast gate.

      This fix resolves the above issue by always resetting
      broadcast gate when there are no more waiters in th queue.


mysys/my_wincond.c:
  * Always reset the broadcast gate if there are no more waiters left.

  Let
    - T be a transactional table and N non-transactional table.
    - B be begin, C commit and R rollback.
    - N be a statement that accesses and changes only N-tables.
    - T be a statement that accesses and changes only T-tables.

In RBR, changes to N-tables that happen early in a transaction are not immediately flushed
upon committing a statement. This behavior may, however, break consistency in the presence
of concurrency since changes done to N-tables become immediately visible to other
connections. To fix this problem, we do the following:

  . B N N T C would log - B N C B N C B T C.
  . B N N T R would log - B N C B N C B T R.

Note that we are not preserving history from the master as we are introducing a commit that
never happened. However, this seems to be more acceptable than the possibility of breaking
consistency in the presence of concurrency.

Let
  - T be a transactional table and N non-transactional table.
  - B be begin, C commit and R rollback.
  - M be a mixed statement, i.e. a statement that updates both T and N.
  - M* be a mixed statement that fails while updating either T or N.

This patch restore the behavior presented in 5.1.37 for rows either produced in
the RBR or MIXED modes, when a M* statement that happened early in a transaction
had their changes written to the binary log outside the boundaries of the
transaction and wrapped in a BEGIN/ROLLBACK. This was done to keep the slave
consistent with with the master as the rollback would keep the changes on N and
undo them on T. In particular, we do what follows:

  . B M* T C would log - B M* R B T C.

Note that, we are not preserving history from the master as we are introducing a
rollback that never happened. However, this seems to be more acceptable than
making the slave diverge. We do not fix the following case:

  . B T M* C would log B T M* C.

The slave will diverge as the changes on T tables that originated from the M
statement are rolled back on the master but not on the slave. Unfortunately, we
cannot simply rollback the transaction as this would undo any uncommitted
changes on T tables.

SBR is not considered in this patch because a failing statement is written to
the binary along with the error code and a slave executes and then rolls back
the statement when it has an associated error code, thus undoing the effects
on T. In RBR and MBR, a full-fledged fix will be pushed after the WL 2687.

Mask part of EXPLAIN output with '#' to account for varying row count estimation.

mysql-test/include/mix1.inc:
  Mask 'rows' column in EXPLAIN output (number varies sometimes between 1 and 2).
mysql-test/r/innodb_mysql.result:
  Update result file after masking of rows estimation in EXPLAIN output.

1. BUG#46256 - drop table with unknown collation crashes innodb

Note: No testcase attached and has to be verified manually

Detailed revision comments:

r5799 | calvin | 2009-09-09 20:47:31 +0300 (Wed, 09 Sep 2009) | 10 lines
branches/5.1: fix bug#46256

Allow tables to be dropped even if the collation is not found,
but issue a warning.

Could not find an easy way to add mysql-test since it requires
changes to charsets and restarting the server. Tests were
executed manually.

Approved by: Heikki (on IM)
r5805 | vasil | 2009-09-10 08:41:48 +0300 (Thu, 10 Sep 2009) | 7 lines
branches/5.1:

Fix a compilation warning caused by c5799:

handler/ha_innodb.cc: In function 'void innobase_get_cset_width(ulint, ulint*, ulint*)':
handler/ha_innodb.cc:830: warning: format '%d' expects type 'int', but argument 2 has type 'ulint'

BUG#44369 - InnoDB: Does not uniformly disallow disallowed column names

Detailed revision comments:

r5741 | jyang | 2009-09-03 07:16:01 +0300 (Thu, 03 Sep 2009) | 5 lines
branches/5.1: Block creating table with column name conflicting
with Innodb reserved key words. (Bug #44369) rb://151 approved
by Sunny Bains.


r5760 | jyang | 2009-09-04 07:07:34 +0300 (Fri, 04 Sep 2009) | 3 lines
branches/5.1: This is to revert change 5741. A return status for
create_table_def() needs to be fixed.

r5834 | jyang | 2009-09-11 00:43:05 +0300 (Fri, 11 Sep 2009) | 5 lines
branches/5.1: Block creating table with column name conflicting
with Innodb reserved key words. (Bug #44369) rb://151 approved
by Sunny Bains.

1. BUG#46000 - using index called GEN_CLUST_INDEX crashes server

Detailed revision comments:

r5895 | jyang | 2009-09-15 03:39:21 +0300 (Tue, 15 Sep 2009) | 5 lines
branches/5.1: Disallow creating index with the name of
"GEN_CLUST_INDEX" which is reserved for the default system
primary index. (Bug #46000) rb://149 approved by Marko Makela.