'flush tables' crashes

The server crashes when 'show procedure status' and 'flush tables' are
run concurrently.

This is caused by the way mysql.proc table is added twice to the list
of table to lock although the requirements on the current locking API
assumes differently.

No test case is submitted because of the nature of the crash which is 
currently difficult to reproduce in a deterministic way.

This is a backport from 5.1

myisam/mi_dbug.c:
  * check_table_is_closed is only used in EXTRA_DEBUG mode but since it is
  iterating over myisam shared data it still needs to be protected by an
  appropriate mutex.
sql/sql_yacc.yy:
  * Since the I_S mechanism is already handling the open and close of 
  mysql.proc there is no need for the method sp_add_to_query_tables.

The 'BEGIN/COMMIT/ROLLBACK' log event could be filtered out if the
database is not selected by --database option of mysqlbinlog command.
This can result in problem if there are some statements in the
transaction are not filtered out.

To fix the problem, mysqlbinlog will output 'BEGIN/ROLLBACK/COMMIT' 
in regardless of the database filtering rules.

client/mysqlbinlog.cc:
  Skip the database check for BEGIN/COMMIT/ROLLBACK log events.
mysql-test/r/mysqlbinlog.result:
  Test result for bug#46998
mysql-test/t/mysqlbinlog.test:
  Added test to verify if the 'BEGIN', 'COMMIT' and 'ROLLBACK' are output
  in regardless of database filtering

          replication
              
MySQL server uses wrong lock type (always TL_READ instead of
TL_READ_NO_INSERT when appropriate) for tables used in
subqueries of UPDATE statement. This leads in some cases to
a broken replication as statements are written in the wrong
order to the binlog.

sql/sql_yacc.yy:
  * Set lock_option to either TL_READ_NO_INSERT or
    TL_READ for any sub-SELECT following UPDATE.
  * Changed line adjusted for parser identation
    rules; code begins at column 13.

Add --help option.

 query
      
The fix for bug 46749 removed the check for OUTER_REF_TABLE_BIT 
and substituted it for a check on the presence of 
Item_ident::depended_from.
Removing it altogether was wrong : OUTER_REF_TABLE_BIT should 
still be checked in addition to depended_from (because it's not 
set in all cases and doesn't contradict to the check of depended_from).
Fixed by returning the old condition back as a compliment to the 
new one.

 - Create the "dummy" thread joinable and wait for it to
   exit before continuing in 'my_thread_global_init'
 - This way we know that the pthread library is initialized
   by one thread only

      
Initialize correctly client flags for the embedded client
Test cases in jp updated to work correctly with embedded server.

When parsing the service installation parameter in 
default_service_handling() make sure the value of the
optional parameter doesn't overwrite it's name.

with gcc 4.3.2
      
This is the fifth patch cleaning up more GCC warnings about
variables used before initialized using the new macro
UNINIT_VAR().

The problem is that argument buffer can be used as result buffer
and it leads to argument value change.
The fix is to use 'old buffer' as result buffer only
if first argument is not constant item.


mysql-test/r/func_str.result:
  test result
mysql-test/t/func_str.test:
  test case
sql/item_strfunc.cc:
  The problem is that argument buffer can be used as result buffer
  and it leads to argument value change.
  The fix is to use 'old buffer' as result buffer only
  if first argument is not constant item.

Updates the results of all the out-dated test suites and adds 
the special mysqltest command to enable innodb for the tests that need it.

            can crash under load

Backport from 5.1.
Does also include key cache fixes from:
Bug 44068 (RESTORE can disable the MyISAM Key Cache)
Bug 40944 (Backup: crash after myisampack)



include/keycache.h:
  Bug#17332 - changing key_buffer_size on a running server
              can crash under load
  Added KEY_CACHE components in_resize and waiting_for_resize_cnt.
myisam/mi_preload.c:
  Bug#17332 - changing key_buffer_size on a running server
              can crash under load
  Added code to allow LOAD INDEX to load indexes of different block size.
mysys/mf_keycache.c:
  Bug#17332 - changing key_buffer_size on a running server
              can crash under load
  .
  Changed resize_key_cache() to not disable the key cache
  after the flush phase. Changed queue handling to use
  standard functions. Wake all threads waiting on resize_queue.
  We can now have read/write threads waiting there (see below).
  .
  Combined add_to_queue() and the wait loops that were always
  following it to the new function wait_on_queue().
  Combined release_queue() and the condition that was always
  preceding it to the new function release_whole_queue().
  .
  Added code to flag and respect the exceptional situation
  BLOCK_IN_EVICTION.
  .
  Rewrote the resize branch of find_key_block().
  .
  Added code to the eviction handling in find_key_block()
  to catch more exceptional cases.
  .
  Changed key_cache_read(), key_cache_insert() and key_cache_write()
  so that they lock keycache->cache_lock whenever the key cache is
  initialized. Checking for a disabled cache and incrementing and
  decrementing the "resize counter" is always done within the lock.
  Locking and unlocking as well as counting the "resize counter" is
  now done once outside the loop. All three functions can now handle
  a NULL return from find_key_block. This happens in the flush phase
  of a resize and demands direct file I/O. Care is taken for
  secondary requests (PAGE_WAIT_TO_BE_READ) to wait in any case.
  Moved block status changes behind the copying of buffer data.
  key_cache_insert() does now read the block if the caller did
  supply less data than a full cache block.
  key_cache_write() does now take care of parallel running flushes
  (BLOCK_FOR_UPDATE, BLOCK_IN_FLUSHWRITE).
  .
  Changed free_block() to un-initialize block variables in the
  correct order and respect an exceptional BLOCK_IN_EVICTION state.
  .
  Changed flushing to take care for parallel running writes.
  Changed flushing to avoid freeing blocks in eviction.
  Changed flushing to consider that parallel writes can move blocks
  from the file_blocks hash to the changed_blocks hash.
  Changed flushing to take care for other parallel flushes.
  Changed flushing to assure that it ends with everything flushed.
  Optimized normal flush at end of statement (FLUSH_KEEP),
  but let other flush types be stringent.
  .
  Added some comments and debugging statements.
mysys/my_static.c:
  Bug#17332 - changing key_buffer_size on a running server
              can crash under load
  Removed an unused global variable.
sql/ha_myisam.cc:
  Bug#17332 - changing key_buffer_size on a running server
              can crash under load
  Moved an automatic (stack) variable to the scope where it is used.
sql/sql_table.cc:
  Bug#17332 - changing key_buffer_size on a running server
              can crash under load
  Changed TL_READ to TL_READ_NO_INSERT in mysql_preload_keys.

Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.


mysql-test/r/explain.result:
  test result
mysql-test/t/explain.test:
  test case
sql/sql_select.cc:
  Memory allocated in TMP_TABLE_PARAM::copy_field is not cleaned up.
  The fix is to clean up TMP_TABLE_PARAM::copy_field array in JOIN::destroy.

                    name as existing view

When trying to create a table with the same name as existing view with
join, mysql server crashes.

The problem is when create table is issued with the same name as view, while
verifying with the existing tables, we assume that base table object is 
created always.

In this case, since it is a view over multiple tables, we don't have the 
mysql derived table object.

Fixed the logic which checks if there is an existing table to not to assume
that table object is created when the base table is view over multiple 
tables.

mysql-test/r/create.result:
  BUG#46384 - mysqld segfault when trying to create table with same 
              name as existing view
  
  Testcase for the bug
mysql-test/t/create.test:
  BUG#46384 - mysqld segfault when trying to create table with same 
              name as existing view
  
  Testcase for the bug
sql/sql_insert.cc:
  BUG#46384 - mysqld segfault when trying to create table with same 
                  name as existing view
      
  Fixed create_table_from_items() method to properly check, if the base table 
  is a view over multiple tables.

                                for CREATE TABLE...LIKE...

Add my_sync.c to mysqltest sources list in CMakeLists.txt

client/CMakeLists.txt:
  BUG#46591 - .frm file isn't sync'd with sync_frm enabled
              for CREATE TABLE...LIKE...
  
  Add my_sync.c to mysqltest sources list

myisamchk tool generates warnings when run on an myisam files (.MYI or .MYD)
This is because of the conversion of max_value for certain options in myisamchk 
from singed long to unsigned long

The max value for the options key_buffer_size, read_buffer_size, write_buffer
_size and sort_buffer_size is given as (long) ~0L which becomes -1  when casted
from signed long to longlong and then casted to ulonglong. When (ulonglong) -1 
is compared with maximal value for GET_ULONG data type, we adjust it to 
(ulonglong) ULONG_MAX and throw the warning.

Fixed by using the right max size.

Max values for the variables (from mysqld.cc)
----------------------------
1. key_buffer_size
   5.0: ULONG_MAX
   5.1: SIZE_T_MAX
   6.0: SIZE_T_MAX

2. read_buffer_size and write_buffer_size
   5.0: INT_MAX32
   5.1: INT_MAX32
   6.0: INT_MAX32

3. sort_buffer_size (aka myisam_sort_buffer_size)
   5.0: UINT_MAX32
   5.1: ULONG_MAX
   6.0: ULONG_MAX

Note: testcase not attached

myisam/myisamchk.c:
  Bug#33785 - myisamchk show warning message
      
  Fixed the Max value for key_buffer_size, read_buffer_size, write_buffer_size and
  sort_buffer_size options

  function,file sql_base.cc

When uncacheable queries are written to a temp table the optimizer must 
preserve the original JOIN structure, because it is re-using the JOIN 
structure to read from the resulting temporary table.
This was done only for uncacheable sub-queries. 
But top level queries can also benefit from this mechanism, specially if 
they're using index access and need a reset.
Fixed by not limiting the saving of JOIN structure to subqueries
exclusively.
Added a new test file to extend the existing (large) subquery.test.

                    CREATE TABLE...LIKE...
      
The mysql server option 'sync_frm' is ignored when table is created with 
syntax CREATE TABLE .. LIKE.. 
      
Fixed by adding the MY_SYNC flag and calling my_sync() from my_copy() when
the flag is set.

In mysql_create_table(), when the 'sync_frm' is set, MY_SYNC flag is passed 
to my_copy(). 
      
Note: TestCase is not attached and can be tested manually using debugger.

client/Makefile.am:
  BUG#46591 - .frm file isn't sync'd with sync_frm enabled for 
              CREATE TABLE...LIKE...
      
  add my_sync to sources as it is used in my_copy() method
include/my_sys.h:
  BUG#46591 - .frm file isn't sync'd with sync_frm enabled for 
              CREATE TABLE...LIKE...
      
  MY_SYNC flag is added to call my_sync() method
mysys/my_copy.c:
  BUG#46591 - .frm file isn't sync'd with sync_frm enabled for 
              CREATE TABLE...LIKE...
      
  my_sync() is method is called when MY_SYNC is set in my_copy()
sql/sql_table.cc:
  BUG#46591 - .frm file isn't sync'd with sync_frm enabled for 
              CREATE TABLE...LIKE...
      
  Fixed mysql_create_like_table() to call my_sync() when opt_sync_frm variable
  is set

with the newer pb2 testing environments

Failing to connect would release parts of the MYSQL struct.
We would then proceed to try again to connect without re-
initializing the struct.

We prevent the unwanted freeing of data we'll still need now.


client/mysqladmin.cc:
  Losing a connection (or not even getting on in the first place) should
  not trash the MYSQL-struct.
  
  Add a lot of comments.
  
  Rewrite re-connection fu.
sql-common/client.c:
  Assert against bad parameters usually caused by de-initing a
  MYSQL-struct without re-initing it again before re-use.

with gcc 4.3.2
      
This patch fixes a number of GCC warnings about variables used
before initialized. A new macro UNINIT_VAR() is introduced for
use in the variable declaration, and LINT_INIT() usage will be
gradually deprecated. (A workaround is used for g++, pending a
patch for a g++ bug.)
      
GCC warnings for unused results (attribute warn_unused_result)
for a number of system calls (present at least in later
Ubuntus, where the usual void cast trick doesn't work) are
also fixed.


client/mysqlmanager-pwgen.c:
  A fix for warn_unused_result, adding fallback to use of
  srand()/rand() if /dev/random cannot be used. Also actually
  adds calls to rand() in the second branch so that it actually
  creates a random password.

When a connection is dropped any remaining temporary table is also automatically
dropped and the SQL statement of this operation is written to the binary log in
order to drop such tables on the slave and keep the slave in sync. Specifically,
the current code base creates the following type of statement:
DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `db`.`table`;

Unfortunately, appending the database to the table name in this manner circumvents
the replicate-rewrite-db option (and any options that check the current database).
To solve the issue, we started writing the statement to the binary as follows:
use `db`; DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `table`;

  field references

This error requires a combination of factors : 
1. An "impossible where" in the outermost SELECT
2. An aggregate in the outermost SELECT
3. A correlated subquery with a WHERE clause that includes an outer 
field reference as a top level WHERE sargable predicate

When JOIN::optimize detects an "impossible WHERE" it will bail out
without doing the rest of the work and initializations. It will not
call make_join_statistics() as well.  And make_join_statistics fills 
in various structures for each table referenced.
When processing the result of the "impossible WHERE" the query must
send a single row of data if there are aggregate functions in it.
In this case the server marks all the aggregates as having received 
no rows and calls the relevant Item::val_xxx() method on the SELECT
list. However if this SELECT list happens to contain a correlated 
subquery this subquery is evaluated in a normal evaluation mode.
And if this correlated subquery has a reference to a field from the 
outermost "impossible where" SELECT the add_key_fields will mistakenly
consider the outer field reference as a "local" field reference when 
looking for sargable predicates.
But since the SELECT where the outer field reference refers to is not
completely initialized due to the "impossible WHERE" in this level
we'll get a NULL pointer reference.
Fixed by making a better condition for discovering if a field is "local"
to the SELECT level being processed. 
It's not enough to look for OUTER_REF_TABLE_BIT in this case since 
for outer references to constant tables the Item_field::used_tables() 
will return 0 regardless of whether the field reference is from the 
local SELECT or not.