columns again

This is follow-up to Bug #54358. Not all occurrences of the bug were fixed.
We need to check all calls to btr_copy_externally_stored_field_prefix_low()
and do the right thing when the pointer to the off-page column is null
(full of zero bytes).

It turns out that only the call to btr_copy_externally_stored_field_prefix()
in row_sel_sec_rec_is_for_blob() needs to be changed.

For fetching complete off-page columns rather than prefixes, the function
btr_rec_copy_externally_stored_field() already checks if the pointer
is null (all-zero).  Two of its callers (row_merge_copy_blobs() and
row_sel_fetch_columns()) are never executed as READ COMMITTED and can
rightfully assert that the fetch succeeded.  The third caller,
row_sel_store_mysql_rec(), already does the right thing.

The calls in row_upd_ext_fetch() and trx_undo_page_fetch_ext() must
expect that the off-page column exists. Update and rollback are
locking operations, never READ UNCOMMITTED.

On Windows, the parameter for number of bytes passed into WriteFile()
and ReadFile() is DWORD. Casting is needed to silence the warning on
64-bit Windows.

Also, adding several asserts to ensure the variable for number of bytes
is no more than 32 bits, even on 64-bit Windows.

This is for InnoDB Plugin.

rb://415
Approved by: Inaam

On Windows, the parameter for number of bytes passed into WriteFile()
and ReadFile() is DWORD. Casting is needed to silence the warning on
64-bit Windows.

Also, adding several asserts to ensure the variable for number of bytes
is no more than 32 bits, even on 64-bit Windows.

This is for built-in InnoDB.

rb://415
Approved by: Inaam

rb://498

Fix handling of update_undo_logs at trx commit. Previously, when
rseg->update_undo_list grows beyond 500 the update_undo_logs were
marked with state TRX_UNDO_TO_FREE which should have been
TRX_UNDO_TO_PURGE.

Approved by: Sunny Bains

Approved by Sunny Bains

"Grantor" columns' data is lost when replicating mysql.tables_priv.
Slave SQL thread used its default user ''@'' as the grantor of GRANT|REVOKE
statements executing on it.

In this patch, current user is put in query log event for all GRANT and REVOKE
statement, SQL thread uses the user in query log event as grantor.


mysql-test/suite/rpl/r/rpl_do_grant.result:
  Add test for this bug.
mysql-test/suite/rpl/t/rpl_do_grant.test:
  Add test for this bug.
sql/log_event.cc:
  Refactoring THD::current_user_used and related functions.
  current_user_used is used to judge if current user should be
  binlogged in query log event. So it is better to call it m_binlog_invoker.
  The related functions are renamed too.
sql/sql_class.cc:
  Refactoring THD::current_user_used and related functions.
  current_user_used is used to judge if current user should be
  binlogged in query log event. So it is better to call it m_binlog_invoker.
  The related functions are renamed too.
sql/sql_class.h:
  Refactoring THD::current_user_used and related functions.
  current_user_used is used to judge if current user should be
  binlogged in query log event. So it is better to call it m_binlog_invoker.
  The related functions are renamed too.
sql/sql_parse.cc:
  Call binlog_invoker() for GRANT and REVOKE statements.

  Simplified cases where a select was used to compare variable against ''

Rows events were applied wrongly on the temporary table with the same name.
But rows events are generated only for base tables. As temporary
table's data never be binlogged on row mode. Normally, base table of the
same name cannot be updated if a temporary table has the same name.
But there are two cases which can generate rows events on 
the base table of same name.
      
Case1: 'CREATE TABLE ... SELECT' statement.
In mixed format, it will generate rows events if it is unsafe.
      
Case2: Drop a transactional temporary table in a transaction
       (happens only on 5.5+).
BEGIN;
DROP TEMPORARY TABLE t1;       # t1 is a InnoDB table
INSERT INTO t1 VALUES(rand()); # t1 is a MyISAM table
COMMIT;
'DROP TEMPORARY TABLE' will be put in the transaction cache and
binlogged after the rows events generated by the 'INSERT' statement.
      
After this patch, slave opens only base table when applying a rows event.

index or foreign key

Approved by Sunny Bains

Fix assorted warnings that are generated in optimized builds.
Most of it is silencing variables that are set but unused.

This patch also introduces the MY_ASSERT_UNREACHABLE macro
which helps the compiler to deduce that a certain piece of
code is unreachable.

include/my_compiler.h:
  Use GCC's __builtin_unreachable if available. It allows
  GCC to deduce the unreachability of certain code paths,
  thus avoiding warnings that, for example, accused that a
  variable could be used without being initialized (due to
  unreachable code paths).

Fix assorted compiler warnings.

include/my_pthread.h:
  Like for pthread_cond_timedwait, the abstime is constant.
mysys/my_gethwaddr.c:
  Instead of using a manual copy that introduce warnings due to
  type mismatch, copy the buffer using memcpy and use memcmp to
  check whether all bytes of the buffer are zeroed.
mysys/thr_mutex.c:
  Like for pthread_cond_timedwait, the abstime is constant.
unittest/mytap/tap.h:
  Introduce a ok() variant that does not take a format argument.
  Since ok() is tagged with a printf attribute, GCC complains if
  the fmt argument is NULL.

Limited to actual bug fix, fixing a while condition
Again confirmed on Linux PPC and on AIX 5.3

Tag or remove unused arguments and variables.

regex/main.c:
  Use the real prototype.
sql/ha_ndbcluster.cc:
  Make conditions less ambiguous.

Ensure that fdatasync is properly declared as on Mac OS X, the
function is available but there is no prototype. Also, port a
fix for a warning from the InnoDB plugin over to the builtin. 

configure.in:
  Check that fdatasync is declared.
mysys/my_sync.c:
  Use fdatasync only if it is declared.
storage/innobase/include/ut0dbg.h:
  Port over from the plugin a fix for a warning.

Just check for all the functions that we are going to use, not a subset
of them.

Reviewed by:	Davi (via IRC)

Tag unused arguments.
      
Approved by: Marko (via IRC)

Tag unused arguments.

Approved by: Marko (via IRC)

Fix assorted compiler warnings on Mac OS X.

BUILD/SETUP.sh:
  Remove -Wctor-dtor-privacy flag to workaround a GCC bug that
  causes it to not properly detect that implicitly generated
  constructors are always public.
cmd-line-utils/readline/terminal.c:
  tgetnum and tgetflag might not take a const string argument.
mysys/my_gethostbyname.c:
  Tag unused arguments.
mysys/my_sync.c:
  Tag unused arguments.

Added some more info in a number of fail cases
(re-commit for administrative reasons)

main.mysqltest skipped on Windows because a perl intentionally does exit(1)
Use exit(2), as exit(1) on Windows is indistinguishable from failing to
execute perl.

           data dictionary confusion

On file systems with case insensitive file names, and
lower_case_table_names set to '2', the server could crash
due to a table definition cache inconsistency.  This is 
the default setting on MacOSX, but may also be set and
used on MS Windows.

The bug is caused by using two different strategies for
creating the hash key for the table definition cache, resulting
in failure to look up an entry which is present in the cache,
or failure to delete an existing entry.  One strategy was to
use the real table name (with case preserved), and the other
to use a normalized table name (i.e a lower case version).

This is manifested in two cases.  One is  during 'DROP DATABASE', 
where all known files are removed.  The removal from
the table definition cache is done via a generated list of
TABLE_LIST with keys (wrongly) created using the case preserved 
name.  The other is during CREATE TABLE, where the cache lookup
is also (wrongly) based on the case preserved name.
   
The fix was to use only the normalized table name when
creating hash keys.


sql/sql_db.cc:
  Normalize table name (i.e lower case it)
sql/sql_table.cc:
  table_name contains the normalized name
  alias contains the real table name

           (variables_debug fails)

The problem was that "SET GLOBAL debug" could cause a crash on Solaris.
The crash happened if the server failed to open the trace file given in 
the "SET GLOBAL debug" statement. This caused an error message to be
printed to stderr containing the process name. However, printing to
stderr crashed the server since the pointer to the process name had
not been initialized.

This patch fixes the problem by initializing the process name 
properly when doing "SET GLOBAL debug".

No test case added as this bug was repeatable with existing test
coverage in variables_debug.test.

row_search_for_mysql(): When a secondary index record might not be
visible in the current transaction's read view and we consult the
clustered index and optionally some undo log records, return the
relevant columns of the clustered index record to MySQL instead of the
secondary index record.

ibuf_insert_to_index_page_low(): New function, refactored from
ibuf_insert_to_index_page().

ibuf_insert_to_index_page(): When we are inserting a record in place
of a delete-marked record and some fields of the record differ, update
that record just like row_ins_sec_index_entry_by_modify() would do.

btr_cur_update_alloc_zip(): Make the function public.

mysql_row_templ_t: Add clust_rec_field_no.

row_sel_store_mysql_rec(), row_sel_push_cache_row_for_mysql(): Add the
flag rec_clust, for returning data at clust_rec_field_no instead of
rec_field_no. Resurrect the debug assertion that the record not be
marked for deletion. (Bug #55626)

[UNIV_DEBUG || UNIV_IBUF_DEBUG] ibuf_debug, buf_page_get_gen(),
buf_flush_page_try():
Implement innodb_change_buffering_debug=1 for evicting pages from the
buffer pool, so that change buffering will be attempted more
frequently.

row_search_for_mysql(): When a secondary index record might not be
visible in the current transaction's read view and we consult the
clustered index and optionally some undo log records, return the
relevant columns of the clustered index record to MySQL instead of the
secondary index record.

REC_INFO_DELETED_FLAG: Move the definition from rem0rec.ic to rem0rec.h.

ibuf_insert_to_index_page_low(): New function, refactored from
ibuf_insert_to_index_page().

ibuf_insert_to_index_page(): When we are inserting a record in place
of a delete-marked record and some fields of the record differ, update
that record just like row_ins_sec_index_entry_by_modify() would do.

mysql_row_templ_t: Add clust_rec_field_no.

row_sel_store_mysql_rec(), row_sel_push_cache_row_for_mysql(): Add the
flag rec_clust, for returning data at clust_rec_field_no instead of
rec_field_no. Resurrect the debug assertion that the record not be
marked for deletion. (Bug #55626)

buf_LRU_free_block(): Refactored from
buf_LRU_search_and_free_block(). This is needed for the
innodb_change_buffering_debug diagnostics.

[UNIV_DEBUG || UNIV_IBUF_DEBUG] ibuf_debug, buf_page_get_gen(),
buf_flush_page_try():
Implement innodb_change_buffering_debug=1 for evicting pages from the
buffer pool, so that change buffering will be attempted more
frequently.

Enable the MySQL maintainer-specific development environment
(which add various warning related options to the compiler
flags) if debugging support is enabled.

config/ac-macros/maintainer.m4:
  Enable the maintainer mode if debug support is enabled.
configure.in:
  Move debug argument to before the maintainer mode check.

because currently hostname stored in db in lowercase.

For crash testing: kill the server without generating core file.

include/my_dbug.h
  Use kill(getpid(), SIGKILL) which cannot be caught by signal handlers.
  All DBUG_XXX macros should be no-ops in optimized mode, do that for DBUG_ABORT as well.
sql/handler.cc
  Kill server without generating core.
sql/log.cc
  Kill server without generating core.

This is a merge from 5.1/builtin to 5.1/plugin of:

  --------------
  revision-id: vasil.dimov@oracle.com-20101018104811-nwqhg9vav17kl5s1
  committer: Vasil Dimov <vasil.dimov@oracle.com>
  timestamp: Mon 2010-10-18 13:48:11 +0300
  message:
    Fix Bug#57252 disabling innobase_stats_on_metadata disables ANALYZE
    
    In order to fix this bug we need to distinguish whether ha_innobase::info()
    has been called from ::analyze() or not. Rename ::info() to ::info_low()
    and add a boolean parameter that tells whether the call is from ::analyze()
    or not. Create a new simple ::info() that just calls
    ::info_low(false => not called from analyze). From ::analyze() instead of
    ::info() call ::info_low(true => called from analyze).
    
    Approved by:	Jimmy (rb://487)
  --------------

In order to fix this bug we need to distinguish whether ha_innobase::info()
has been called from ::analyze() or not. Rename ::info() to ::info_low()
and add a boolean parameter that tells whether the call is from ::analyze()
or not. Create a new simple ::info() that just calls
::info_low(false => not called from analyze). From ::analyze() instead of
::info() call ::info_low(true => called from analyze).

Approved by:	Jimmy (rb://487)

          replication aborts

When recieving a 'SLAVE STOP' command, slave SQL thread will roll back the
transaction and stop immidiately if there is only transactional table updated,
even through 'CREATE|DROP TEMPOARY TABLE' statement are in it. But These
statements can never be rolled back. Because the temporary tables to the user
session mapping remain until 'RESET SLAVE', Therefore it will abort SQL thread
with an error that the table already exists or doesn't exist, when it restarts
and executes the whole transaction again.

After this patch, SQL thread always waits till the transaction ends and then stops,
if 'CREATE|DROP TEMPOARY TABLE' statement are in it.

mysql-test/extra/rpl_tests/rpl_stop_slave.test:
  Auxiliary file which is used to test this bug.
mysql-test/suite/rpl/t/rpl_stop_slave.test:
  Test case for this bug.
sql/slave.cc:
  Checking if OPTION_KEEP_LOG is set. If it is set, SQL thread should wait
  until the transaction ends.
sql/sql_parse.cc:
  Add a debug point for testing this bug.

It was added in jimmy.yang@oracle.com-20100902004302-m5ax9qo0ne2msdfx