------------------------------------------------------------
revno 2876.244.305
revision id marko.makela@oracle.com-20110413082211-e6ouhjz5rmqxcqap
parent  marko.makela@oracle.com-20110413075948-kvytmc37ye1nt7d9
committer  Marko Mäkelä <marko.makela@oracle.com>
branch nick 5.6-innodb
timestamp Wed 2011-04-13 11:22:11 +0300
message:
  Suppress the Bug #58815 (Bug #11765812) assertion failure.

  buf_page_get_gen(): Introduce BUF_GET_POSSIBLY_FREED for suppressing the
  check that the file page must not have been freed.

  btr_estimate_n_rows_in_range_on_level(): Pass BUF_GET_POSSIBLY_FREED and
  explain in the comments why this is needed and why it should be mostly
  harmless to ignore the problem. If InnoDB had always initialized all
  unused fields in data files, no problem would exist.

  This change does not fix the bug, it just "shoots the messenger".

  rb:647 approved by Jimmy Yang

Automerged bzr bundle from bug report into latest mysql-5.5.

In RBR and in case of converting blob fields, the space allocated
while unpacking into the conversion field was not freed after
copying from it into the real field.

We fix this by freeing the conversion field when the conversion
table is not needed anymore (on close_tables_to_lock).

from 5.1->5.5.

when selecting from I_S and views exist, in SP.

Symptoms: re-execution of prepared statement (or statement in a stored
routine) which read from one of I_S tables and which in order to fill
this I_S table had to open a view led to increasing memory consumption.

What happened in this situation was that during the process of view
opening for purpose of I_S filling view-related structures (like its
LEX) were allocated on persistent MEM_ROOT of prepared statement (or 
stored routine). Since this MEM_ROOT is not freed until prepared
statement deallocation (or expulsion of stored routine from the cache)
and code responsible for filling I_S is not able to re-use results of
view opening from previous executions this allocation ended up in
memory hogging.

This patch solves the problem by ensuring that when a view opened
for the purpose of I_S filling all its structures are allocated on
non-persistent runtime MEM_ROOT. This is achieved by activating a
temporary Query_arena bound to this MEM_ROOT.
Since this step makes impossible linking of view structures into
LEX of our prepared statement (or stored routine statement) this
patch also changes code filling I_S table to install a proxy LEX
before trying to open a view or a table. Consequently some code
which was responsible for backing-up/restoring parts of LEX when
view/table was opened during filling of I_S table became redundant
and was removed.

This patch doesn't contain test case for this bug as it is hard
to test memory hogging in our test suite.

Manual merged mysql-5.1-gca into latest mysql-5.5.

Conflicts
=========
Text conflict in mysql-test/suite/rpl/r/rpl_relayspace.result
Text conflict in mysql-test/suite/rpl/t/rpl_relayspace.test

Automerge mysql-5.1-gca into latest mysql-5.1.

              VM-WIN2003-32-A, SLES10-IA64-A 
      
The test case waits for master_pos_wait not to timeout, which
means that the deadlock between SQL and IO threads was 
succesfully and automatically dealt with.
      
However, very rarely, master_pos_wait reports a timeout. This
happens because the time set for master_pos_wait to wait was
too small (6 seconds). On slow test env this could be a 
problem.
      
We fix this by setting the timeout inline with the one used
in sync_slave_with_master (300 seconds). In addition we 
refactored the test case and refined some comments.

Issue:
While running embedded server, if client issues TEE command (\T foo/bar) and
"foo/bar" directory doesn't exist, it is suppose to give error. But it was
aborting.  This was happening because wrong error handler was being called.

Solution:
Modified calls to correct error handler. In embedded server case, there are 
two error handler (client and server) which are supposed to be called based 
on which context code is in. If it is in client context, client error handler
should be called otherwise server.

Test case:
Test case automation is not possible as current (following) code doesn't 
allow '\T' to be executed from command line (OR command read from a file):
[client/mysql.cc]
...
 static int
 com_tee(String *buffer __attribute__((unused)),
         char *line __attribute__((unused)))
 {
   char file_name[FN_REFLEN], *end, *param;

   if (status.batch) << THIS IS TRUE WHILE EXECUTING FROM COMMAND LINE.
      return 0;
 ...
So, not adding test case in GA. WIll add a test case in mysql-trunk after 
removing above code so that this could be properly tested before GA.

libmysqld/lib_sql.cc:
  Added code to call client/server error handler based on in control is in 
  client/server code respectively.
sql/mysql_priv.h:
  Added comments for THR_THD, THR_MALLOC keys.
sql/sql_class.cc:
  Function definition of new function restore_global to removes thread specific 
  data from stack (which was stored in store global).
sql/sql_class.h:
  Function declaration of new function restore_global.

or Oracle Bug #11766513.

trx_undo_free_prepared(): Do not acquire or release trx->rseg->mutex.
This code is invoked in the single-threaded part of shutdown, therefore
a mutex is not needed.

There are two problems:
1. There is a missing check for 'year' parameter(year can not be greater than 9999) in
   makedate function. fix: added check that year can not be greater than 9999.
2. There is a missing check for zero date in from_days() function.
   fix: added zero date check into Item_func_from_days::get_date()
   function.


mysql-test/r/func_time.result:
  test case
mysql-test/t/func_time.test:
  test case
sql/item_timefunc.cc:
  --added check that year can not be greater than 9999 for makedate() function
  --added zero date check into Item_func_from_days::get_date() function

node->modification_counter == node->flush_counter

This bug (an assertion failure during a crash-like shutdown)
was introduced by the fix of
Bug#12323643 Clean up the InnoDB thread shutdown and assertions (WL#5136).

fil_node_close_file(): Relax the failing assertion
when innodb_fast_shutdown=2.

In sql_class.cc, 'row_count', of type 'ha_rows', was used as last argument for
ER_TRUNCATED_WRONG_VALUE_FOR_FIELD which is
"Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld".
So 'ha_rows' was used as 'long'.
On SPARC32 Solaris builds, 'long' is 4 bytes and 'ha_rows' is 'longlong' i.e. 8 bytes.
So the printf-like code was reading only the first 4 bytes.
Because the CPU is big-endian, 1LL is 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01
so the first four bytes yield 0. So the warning message had "row 0" instead of
"row 1" in test outfile_loaddata.test:
-Warning	1366	Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 1
+Warning	1366	Incorrect string value: '\xE1\xE2\xF7' for column 'b' at row 0

All error-messaging functions which internally invoke some printf-life function
are potential candidate for such mistakes.
One apparently easy way to catch such mistakes is to use
ATTRIBUTE_FORMAT (from my_attribute.h).
But this works only when call site has both:
a) the format as a string literal
b) the types of arguments.
So:
  func(ER(ER_BLAH), 10);
will silently not be checked, because ER(ER_BLAH) is not known at
compile time (it is known at run-time, and depends on the chosen
language).
And
  func("%s", a va_list argument);
has the same problem, as the *real* type of arguments is not
known at this site at compile time (it's known in some caller).
Moreover,
  func(ER(ER_BLAH));
though possibly correct (if ER(ER_BLAH) has no '%' markers), will not
compile (gcc says "error: format not a string literal and no format
arguments").

Consequences:
1) ATTRIBUTE_FORMAT is here added only to functions which in practice
take "string literal" formats: "my_error_reporter" and "print_admin_msg".
2) it cannot be added to the other functions: my_error(),
push_warning_printf(), Table_check_intact::report_error(),
general_log_print().

To do a one-time check of functions listed in (2), the following
"static code analysis" has been done:
1) replace
  my_error(ER_xxx, arguments for substitution in format)
with the equivalent
  my_printf_error(ER_xxx,ER(ER_xxx), arguments for substitution in
format),
so that we have ER(ER_xxx) and the arguments *in the same call site*
2) add ATTRIBUTE_FORMAT to push_warning_printf(),
Table_check_intact::report_error(), general_log_print()
3) replace ER(xxx) with the hard-coded English text found in
errmsg.txt (like: ER(ER_UNKNOWN_ERROR) is replaced with
"Unknown error"), so that a call site has the format as string literal
4) this way, ATTRIBUTE_FORMAT can effectively do its job
5) compile, fix errors detected by ATTRIBUTE_FORMAT
6) revert steps 1-2-3.
The present patch has no compiler error when submitted again to the
static code analysis above.
It cannot catch all problems though: see Field::set_warning(), in
which a call to push_warning_printf() has a variable error
(thus, not replacable by a string literal); I checked set_warning() calls
by hand though.

See also WL 5883 for one proposal to avoid such bugs from appearing
again in the future.

The issues fixed in the patch are:
a) mismatch in types (like 'int' passed to '%ld')
b) more arguments passed than specified in the format.
This patch resolves mismatches by changing the type/number of arguments,
not by changing error messages of sql/share/errmsg.txt. The latter would be wrong,
per the following old rule: errmsg.txt must be as stable as possible; no insertions
or deletions of messages, no changes of type or number of printf-like format specifiers,
are allowed, as long as the change impacts a message already released in a GA version.
If this rule is not followed:
- Connectors, which use error message numbers, will be confused (by insertions/deletions
of messages)
- using errmsg.sys of MySQL 5.1.n with mysqld of MySQL 5.1.(n+1)
could produce wrong messages or crash; such usage can easily happen if
installing 5.1.(n+1) while /etc/my.cnf still has --language=/path/to/5.1.n/xxx;
or if copying mysqld from 5.1.(n+1) into a 5.1.n installation.
When fixing b), I have verified that the superfluous arguments were not used in the format
in the first 5.1 GA (5.1.30 'bteam@astra04-20081114162938-z8mctjp6st27uobm').
Had they been used, then passing them today, even if the message doesn't use them
anymore, would have been necessary, as explained above.

include/my_getopt.h:
  this function pointer is used only with "string literal" formats, so we can add
  ATTRIBUTE_FORMAT.
mysql-test/collections/default.experimental:
  test should pass now
sql/derror.cc:
  by having a format as string literal, ATTRIBUTE_FORMAT check becomes effective.
sql/events.cc:
  Change justified by the following excerpt from sql/share/errmsg.txt:
  ER_EVENT_SAME_NAME
          eng "Same old and new event name"
  ER_EVENT_SET_VAR_ERROR
          eng "Error during starting/stopping of the scheduler. Error code %u"
sql/field.cc:
  ER_TOO_BIG_SCALE 42000 S1009
          eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu."
  ER_TOO_BIG_PRECISION 42000 S1009
          eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu."
  ER_TOO_BIG_DISPLAYWIDTH 42000 S1009
          eng "Display width out of range for column '%-.192s' (max = %lu)"
sql/ha_ndbcluster.cc:
  ER_OUTOFMEMORY HY001 S1001
          eng "Out of memory; restart server and try again (needed %d bytes)"
  (sizeof() returns size_t)
sql/ha_ndbcluster_binlog.cc:
  Too many arguments for:
  ER_GET_ERRMSG  
          eng "Got error %d '%-.100s' from %s"
  Patch by Jonas Oreland.
sql/ha_partition.cc:
  print_admin_msg() is used only with a literal as format, so ATTRIBUTE_FORMAT
  works.
sql/handler.cc:
  ER_OUTOFMEMORY HY001 S1001
          eng "Out of memory; restart server and try again (needed %d bytes)"
  (sizeof() returns size_t)
sql/item_create.cc:
  ER_TOO_BIG_SCALE 42000 S1009
          eng "Too big scale %d specified for column '%-.192s'. Maximum is %lu."
  ER_TOO_BIG_PRECISION 42000 S1009
          eng "Too big precision %d specified for column '%-.192s'. Maximum is %lu."
  'c_len' and 'c_dec' are char*, passed as %d !! We don't know their value
  (as strtoul() failed), but they are likely big, so we use INT_MAX.
  'len' is ulong.
sql/item_func.cc:
  ER_WARN_DATA_OUT_OF_RANGE 22003 
          eng "Out of range value for column '%s' at row %ld"
  ER_CANT_FIND_UDF  
          eng "Can't load function '%-.192s'"
sql/item_strfunc.cc:
  ER_TOO_BIG_FOR_UNCOMPRESS  
          eng "Uncompressed data size too large; the maximum size is %d (probably, length of uncompressed data was corrupted)"
  max_allowed_packet is ulong.
sql/mysql_priv.h:
  sql_print_message_func is a function _pointer_.
sql/sp_head.cc:
  ER_SP_RECURSION_LIMIT
          eng "Recursive limit %d (as set by the max_sp_recursion_depth variable) was exceeded for routine %.192s"
  max_sp_recursion_depth is ulong
sql/sql_acl.cc:
  ER_PASSWORD_NO_MATCH 42000 
          eng "Can't find any matching row in the user table"
  ER_CANT_CREATE_USER_WITH_GRANT 42000
          eng "You are not allowed to create a user with GRANT"
sql/sql_base.cc:
  ER_NOT_KEYFILE  
          eng "Incorrect key file for table '%-.200s'; try to repair it"
  ER_TOO_MANY_TABLES  
          eng "Too many tables; MySQL can only use %d tables in a join"
  MAX_TABLES is size_t.
sql/sql_binlog.cc:
  ER_UNKNOWN_ERROR  
          eng "Unknown error"
sql/sql_class.cc:
  ER_TRUNCATED_WRONG_VALUE_FOR_FIELD  
          eng "Incorrect %-.32s value: '%-.128s' for column '%.192s' at row %ld"
  WARN_DATA_TRUNCATED 01000 
          eng "Data truncated for column '%s' at row %ld"
sql/sql_connect.cc:
  ER_HANDSHAKE_ERROR 08S01 
          eng "Bad handshake"
  ER_BAD_HOST_ERROR 08S01 
          eng "Can't get hostname for your address"
sql/sql_insert.cc:
  ER_WRONG_VALUE_COUNT_ON_ROW 21S01 
          eng "Column count doesn't match value count at row %ld"
sql/sql_parse.cc:
  ER_WARN_HOSTNAME_WONT_WORK  
          eng "MySQL is started in --skip-name-resolve mode; you must restart it without this switch for this grant to work"
  ER_TOO_HIGH_LEVEL_OF_NESTING_FOR_SELECT
  	eng "Too high level of nesting for select"
  ER_UNKNOWN_ERROR  
          eng "Unknown error"
sql/sql_partition.cc:
  ER_OUTOFMEMORY HY001 S1001
          eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_plugin.cc:
  ER_OUTOFMEMORY HY001 S1001
          eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_prepare.cc:
  ER_OUTOFMEMORY HY001 S1001
          eng "Out of memory; restart server and try again (needed %d bytes)"
  ER_UNKNOWN_STMT_HANDLER  
          eng "Unknown prepared statement handler (%.*s) given to %s"
  length value (for '%.*s') must be 'int', per the doc of printf()
  and the code of my_vsnprintf().
sql/sql_show.cc:
  ER_OUTOFMEMORY HY001 S1001
          eng "Out of memory; restart server and try again (needed %d bytes)"
sql/sql_table.cc:
  ER_TOO_BIG_FIELDLENGTH 42000 S1009
          eng "Column length too big for column '%-.192s' (max = %lu); use BLOB or TEXT instead"
sql/table.cc:
  ER_NOT_FORM_FILE  
          eng "Incorrect information in file: '%-.200s'"
  ER_COL_COUNT_DOESNT_MATCH_PLEASE_UPDATE
          eng "Column count of mysql.%s is wrong. Expected %d, found %d. Created with MySQL %d, now running %d. Please use mysql_upgrade to fix this error."
  table->s->mysql_version is ulong.
sql/unireg.cc:
  ER_TOO_LONG_TABLE_COMMENT
    eng "Comment for table '%-.64s' is too long (max = %lu)"
  ER_TOO_LONG_FIELD_COMMENT
    eng "Comment for field '%-.64s' is too long (max = %lu)"
  ER_TOO_BIG_ROWSIZE 42000 
          eng "Row size too large. The maximum row size for the used table type, not counting BLOBs, is %ld. You have to change some columns to TEXT or BLOBs"

- Added an alterantive search path for WiX components
- Added a custom welcome dialog to include the copyright line (Bug#59805 and Bug#59512)
- Excluded some binaries to make package smaller, in effect makig the
  "essentials" package obsolete
- Added a bit more error checking when running the WiX tools

WiX XML changes done by Johannes Taxacher

notifying PB2 the test fails through mysql-test/collections/default.experimental.

FREED IN FLUSH_READ_LOCK (VALGRIND WARNING).

The problem was that under some circustances the memory allocated
for Query_tables_list::sroutines was not freed properly.
The cause of this problem was the absence of
LEX::restore_backup_query_tables_list() call in one of the branches
in mysql_table_grant() function.

      
Impementing Test Review Comment.
      
Bug test scenario:      
SELECT is not returning result set for "equal" (=) and "NULL safe equal
operator"  (<=>) on BIT data type. Extending this scenario for all data types

Before this fix, a thread instrumented for the performance schema,
that would perform file io operations, could crash inside the LF_HASH
implementation, in cases when my_thread_init is not called.

The crash itself has not been reported in 5.5 but similar crashes have
been found in 5.6-based development branches, using LF_HASH for
more instrumentation.

The possibility of a crash in 5.5 is confirmed by code analysis.

The problem is that, when my_thread_init() is not called,
which can happen for threads in storage engines or thirs party code,
my_thread_var is NULL.

Using my_thread_var->stacks_ends_here in mysys/lf_alloc-pin.c is unsafe.

Given that my_thread_var is used:
- only for stacks_ends_here
- only on platform with HAVE_ALLOCA
- only when there is enough room on the stack
and given that the LF_HASH implementation has a fallback
algorythm implemented already when using alloca is not possible,
using my_thread_var->stacks_ends_here is in fact not a strict requirement,
and can be relaxed.

The fix is to:
- test explicitly if my_thread_var is NULL, to account for cases
  when my_thread_init() is not used by the calling thread.
- not use alloca in this case, and rely on the fall back code already in place.
  so that the LF_HASH can be supported even without my_thread_init().

The implementation of mysys/lf_alloc-pin.c has been fixed to support this new usage.
The units tests in unittest/mysys/lf-t.c have been adjusted accordingly.