field references

This error requires a combination of factors : 
1. An "impossible where" in the outermost SELECT
2. An aggregate in the outermost SELECT
3. A correlated subquery with a WHERE clause that includes an outer 
field reference as a top level WHERE sargable predicate

When JOIN::optimize detects an "impossible WHERE" it will bail out
without doing the rest of the work and initializations. It will not
call make_join_statistics() as well.  And make_join_statistics fills 
in various structures for each table referenced.
When processing the result of the "impossible WHERE" the query must
send a single row of data if there are aggregate functions in it.
In this case the server marks all the aggregates as having received 
no rows and calls the relevant Item::val_xxx() method on the SELECT
list. However if this SELECT list happens to contain a correlated 
subquery this subquery is evaluated in a normal evaluation mode.
And if this correlated subquery has a reference to a field from the 
outermost "impossible where" SELECT the add_key_fields will mistakenly
consider the outer field reference as a "local" field reference when 
looking for sargable predicates.
But since the SELECT where the outer field reference refers to is not
completely initialized due to the "impossible WHERE" in this level
we'll get a NULL pointer reference.
Fixed by making a better condition for discovering if a field is "local"
to the SELECT level being processed. 
It's not enough to look for OUTER_REF_TABLE_BIT in this case since 
for outer references to constant tables the Item_field::used_tables() 
will return 0 regardless of whether the field reference is from the 
local SELECT or not.

with gcc 4.3.2
      
This patch fixes a number of GCC warnings about variables used
before initialized. A new macro UNINIT_VAR() is introduced for
use in the variable declaration, and LINT_INIT() usage will be
gradually deprecated. (A workaround is used for g++, pending a
patch for a g++ bug.)
      
GCC warnings for unused results (attribute warn_unused_result)
for a number of system calls (present at least in later
Ubuntus, where the usual void cast trick doesn't work) are
also fixed.


client/mysqlmanager-pwgen.c:
  A fix for warn_unused_result, adding fallback to use of
  srand()/rand() if /dev/random cannot be used. Also actually
  adds calls to rand() in the second branch so that it actually
  creates a random password.

When a connection is dropped any remaining temporary table is also automatically
dropped and the SQL statement of this operation is written to the binary log in
order to drop such tables on the slave and keep the slave in sync. Specifically,
the current code base creates the following type of statement:
DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `db`.`table`;

Unfortunately, appending the database to the table name in this manner circumvents
the replicate-rewrite-db option (and any options that check the current database).
To solve the issue, we started writing the statement to the binary as follows:
use `db`; DROP /*!40005 TEMPORARY */ TABLE IF EXISTS `table`;

The crash happens because select_union object is used as result set
for queries which have derived tables.
select_union use temporary table as data storage and if
fields count exceeds 10(count of values for procedure ANALYSE())
then we get a crash on fill_record() function.


mysql-test/r/analyse.result:
  test result
mysql-test/r/subselect.result:
  result fix
mysql-test/t/analyse.test:
  test case
mysql-test/t/subselect.test:
  test fix
sql/sql_yacc.yy:
  The crash happens because select_union object is used as result set
  for queries which have derived tables.
  select_union use temporary table as data storage and if
  fields count exceeds 10(count of values for procedure ANALYSE())
  then we get a crash on fill_record() function.

The code was using a special global buffer for the value of IS NULL ranges.
This was not always long enough to be copied by a regular memcpy. As a 
result read buffer overflows may occur.
Fixed by setting the null byte to 1 and setting the rest of the field disk image
to NULL with a bzero (instead of relying on the buffer and memcpy()).

         decrease for INSERTs


Bulk inserts (multiple row, CREATE ... SELECT, INSERT ... SELECT) into
MyISAM tables were performed inefficiently. This was mainly affecting
use cases where read_buffer_size was considerably large (>256K) and low
number of rows was inserted (e.g. 30-100).

The problem was that during I/O cache initialization (this happens
before each bulk insert) allocated I/O buffer was unnecessarily
initialized to '\0'.

This was happening because of mess in flag values. MyISAM informs I/O
cache to wait for free space (if out of disk space) by passing
MY_WAIT_IF_FULL flag. Since MY_WAIT_IF_FULL and MY_ZEROFILL have the
same values, memory allocator was initializing memory to '\0'.

The performance gain provided with this patch may only be visible with
non-debug binaries, since safemalloc always initializes allocated memory
to 0xA5A5...

mysys/mf_iocache.c:
  Remove MY_WAIT_IF_FULL from myflags before calling my_malloc
  to prevent conflict with MY_ZEROFILL.

The check for stack overflow was independent of the size of the 
structure stored in the heap. 
Fixed by adding sizeof(PARAM) to the requested free heap size.

view manipulations
      
The bespoke flag was not properly reset after last call to 
fill_record. Fixed by resetting in caller mysql_update.

mysql-test/r/auto_increment.result:
  Bug#46616: Test result.
mysql-test/t/auto_increment.test:
  Bug#46616: Test case.
sql/sql_update.cc:
  Bug#46616: Fix.

view that has Group By
      
Table access rights checking function check_grant() assumed
that no view is opened when it's called.
This is not true with nested views where the inner view
needs materialization. In this case the view is already 
materialized when check_grant() is called for it.
This caused check_grant() to not look for table level
grants on the materialized view table.
Fixed by checking if a view is already materialized and if 
it is check table level grants using the original table name
(not the ones of the materialized temp table).

Bug#45243: crash on win in sql thread clear_tables_to_lock() -> free()
Bug#45242: crash on win in mysql_close() -> free()
Bug#45238: rpl_slave_skip, rpl_change_master failed (lost connection) for STOP SLAVE
Bug#46030: rpl_truncate_3innodb causes server crash on windows
Bug#46014: rpl_stm_reset_slave crashes the server sporadically in pb2

When killing a user session on the server, it's necessary to
interrupt (notify) the thread associated with the session that
the connection is being killed so that the thread is woken up
if waiting for I/O. On a few platforms (Mac, Windows and HP-UX)
where the SIGNAL_WITH_VIO_CLOSE flag is defined, this interruption
procedure is to asynchronously close the underlying socket of
the connection.

In order to enable this schema, each connection serving thread
registers its VIO (I/O interface) so that other threads can
access it and close the connection. But only the owner thread of
the VIO might delete it as to guarantee that other threads won't
see freed memory (the thread unregisters the VIO before deleting
it). A side note: closing the socket introduces a harmless race
that might cause a thread attempt to read from a closed socket,
but this is deemed acceptable.

The problem is that this infrastructure was meant to only be used
by server threads, but the slave I/O thread was registering the
VIO of a mysql handle (a client API structure that represents a
connection to another server instance) as a active connection of
the thread. But under some circumstances such as network failures,
the client API might destroy the VIO associated with a handle at
will, yet the VIO wouldn't be properly unregistered. This could
lead to accesses to freed data if a thread attempted to kill a
slave I/O thread whose connection was already broken.

There was a attempt to work around this by checking whether
the socket was being interrupted, but this hack didn't work as
intended due to the aforementioned race -- attempting to read
from the socket would yield a "bad file descriptor" error.

The solution is to add a hook to the client API that is called
from the client code before the VIO of a handle is deleted.
This hook allows the slave I/O thread to detach the active vio
so it does not point to freed memory.

server-tools/instance-manager/mysql_connection.cc:
  Add stub method required for linking.
sql-common/client.c:
  Invoke hook.
sql/client_settings.h:
  Export hook.
sql/slave.cc:
  Introduce hook that clears the active VIO before it is freed
  by the client API.

        
Replication SQL thread does not set database default charset to 
thd->variables.collation_database properly, when executing LOAD DATA binlog.
This bug can be repeated by using "LOAD DATA" command in STATEMENT mode.
        
This patch adds code to find the default character set of the current database 
then assign it to thd->db_charset when slave server begins to execute a relay log.
The test of this bug is added into rpl_loaddata_charset.test 

The problem is that the lexer could inadvertently skip over the
end of a query being parsed if it encountered a malformed multibyte
character. A specially crated query string could cause the lexer
to jump up to six bytes past the end of the query buffer. Another
problem was that the laxer could use unfiltered user input as
a signed array index for the parser maps (having upper and lower
bounds 0 and 256 respectively).

The solution is to ensure that the lexer only skips over well-formed
multibyte characters and that the index value of the parser maps
is always a unsigned value.

mysql-test/r/ctype_recoding.result:
  Update test case result: ending backtick is not skipped over anymore.
sql/sql_lex.cc:
  Characters being analyzed must be unsigned as they can be
  used as indexes for the parser maps. Only skip over if the
  string is a valid multi-byte sequence.
tests/mysql_client_test.c:
  Add test case for Bug#45010

- Remove offensive quotes.

Install procedure does not copy *.inc files located under the mysql-test/t directory.
Therefore, this patch moves the rpl_trigger.inc to the mysql-test/include directory.

The test case fails sporadically on Windows while trying to overwrite an unused
binary log. The problem stems from the fact that MySQL on Windows does not
immediately unlock/release a file while the process that opened and closed it is
still running. In BUG 38603, this issue was circumvented by stopping the MySQL
process, copying the file and then restarting the MySQL process. 

Unfortunately, such facilities are not available in the 5.0.  Other approaches
such as stopping the slave and issuing change master do not work because the relay
log file and index are not closed when a slave is stopped. So to fix the problem,
we simply don't run on windows the part of the test that was failing.

- Define and pass compile time path variables as pre-processor definitions to 
  mimic the makefile build.
- Set new CMake version and policy requirements explicitly.
- Changed DATADIR to MYSQL_DATADIR to avoid conflicting definition in 
  Platform SDK header ObjIdl.h which also defines DATADIR.

correctly if the compiler optimizes too clever.

This has happaned on HP-UX 11.23 (IA64) at optimization
level "+O2", causing bug#42213:
   Check for "stack overrun" doesn't work, server crashes

Fix it by adding a pragma that prevents this optimization.
As a result, it should be safe to use "+O2" on this platform
(unless there is some other, optimizer-related, bug which
is just currently masked because we use resudec optimization).


config/ac-macros/misc.m4:
  Our autoconf function "MYSQL_STACK_DIRECTION" is meant to
  determine whether the stack grows towards higher or towards
  lower addresses.
  It does this by comparing the addresses of a variable
  (which is local to a recursive function) on different
  nesting levels.
  
  This approach requires that the function is really
  implemented as a recursive function, with each nested call
  allocating a new stack frame containing the local variable.
  If, however, the compiler is optimizing so clever that the
  recursive function is implemented by a loop, then this
  test will not produce correct results.
  
  This has happened on HP-UX 11.23 (IA64) when HP's compiler
  was called with optimization "+O2" (not with "+O1"),
  reported as bug#42213.
  
  Rather than starting a race with the compiler and making
  the function so complicated that this optimization does
  not happen, the idea is to prevent the optimization
  by adding a pragma. For HP, this is "#pragma noinline".
  
  If we encounter other compilers which also optimize
  too clever, we may add their pragmas here.
  
  It is a debatable issue whether such pragmas should be
  guarded by conditional compiling or not, the reviewers
  voted to do it.
  It seems HP has different compilers, "ANSI C" and "aCC",
  on the affected platform "__HP_cc" ("ANSI C") is predefined.
  To be on the safe side, the pragma will also take effect
  if HP's "aCC" compiler is used, or any other compiler on HP-UX.

            compression 
 
Since uint3korr() may read 4 bytes depending on build flags and 
platform, allocate 1 extra "safety" byte in the network buffer 
for cases when uint3korr() in my_real_read() is called to read
last 3 bytes in the buffer. 
 
It is practically hard to construct a reliable and reasonably 
small test case for this bug as that would require constructing 
input stream such that a certain sequence of bytes in a 
compressed packet happens to be the last 3 bytes of the network 
buffer. 


sql/net_serv.cc:
  Allocate 1 extra "safety" byte in the network buffer for cases 
  when uint3korr() is used to read last 3 bytes in the buffer.

The maximum value of the max_join_size variable is set by converting
a signed type (long int) with negative value (-1) to a wider unsigned
type (unsigned long long), which yields the largest possible value of
the wider unsigned type -- as per the language conversion rules. But,
depending on build options, the type of the max_join_size might be a
shorter type (ha_rows - unsigned long) which causes the warning to be
thrown once the large value is truncated to fit.

The solution is to ensure that the maximum value of the variable is
always set to the maximum value of integer type of max_join_size.

Furthermore, it would be interesting to always have a fixed type for
this variable, but this would incur in a change of behavior which is
not acceptable for a GA version. See Bug#35346.

sql/mysqld.cc:
  Set max value for type.

Post-merge fix: test case could fail due to a conversion of the
max_join_size value to a integer. Fixed by preserving the value
as a string for comparison purposes.

tests/mysql_client_test.c:
  Preserve max_join_size value as a string instead of converting
  it to a integer -- value can be larger then the type used.