mysql_client_binlog_statement

Problem: server may read from unassigned memory performing
"wrong" BINLOG queries.

Fix: never read from unassigned memory.


mysql-test/suite/binlog/r/binlog_base64_flag.result:
  Fix for bug #54393: crash and/or valgrind errors in
  mysql_client_binlog_statement
    - test result.
mysql-test/suite/binlog/t/binlog_base64_flag.test:
  Fix for bug #54393: crash and/or valgrind errors in
  mysql_client_binlog_statement
    - test case.
sql/sql_binlog.cc:
  Fix for bug #54393: crash and/or valgrind errors in
  mysql_client_binlog_statement
    - coded_len should not count trailing '/0';
    - never read from unassigned memory.

line exceeds the limit

Upmerge the fix from 5.0 to 5.1

           line exceeds the limit

The number and/or names of our files for the main test suite
(contents of "mysql-test/t/") now exceeds the command line
length limit on AIX.
Solve the problem by using separate "cp" commands for the
various file name extensions.

This is the fix for 5.1, where only the behaviour on upgrade is changed:
If the server was stopped when the upgrade begins, we assume the
administrator is taking manual action, so we do not start the (new)
server at the end of the upgrade.
We still install the start/stop script, so it will be started on reboot.


support-files/mysql.spec.sh:
  In the "pre" section of the spec file, check the server status, and write
  the result to a file.
  In the "post" section, evaluate the status file, and start the server if
  it was running during status analysis.
  In 5.1, we start the server if there is no status file (which will happen
  on first installation, when there is no data directory yet).

root cause of 52208 resulted in another test failure
in 5.1.

Reorder code breaks when finding tests skipped due to --skip-rpl etc.
Add simple test that master_opt is non-empty

Kill mysqltest and call mtr_kill_leftovers() before terminating

In Prepared_statement::prepare() bail out as soon as
parser_state.init() fails, trying to continue leads to crashes.

conflicts:
   conflict      sql/sql_parse.cc

Item*) at opt_sum.cc:305
      
Queries applying MIN/MAX functions to indexed columns are
optimized to read directly from the index if all key parts
of the index preceding the aggregated key part are bound to
constants by the WHERE clause. A prefix length is also
produced, equal to the total length of the bound key
parts. If the aggregated column itself is bound to a
constant, however, it is also included in the prefix.

Such full search keys are read as closed intervals for
reasons beyond the scope of this bug. However, the procedure
missed one case where a key part meant for use as range
endpoint was being overwritten with a NULL value destined
for equality checking. In this case the key part was
overwritten but the range flag remained, causing open
interval reading to be performed.

Bug was fixed by adding more stringent checking to the
search key building procedure (matching_cond) and never
allow overwrites of range predicates with non-range
predicates.

An assertion was added to make sure open intervals are never
used with full search keys.

            strict aliasing violations.

One somewhat major source of strict-aliasing violations and
related warnings is the SQL_LIST structure. For example,
consider its member function `link_in_list` which takes
a pointer to pointer of type T (any type) as a pointer to
pointer to unsigned char. Dereferencing this pointer, which
is done to reset the next field, violates strict-aliasing
rules and might cause problems for surrounding code that
uses the next field of the object being added to the list.

The solution is to use templates to parametrize the SQL_LIST
structure in order to deference the pointers with compatible
types. As a side bonus, it becomes possible to remove quite
a few casts related to acessing data members of SQL_LIST.

sql/handler.h:
  Use the appropriate template type argument.
sql/item.cc:
  Remove now-unnecessary cast.
sql/item_subselect.cc:
  Remove now-unnecessary casts.
sql/item_sum.cc:
  Use the appropriate template type argument.
  Remove now-unnecessary cast.
sql/mysql_priv.h:
  Move SQL_LIST structure to sql_list.h
  Use the appropriate template type argument.
sql/sp.cc:
  Remove now-unnecessary casts.
sql/sql_delete.cc:
  Use the appropriate template type argument.
  Remove now-unnecessary casts.
sql/sql_derived.cc:
  Remove now-unnecessary casts.
sql/sql_lex.cc:
  Remove now-unnecessary casts.
sql/sql_lex.h:
  SQL_LIST now takes a template type argument which must
  match the type of the elements of the list. Use forward
  declaration when the type is not available, it is used
  in pointers anyway.
sql/sql_list.h:
  Rename SQL_LIST to SQL_I_List. The template parameter is
  the type of object that is stored in the list.
sql/sql_olap.cc:
  Remove now-unnecessary casts.
sql/sql_parse.cc:
  Remove now-unnecessary casts.
sql/sql_prepare.cc:
  Remove now-unnecessary casts.
sql/sql_select.cc:
  Remove now-unnecessary casts.
sql/sql_show.cc:
  Remove now-unnecessary casts.
sql/sql_table.cc:
  Remove now-unnecessary casts.
sql/sql_trigger.cc:
  Remove now-unnecessary casts.
sql/sql_union.cc:
  Remove now-unnecessary casts.
sql/sql_update.cc:
  Remove now-unnecessary casts.
sql/sql_view.cc:
  Remove now-unnecessary casts.
sql/sql_yacc.yy:
  Remove now-unnecessary casts.
storage/myisammrg/ha_myisammrg.cc:
  Remove now-unnecessary casts.

           strict aliasing violations.

Essentially, the problem is that large parts of the server were
developed in simpler times (last decades, pre C99 standard) when
strict aliasing and compilers supporting such optimizations were
rare to non-existent. Thus, when compiling the server with a modern
compiler that uses strict aliasing rules to perform optimizations,
there are several places in the code that might trigger undefined
behavior.

As evinced by some recent bugs, GCC does a somewhat good of job
misoptimizing such code, but on the other hand also gives warnings
about suspicious code. One problem is that the warnings aren't
always accurate, yet we can't afford to just shut them off as we
might miss real cases. False-positive cases are aggravated mostly
by casts that are likely to trigger undefined behavior.

The solution is to start a cleanup process focused on fixing and
reducing the amount of strict-aliasing related warnings produced
by GCC and others compilers. A good deal of noise reduction can
be achieved by just removing useless casts that are product of
historical cruft and are likely to trigger undefined behavior if
dereferenced.

client/mysql.cc:
  Remove now-unnecessary casts.
  Break up large strings.
client/mysql_upgrade.c:
  Remove now-unnecessary casts.
client/mysqladmin.cc:
  Remove now-unnecessary casts.
  Break up large strings.
client/mysqlbinlog.cc:
  Remove now-unnecessary casts.
client/mysqlcheck.c:
  Remove now-unnecessary casts.
client/mysqldump.c:
  Remove now-unnecessary casts.
client/mysqlimport.c:
  Remove now-unnecessary casts.
client/mysqlshow.c:
  Remove now-unnecessary casts.
client/mysqlslap.c:
  Remove now-unnecessary casts.
client/mysqltest.cc:
  Remove now-unnecessary casts.
extra/comp_err.c:
  Remove now-unnecessary casts.
extra/my_print_defaults.c:
  Remove now-unnecessary casts.
  Break up large strings.
extra/mysql_waitpid.c:
  Remove now-unnecessary casts.
extra/perror.c:
  Remove now-unnecessary casts.
extra/resolve_stack_dump.c:
  Remove now-unnecessary casts.
extra/resolveip.c:
  Remove now-unnecessary casts.
include/my_getopt.h:
  Use a void pointer type as the opaque type to avoid problems with type
  incompatibility -- GCC issues warnings when the type name is not type
  compatible with a operand. As a side bonus, a explicit cast won't be
  necessary anymore.
include/sslopt-longopts.h:
  Remove now-unnecessary casts.
  Break up large strings.
mysys/my_getopt.c:
  Update opaque type and introduce a type definition for the
  argument to my_getopt_register_get_addr.
server-tools/instance-manager/options.cc:
  Remove now-unnecessary casts.
sql/mysqld.cc:
  Remove now-unnecessary casts.
  Break up large strings.
  Update mysql_getopt_value prototype (the old prototype
  was different from the definition anyway).
sql/sql_plugin.cc:
  The type of a pointer to a function must be compatible with the
  pointed-to function type, otherwise the behavior is undefined.
sql/table.cc:
  The variable buf pointer to pointer to pointer to constant char
  could improperly alias a incompatible type in call to fix_type_
  pointers. Since this was actually dead code, it is simply removed.
sql/unireg.cc:
  Remove call to get_form_pos. The code creates a new FRM file which
  is always truncated and writes the form position as 0. Hence, no
  need to retrieve it, we now for sure it is 0.
storage/archive/archive_reader.c:
  Remove now-unnecessary casts.
storage/myisam/ft_nlq_search.c:
  Read weight directly from the buffer.
storage/myisam/fulltext.h:
  Add explanation about the type duality of a key buffer.
  Add accessor macro to retrieve a FT float value.
storage/myisam/mi_test1.c:
  Remove now-unnecessary casts.
storage/myisam/myisam_ftdump.c:
  Read weight directly from the buffer.
storage/myisam/myisamchk.c:
  Remove now-unnecessary casts.
storage/myisam/myisamlog.c:
  A pointer to char was used to alias a pointer to pointer to
  unsigned char, thus violating strict aliasing rules.
storage/myisam/myisampack.c:
  Remove now-unnecessary casts.
strings/decimal.c:
  Remove aliasing violation, printing the value is enough for
  debugging purposes.
tests/mysql_client_test.c:
  Remove now-unnecessary casts.

The bug was caused by buffered output. Flushing resolved it.
We still recommend to allways call plan().
  
Also fix some compile warnings (formal parameter different from declaration)


unittest/examples/Makefile.am:
  Omit core-t, since it will always fail.
unittest/examples/no_plan-t.c:
  Comment that we recommend calling plan(NO_PLAN)
unittest/mytap/tap.c:
  Use the named constant NO_PLAN
  Flush all output.
unittest/mytap/tap.h:
  Change documentation for the plan() function.

Addendum: Work around a compilation failure on Windows due to
          windows.h not being added to the global namespace.

extra/yassl/include/lock.hpp:
  Move windows.h inclusion into the global namespace.

Problem: the server missed the fact that one can read from 
2 indexes alternately using HANDLER interface.

Fix: check if the same (initialized) index is involved
reading next/prev values from the index.


mysql-test/r/handler_myisam.result:
  Fix for bug #54007: assert in ha_myisam::index_next, HANDLER
    - test result.
mysql-test/t/handler_myisam.test:
  Fix for bug #54007: assert in ha_myisam::index_next, HANDLER
    - test case.
sql/sql_handler.cc:
  Fix for bug #54007: assert in ha_myisam::index_next, HANDLER
    - check if we use the same (initialized) index 
  to read next/prev values from the index.

config/ac-macros/ssl.m4:
  Enable yaSSL thread safety if linking with the server or a
  thread safe client library. Avoids building a thread safe
  yaSSL when only building a non-thread safe client library.

include/my_sys.h:
  Remove stray semicolon.

The problem was that the bundled yaSSL library was being built
without thread safety support regardless of the thread safeness
of the compoments linked with it.

The solution is to enable yaSSL thread safety support if any
component (server or client) is to be built with thread support.

Also, generate new certificates for yaSSL's test suite.

config/ac-macros/yassl.m4:
  Enable yaSSL thread safety if linking with the server or a
  thread safe client library. Avoids building a thread safe
  yaSSL when only building a non-thread safe client library.
extra/yassl/CMakeLists.txt:
  Always enable for Windows builds.
extra/yassl/certs/ca-cert.pem:
  New certificate, previous one expired.
extra/yassl/certs/client-cert.der:
  New certificate, previous one expired.
extra/yassl/certs/client-cert.pem:
  New certificate, previous one expired.
extra/yassl/certs/dsa-cert.pem:
  New certificate, previous one expired.
extra/yassl/certs/server-cert.pem:
  New certificate, previous one expired.
extra/yassl/include/lock.hpp:
  Rename MULTI_THREAD to YASSL_THREAD_SAFE.
extra/yassl/src/Makefile.am:
  Use CXXFLAGS to set thread related definitions as the lock header
  (lock.hpp) has no local dependencies.
extra/yassl/src/lock.cpp:
  Rename MULTI_THREAD to YASSL_THREAD_SAFE.
extra/yassl/taocrypt/CMakeLists.txt:
  Always enable for Windows builds.
extra/yassl/taocrypt/benchmark/Makefile.am:
  Pass thread related CXXFLAGS.
extra/yassl/taocrypt/src/Makefile.am:
  Pass thread related CXXFLAGS.
extra/yassl/taocrypt/test/Makefile.am:
  Pass thread related CXXFLAGS.
extra/yassl/taocrypt/test/memory.cpp:
  Rename MULTI_THREAD to YASSL_THREAD_SAFE.
extra/yassl/testsuite/Makefile.am:
  Pass thread related CXXFLAGS.

Logging slow stored procedures caused the slow log to write 
very large lock times. The lock times was a result of a 
negative number being cast to an unsigned integer.
The reason the lock time appeard negative was because 
one of the measurements points was reset after execution
causing it to change order with the start time of the 
statement.
      
This bug is related to bug 47905 which in turn was 
introduced because of a joint fix for 12480,12481,12482 and 11587.

The fix is to only reset the start_time before any statement
execution in a SP while not resetting start_utime or
utime_after_lock which are used for measuring the 
performance of the SP. Start_time is used to set the
timestamp on the replication event which controlls how
the slave interprets time functions like NOW().

The problem is in the Item_func_isnull::update_used_tables() function,
bracket is at the wrong place. Because of that isnull item erroneously
is treated as const item. The fix is to set brackets in the right place.


mysql-test/r/func_isnull.result:
  test case
mysql-test/t/func_isnull.test:
  test case
sql/item_cmpfunc.h:
  set brackets in the right place.

when an out-of-supported-range date is detected.

NET::skip_big_packet isn't defined for the embedded server,
hide it in such a case.


sql/sql_connect.cc:
  Fix for bug #53912: Fails to build from source
    - hide net.skip_big_packet for the embedded server,
  as it isn't defined there.

            
Some of the server implementations don't support dates later
than 2038 due to the internal time type being 32 bit.
Added checks so that the server will refuse dates that cannot
be handled by either throwing an error when setting date at 
runtime or by refusing to start or shutting down the server if 
the system date cannot be stored in my_time_t.

Patch for bug#52913: Inserted check if mysqlhotcopy script is existing as requested by review. DIsabled the test until bug#54129 will befixed.

Post-push fix.