WITH LARGE BUFFER POOL

(Note: this a backport of revno:3472 from mysql-trunk)

rb://845
approved by: Marko

  When dropping a table (with an .ibd file i.e.: with
  innodb_file_per_table set) we scan entire LRU to invalidate pages from
  that table. This can be painful in case of large buffer pools as we hold
  the buf_pool->mutex for the scan. Note that gravity of the problem does
  not depend on the size of the table. Even with an empty table but a
  large and filled up buffer pool we'll end up scanning a very long LRU
  list.
  
  The fix is to scan flush_list and just remove the blocks belonging to
  the table from the flush_list, marking them as non-dirty. The blocks
  are left in the LRU list for eventual eviction due to aging. The
  flush_list is typically much smaller than the LRU list but for cases
  where it is very long we have the solution of releasing the
  buf_pool->mutex after scanning 1K pages.
  
  buf_page_[set|unset]_sticky(): Use new IO-state BUF_IO_PIN to ensure
  that a block stays in the flush_list and LRU list when we release
  buf_pool->mutex. Previously we have been abusing BUF_IO_READ to achieve
  this.

The predicate is re-written from
((`test`.`g1`.`a` = geometryfromtext('')) or ...
to
((`test`.`g1`.`a` = <cache>(geometryfromtext(''))) or ...

The range optimizer calls save_in_field_no_warnings, in order to fetch keys.
save_in_field_no_warnings returns 0 because of the cache wrapper,
and get_mm_leaf() proceeded to call Field_blob::get_key_image() 
which accesses un-initialized data.




mysql-test/r/gis.result:
  New test case.
mysql-test/t/gis.test:
  New test case.
sql/item.cc:
  If we have cached a null_value, then verify that the Field can accept it.

handle_segfault is the signal handler code of mysqld.  however, it makes
calls to potentially unsafe functions localtime_r, fprintf, fflush.



include/my_stacktrace.h:
  Add safe versions of itoa() write() and snprintf().
libmysqld/CMakeLists.txt:
  Move signal handler to separate file.
mysys/stacktrace.c:
  Remove unsafe function calls.
sql/CMakeLists.txt:
  Move signal handler to separate file.
sql/mysqld.cc:
  Move signal handler to separate file.
sql/set_var.h:
  Add missing #include dependency.
sql/sys_vars.cc:
  Cleanup .h and .cc files.
sql/sys_vars.h:
  Cleanup .h and .cc files.

We can have dirty pages during a fast shutdown. Relax the assertion.

Includes fix for strace-client and restricted to strace and linux only.
******

Refactored the test case: hardened and extended it. Created test inc file
to abstract the task of relocating binlogs.

Also, disabled it on windows for not cluttering the test case any further, 
as it depends heavily on doing filesystem operations and path handling.

mysql-test/include/relocate_binlogs.inc:
  Auxiliar include file that performs the relocation of binary logs
  listed in an index file.

Post-push fix: build break on windows/optimized

Bug#11761576 54082: HANDLE_SEGFAULT MAKES USE OF UNSAFE FUNCTIONS

handle_segfault is the signal handler code of mysqld.  however, it makes
calls to potentially unsafe functions localtime_r, fprintf, fflush.



include/my_stacktrace.h:
  Add safe versions of itoa() write() and snprintf().
libmysqld/CMakeLists.txt:
  Move signal handler to separate file.
mysys/stacktrace.c:
  Remove unsafe function calls.
sql/CMakeLists.txt:
  Move signal handler to separate file.
sql/Makefile.am:
  Move signal handler to separate file.
sql/mysqld.cc:
  Move signal handler to separate file.
sql/signal_handler.cc:
  Remove unsafe function calls.

There was memory leak when running some tests on PB2.
The reason of the failure is an early return from change_master()
that was supposed to deallocate a dyn-array.

Fixed with relocating the dyn-array's destructor at ~LEX() that is
the end of the session, per Gleb's patch idea.
Two optimizations were done: the static buffer for the dyn-array to base on,
and the array initialization is called precisely when it's necessary rather than
per each CHANGE-MASTER as before.
 

mysql-test/suite/rpl/t/rpl_empty_master_host.test:
  the test is binlog-format insensitive so it will be run with MIXED mode only.
sql/sql_lex.cc:
  the new flag is initialized.
sql/sql_lex.h:
  A new bool flag new member to LEX.mi is added to stay UP since after
  LEX.mi.repl_ignore_server_ids dynarray initialization was called
  for the first time on the session. So it is set once and its life time 
  is session.
  
  The array is destroyed at the end of the session.
sql/sql_repl.cc:
  dyn-array destruction is relocated to ~LEX.
sql/sql_yacc.yy:
  Refining logics of Lex->mi.repl_ignore_server_ids initialization.
  The array is initialized once a corresponding option in CHANGE MASTER token sequence
  is found.
  The fact of initialization is memorized into the new flag.

readline.cc: In function char* batch_readline(LINE_BUFFER*):
readline.cc:60:9: error: out_length may be used uninitialized in this function
log.cc: In function int find_uniq_filename(char*):
log.cc:1857:8: error: number may be used uninitialized in this function

Automerged approved bzr bundle into latest mysql-5.5.

                    WITH MYISAM_USE_MMAP ENABLED

MySQL server can crash due to segmentation fault when
started with myisam_use_mmap.

The reason behind this being, while making a request to
unmap (munmap) the previously mapped memory (mmap), the
size passed was 7 bytes larger than the size requested at
the time of mapping. This can eventually unmap the adjacent
memory mapped block, belonging to some other memory-map pool.
Hence the subsequent call to mmap can map a region which was
still a valid memory mapped area.

Fixed by removing the extra 7-byte margin which was erroneously
added to the size, used for unmappping.


storage/myisam/mi_close.c:
  Bug#11756764 48726: MYSQLD KEEPS CRASHING WITH SIGSEGV
                      WITH MYISAM_USE_MMAP ENABLED
  
  Added a condition to call _mi_unmap_file() in case
  of compressed records. mi_munmap_file() is called
  otherwise.
storage/myisam/mi_packrec.c:
  Bug#11756764 48726: MYSQLD KEEPS CRASHING WITH SIGSEGV
                      WITH MYISAM_USE_MMAP ENABLED
  
  mi_dynmap_file() function, after successfully executing
  mmap, stores the total size in info->s->mapped_length
  variable. Now, if mi_dynmap_file() is invoked with a size
  with an extra 7-byte margin (MEMMAP_EXTRA_MARGIN),
  the margin will eventually also get stored in mapped_length.
  So, un-mapping function can simply use the value stored in
  mapped_length in order to unmap the previously mapped
  region.

              BIN LOG HAS BEEN MOVED 

When moving the binary/relay log files from one location to
another and restarting the server with a different log-bin or
relay-log paths, would cause the startup process to abort. The
root cause was that the server would not be able to find the log
files because it would consider old paths for entries in the
index file instead of the new location.  What's even worse, the
relative paths would not be considered relative to the path
provided in log-bin and relay-log, but to mysql_data_dir.
      
We fix the cases where the server contains relative paths. When
the server is reading from the index file, it checks whether the
entry contains relative paths. If it does, we replace it with the
absolute path set in log-bin/relay-log option. Absolute paths
remain unchanged and the index must be manually edited to
consider the new log-bin and/or relay-log path (this should be
documented). This is a fix for a GA version, that does not break
behavior (that much).
      
For development versions, we should go with Zhenxing's approach 
that removes paths altogether from index files.

mysql-test/include/begin_include_file.inc:
  Added parameter to keep the begin_include_file.inc silent. Useful when 
  including scripts that contain platform dependent parameters, for example:
  
  --let $rpl_server_parameters=--log-bin=$tmpdir/slave-bin --relay-log=$tmpdir/slave-relay-bin
  --let $keep_include_silent=1
  source include/rpl_start_server.inc;
  --let $keep_include_silent=0
  
  We want the paths ($tmpdir/slave-bin and $tmpdir/slave-relay-bin) not to be in the 
  result file.
mysql-test/suite/rpl/t/rpl_binlog_index.test:
  Test case.
sql/log.cc:
  When finding the corresponding log entry in the index file, we first 
  normalize the paths before doing the comparison. This will make relative
  paths to be turned into absolute paths (based on the opt_bin_logname or
  opt_relay_logname) and then compared against also, expanded paths entered, 
  through CHANGE MASTER for instance.
sql/log.h:
  Added normalize_binlog_name, which turns relative paths, into absolute paths
  given the parameter: is_relay_log ? opt_relay_logname : opt_bin_logname .
sql/mysqld.cc:
  Exposing opt_bin_logname.
sql/mysqld.h:
  Exposing opt_bin_logname.

Automerged against latest mysql-5.5.

  
When passing an empty user to the connect function will cause
valgrind warnings. Seems that the client code is not prepared 
to handle empty users. On 5.6 this can even be triggered by 
START SLAVE PASSWORD='...'; i.e., without setting USER='...' on
the START SLAVE command (see WL#4143 for details on the new
additional START SLAVE commands).
  
To fix this, we disallow empty users when configuring the slave
connection parameters (this decision might be revisited if the 
client code accepts empty users in the future).

sql/slave.cc:
  We throw an error if an empty user is supplied to the connection 
  function.

                      AND HANG IN SHOW TABLE STATUS.

ISSUE: Table corruption due to concurrent queries.
       Different threads running insert and check
       query leads to table corruption. Not properly locked,
       rows are inserted in between check query.

SOLUTION: In check query mutex lock is acquired
          for a longer time to handle concurrent
          insert and check query.

NOTE: Additionally we backported the fix for CHECKSUM
      issue(bug#11758979).

innodb_change_buffering_debug prevents creating the Dup Key scenario on
windows

sporadically on 5.1. See Bug#12584161.

Test runs successfully on 5.5/trunk, so this changeset will
be null-merged.

Fix merge issues after discussing with Marko.

Remove btr_cur_t::ibuf_cnt. The only dependence on cursor->ibuf_cnt
was ibuf_set_entry_counter(). That code path was removed in the
original bug fix (marko.makela@oracle.com-20111107072802-dgwagejlpub0rjkd).

rb:819 approved by Jimmy Yang

A patch for alter_table-big.test has been committed earlier.
This is a patch for create-big.test:
The test used to time-out after 900 seconds. 
It relied on debug sleeps that are no longer present in the 
code. Since the sleeps are long gone, fixing the problem didn't 
involve just updating the result file or using macro 
"show_binlog_events2.inc" instead of "show binlog events" 
statement. The test needed to be rewritten using debug sync 
points, and result then needed to be updated.
So, the sleeps have been replaced by debug_sync points and the test execution time has 
been reduced significantly.

  Disable federated_plugin test for embedded, like other federated tests
  Also removed redundant include/not_embedded.inc from federated.test

Setting query_cache_size to larger values might fail depending on the memory 
pressure being put on the system. This can be seen on pushbuild as the test 
case query_cache_size_basic tries to allocate a +3GB query cache, which 
succeeds in some machines and fails in others.

So this part of the code where the test case tries to allocate +3GB query cache has been 
disabled for now to get the test running on pb2.

rb://608 approved by Sunny Bains

  
rb://816
approved by: Marko Makela
  
The title is misleading. This bug was actually introduced by
bug 12635227 and was unearthed by a later optimization.
We need to free buf_page_t structs that we are allocating using
malloc() at shutdown.

I manually checked that all the conflicting InnoDB changes are in 5.5 already.
Two things I am not sure about - I commented them with XXX in this patch.
I will further check with the authors of the changesets whether these things
should be present or not.