Reduce ibuf_mutex and ibuf_pessimistic_insert_mutex contention further.

Protect ibuf->empty by the insert buffer root page latch, not ibuf_mutex.

ibuf_tree_root_get(): Assert that ibuf_mutex is owned by the
caller. Assert that the stamped page number is correct. Assert that
ibuf->empty agrees with the root page.

ibuf_size_update(): Do not update ibuf->empty.

ibuf_init_at_db_start(): Update ibuf->empty while holding the root page latch.

ibuf_add_free_page(): Return TRUE/FALSE instead of DB_SUCCESS/DB_STRONG_FAIL.

ibuf_remove_free_page(): Release ibuf_pessimistic_insert_mutex as
early as possible.

ibuf_contract_ext(): Rely on a dirty read of ibuf->empty, unless the
server is being shut down. Never acquire ibuf_mutex. Eliminate n_stored.

ibuf_contract_after_insert(): Never acquire ibuf_mutex. Perform dirty
reads of ibuf->size and ibuf->max_size.

ibuf_insert_low(): Only acquire ibuf_mutex for mode==BTR_MODIFY_TREE.
Perform dirty reads of ibuf->size and ibuf->max_size. Update
ibuf->empty while holding the root page latch.

ibuf_delete_rec(): Update ibuf->empty while holding the root page latch.

ibuf_is_empty(): Release ibuf_mutex earlier.

regression in Bug #54914, but it does speed up the execution for
innodb_change_buffering=inserts.

ibuf_add_ops(), ibuf_merge_or_delete_for_page(),
ibuf_delete_for_discarded_space(): Use atomic built-ins instead of
ibuf_mutex, when available.

ibuf_add_free_page(), ibuf_remove_free_page(), ibuf_contract_ext():
Release ibuf_mutex earlier.

ibuf_free_excess_pages(): Release ibuf_mutex before a conditional branch.

ibuf_insert_low(): Release ibuf_mutex before a conditional
branch. Create ibuf_entry before re-acquiring ibuf_mutex. Simplify a
loop to reduce code footprint. Release ibuf_mutex before mtr_commit()
[btr_pcur_close()].

ibuf_is_empty(): Release ibuf_mutex before mtr_commit().

called on any table. This patch fixes this by only doing this full
check in debug versions.

bug#55716
rb://423
approved by: Marko

pages that it wants to flush then we should honor that value as in
not going beyond that in our eagerness to flush the neighbors of
the selected victim.

            DELETE statement

Single-table delete ordered by a field that has a hash-type index
may cause an assertion failure or a crash.

An optimization added by the fix for the bug 36569 forced the
optimizer to use ORDER BY-compatible indices when applicable.

However, the existence of unsorted indices (HASH index algorithm
for some engines such as MEMORY/HEAP, NDB) was ignored.

The test_if_order_by_key function has been modified to skip
unsorted indices.


mysql-test/r/heap_hash.result:
  Test case for bug #55472.
mysql-test/t/heap_hash.test:
  Test case for bug #55472.
sql/sql_select.cc:
  Bug #55472: Assertion failed in heap_rfirst function of hp_rfirst.c on
              DELETE statement
  
  The test_if_order_by_key function has been modified to skip
  unsorted indices.

The problem was that the optimize method of the ARCHIVE storage
engine was not preserving the FRM embedded in the ARZ file when
rewriting the ARZ file for optimization. The ARCHIVE engine stores
the FRM in the ARZ file so it can be transferred from machine to
machine without also copying the FRM -- the engine restores the
embedded FRM during discovery.

The solution is to copy over the FRM when rewriting the ARZ file.
In addition, some initial error checking is performed to ensure
garbage is not copied over.


mysql-test/t/archive.test:
  Add test case for Bug#45377.
storage/archive/azio.c:
  Add error checking to ensure that the I/O operations are
  successful.
storage/archive/ha_archive.cc:
  Copy over the embedded FRM.

Workaround a interface problem with the atomic macros that was
causing warnings. The correct type is retrieved using typeof if
compiling with GCC.

Remove ASM for MC68000 and Vax.

Remove 32-bit SPARC specific code.

into actual tree. No conflicts.

rb://408 approved by Sunny Bains

* Fixed obvious errors (HAVE_BROKEN_PREAD is not true for on any
of systems we use, definitely not on HPUX)

* Remove other junk flags for OSX and HPUX

* Avoid checking type sizes in universal builds on OSX, again 
(CMake2.8.0 fails is different architectures return different results)

* Do not compile template instantiation stuff unless 
EXPLICIT_TEMPLATE_INSTANTIATION is used.

* Some cleanup (make gen_lex_hash simpler, avoid dependencies)

* Exclude some unused files from compilation (strtol.c etc)

Fix some issues with WiX packaging, particularly 
major upgrade and change scenarios.

* remember binary location and data location
(for major upgrade)

* use custom UI, which is WiX Mondo extended 
for major upgrade dialog (no feature selection
screen shown on major upgrade, only upgrade
confirmation). This is necessary to prevent
changing installation path during upgrade
(services are not reregistered, so they would 
have invalid binary path is it is changed)

* Hide datafiles that are installed into 
ProgramFiles, show ones that are installed
in ProgramData

* Make MSI buildable with nmake

* Fix autotools "make dist"




Makefile.am:
  Fix autotools "make dist"
configure.in:
  Fix autotools "make dist"
packaging/Makefile.am:
  Fix autotools "make dist"
packaging/WiX/CMakeLists.txt:
  Use custom UI, for major upgrades
packaging/WiX/CPackWixConfig.cmake:
  Show user editable datafiles in feature selection dialog, 
  not datafiles installed into ProgramFiles directory
packaging/WiX/create_msi.cmake.in:
  Use custom UI, fix nmake build for installer
packaging/WiX/custom_ui.wxs:
  Use custom UI
packaging/WiX/extra.wxs.in:
  Show user editable datafiles in feature selection dialog, 
  not datafiles installed into ProgramFiles directory
packaging/WiX/mysql_server.wxs.in:
  Remember install locations of binaries and 
  user editable datafiles.

Fix assorted warnings in order for the warning-mode to be effective.

Post-merge fix: remove remaining casts which are now
unnecessary and are actually causing warnings.

Remove wrappers around inline -- static inline is used without
wrappers throughout the source code. We rely on the compiler or
linker to eliminate unused static functions.

Remove workarounds for ancient systems.

mysys/default.c:
  Make cast more explicit to ensure a correct offset.

Remove unused string functions.

include/m_string.h:
  Remove prototypes for unused string functions and for functions
  that do not exist anymore.

Remove unused macros or macro which are always defined.

include/my_global.h:
  Remove unused macros and move macros which aren't used globally.

Remove the obsolete and buggy bmove512, use memcpy instead.

Remove the ancient and dead raid code. By now, even the server side
has been removed.

Remove unused source code and associated paraphernalia.

Remove Windows related files which aren't used anymore.

Remove unused variables.

Remove code that has been disabled for a long time.

A change in the default values of some config parameters
caused this test to fail, adjust the test and make it more
robust so it does not fail for the same reason in the future.

           sporadically

There are two problems:

1. When closing temporary tables, during the THD clean up - and
   after the session connection was already closed, there is a
   chance we can push an error into the THD diagnostics area, if
   the writing of the implicit DROP event to the binary log fails
   for some reason. As a consequence an assertion can be
   triggered, because at that point the diagnostics area is
   already set.

2. Using push_warning with MYSQL_ERROR::WARN_LEVEL_ERROR is a 
   bug.

Given that close_temporary_tables is mostly called from
THD::cleanup - ie, with the session already closed, we fix
problem #1 by allowing the diagnostics area to be
overwritten. There is one other place in the code that calls
close_temporary_tables - while applying Start_log_event_v3. To
cover that case, we make close_temporary_tables to return the
error, thus, propagating upwards in the stack.

To fix problem #2, we replace push_warning with sql_print_error.

sql/log_event.cc:
  Added handling of error returned by close_temporary_tables to
  Start_log_event_v3::do_apply_event.
sql/sql_base.cc:
  Three changes to close_temporary_tables:
  1. it returns a boolean now (instead of void)
  2. it uses sql_print_error instead of push_warning when writing to
     binary log fails
  3. we set can_overwrite_status before writing to the binary log,
     thence not risking triggering an assertion by any other push
     into diagnostics area happening inside mysql_bin_log.write.
sql/sql_base.h:
  Changed the interface of close_temporary_tables so that it returns
  bool instead of void.

release a dirty page in the middle of a mini-transaction. Replace the code
with an assertion that checks for this condition.

Original svn revision was: r6330.

      
Silence the UNIV_SYNC_DEBUG assertion failure while upgrading old data files
to multiple rollback segments during server startup. Because the upgrade
takes place while InnoDB is running a single thread, we can safely ignore the
latching order checks without fearing deadlocks.
      
innobase_start_or_create_for_mysql(): Set srv_is_being_started = FALSE
only after trx_sys_create_rsegs() has completed.
      
sync_thread_add_level(): If srv_is_being_started, ignore latching order
violations for SYNC_TRX_SYS_HEADER and SYNC_IBUF_BITMAP.

Create all the non-IO threads after creating the extra rollback segments.

Patch originally from Marko with some additions by Sunny.