creating indexes.  Lock the user table inside the user transaction.

enum trx_dict_op: Remove TRX_OP_INDEX_MAY_WAIT.

ha_innobase::add_index(): Lock the user tables within prebuilt->trx.
Commit the data dictionary transaction before creating indexes.

ha_innobase::final_drop_index(): Lock the user table within prebuilt->trx.

for determining if an index is the insert buffer B-tree.

clustered index is delete-marked, return FALSE without comparing any columns.

continuation pages containing compressed BLOBs.  The first compressed
BLOB page will be of type FIL_PAGE_TYPE_ZBLOB.

to the undo log, also store the original length of the column, so that the
changes will be correctly undone in transaction rollback or when fetching
previous versions of the row.

innodb-zip.test: New file, for tests of the compression.

upd_field_t: Add orig_len, the original length of new_val.

btr_push_update_extern_fields(): Restore the original prefix of the column.
Add the parameter heap where memory will be allocated if necessary.

trx_undo_rec_get_col_val(): Add the output parameter orig_len.

trx_undo_page_report_modify_ext(): New function: Write an externally
stored column to the undo log.  This is only called from
trx_undo_page_report_modify(), and this is the only caller of
trx_undo_page_fetch_ext().

trx_undo_update_rec_get_update(): Read the original length of the column
prefix to upd_field->orig_len.

 
Include univ.i in mysql_addons.h because univ.i includes innodb_redefine.h
which is needed to rename the functions in this file with ibd_ prefix in
the dynamic InnoDB plugin. Aka the renaming magic, required for the dynamic
plugin to work.

Approved by:	Marko

Cast to  ib_uint64_t the mask for align down before doing the '~' operation on it.

Reviewed by: Heikki

row_sel_get_clust_rec(), return if a previous version of the clustered index
record does not exist in the read view.  This bug was made and found by
Heikki.  It was harmless as long as the clustered index record contained
a long enough local prefix of externally stored columns, to allow secondary
index records to be built without fetching BLOBs.

InnoDB transactions should never look at a clustered index record whose
transaction identifier (DB_TRX_ID) is not visible in the transaction's
read view.

btr_page_get_sure_split_rec(): Remove the check if insert_size
exceeds free_space.

btr_page_split_and_insert(): If a compressed page has already been split,
avoid further splits by inserting the record to an empty page.  As a
performance optimization, avoid invoking btr_page_insert_fits() on
compressed tables.

is not guaranteed to fit on a compressed page, instead of letting
an assertion fail.

Update ut_total_allocated_memory.

buf_pool_free(): New function: Free all chunks of the buffer pool.

innobase_shutdown_for_mysql(): Call buf_pool_free() right before
ut_free_all_mem().

stored columns (BLOBs).

btr_copy_blob_prefix(), btr_copy_zblob_prefix(),
btr_copy_externally_stored_field_prefix_low(),
btr_copy_externally_stored_field_prefix(),
btr_copy_externally_stored_field(),
btr_rec_copy_externally_stored_field():
Note that the page containing the clustered index record that points to
the BLOB must be latched.

btr_copy_zblob_prefix(): Note that there is no latch on the page, and thus
all accesses to a given page via this function must be covered by the same
set of locks or latches.

btr_copy_zblob_prefix(): Note that the block acquired by
buf_page_get_zip() is protected by an exclusive table lock or
or by a latch on the clustered index record.

record is protected by a latch or a lock.

dtuple_copy(): New function: Copy a data tuple.

row_upd_replace(): New function: Apply on a row an update vector that
was built for the clustered index.  Set up a cache of externally stored
column prefixes if needed.

undo_node_t: Add the fields undo_row, undo_ext.

row_undo_search_clust_to_pcur(): Initialize undo_row and undo_ext.

row_undo_mod_upd_exist_sec(): Instead of fetching prefixes of
externally stored columns, use the undo_row and undo_ext that were
initialized in row_undo_search_clust_to_pcur().

upd_node_t: Remove the field n_ext.  Add the fields upd_row and upd_ext.

row_upd_store_row(): Initialize the upd_row and upd_ext fields of upd_node_t.

row_upd_sec_index_entry(), row_upd_clust_rec_by_insert(): Instead of
fetching prefixes of externally stored columns, use the upd_row
and upd_ext that were initialized in row_upd_store_row().

so that it will not be uninitialized when the undo record is being
processed.

 
Move the column trx_weight near the end in the
information_schema.innodb_trx table.

Suggested by:	Ken
Approved by:	Marko

acquiring buf_pool_mutex.  This avoids triggering the debug assertion
that was added in r2227.

Forward port of r2236
Introduce retry/sleep logic as a workaround for a transient bug
where ::open fails for partitioned tables randomly if we are using
one file per table.  (Bug #33349)


Reviewed by: Heikki

the clustered index record before calling row_upd_index_replace_new_col_vals().

row_build(), row_upd_index_replace_new_col_vals_index_pos(), and
row_upd_index_replace_new_col_vals() are safe.

btr_cur_optimistic_update(), btr_cur_pessimistic_update(): Note that
the B-tree page of the clustered index record is latched in mtr.

trx_undo_prev_version_build(): Add const qualifiers to index_rec
and rec.  Note that the page of index_rec is latched in index_mtr.

row_vers_impl_x_locked_off_kernel(), row_vers_old_has_index_entry():
Note that the stack of versions is locked by mtr and thus it is
safe to call row_build().

btr_rec_copy_externally_stored_field() are safe.

row_merge_copy_blobs(): Note that the table is locked during index creation.
Therefore, none of its BLOBs can be freed.

row_sel_fetch_columns(): Note that rec must be protected by a page latch.
Add const qualifier to rec.

row_sel_get_clust_rec(): Note that the clustered index record is protected
by a page latch that was acquired when the persistent cursor was positioned
and that the latch will be freed by mini-transaction commit.

row_sel_try_search_shortcut(): Check the delete-mark flag before fetching
the columns.  Note that the clustered index record is protected
by a page latch that was acquired when the persistent cursor was positioned
and that the latch will be freed by mini-transaction commit.

row_sel(), row_search_for_mysql(): Note that the clustered index record
is protected by a page latch that was acquired when the persistent cursor
was positioned and that the latch will be freed by mini-transaction commit.

row_sel_field_store_in_mysql_format(): Add const qualifier to data.

row_sel_store_mysql_rec(), row_sel_push_cache_row_for_mysql():
Add const qualifier to rec.  Note that rec must be protected by a page latch.

be protected from being deleted while any externally stored columns are
being fetched.

row_ext_create(),
row_upd_index_replace_new_col_vals(),
row_upd_index_replace_new_col_vals_index_pos(),
row_sel_sec_rec_is_for_blob(),
row_sel_sec_rec_is_for_clust_rec(): Note that the clustered
index record must be covered by a lock or a page latch.

row_upd_sec_index_entry(), row_upd_clust_rec_by_insert(),
row_undo_mod_upd_exist_sec(): Note that these functions probably
do not obtain an appropriate lock on the clustered index record
before fetching any externally stored columns.

of the buffer pool mutex.  Apparently, it is temporarily released also
in older versions of MySQL/InnoDB for the duration of the
btr_search_drop_page_hash_index() call [in buf_LRU_free_block()].

failure.

the buffer pool mutex.  The instrumentation can be activated by
defining UNIV_DEBUG or UNIV_BUF_DEBUG at compilation time.

buf_pool_mutex_exit_forbidden: New variable.  When this is nonzero,
an assertion will fail in buf_pool_mutex_exit().

buf_pool_mutex_exit_forbid(): Macro for declaring that the buffer pool
mutex must not be released.  Calls may be nested.

buf_pool_mutex_exit_allow(): Macro for declaring that the buffer pool
mutex may be released.  Calls may be nested.

buf_LRU_search_and_free_block(): Prohibit buf_pool_mutex_exit() in the
scope of the function.

buf_LRU_free_block(): Prohibit buf_pool_mutex_exit() in buf_buddy_alloc()
and buf_buddy_free().

buf_LRU_block_remove_hashed_page(): Prohibit buf_pool_mutex_exit()
in buf_buddy_free().

buf_pool->mutex: Rename to buf_pool_mutex, so that the wrappers will have
to be used when changes are merged from other source trees.

buf_pool->zip_mutex: Rename to buf_pool_zip_mutex.

buf_pool_mutex_own(), buf_pool_mutex_enter(), buf_pool_mutex_exit():
Wrappers for buf_pool_mutex.

ha_innobase::final_drop_index(): If row_merge_drop_table() fails, clear
the to_be_dropped flags.  This was the error fixed in this commit; the rest
is just additional safety.

ha_innobase::final_drop_index(): After dropping the flagged indexes,
assert that none of the remaining indexes are flagged to_be_dropped.

ha_innobase::prepare_drop_index(): Assert that no index has been flagged
for deletion.  When checking foreign key constraints, simply traverse the
list of indexes and check if any of the indexes that were just flagged
to_be_dropped.  On error, clear the to_be_dropped flags with simple list
traversal.

attempting to reorganize the page, not after the reorganization has succeeded.

(Revisions 2146:2155 were already merged.)

indexes.

buf_buddy_min_n_frames into two, also buf_buddy_max_n_frames.
Set the default values in such a way that a strict LRU policy will apply
for replacing compressed or uncompressed page frames in the buffer pool.

These parameters have not yet been exposed to the MySQL layer.

Add a call to rec_offs_any_extern() as an optimization.

decompression failures.

page_zip_fields_decode(): Add page_zip_fail() diagnostics.

page_zip_apply_log(): Remove double space in page_zip_fail() printout.

page_zip_decompress_node_ptrs(), page_zip_decompress_sec(),
page_zip_decompress_clust_ext(), page_zip_decompress_clust(),
page_zip_decompress(): Add page_zip_fail() diagnostics for inflate()
failures and other errors.

TRX_RSEG_N_SLOTS: Make the definition dependent of UNIV_PAGE_SIZE.

FSP_EXTENT_SIZE: Define as 1 megabyte, as assumed by the code elsewhere.

fseg_create_general(): Add a debug assertion against page overflow.

UNIV_PAGE_SIZE than the default 16384.

MEM_BLOCK_STANDARD_SIZE: Cap to MEM_MAX_ALLOC_IN_BUF when UNIV_PAGE_SIZE
is less than 16384.

ha_create_func(): Invoke mem_heap_create_in_btr_search() with
ut_min(4096, MEM_MAX_ALLOC_IN_BUF) instead of 4096.  The memory
will be allocated from the buffer pool, and with UNIV_PAGE_SIZE
defined to 4096, there would not be any space for the mem_heap
data structure overhead.

Change the format of TRX_IDs in INFORMATION_SCHEMA tables from DEC to
HEX.

The current TRX_IDs are hard to remember and track down: 426355, 428466,
428566, etc.

In HEX:
* there are less "digits", the strings are shorter;
* since there are 16 instead of 10 "digits", the chance of having
  repeating ones are smaller.

The above look like 68173, 689B2, 68A16 in HEX.

Discussed with:	Ken
Approved by:	Heikki (via IM)