externally stored columns.

page_zip_compress_clust_ext(), page_zip_apply_log_ext():
New functions.

page_zip_compress_clust(), page_zip_apply_log(): Check rec_offs_any_extern()
and avoid invoking the costly loop in most cases.

Similar optimizations can be made in page_zip_decompress_clust() and
page_zip_write_rec().

by adding the REC_OFFS_EXTERNAL flag to rec_offs_base(offsets)[0].

This reduces the processor usage of page_zip_write_rec() by about 40%
in one test case.  The code could be sped up further by testing
rec_offs_any_extern() outside of loops that check rec_offs_nth_extern().
The vast majority of records does not contain any externally stored columns.

and introduce enum lock_mode.

lock_mode_stronger_or_eq(), lock_mode_compatible(): Replace if-then-else
chain with a bitwise and against a constant.

page_cur_insert_rec_zip_reorg(): New function: Recompress or
reorganize a compressed page.

page_cur_insert_rec_zip(): New function: insert a record to
a compressed page.

page_cur_insert_rec_low(): Only handle inserts to uncompressed pages.

trx_undo_page_report_modify(), trx_undo_report_row_operation():
Add const qualifier to the parameter rec.  Remove some local variables.

trx_undo_report_row_operation(): Invoke rec_get_offsets() only once.

and optimize one test.

eliminate some local variables, and add branching hints.

to tables.

dict_mem_table_add_col(): Add the parameter "heap" for temporary memory
allocation.  Allow it and "name" to be NULL.  These parameters are NULL
when creating dummy indexes.

dict_add_col_name(): Remove calls to ut_malloc() and ut_free().

dict_table_get_col_name(): Allow table->col_names to be NULL.

dict_table_add_system_columns(), dict_table_add_to_cache():
Add the parameter "heap".

page_zip_set_alloc(): New function, to configure zlib to use a memory heap.

btr_store_big_rec_extern_fields(): Reduce memLevel, so that a heap of
256 kilobytes will suffice.  Use page_zip_set_alloc().

btr_copy_externally_stored_field_prefix_low(): Use page_zip_set_alloc().

page_zip_compress(), page_zip_decompress(): Use page_zip_set_alloc().

of HASH_DELETE().

HASH_ASSERT_VALID(), HASH_INVALIDATE(): New macros, used in HASH_DELETE()
and HASH_SEARCH().

buf_page_init_low(): Add HASH_INVALIDATE(bpage, hash);

buf_page_init(): Call buf_page_init_low() before HASH_INSERT(), so that
bpage->hash will not be invalidated.

for extra validation in UNIV_DEBUG builds.

block->page.buf_fix_count.  This mistake was made in r1255.
When access to the block is requested by some other thread,
buf_fix_count can be incremented during the execution of
buf_zip_decompress().

Replace some while loops with do..while or for loops.

lock_move_rec_list_start(): Replace 2 with PAGE_HEAP_NO_USER_LOW.

initialize all the fields of the control block.

buf_block_init_low(), buf_page_init_low(): New functions, used in
buf_page_init_for_backup_restore(), buf_page_init(), and
buf_page_init_for_read().

Declare some local variables in narrower scope.

lock_move_rec_list_start(): Check for orphan locks #ifdef UNIV_DEBUG

orphaned locks outside the "for" loop.  This mistake was made in r1252.

and implement UNIV_DEBUG diagnostics for orphaned locks.

inside #ifdef UNIV_DEBUG_LOCK_VALIDATE.

lock_rec_validate_page(): Assert that page_find_rec_with_heap_no()
returns non-NULL.

lock_rec_get_nth_bit(), lock_rec_set_nth_bit(), lock_rec_reset_nth_bit(),
lock_rec_copy(): Simplify the code.

buf_buddy_relocate_block(): New function, split from buf_buddy_relocate().
Return FALSE when buf_page_can_relocate() returns FALSE.

Fix some formatting in buf_buddy_free_low() and buf_page_init_for_read().

uncompressed page frames.

buf_page_get_zip(), buf_page_release_zip(): New functions, used
by btr_copy_zblob_prefix().

btr_copy_zblob_prefix(): Do not call inflateEnd().

buf_LRU_free_block(): Do not allocate the control block when
a compressed-only page is being freed.

buf_validate(): Note that buf_pool->zip_clean may contain blocks
with io_fix == BUF_IO_READ.

compressed page also when freeing an unmodified uncompressed page.

btr_blob_free(): New function to commit a mini-transaction and to free
an uncompressed BLOB block, or the entire block.

btr_store_big_rec_extern_fields(): Replace the existing code with
btr_blob_free().  The old code may have contained a race condition.

btr_free_externally_stored_field(): Completely free the buffer blocks
allocated for the freed BLOB.

buf_page_get_gen().  This saves one mutex operation per block request.

buf_page_get_gen(), various macros and functions: Add parameter zip_size.

btr_node_ptr_get_child(): Add parameter index.

fil_space_get_latch(): Add optional output parameter zip_size.

fil_space_get_zip_size(): Return 0 for space id==0, because the
system tablespace is never compressed.

fsp_header_init(): Remove the parameter zip_size.

ibuf_free_excess_pages(): Remove the parameter zip_size.

trx_rseg_t, trx_undo_t: Add field zip_size.

xdes_lst_get_next(): Remove, unused.

Split the code to subroutines.

btr_copy_blob_prefix(): New function: copy the prefix of the externally
stored part of an uncompressed BLOB.

btr_copy_zblob_prefix(): New function: copy the prefix of the externally
stored part of a compressed BLOB.

btr_blob_get_part_len(), btr_blob_get_next_page_no(): Add const qualifier
to the parameter.

that also exists in uncompressed form in the buffer pool, pass the
compressed frame to fil_io() instead of the uncompressed one when
not using the doublewrite buffer.

page checksum of compressed-only blocks.  Pass the compressed page frame
to fil_io() when needed.

page_zip_calc_checksum(): Skip also FIL_PAGE_LSN and FIL_PAGE_FILE_FLUSH_LSN.

buf_flush_init_for_writing(): Expect page to be non-NULL.

and to be flushed to disk.

buf_LRU_free_block(): Enable the freeing of uncompressed pages of
compressed tablespaces.

trx_doublewrite->buf_block_arr[]: Change the type from buf_block_t*
to buf_page_t*.

buf_flush_ready_for_flush(): Add debug assertion.

buf_flush_buffered_writes(), buf_flush_try_page(): Support blocks of type
BUF_BLOCK_ZIP_DIRTY.

buf_flush_post_to_doublewrite_buf(), buf_flush_write_block_low():
Change the type of the parameter from buf_block_t* to buf_page_t*.

buf_flush_init_for_writing(): Allow page to be NULL if page_zip_ is non-NULL.

compressed page) is not protected by block_mutex.  Before releasing
buf_pool->mutex, we can simply assign b->io_fix.  After reacquiring
buf_pool->mutex, we also have to acquire buf_pool->zip_mutex.

of a block, do not release buf_pool->mutex between the time the old control
block is removed from buf_pool->page_hash and the new control block is
added to it.  Prevent operations on the compressed-only block while calling
btr_search_drop_page_hash_index() or page_zip_calc_checksum().

buf_LRU_invalidate_tablespace(): Revert the change done in r1223.
buf_pool->zip_mutex will have been released by
buf_LRU_block_remove_hashed_page() when it returns BUF_BLOCK_ZIP_FREE.

acquired by the caller.

buf_LRU_invalidate_tablespace(): Do not leak buf_pool->zip_mutex when
rescanning the LRU list.

of a dirty block, relocate the control blocks on buf_pool->flush_list
instead of calling buf_flush_insert_into_flush_list().

invoking buf_buddy_alloc().  An IO-fixed block must also be x-latched
when block->mutex and buf_pool->mutex are released.

buf_page_create(): Protect the block during buf_buddy_alloc() by
IO-fixing and X-latching it.

should have been added in r1218.

buf_zip_decompress().  This will prevent btr_cur_search_to_nth_level()
from accessing a half-decompressed leaf page.  Internal B-tree nodes
(non-leaf pages) in B-tree searches are only protected by the tree
latch (index->lock) and by buffer-fixing.