to mysql-5.5

rb://979 approved by Marko

conflict in "sql/filesort.cc" solved manually.

Problem:

Field_set::val_str in case of an empty SET value
returned a String with str_length==0 and Ptr==0,
which is not expected by some pieces of the code.

Fix:
      
Returning an empty string with str_length==0 and Ptr=="",
like Field_enum does.

  Doing this for Sayantan whose VPN is not working ATM

The test case must insert all the records using a single transaction. Otherwise the test 
case takes more than 15 minutes and will time out in pb2 and mtr.  

FROM BUFFER POOL

rb://975
approved by: Marko Makela

There is a race in lock_validate() where we try to access a page
without ensuring that the tablespace stays valid during the operation
i.e.: it is not deleted. This patch tries to fix that by using an
existing flag (the flag is renamed to make it's name more generic
in line with it's new use).

rb://976
approved by: Marko Makela

Add an assertion to ensure that string overflow is not happening.
Pointed by Coverity analysis.

IN OS_THREAD_EQ

rb://977
approved by: Marko Makela

rw_lock::writer_thread field contains the thread id of current x-holder
or wait-x thread. This field is un-initialized at lock creation and is
written to for the first time when an attempt is made to x-lock.

Current code considers ::writer_thread as valid memory region only when
the lock is held in x-mode (or there is an x-waiter). This is an
overkill and it generates valgrind warnings.

The fix is to consider ::writer_thread as valid memory region once it
has been written to.

Reasoning:
==========
The ::writer_thread can be safely considered valid because:

* We only ever do comparison with current calling threads id.
* We only ever do comparison when ::recursive flag is set
* We always unset ::recursive flag in x-unlock
* Same thread cannot be unlocking and attempting to lock at the same
time
* thread_id recycling is not an issue because before an id is recycled
the thread must leave innodb meaning it must release all locks meaning
it must unset ::recursive flag.

Pre-requisite patch.
Add end-of-file marker to 5.5 version of errmsg-utf8.txt.

AND SAVEPOINT.
      
The bug was introduced by the patch for bug#11766752. This patch sets too
strong condition on XA state for SAVEPOINT statement that disallows its
execution during XA transaction. But since the statement SAVEPOINT doesn't
imply implicit commit we can allow its handling during XA transaction.

The patch explicitly check for transaction state against states XA_NOTR
and XA_ACTIVE for which the handling of statement SAVEPOINT for XA
transaction is allowed.

mysql-test/t/xa.test:
  Testcase was adjusted for bug#13737343. Now SAVEPOINT is allowed for XA
  transactions in ACTIVE state.

Updated code comments according to reviewers requests.

We are trying to sort a lot of text/blob fields, 
so the buffer is indeed too small.
Memory available = thd->variables.sortbuff_size = 262144
min_sort_memory = param.sort_length*MERGEBUFF2 = 292245
So the decision to abort the query is correct.

filesort() calls my_error(), the error is reported.
But, since we have DELETE IGNORE ... the error is converted to a warning by
THD::raise_condition

filesort currently expects an error to be recorded in the THD diagnostics
area.
If we lift this restriction (remove the assert) we end up in the familiar
void Protocol::end_statement()
  default:
    DBUG_ASSERT(0);

The solution seems to be to call my_error(ME_FATALERROR) in filesort,
so that the error is propagated as an error rather than a warning.


mysql-test/r/filesort_debug.result:
  New test case.
mysql-test/t/filesort_debug.test:
  New test case.

               ENOUGH - CONCAT() HACKS. ALSO WRONG
               ERROR MESSAGE WHILE TRYING TO CREATE
               A VIEW ON A NON EXISTING DATABASE

PROBLEM:
The first part of the problem is concluded as not a
bug, as 'concat' is not a reserved word and it is
completely valid to create a view with the name
'concat'.

The second issue is, while trying to create a view on
a non existing database, we are not giving a proper error
message.

FIX:
We have added a check for the database existence while
trying to create a view. This check would give an error
as 'unknown database' when the database does not exist.

This patch is a backport of the patch for Bug#13601606


mysql-test/r/view.result:
  Added test case result of Bug#12626844
mysql-test/t/view.test:
  Added test case for Bug#12626844
sql/sql_view.cc:
  Added a check for database existence in mysql_create_view

Automerging from mysql-5.1.

Adding missing sync_slave_with_master to the test case.

Fixing waiting condition. "enter_cond" text changed in 5.5+ (when the SQL thread
is waiting for more events).

Automerge changes to the test case from mysql-5.1.

Hardening the test case:
  - including a diff_tables at the end.
  - increasing the tolerance on the relay limit size.

Manual merge from mysql-5.1 into mysql-5.5.

CONFLICTS
=========
  Text conflict in sql/log.cc
  Text conflict in sql/slave.cc

Automerge with mysql-5.1.

IN LOCK_VALIDATE()

rb://917
approved by: Marko Makela

In lock_validate() the limit is used to release the kernel_mutex during
the validation, to obey the latching order.
If we do the limit++ then we are rechecking the same lock most times on
each iteration because limit is being incremented by one and
<space, page_no> will nearly always be > limit. If we set the limit
correctly to (space, page+1) then we are actually making progress
during the iteration.

BUG#64503: mysql frequently ignores --relay-log-space-limit

When the SQL thread goes to sleep, waiting for more events, it sets
the flag ignore_log_space_limit to true. This gives the IO thread a
chance to queue some more events and ultimately the SQL thread will be
able to purge the log once it is rotated. By then the SQL thread
resets the ignore_log_space_limit to false. However, between the time
the SQL thread has set the ignore flag and the time it resets it, the
IO thread will be queuing events in the relay log, possibly going way
over the limit.

This patch makes the IO and SQL thread to synchronize when they reach
the space limit and only ask for one event at a time. Thus the SQL
thread sets ignore_log_space_limit flag and the IO thread resets it to
false everytime it processes one more event. In addition, everytime
the SQL thread processes the next event, and the limit has been
reached, it checks if the IO thread should rotate. If it should, it
instructs the IO thread to rotate, giving the SQL thread a chance to
purge the logs (freeing space). Finally, this patch removes the
resetting of the ignore_log_space_limit flag from purge_first_log,
because this is now reset by the IO thread every time it processes the
next event when the limit has been reached.

If the SQL thread is in a transaction, it cannot purge so, there is no
point in asking the IO thread to rotate. The only thing it can do is
to ask for more events until the transaction is over (then it can ask
the IO to rotate and purge the log right away). Otherwise, there would
be a deadlock (SQL would not be able to purge and IO thread would not
be able to queue events so that the SQL would finish the transaction).

truncating, inserting the same set of rows. When a table is 
re-created with the same set of rows, the data file size must
not grow.  

rb:968
Approved by Marko. 

Fix - Changed the implementation of the condition check from the result file
      to using an assert.

mysql-test/suite/rpl/r/rpl_report_port.result:
  Updated the result file.
mysql-test/suite/rpl/t/rpl_report_port.test:
  Changed from the a condtional check to an assert.

This bug has been there at least since MySQL 4.0.9. (Before 4.0.9, the
code probably was even more severely broken.)

btr_pcur_restore_position(): When cursor restoration fails, before
invoking btr_pcur_store_position() move to the previous or next record
unless cursor->rel_pos==BTR_PCUR_ON or the record was not a user
record.

This bug can cause skipped records when btr_pcur_store_position() is
called on the last record of a page. A symptom would be record count
mismatch in CHECK TABLE, or failure to find a record to delete-mark or
update or purge. The following operations should be affected by the
bug:

* row_search_for_mysql(): SELECT, UPDATE, REPLACE, CHECK TABLE,
  (almost anything else than INSERT)

* foreign key CASCADE operations

* row_merge_read_clustered_index(): index creation (since MySQL 5.1
  InnoDB Plugin)

* multi-threaded purge (after MySQL 5.5): not sure, but it might fail
  to purge some records

Not all callers of btr_pcur_restore_position() should be affected.
Anything that asserts or checks that restoration succeeds is
unaffected. For example, cursor restoration on the change buffer tree
should always succeed, because access is being protected by additional
latches. Likewise, rollback, or any code accesses data dictionary
tables while holding dict_sys->mutex should be safe.

rb:967 approved by Jimmy Yang

storage/innobase/include/sync0rw.ic:
  Prerequisite for compiling with gcc4 on solaris: ignore result from
  os_compare_and_swap_ulint
storage/myisam/mi_dynrec.c:
  Prerequisite for compiling with gcc4 on solaris: cast to void*