post-fixes: prevent semi-related overflow, additional comments


mysys/mf_pack.c:
  extra comments
sql/log.cc:
  prevent overflow (length parameter of strmake() should
  never become < 0)
sql/sql_show.cc:
  additional comments
sql/unireg.cc:
  additional comments

strmake() calls are easy to get wrong. Add checks in extra
debug mode to identify possible exploits.

Remove some dead code.

Remove some off-by-one errors identified with new checks.


sql/log.cc:
  fix off-by-one buffer-length argument to prevent stack smashing
sql/repl_failsafe.cc:
  fix off-by-one buffer-length argument to prevent stack smashing
sql/set_var.cc:
  fix off-by-one buffer-length argument to prevent stack smashing
  (already approved, backports #31588)
sql/sql_show.cc:
  misdimensioned buffers: functions further down the callstack
  expect bufsize of FN_REFLEN
sql/unireg.cc:
  When EXTRA_DEBUG is enabled, strmake() will write funny patterns to
  buffers it operates on to identify possibly overflows. This leads to
  badness in mysql_create_frm(), so we explicitly put any unused bytes
  (back) into a defined state. Not a bug-fix, but part of the strmake()
  bug detector.
strings/strmake.c:
  strmake() takes maximum string length rather than buffer-length
  (string length + 1 to accomodate \0 terminator) as argument.
  Since this is easy to get wrong, add extra debug code to identify
  off-by-ones so we can prevent stack smashing.
  
  Alternative "BAD_STRING_COMPILER" removed after checking
  with Monty.

into  mysql.com:/home/hf/work/30286/my41-30286

into  mysql.com:/home/hf/work/30286/my41-30286

As the result of DOUBLE claculations can be bigger
than DBL_MAX constant we use in code, we shouldn't use this constatn
as a biggest possible value.
Particularly the rtree_pick_key function set 'min_area= DBL_MAX' relying
that any rtree_area_increase result will be less so we return valid
key. Though in rtree_area_increase function we calculate the area
of the rectangle, so the result can be 'inf' if the rectangle is
huge enough, which is bigger than DBL_MAX.

Code of the rtree_pick_key modified so we always return a valid key.


myisam/rt_index.c:
  Bug #30286 spatial index cause corruption and server crash!
  
  always set the best_key with the first key we get, so we always return
  somthing valid.
myisam/rt_mbr.c:
  Bug #30286 spatial index cause corruption and server crash!
  
  function comment extended
mysql-test/r/gis-rtree.result:
  Bug #30286 spatial index cause corruption and server crash!
  test result
mysql-test/t/gis-rtree.test:
  Bug #30286 spatial index cause corruption and server crash!
  test case

end-of-line check missed in Gis_read_stream::get_next_word,
what can lead to crashes (expecially with NULL strings).

End-of-line check added


sql/gstream.cc:
  Bug #30955 geomfromtext() crasher
mysql-test/r/gis.result:
  Bug #30955 geomfromtext() crasher.
  
  test result
mysql-test/t/gis.test:
  Bug #30955 geomfromtext() crasher.
  
  test case

in get_index_for_order(), don't walk over the end of the index key parts
when matching index description and needed ordering.


mysql-test/r/delete.result:
  BUG#30385: Testcase
mysql-test/t/delete.test:
  BUG#30385: Testcase

'mysqld_wait_started' don't return prematurely because of an old
pidfile

mysql-test/lib/mtr_misc.pl:
  Add function 'mtr_rmtree' it will try 'rmtree' and if that fails (most likely
  due to permission problems we will fun File::find to chmod all files and dirs
  to 0777 and then delete.
mysql-test/mysql-test-run.pl:
  Use 'mtr_rmtree' in favour of 'rmtree'

into  synthia.local:/home/mydev/mysql-4.1-axmrg

into  pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-4.1-build

into  pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-4.1

into  trift2.:/MySQL/M41/push-4.1

into  chilla.local:/home/mydev/mysql-4.1-axmrg

into  production.mysql.com:/usersnfs/jperkin/bk/trees/build/mysql-4.1

  Fix typo in usage.


myisam/myisamchk.c:
  Fix typo in usage.

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/41

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/41

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/41

into  production.mysql.com:/usersnfs/jperkin/bk/mysql-4.1-maint

which does not work.  Removing these attempted privileges makes
this identical to option 5 so remove it completely.  The spirit
of the program appears to be aimed at database privileges, so do
not add another option for granting global privileges as it may
be unexpected.  Fixes bug#14618 (same as previous patch, this
time applied to -maint tree).


scripts/mysql_setpermission.sh:
  Option 6 tries to apply global privileges at the database
  level which does not work - remove it.

When using concurrent insert with parallel index reads, it could
happen that reading sessions found keys that pointed to records
yet to be written to the data file. The result was a report of
a corrupted table. But it was false alert.

When inserting a record in a table with indexes, the keys are
inserted into the indexes before the record is written to the data
file. When the insert happens concurrently to selects, an
index read can find a key that references the record that is not
yet written to the data file. To avoid any access to such record,
the select saves the current end of file position when it starts.
Since concurrent inserts are always appended at end of the data
file, the select can easily ignore any concurrently inserted record.

The problem was that the ignore was only done for non-exact key
searches (partial key or using >, >=, < or <=).

The fix is to ignore concurrently inserted records also for
exact key searches.

No test case. Concurrent inserts cannot be tested with the test
suite. Test cases are attached to the bug report.


myisam/mi_rkey.c:
  Bug#29838 - myisam corruption using concurrent select ... and update
  Fixed mi_rkey() to always ignore records beyond saved eof.

into  mysql.com:/home/hf/work/29717/my41-29717


sql/sql_select.cc:
  Auto merged

 SELECT statement itself returns empty.

As a result of this bug 'SELECT AGGREGATE_FUNCTION(fld) ... GROUP BY'
can return one row instead of an empty result set.

When GROUP BY only has fields of constant tables
(with a single row), the optimizer deletes the group_list.
After that we lose the information about whether we had an
GROUP BY statement. Though it's important
as SELECT min(x) from empty_table; and
   SELECT min(x) from empty_table GROUP BY y; have to return
different results - the first query should return one row,
second - an empty result set.
So here we add the 'group_optimized_away' flag to remember this case
when GROUP BY exists in the query and is removed
by the optimizer, and check this flag in end_send_group()


mysql-test/r/group_by.result:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test result
mysql-test/r/insert_select.result:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test result
mysql-test/t/group_by.test:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  This is additional testcase that is more basic than the
  original bug's testcase and has the same reason.
mysql-test/t/insert_select.test:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  test case
sql/sql_select.cc:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  Remember the 'GROUP BY was optimized away' case in the JOIN::group_optimized
  and check this in the end_send_group()
sql/sql_select.h:
  Bug #29717 INSERT INTO SELECT inserts values even if
   SELECT statement itself returns empty.
  
  JOIN::group_optimized member added to remember the 'GROUP BY optimied away'
  case

  Backport of correction for Mac OS X build problem, global variable not
  initiated is "common" and can't be used in shared libraries, unless
  special flags are used (bug#26218)


mysys/my_pthread.c:
  Backport of correction for Mac OS X build problem, global variable not
  initiated is "common" and can't be used in shared libraries, unless
  special flags are used (bug#26218)

into  pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-4.1

into  debian.(none):/M41/push-4.1

to 150 or 107 characters for those messages which are generated
by the embedded server during release builds.

This fixes bug#16635:
   Error messages wrong: absolute path names, "%s" format code

See the bug report or the changelog for "sql/share/english/errmsg.txt"
for instructions how to do that with other languages, 
even at the customer site, and for the restrictions to keep.


sql/share/english/errmsg.txt:
  The embedded server uses absolute path names in its error messages,
  in the release build environment these exceed the 64 character limit
  which the format strings for the error messages impose (bug#16635).
  
  But when the messages are output, the server does the "printf()" 
  internally in a 256 character buffer; the constant text and the
  expanded variables (strings, error number) must fit into this.
  
  (If the buffer would overflow, a format specification will not be
  expanded but just copied with its code, and the message output
  will just contain '%s' or '%d' where a value is expected.)
  
  So the string lengths are increased to 150 characters in those messages
  which are issued by the embedded server during release tests
  and contain 1 (one) path name,
  but only to 107 in the "rename" message which contains 2 (two).
  
  This solves bug#16635 for the release builds.
  
  For other languages used by OEM customers, similar fixes may be needed,
  but we cannot test them.
  
  These fixes can be done even in a binary installation at the customer site
  by following these steps:
     cd <<install-root>>/share
     $EDITOR <<lang>>/errmsg.txt
     ../../bin/comp_err -C./charsets/ <<lang>>/errmsg.txt <<lang>>/errmsg.sys
  and then restarting the server.

into  mysql.com:/home/hf/work/29494/my41-29494

This bug manifested itself for join queries with GROUP BY and HAVING clauses
whose SELECT lists contained DISTINCT. It occurred when the optimizer could
deduce that the result set would have not more than one row.
The bug could lead to wrong result sets for queries of this type because
HAVING conditions were erroneously ignored in some cases in the function
remove_duplicates.   


mysql-test/r/having.result:
  Added a test case for bug #29911.
mysql-test/t/having.test:
  Added a test case for bug #29911.

unpack_fields() didn't expect NULL_LENGHT in the field's descriptions.
In this case we get NULL in the resulting string so cannot use
strdup_root to make a copy of it.
strdup_root changed with strmake_root as it's NULL-safe


sql-common/client.c:
  Bug #29494 Field packet with NULL fields crashes libmysqlclient
  
  strdup_root changed with strmake_root in unpack_fields()

into  pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-4.1

into  ramayana.hindu.god:/home/tsmith/m/bk/maint/41

into  sin.intern.azundris.com:/home/tnurnberg/27198/41-27198


sql/mysql_priv.h:
  Bug #27198: Error returns from time() are ignored
  
  manual merge
sql/sql_class.h:
  Bug #27198: Error returns from time() are ignored
  
  manual merge

into  chilla.local:/home/mydev/mysql-4.1-axmrg

gettimeofday() can fail and presumably, so can time().
Keep an eye on it.

Since we have no data on this at all so far, we just
retry on failure (and log the event), assuming that
this is just an intermittant failure. This might of
course hang the threat until we succeed. Once we know
more about these failures, an appropriate more clever
scheme may be picked (only try so many times per thread,
etc., if that fails, return last "good" time() we got or
some such).  Using sql_print_information() to log as this
probably only occurs in high load scenarios where the debug-
trace likely is disabled (or might interfere with testing
the effect).  No test-case as this is a non-deterministic
issue.


sql/mysql_priv.h:
  Bug#27198: Error returns from time() are ignored
  
  move declarations for log.cc to before inclusion of
  sql_class.h as we now use sql_print_information() in
  there.
sql/sql_class.h:
  Bug#27198: Error returns from time() are ignored
  
  gettimeofday() can fail and presumably, so can time().
  Keep an eye on it.

into  pippilotta.erinye.com:/shared/home/df/mysql/build/mysql-4.1

into  sita.local:/Users/tsmith/m/bk/maint/41

into  chilla.local:/home/mydev/mysql-4.1-axmrg