• unknown's avatar
    Fixed bug#36005: server crashes inside NOT IN clause subquery with · 5bb4282e
    unknown authored
                     impossible WHERE/HAVING clause
                     (subselect_single_select_engine::exec).
    
    Allocation and initialization of joined table list t1, t2... of
    subqueries like:
    
        NOT IN (SELECT ... FROM t1,t2,... WHERE 0)
    
    is optimized out, however server tries to traverse this list.
    
    
    mysql-test/r/subselect3.result:
      Added test case for bug#36005.
    mysql-test/t/subselect3.test:
      Added test case for bug#36005.
    sql/sql_select.cc:
      Fixed bug#36005.
      
      1. JOIN::prepare initializes JOIN::table counter (actually a size
         of the JOIN::join_tab array) and sets it to a number of joined tables.
      
      2. The make_join_statistics function (when called from JOIN::optimize)
         allocates and fills the JOIN::join_tab array.
         However, when optimizing subselect has impossible (definite false)
         WHERE or HAVING clause, optimizer skips call to make_join_statistics
         and leaves JOIN::join_tab == NULL.
      
      3. subselect_single_select_engine::exec does traversal of the JOIN::join_tab
         array and the server dies because array is not allocated but array
         counter is greater than 0.
      
      The JOIN::optimize method has been modified to reset the JOIN::table
      counter to 0 in cause of impossible WHERE/HAVING clause.
    5bb4282e
sql_select.cc 499 KB