Fix for BUG#8576

The problem was in different representations of double variables depending on
platform/compiler/compile options. In some cases double variables are represented by
64 bits (while in memory), or by 80 bits (while in FPU register). As a result equal
values are not considered "==". As many sources point out,  doubles should not be
compared by '==' for this reason. This fix subtracts the scaled minimal double
value X such that 1 + X != 1, to ensure that the inequality holds in any case.

sql/opt_range.cc

@@ -7027,8 +7027,12 @@ get_best_group_min_max(PARAM *param, SEL_TREE *tree)
                        cur_group_key_parts, tree, cur_index_tree,
                        cur_quick_prefix_records, have_min, have_max,
                        &cur_read_cost, &cur_records);
-
-    if (cur_read_cost < best_read_cost)
+    /*
+      If cur_read_cost is lower than best_read_cost use cur_index.
+      Do not compare doubles directly because they may have different
+      representations (64 vs. 80 bits).
+    */
+    if (cur_read_cost < best_read_cost - (DBL_EPSILON * cur_read_cost))
     {
       index_info= cur_index_info;
       index= cur_index;