Commit 613b4aa1 authored by Dave Gosselin's avatar Dave Gosselin

MDEV-34969: test fail main.spatial_utility_function_simplify

On aarch64 and when computing the perpendicular distance, we need to
avoid the fmsub (Fused Multiply-Subtract) because it can return
slightly different precision results when evaluating expressions
like
  x = a - (b * c)
such as we did (before this patch) in perpendicular-distance.  Instead,
we now store the result of the multiplication (b * c) and then subtract
it from a which avoids (in all build types) fmsub.

This error occurs because the C++ standard allows implementations to
use higher precision throughout calculations before storing to the destination
type.  This can allow for differences in the final result if using higher
precision meaningfully changes intermediate values which is what happened in
our case.  By breaking the operation into two, we prevent the use case where
fmsub applies, even for optimized builds under our current build flags
configuration.

See also
  https://github.com/dotnet/runtime/issues/64591
  https://stackoverflow.com/questions/51124436/strange-issue-with-floating-point-accuracy-on-arm64
parent 7391f714
......@@ -179,8 +179,10 @@ static double perpendicular_distance(const st_point_2d& point,
double point_vector_dot_product= ((difference_x * point_vector_x) +
(difference_y * point_vector_y));
double adjusted_x= point_vector_x - (point_vector_dot_product * difference_x);
double adjusted_y= point_vector_y - (point_vector_dot_product * difference_y);
double adjusted_x= point_vector_dot_product * difference_x;
adjusted_x= point_vector_x - adjusted_x;
double adjusted_y= point_vector_dot_product * difference_y;
adjusted_y= point_vector_y - adjusted_y;
return sqrt((adjusted_x * adjusted_x) + (adjusted_y * adjusted_y));
}
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment