MDEV-34969: test fail main.spatial_utility_function_simplify
On aarch64 and when computing the perpendicular distance, we need to avoid the fmsub (Fused Multiply-Subtract) because it can return slightly different precision results when evaluating expressions like x = a - (b * c) such as we did (before this patch) in perpendicular-distance. Instead, we now store the result of the multiplication (b * c) and then subtract it from a which avoids (in all build types) fmsub. This error occurs because the C++ standard allows implementations to use higher precision throughout calculations before storing to the destination type. This can allow for differences in the final result if using higher precision meaningfully changes intermediate values which is what happened in our case. By breaking the operation into two, we prevent the use case where fmsub applies, even for optimized builds under our current build flags configuration. See also https://github.com/dotnet/runtime/issues/64591 https://stackoverflow.com/questions/51124436/strange-issue-with-floating-point-accuracy-on-arm64
Showing
Please register or sign in to comment