iio: afe: rescale: fix accuracy for small fractional scales
The approximation caused by integer divisions can be costly on smaller scale values since the decimal part is significant compared to the integer part. Switch to an IIO_VAL_INT_PLUS_NANO scale type in such cases to maintain accuracy. Signed-off-by: Liam Beguin <liambeguin@gmail.com> Reviewed-by: Peter Rosin <peda@axentia.se> Reviewed-by: Andy Shevchenko <andy.shevchenko@gmail.com> Link: https://lore.kernel.org/r/20220213025739.2561834-5-liambeguin@gmail.comSigned-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Showing
Please register or sign in to comment