Commit b9f918a3 authored by Rasmus Villemoes's avatar Rasmus Villemoes Committed by David Howells

MPILIB: Fix comparison of negative MPIs

If u and v both represent negative integers and their limb counts
happen to differ, mpi_cmp will always return a positive value - this
is obviously bogus. u is smaller than v if and only if it is larger in
absolute value.
Signed-off-by: default avatarRasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: default avatarDavid Howells <dhowells@redhat.com>
Acked-by: default avatarDmitry Kasatkin <dmitry.kasatkin@gmail.com>
parent 98dbbcba
...@@ -57,7 +57,7 @@ int mpi_cmp(MPI u, MPI v) ...@@ -57,7 +57,7 @@ int mpi_cmp(MPI u, MPI v)
if (usize != vsize && !u->sign && !v->sign) if (usize != vsize && !u->sign && !v->sign)
return usize - vsize; return usize - vsize;
if (usize != vsize && u->sign && v->sign) if (usize != vsize && u->sign && v->sign)
return vsize + usize; return vsize - usize;
if (!usize) if (!usize)
return 0; return 0;
cmp = mpihelp_cmp(u->d, v->d, usize); cmp = mpihelp_cmp(u->d, v->d, usize);
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment