• Brian Kessler's avatar
    math/big: handle alias of cofactor inputs in GCD · a73abca3
    Brian Kessler authored
    If the variables passed in to the cofactor arguments of GCD (x, y)
    aliased the input arguments (a, b), the previous implementation would
    result in incorrect results for y.  This change reorganizes the calculation
    so that the only case that need to be handled is when y aliases b, which
    can be handled with a simple check.
    
    Tests were added for all of the alias cases for input arguments and and
    and irrelevant test case for a previous binary GCD calculation was dropped.
    
    Fixes #30217
    
    Change-Id: Ibe6137f09b3e1ae3c29e3c97aba85b67f33dc169
    Reviewed-on: https://go-review.googlesource.com/c/162517
    Run-TryBot: Robert Griesemer <gri@golang.org>
    Reviewed-by: default avatarRobert Griesemer <gri@golang.org>
    TryBot-Result: Gobot Gobot <gobot@golang.org>
    a73abca3
int_test.go 50.9 KB