cmd/go: suppress errors in package-to-module queries if the package is already found

In CL 173017, I changed the package-to-module query logic to query all
possible module paths in parallel in order to reduce latency. (For
long package paths, most such paths will not exist and will fail with
little overhead.)

The module resolution algorithm treats various kinds of non-existence
as “soft errors”, to be reported only if package resolution fails, but
treats any remaining errors as hard errors that should fail the query.

Unfortunately, that interacted badly with the +incompatible version
validation added in CL 181881, causing a regression in the 'direct'
fetch path for modules using the “major branch” layout¹ with a post-v1
version on the repository's default branch. Because we did not
interpret a mismatched module path as “no such module”, a go.mod file
specifying the path 'example.com/foo/v2' would cause the search for
module 'example.com/foo' to error out. (That regression was not caught
ahead of time due to a lack of test coverage for 'go get' on a package
within a /vN module.)

The promotion of hard errors during parallel search also made the 'go'
command less tolerant of servers that advertise 'go-import' tags for
nonexistent repositories. CL 194561 mitigated that problem for HTTP
servers that return code 404 or 410 for a nonexistent repository, but
unfortunately a few servers in common use (notably GitLab and
pre-1.9.3 releases of Gitea) do not.

This change mitigates both of those failure modes by ignoring
“miscellaneous” errors from shorter module paths if the requested
package pattern was successfully matched against a module with a
longer path.

¹https://research.swtch.com/vgo-module#from_repository_to_modules

Updates #34383
Updates #34094

Change-Id: If37dc422e973eba13f3a3aeb68bc7b96e2d7f73d
Reviewed-on: https://go-review.googlesource.com/c/go/+/197059
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>

src/cmd/go/internal/modload/query.go

@@ -471,10 +471,20 @@ func queryPrefixModules(candidateModules []string, queryModule func(path string)
 					notExistErr = rErr
 				}
 			} else if err == nil {
+				if len(found) > 0 {
+					// golang.org/issue/34094: If we have already found a module
+					// containing the target package, ignore errors for modules with
+					// shorter paths.
+
+					// golang.org/issue/34383 is a special case of this: if we have
+					// already found example.com/foo/v2@v2.0.0 with a matching go.mod
+					// file, ignore the error from example.com/foo@v2.0.0.
+				} else {
 					err = r.err
 				}
 			}
 		}
+	}
 
 	// TODO(#26232): If len(found) == 0 and some of the errors are 4xx HTTP
 	// codes, have the auth package recheck the failed paths.

src/cmd/go/testdata/script/mod_get_major.txt

+[!net] skip
+[!exec:git] skip
+
+env GO111MODULE=on
+env GOPROXY=direct
+env GOSUMDB=off
+
+# golang.org/issue/34383: if a module path ends in a major-version suffix,
+# ensure that 'direct' mode can resolve the package to a module.
+
+go get -d vcs-test.golang.org/git/v3pkg.git/v3@v3.0.0
+
+go list -m vcs-test.golang.org/git/v3pkg.git/v3
+stdout '^vcs-test.golang.org/git/v3pkg.git/v3 v3.0.0$'
+
+-- go.mod --
+module example.com
+go 1.13