cmd/doc: skip directories like other go tools

It was skipping dirs starting with ".", but it was missing the "_"
prefix and the "testdata" name. From "go help packages":

	Directory and file names that begin with "." or "_" are ignored
	by the go tool, as are directories named "testdata".

Before the change:

	$ go doc z # using src/cmd/go/testdata/testvendor/src/q/z
	package z // import "."

After the fix, it falls back to the current directory, as expected when
a single argument isn't found as a package in $GOPATH.

TestMain needs a small adjustment to keep the tests working, as now
their use of cmd/doc/testdata would normally not work.

This is the second try for this fix; the first time around, we included
cmd/doc/testdata to the dirs list by sending it to the channel via a
goroutine. However, that can end up in a send to a closed channel, if
GOROOT is a very small directory tree or missing.

To avoid that possibility, include the extra directory by pre-populating
the paths list, before the walking of GOROOT and GOPATH actually starts.

Fixes #24462.

Change-Id: I3b95b6431578e0d5cbb8342f305debc4ccb5f656
Reviewed-on: https://go-review.googlesource.com/109216
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Elias Naur <elias.naur@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

src/cmd/doc/dirs.go

@@ -24,8 +24,11 @@ type Dirs struct {
 
 var dirs Dirs
 
-func dirsInit() {
+// dirsInit starts the scanning of package directories in GOROOT and GOPATH. Any
+// extra paths passed to it are included in the channel.
+func dirsInit(extra ...string) {
 	dirs.paths = make([]string, 0, 1000)
+	dirs.paths = append(dirs.paths, extra...)
 	dirs.scan = make(chan string)
 	go dirs.walk()
 }
@@ -97,8 +100,9 @@ func (d *Dirs) bfsWalkRoot(root string) {
 					continue
 				}
 				// Entry is a directory.
-				// No .git or other dot nonsense please.
-				if strings.HasPrefix(name, ".") {
+
+				// The go tool ignores directories starting with ., _, or named "testdata".
+				if name[0] == '.' || name[0] == '_' || name == "testdata" {
 					continue
 				}
 				// Remember this (fully qualified) directory for the next pass.

src/cmd/doc/doc_test.go

@@ -16,10 +16,18 @@ import (
 )
 
 func TestMain(m *testing.M) {
-	// otherwise the tests are brittle, as they may give unexpected
-	// output or errors when a suffix match with GOPATH takes place
+	// Clear GOPATH so we don't access the user's own packages in the test.
 	buildCtx.GOPATH = ""
-	dirsInit()
+
+	// Add $GOROOT/src/cmd/doc/testdata explicitly so we can access its contents in the test.
+	// Normally testdata directories are ignored, but sending it to dirs.scan directly is
+	// a hack that works around the check.
+	testdataDir, err := filepath.Abs("testdata")
+	if err != nil {
+		panic(err)
+	}
+	dirsInit(testdataDir)
+
 	os.Exit(m.Run())
 }