Commit 6b1ac829 authored by Brad Fitzpatrick's avatar Brad Fitzpatrick

cmd/dist: allow builders to control granularity of test/ directory sharding

Even with 10 shards on builders, it still takes about ~2.5 minutes per
shard (and getting slower all the time as the test/ directory grows).
I'm currently experimenting with massively sharding out testing on
Cloud Run (each dist test & normal TestFoo func all running in
parallel), and in such a setup, 2.5 minutes is an eternity. I'd like
to increase that dist test's sharding from 10 to more like 1,000.

Updates golang/go#31834

Change-Id: I8b02989727793b5b5b2013d67e1eb01ef4786e28
Reviewed-on: https://go-review.googlesource.com/c/go/+/175297Reviewed-by: default avatarDmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
parent ed7f323c
......@@ -709,11 +709,14 @@ func (t *tester) registerTests() {
if goos != "android" && !t.iOS() {
// Only start multiple test dir shards on builders,
// where they get distributed to multiple machines.
// See issue 20141.
// See issues 20141 and 31834.
nShards := 1
if os.Getenv("GO_BUILDER_NAME") != "" {
nShards = 10
}
if n, err := strconv.Atoi(os.Getenv("GO_TEST_SHARDS")); err == nil {
nShards = n
}
for shard := 0; shard < nShards; shard++ {
shard := shard
t.tests = append(t.tests, distTest{
......
......@@ -3,6 +3,17 @@
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
# Environment variables that control run.bash:
#
# GO_TEST_SHARDS: number of "dist test" test shards that the
# $GOROOT/test directory will be sliced up into for parallel
# execution. Defaults to 1, unless GO_BUILDER_NAME is also specified,
# in which case it defaults to 10.
#
# GO_BUILDER_NAME: the name of the Go builder that's running the tests.
# Some tests are conditionally enabled or disabled based on the builder
# name or the builder name being non-empty.
set -e
eval $(go env)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment