Suggested during code reviews of last 15 CLs (or so).

Change-Id: If780f6eb47a7a31df133c64d5dcf0eaf04d8447b
Reviewed-on: https://go-review.googlesource.com/10675Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

stkcheck is flow-insensitive: It processes calls in PC order.
Since morestack was always the first call in a function,
it was a safe, conservative approximation to simply adjust stack
space as we went, recognizing morestack when it showed up.

Subsequent CLS will rearrange the function prologue;
morestack may no longer be the first call in a function.

Introducing flow-sensitivity to stkcheck would allow this,
and possibly allow a smaller stackguard.
It is also a high risk change and possibly expensive.

Instead, assume that all calls to morestack occur as
part of the function prologue, no matter where they
are located in the program text.

Updates #10587.

Change-Id: I4dcdd4256a980fc4bc433a68a10989ff57f7034f
Reviewed-on: https://go-review.googlesource.com/10496
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #10209.

Change-Id: I248434f9195c868befd1ed8a6000a9cac72d1df8
Reviewed-on: https://go-review.googlesource.com/10263Reviewed-by: Ian Lance Taylor <iant@golang.org>

Otherwise subsequent tests won't see any modified GOROOT.

With this CL I can move my GOROOT, set GOROOT to the new location, and
the runtime tests pass. Previously the crash_tests would instead look
for the GOROOT baked into the binary, instead of the env var:

--- FAIL: TestGcSys (0.01s)
        crash_test.go:92: building source: exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go
--- FAIL: TestGCFairness (0.01s)
        crash_test.go:92: building source: exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go
--- FAIL: TestGdbPython (0.07s)
        runtime-gdb_test.go:64: building source exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go
--- FAIL: TestLargeStringConcat (0.01s)
        crash_test.go:92: building source: exit status 2
                go: cannot find GOROOT directory: /home/bradfitz/go

Update #10029

Change-Id: If91be0f04d3acdcf39a9e773a4e7905a446bc477
Reviewed-on: https://go-review.googlesource.com/10685Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

The only unreviewed change is in mparith3.go.

Change-Id: Iec0885e7688981cbaed04c152dc9b1c7032677e6
Reviewed-on: https://go-review.googlesource.com/10665Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Robert Griesemer <gri@golang.org>

Change-Id: Ie38742cddc5a256e2f0fc0f720c0ed2f1b2e1bca
Reviewed-on: https://go-review.googlesource.com/10664Reviewed-by: Alan Donovan <adonovan@google.com>

m was being resliced as the result of looking for the first
non zero word of the mantissa, however m was not used later
in printing.

Spotted by Gordon Klaus, https://groups.google.com/forum/#!topic/golang-nuts/MdDLbvOjb4o

Change-Id: Ifbebb51ea5e0d86cb8e0422eb184b8634639a733
Reviewed-on: https://go-review.googlesource.com/10604Reviewed-by: Robert Griesemer <gri@golang.org>

The build ID is an opaque token supplied by the build system.
The compiler writes it out early in the Go export metadata
(the second line), in a way that does not bother existing readers.

The intent is that the go command can use this to store information
about the sources for the generated code, so that it can detect
stale packages even in cases (like removed files) where mtimes fail.

Change-Id: Ib5082515d6cde8a07a8d4b5c69d1e8e4190cb5e1
Reviewed-on: https://go-review.googlesource.com/9153Reviewed-by: Ian Lance Taylor <iant@golang.org>

This one didn't get written out.

Change-Id: Iee173861fb4dc7cafa64ba5f601f4664b6e8da4e
Reviewed-on: https://go-review.googlesource.com/10681Reviewed-by: Russ Cox <rsc@golang.org>

In particular, this avoids moving the mtime on runtime/zversion.go
forward unless the file is out of date. In turn, this makes cross compiles
that run dist multiple times coexist nicely.

(It's no longer necessary to run dist multiple times to set up cross compiles,
but people still might, and it's easy to fix regardless.)

Fixes #4749.

Change-Id: Id430525f168f106bc4b821ca74b2ca498a748f14
Reviewed-on: https://go-review.googlesource.com/9152Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

It was mishandling conjunctions containing negations.

Change-Id: Ife571b28416870ba2ceadbdac5ecb4670432bba1
Reviewed-on: https://go-review.googlesource.com/9151Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #8809.

Change-Id: Id443fd406e9c611d5dfabc71a98eb71d1cc0972c
Reviewed-on: https://go-review.googlesource.com/9150Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I6476284a2cf80d50bd0a57fd9a0de9bc74273c7e
Reviewed-on: https://go-review.googlesource.com/10680Reviewed-by: Russ Cox <rsc@golang.org>

See golang.org/s/go14internal.

Fixes #10479.

Change-Id: I8c657dadeb5f10af060f22dedc15c1af989d4519
Reviewed-on: https://go-review.googlesource.com/9156Reviewed-by: Rob Pike <r@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 160
$

Change-Id: Ib0bd4230c8547f36972b2a9d81ba3eca81496e39
Reviewed-on: https://go-review.googlesource.com/10537Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

This is follow-up to CL10607

- Refactor AddParseTree() to use t.associate()
- Refactor Parse() to use AddParseTree() to put entries into common structure
- Clone() should not put entry in t.tmpl for undefined template
- Clarify documentation for Templates()
- Clarify documentation for AddParseTree() to include the error case

Updates #10910
Uodates #10926

Includes test cases for most of the above changes

Change-Id: I25b2fce6f9651272866f881acf44e4dbca04a4a8
Reviewed-on: https://go-review.googlesource.com/10622Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: Ie7e2cee8cec101560bd5dd013b23969278f89b12
Reviewed-on: https://go-review.googlesource.com/10536Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: I22bcea8099f308298c9db75c937f35e7fca906f1
Reviewed-on: https://go-review.googlesource.com/10535Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: I7decd950fe068c0f294c6c9bff07ef809c394429
Reviewed-on: https://go-review.googlesource.com/10534Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: I1e2f17bfee0e6ca7213706c4cef8d990f4461915
Reviewed-on: https://go-review.googlesource.com/10533Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 168
$

Change-Id: If624a2d72ec04ef30a1bc7ce76c0d61a526d8a37
Reviewed-on: https://go-review.googlesource.com/10532Reviewed-by: Ian Lance Taylor <iant@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 176
$

Change-Id: Ibf1ab531a60d4af8a0c242c0e504f4fd50cd5b36
Reviewed-on: https://go-review.googlesource.com/10530Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 192
$

Change-Id: I8f0c1a3cc2bf9c8eff02bbd8d061ff98affc9eb0
Reviewed-on: https://go-review.googlesource.com/10529Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 200
$

Change-Id: Iba4e88eac6bee3e2349e818a5a2326deabcb96f9
Reviewed-on: https://go-review.googlesource.com/10528Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 224
$

Change-Id: Id0969e8df99c43a5f6f8d77a38f20a71a467e7c6
Reviewed-on: https://go-review.googlesource.com/10527Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 232
$

Change-Id: I4be025f4ec11f882f24ae7582821d36d3b122b77
Reviewed-on: https://go-review.googlesource.com/10526Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Also add a reference to the strings blog post.

Fixes #11045.

Change-Id: Ic0a8908cbd7b51a36d104849fa0e8abfd54de2b9
Reviewed-on: https://go-review.googlesource.com/10662Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #10924

Change-Id: I0caf5f8f82682ee48e95270d43328550bfd8b6e1
Reviewed-on: https://go-review.googlesource.com/10440Reviewed-by: Rob Pike <r@golang.org>

Sending out the conversion of a single test to get comments on the
overall approach.  Converting more tests will follow.

Change-Id: I4755442d08aeb6f74c46856ae406fec41cf8d5dc
Reviewed-on: https://go-review.googlesource.com/10464
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

$ sizeof -p cmd/compile/internal/gc Node
Node 240
$

Change-Id: Id12710c480ed4e0a5bf4f5006f6bd56ef91a2af1
Reviewed-on: https://go-review.googlesource.com/10525Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 248
$

Change-Id: I0fbfeb0d0b36e225eb282fce9e480a96ec1d278f
Reviewed-on: https://go-review.googlesource.com/10524Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 256
$

Change-Id: I89ac8bbe077664aa076092bfd096947e84c0624c
Reviewed-on: https://go-review.googlesource.com/10523Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 264
$

Change-Id: I5c90089dcf5df51c874250f28a1bc3ec32f764b9
Reviewed-on: https://go-review.googlesource.com/10522Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 272
$

Change-Id: I3d9b67eebfc0be0a4b9768d3de3dc76300abd89c
Reviewed-on: https://go-review.googlesource.com/10521Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

$ sizeof -p cmd/compile/internal/gc Node
Node 288
$

Change-Id: I4e316efa246132b3faa3a892e4fe9c9039250665
Reviewed-on: https://go-review.googlesource.com/10520Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

It was an oversight (but as linux/arm64 doesn't support internal
linking and always use external linking with cgo, no harm is done.)

Change-Id: Ie5f2b445cb67a8e63d6b868e63379c68847554f9
Reviewed-on: https://go-review.googlesource.com/10636Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Currently the GODEBUG=gctrace=1 trace line includes "@n.nnns" to
indicate the time that the GC cycle ended relative to the time the
program started. This was meant to be consistent with the utilization
as of the end of the cycle, which is printed next on the trace line,
but it winds up just being confusing and unexpected.

Change the trace line to include the time that the GC cycle started
relative to the time the program started.

Change-Id: I7d64580cd696eb17540716d3e8a74a9d6ae50650
Reviewed-on: https://go-review.googlesource.com/10634Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Due to the requirements of parsing template definitions that mention
other templates that are not yet defined, a Template can be in two states:
defined and undefined. Thus, although one calls New, the resulting
template has no definition even though it exists as a data structure.

Thus, for example, will return nil for a template that is named but not
yet defined.

Fixes #10910
Fixes #10926

Clarify the documentation a little to explain this,
Also tidy up the code a little and remove a spurious call to init.

Change-Id: I22cc083291500bca424e83dc12807e0de7b00b7a
Reviewed-on: https://go-review.googlesource.com/10641Reviewed-by: Andrew Gerrand <adg@golang.org>

This commit implements stack barriers to minimize the amount of
stack re-scanning that must be done during mark termination.

Currently the GC scans stacks of active goroutines twice during every
GC cycle: once at the beginning during root discovery and once at the
end during mark termination. The second scan happens while the world
is stopped and guarantees that we've seen all of the roots (since
there are no write barriers on writes to local stack
variables). However, this means pause time is proportional to stack
size. In particularly recursive programs, this can drive pause time up
past our 10ms goal (e.g., it takes about 150ms to scan a 50MB heap).

Re-scanning the entire stack is rarely necessary, especially for large
stacks, because usually most of the frames on the stack were not
active between the first and second scans and hence any changes to
these frames (via non-escaping pointers passed down the stack) were
tracked by write barriers.

To efficiently track how far a stack has been unwound since the first
scan (and, hence, how much needs to be re-scanned), this commit
introduces stack barriers. During the first scan, at exponentially
spaced points in each stack, the scan overwrites return PCs with the
PC of the stack barrier function. When "returned" to, the stack
barrier function records how far the stack has unwound and jumps to
the original return PC for that point in the stack. Then the second
scan only needs to proceed as far as the lowest barrier that hasn't
been hit.

For deeply recursive programs, this substantially reduces mark
termination time (and hence pause time). For the goscheme example
linked in issue #10898, prior to this change, mark termination times
were typically between 100 and 500ms; with this change, mark
termination times are typically between 10 and 20ms. As a result of
the reduced stack scanning work, this reduces overall execution time
of the goscheme example by 20%.

Fixes #10898.

The effect of this on programs that are not deeply recursive is
minimal:

name                   old time/op    new time/op    delta
BinaryTree17              3.16s ± 2%     3.26s ± 1%  +3.31%  (p=0.000 n=19+19)
Fannkuch11                2.42s ± 1%     2.48s ± 1%  +2.24%  (p=0.000 n=17+19)
FmtFprintfEmpty          50.0ns ± 3%    49.8ns ± 1%    ~     (p=0.534 n=20+19)
FmtFprintfString          173ns ± 0%     175ns ± 0%  +1.49%  (p=0.000 n=16+19)
FmtFprintfInt             170ns ± 1%     175ns ± 1%  +2.97%  (p=0.000 n=20+19)
FmtFprintfIntInt          288ns ± 0%     295ns ± 0%  +2.73%  (p=0.000 n=16+19)
FmtFprintfPrefixedInt     242ns ± 1%     252ns ± 1%  +4.13%  (p=0.000 n=18+18)
FmtFprintfFloat           324ns ± 0%     323ns ± 0%  -0.36%  (p=0.000 n=20+19)
FmtManyArgs              1.14µs ± 0%    1.12µs ± 1%  -1.01%  (p=0.000 n=18+19)
GobDecode                8.88ms ± 1%    8.87ms ± 0%    ~     (p=0.480 n=19+18)
GobEncode                6.80ms ± 1%    6.85ms ± 0%  +0.82%  (p=0.000 n=20+18)
Gzip                      363ms ± 1%     363ms ± 1%    ~     (p=0.077 n=18+20)
Gunzip                   90.6ms ± 0%    90.0ms ± 1%  -0.71%  (p=0.000 n=17+18)
HTTPClientServer         51.5µs ± 1%    50.8µs ± 1%  -1.32%  (p=0.000 n=18+18)
JSONEncode               17.0ms ± 0%    17.1ms ± 0%  +0.40%  (p=0.000 n=18+17)
JSONDecode               61.8ms ± 0%    63.8ms ± 1%  +3.11%  (p=0.000 n=18+17)
Mandelbrot200            3.84ms ± 0%    3.84ms ± 1%    ~     (p=0.583 n=19+19)
GoParse                  3.71ms ± 1%    3.72ms ± 1%    ~     (p=0.159 n=18+19)
RegexpMatchEasy0_32       100ns ± 0%     100ns ± 1%  -0.19%  (p=0.033 n=17+19)
RegexpMatchEasy0_1K       342ns ± 1%     331ns ± 0%  -3.41%  (p=0.000 n=19+19)
RegexpMatchEasy1_32      82.5ns ± 0%    81.7ns ± 0%  -0.98%  (p=0.000 n=18+18)
RegexpMatchEasy1_1K       505ns ± 0%     494ns ± 1%  -2.16%  (p=0.000 n=18+18)
RegexpMatchMedium_32      137ns ± 1%     137ns ± 1%  -0.24%  (p=0.048 n=20+18)
RegexpMatchMedium_1K     41.6µs ± 0%    41.3µs ± 1%  -0.57%  (p=0.004 n=18+20)
RegexpMatchHard_32       2.11µs ± 0%    2.11µs ± 1%  +0.20%  (p=0.037 n=17+19)
RegexpMatchHard_1K       63.9µs ± 2%    63.3µs ± 0%  -0.99%  (p=0.000 n=20+17)
Revcomp                   560ms ± 1%     522ms ± 0%  -6.87%  (p=0.000 n=18+16)
Template                 75.0ms ± 0%    75.1ms ± 1%  +0.18%  (p=0.013 n=18+19)
TimeParse                 358ns ± 1%     364ns ± 0%  +1.74%  (p=0.000 n=20+15)
TimeFormat                360ns ± 0%     372ns ± 0%  +3.55%  (p=0.000 n=20+18)

Change-Id: If8a9bfae6c128d15a4f405e02bcfa50129df82a2
Reviewed-on: https://go-review.googlesource.com/10314Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Currently there's a race between stopg scanning another G's stack and
the G reaching a preemption point and scanning its own stack. When
this race occurs, the G's stack is scanned twice. Currently this is
okay, so this race is benign.

However, we will shortly be adding stack barriers during the first
stack scan, so scanning will no longer be idempotent. To prepare for
this, this change ensures that each stack is scanned only once during
each GC phase by checking the flag that indicates that the stack has
been scanned in this phase before scanning the stack.

Change-Id: Id9f4d5e2e5b839bc3f200ec1723a4a12dd677ab4
Reviewed-on: https://go-review.googlesource.com/10458Reviewed-by: Rick Hudson <rlh@golang.org>