In CL 173938, the linker's object file reader was switched over to
selectively create strings backed with read-only mmap'd memory.
In the process a call to r.rd.Offset() was added to readSymName(),
which greatly increased the number of system calls (Offset does a
seek system call).

This patch changes the object file reader so that all reads are done
directly from the mmap'd data if it is present, and adds logic to keep
track of the offset within the rodata consumed so far. Doing this gets
rid of the calls to r.rd.Offset() and the corresponding seek system
calls.

Also as part of this change, hoist the calls to objabi.PathToPrefix
up into the initial setup code for object reading, and store the
result in the reader (since objabi.PathToPrefix was also coming up
as hot in the profile).

Numbers for this change from compilebench:

benchmark                 old ns/op       new ns/op       delta
BenchmarkTemplate         172053975       170357597       -0.99%
BenchmarkUnicode          64564850        64333653        -0.36%
BenchmarkGoTypes          627931042       628043673       +0.02%
BenchmarkCompiler         2982468893      2924575043      -1.94%
BenchmarkSSA              9701681721      9799342557      +1.01%
BenchmarkFlate            106847240       107509414       +0.62%
BenchmarkGoParser         132082319       130734905       -1.02%
BenchmarkReflect          386810586       383036621       -0.98%
BenchmarkTar              154360072       152670594       -1.09%
BenchmarkXML              217725693       216858727       -0.40%
BenchmarkLinkCompiler     908813802       734363234       -19.20%
BenchmarkStdCmd           32378532486     31222542974     -3.57%

Fixes #31898.

Change-Id: Ibf253a52ce9213325f42b1c2b20d0410f5c88c3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/176039Reviewed-by: Cherry Zhang <cherryyz@google.com>

Fixes #31917

Change-Id: I794e457b2245d355e2df5077078c67aa09e00ff9
Reviewed-on: https://go-review.googlesource.com/c/go/+/175920
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

f+0(SB) is a non-standard but acceptable alias for f(SB).

Fixes #30968.

Change-Id: I499ccee4d3ff3ab4e47f75d99407aace858e59aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/174537Reviewed-by: Austin Clements <austin@google.com>

The most common failure mode of the current std/cmd setup is
going to be people running "go get m@latest" and then not running
"go mod vendor" and being confused about getting the old m.
Diagnose and report what to do.

Also, having done the check, when in the standard library,
switch the go command to -mod=vendor mode.
This avoids some network accesses I saw when running
'go clean -modcache' before doing some work in cmd.

Change-Id: I0ba4a66637b67225a9b97a1c89f26f9015b41673
Reviewed-on: https://go-review.googlesource.com/c/go/+/174528
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

And use it in two internal windows packages, so they don't show up in
"go list std" or binary releases on non-Windows platforms.

Fixes #31920

Change-Id: Iaa292b6015c9d7310dd677c9e296006440ba5e27
Reviewed-on: https://go-review.googlesource.com/c/go/+/175983Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

noteRule is useful when you're trying to debug
a particular rule, or get a general sense for
how often a rule fires overall.

It is less useful if you're trying to figure
out which functions might be useful to benchmark
to ascertain the impact of a newly added rule.

Enter countRule. You use it like noteRule,
except that you get per-function summaries.

Sample output:

 # runtime
(*mspan).sweep: idx1=1
evacuate_faststr: idx1=1
evacuate_fast32: idx1=1
evacuate: idx1=2
evacuate_fast64: idx1=1
sweepone: idx1=1
purgecachedstats: idx1=1
mProf_Free: idx1=1

This suggests that the map benchmarks
might be good to run for this added rule.

Change-Id: Id471c3231f1736165f2020f6979ff01c29677808
Reviewed-on: https://go-review.googlesource.com/c/go/+/167088
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Updates #31920

Change-Id: Ie24ed5bab249e2f90d1740f42a8b8d94fd0983f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176019Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

compilebench depends on the legacy heap profile format to read the
allocation stats of build tools. We're adding a benchmark for the
linker to compilebench, so we need the linker to emit the heap profile
in the legacy format.

This is the linker equivalent of CL 35484, which did this for the
compiler.

Change-Id: I16ad60c4f0fd80b4b6d608a5677ebe04e1fb5e5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/176057
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

When running go build with the flag -mod=readonly, it fails the build if
go.sum files requires updating. This ensures that CI/CD systems get a
complete go.sum file so that they'd never hit a notary,
assuming the CI/CD system passes the above flag.
I am not familiar with the entire codebase but I assume goSum.dirty
will always be true if go.sum has any missing lines.

Fixes #30667

Change-Id: I767d3b594055d8c10048f4c68e6687c94bb0545c
Reviewed-on: https://go-review.googlesource.com/c/go/+/166237Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

In IDE-like applications (and also in tests), there is often a need to
type-check an expression as if it appeared at a particular position in
the source code of an already typed-checked package.

Eval was added to address this need, but did so only partially,
stopping short of exposing a type-annotated expression tree. This
makes it impossible to resolve an expression such as new(T).x and
discover what Object x refers to.  CheckExpr exposes that generality.
Eval is now implemented in terms of CheckExpr.

This change includes a test that demonstrates the object resolution
functionality just described.

Historical context:
- https://go-review.googlesource.com/c/tools/+/10800
- https://codereview.appspot.com/10748044/

Change-Id: I715ba934b9fc0c9ceb61270e20c5f91f4eff20c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/144677
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

benchmark             old ns/op     new ns/op     delta
BenchmarkRatCmp-4     154           77.9          -49.42%

Change-Id: I932710ad8b6905879e232168b1777927f86ba22a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175460
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Currently we say that a negative index means no match,
but we don't say how "no match" is expressed when 'Index'
is not present. Say how it is expressed.

Change-Id: I82b6c9038557ac49852ac03642afc0bc545bb4a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/175677Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I2f2a70dd9279ab95fdabee51579d49363a9f65b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/176018
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I8563a20a4ba43cee7d4b73377c405a6ff12636e5
GitHub-Last-Rev: 0dae408845c7cf42667a65fff6f7d1a382e2d320
GitHub-Pull-Request: golang/go#31914
Reviewed-on: https://go-review.googlesource.com/c/go/+/176017Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Satisfies the Apple Notary.

Fixes #30488

Change-Id: I91cf2d706a3ebe79bafdb759a0d32266ed6b9096
Reviewed-on: https://go-review.googlesource.com/c/go/+/175918
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

The compiler output shown in the doc is now quite old
(most of the changes happened in Go 1.5).
Update it to be more like what users will actually see.

Also explain how to get literal machine code again.

Prompted by #30968.

Change-Id: I0ce139c3fe299ccc43e85b6aca81c6e0aac1a2df
Reviewed-on: https://go-review.googlesource.com/c/go/+/175757
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

Noticed while working on issue 30488.

Change-Id: Ia3655e07c939d03925b3560eeba24c60e24c136c
Reviewed-on: https://go-review.googlesource.com/c/go/+/175917
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Gccgo's implementation of direct interface types has bugs that
causes reflect Call of method from Type.Method fail. CL 175837
and CL 175798 fix the bug. This CL adds a test.

Change-Id: I4e5f2cb96304c1ac7be04ca6d2851bac52b8eb24
Reviewed-on: https://go-review.googlesource.com/c/go/+/175880
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The PNG decode path attempts to handle paletted images that refer to
non-existent palette indicies. This PR fixes a corner case were images
that have exactly 255 palette colors and an IDAT chunk that references
palette index 255 produce an invalid image such that invoking At on
the pixel(s) in question causes an index out of range panic.

Fixes #31830

Change-Id: I34c44d9de5b9d76fe8c45c04e866fbc7f51f2a9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/175397
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Nigel Tao <nigeltao@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

For the case where the addresses of parameter z and x of the function
shlVU overlap and the address of z is greater than x, x (input value)
can be polluted during the calculation when the high words of x are
overlapped with the low words of z (output value).

Fixes #31084

Change-Id: I9bb0266a1d7856b8faa9a9b1975d6f57dece0479
Reviewed-on: https://go-review.googlesource.com/c/go/+/169780
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

This change is a simple work-around to avoid a compiler crash
and provide a reasonable error message. A future change should
fix the root cause for this problem.

Fixes #23823.

Change-Id: Ifc80d9f4d35e063c378e54d5cd8d1cf4c0d2ec6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175518Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I4a70f4a52f84cf50f99939351319504b1c5dff76
Reviewed-on: https://go-review.googlesource.com/c/go/+/175777
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

I assume this was for debugging purposes.

Updates #31624

Change-Id: Ie158fde0574c9bbbd9d1b684f51af5681974aff7
Reviewed-on: https://go-review.googlesource.com/c/go/+/175449
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #31779

Change-Id: Iae80d9adcb39d12c36c525fc2738625cadcc8e41
Reviewed-on: https://go-review.googlesource.com/c/go/+/174838
Run-TryBot: Baokun Lee <nototon@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

Followup to CL 157961 and CL 158457.
Finish the list of operators and punctuation
that disable semicolon insertion at end-of-line
The reported case was "(" but "." was also missing.

Fixes #31017.

Change-Id: I0c06443f38dc8250c62e3aadd104abfa0e3be074
Reviewed-on: https://go-review.googlesource.com/c/go/+/174524
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

No point to s[:].

Change-Id: I9ba5483010180015555ecbed87c1ac82903fd9dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/175277Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The crypto/tls and crypto/x509 APIs leak PublicKey and PrivateKey types,
so in order to add support for Ed25519 certificates we need the ed25519
package in the stdlib.

It's also a primitive that's reasonable to use directly in applications,
as it is a modern, safe and fast signing algorithm, for which there
aren't higher level APIs. (The nacl/sign API is limiting in that it
repeats the message.)

A few docs changes will come in a follow-up, and a CL will land on
golang.org/x/crypto/ed25519 to make it a type alias wrapper on Go 1.13+.

Updates #25355

Change-Id: I057f20cc7d1aca2b95c29ce73eb03c3b237e413f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174945
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>

These changes add tests for URLHostname.

Change-Id: Ie474516401a2236a9be65fb5c4e478322b1a199c
GitHub-Last-Rev: 18f2d597be960c4d13cde12fef5d115926bff7bd
GitHub-Pull-Request: golang/go#31832
Reviewed-on: https://go-review.googlesource.com/c/go/+/175142Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Even with 10 shards on builders, it still takes about ~2.5 minutes per
shard (and getting slower all the time as the test/ directory grows).
I'm currently experimenting with massively sharding out testing on
Cloud Run (each dist test & normal TestFoo func all running in
parallel), and in such a setup, 2.5 minutes is an eternity. I'd like
to increase that dist test's sharding from 10 to more like 1,000.

Updates golang/go#31834

Change-Id: I8b02989727793b5b5b2013d67e1eb01ef4786e28
Reviewed-on: https://go-review.googlesource.com/c/go/+/175297Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Most changes are removing redundant declaration of type when direct
instantiating value of map or slice, e.g. []T{T{}} become []T{{}}.

Small changes are removing the high order of subslice if its value
is the length of slice itself, e.g. T[:len(T)] become T[:].

The following file is excluded due to incompatibility with go1.4,

- src/cmd/compile/internal/gc/ssa.go

Change-Id: Id3abb09401795ce1e6da591a89749cba8502fb26
Reviewed-on: https://go-review.googlesource.com/c/go/+/166437
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Before this CL we used to panic with "nil pointer dereference" because
the value we're calling assignTo on is the zero Value. Provide a better
error message.

Fixes #28748

Change-Id: I7dd4c9e30b599863664d91e78cc45878d8b0052e
Reviewed-on: https://go-review.googlesource.com/c/go/+/175440
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This change adds a new sysHugePage function to provide the equivalent of
Linux's madvise(MADV_HUGEPAGE) support to the runtime. It then uses
sysHugePage to mark a newly-coalesced free span as backable by huge
pages to make the freeHugePages approximation a bit more accurate.

The problem being solved here is that if a large free span is composed
of many small spans which were coalesced together, then there's a chance
that they have had madvise(MADV_NOHUGEPAGE) called on them at some point,
which makes freeHugePages less accurate.

For #30333.

Change-Id: Idd4b02567619fc8d45647d9abd18da42f96f0522
Reviewed-on: https://go-review.googlesource.com/c/go/+/173338
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

Now that the treap is first-fit, we can make a nice optimization.
Mainly, since we know that span splitting doesn't modify the relative
position of a span in a treap, we can actually modify a span in-place
on the treap. The only caveat is that we need to update the relevant
metadata.

To enable this optimization, this change introduces a mutate method on
the iterator which takes a callback that is passed the iterator's span.
The method records some properties of the span before it calls into the
callback and then uses those records to see what changed and update
treap metadata appropriately.

Change-Id: I74f7d2ee172800828434ba0194d3d78d3942acf2
Reviewed-on: https://go-review.googlesource.com/c/go/+/174879
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>

This change tracks the number of potential free and unscavenged huge
pages which will be used to inform the rate at which scavenging should
occur.

For #30333.

Change-Id: I47663e5ffb64cac44ffa10db158486783f707479
Reviewed-on: https://go-review.googlesource.com/c/go/+/170860
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

This change adds two new treap iteration types: one for large
unscavenged spans (contain at least one huge page) and one for small
unscavenged spans. This allows us to scavenge the huge spans first by
first iterating over the large ones, then the small ones.

Also, since we now depend on physHugePageSize being a power of two,
ensure that that's the case when it's retrieved from the OS.

For #30333.

Change-Id: I51662740205ad5e4905404a0856f5f2b2d2a5680
Reviewed-on: https://go-review.googlesource.com/c/go/+/174399
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

This change introduces a treapIterFilter type which represents the
power set of states described by a treapIterType.

This change then adds a treapIterFilter field to each treap node
indicating the types of spans that live in that subtree. The field is
maintained via the same mechanism used to maintain maxPages. This allows
pred, succ, start, and end to be judicious about which subtrees it will
visit, ensuring that iteration avoids traversing irrelevant territory.

Without this change, repeated scavenging attempts can end up being N^2
as the scavenger walks over what it already scavenged before finding new
spans available for scavenging.

Finally, this change also only scavenges a span once it is removed from
the treap. There was always an invariant that spans owned by the treap
may not be mutated in-place, but with this change violating that
invariant can cause issues with scavenging.

For #30333.

Change-Id: I8040b997e21c94a8d3d9c8c6accfe23cebe0c3d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174878
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

This change modifies the treap implementation to support holding all
spans in a single treap, instead of keeping them all in separate treaps.

This improves ergonomics for nearly all treap-related callsites.
With that said, iteration is now more expensive, but it never occurs on
the fast path, only on scavenging-related paths.

This change opens up the opportunity for further optimizations, such as
splitting spans without treap removal (taking treap removal off the span
allocator's critical path) as well as improvements to treap iteration
(building linked lists for each iteration type and managing them on
insert/removal, since those operations should be less frequent).

For #30333.

Change-Id: I3dac97afd3682a37fda09ae8656a770e1369d0a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174398
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

Fixes #31775

`

Change-Id: I59c4e90f20d1b31161c259680b48b7be7218bf58
Reviewed-on: https://go-review.googlesource.com/c/go/+/175017
Run-TryBot: Baokun Lee <nototon@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

Change-Id: I9986a323db2a8f5fa74b071cfd04e8c786da0cb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/175438Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Icca4495f727e3921b717a4bbb441cd832d321d46
Reviewed-on: https://go-review.googlesource.com/c/go/+/175439Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>