This change fixes an error in HACKING.md which claims all pointers
which live in unmanaged memory but point to the heap must be marked
as GC roots explicitly by runtime.markroot. This isn't technically
necessary if the pointer is accessible through a global variable.

Change-Id: I632b25272fdb2f789c5259dd1685d517f45fd435
Reviewed-on: https://go-review.googlesource.com/c/151539Reviewed-by: Rick Hudson <rlh@golang.org>

In TestObjImporter skip tests that use type aliases when using a
version of gccgo before GCC 7, since that is when type aliases were
added.

Fixes #29006

Change-Id: I676bae9f023931cf95ac9b4d4de893fe8517af9b
Reviewed-on: https://go-review.googlesource.com/c/152078Reviewed-by: Than McIntosh <thanm@google.com>

TestInstallationImporter checks that it can read the export data for a
list of known standard library packages. It was failing on the SmartOS
builder which has GCC 4.7 installed. Skip packages that did not exist
in GCC 4.7. Most packages are still there and the missing packages are
fairly simple, so this doesn't really affect test quality.

Updates #29006

Change-Id: If7ae6f83d51d40168a9692acb0b99c9bf21f2a4d
Reviewed-on: https://go-review.googlesource.com/c/152077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fix a bug in the reading of elderly export data. In such export data
when reading type information it's possible to encounter a named type N1
defined as a typedef of some other named type N2 at a point when the
underying type of N1 has not yet been finalized. Handle this case by
generating a fixup, then process fixups at the end of parsing to
set the correct underlying type.

Fixes #29006.

Change-Id: I6a505c897bd95eb161ee04637bb6eebad9f20d52
Reviewed-on: https://go-review.googlesource.com/c/151997
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #28960

Change-Id: I0d049d4776dc42ef165a1da15f63de08677fbb85
Reviewed-on: https://go-review.googlesource.com/c/151661
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>

signatureSchemesForCertificate was written to be used with TLS 1.3, but
ended up used for TLS 1.2 client certificates in a refactor. Since it
only supported TLS 1.3 signature algorithms, it would lead to no RSA
client certificates being sent to servers that didn't support RSA-PSS.

TestHandshakeClientCertRSAPKCS1v15 was testing *specifically* for this,
but alas the OpenSSL flag -verify accepts an empty certificates list as
valid, as opposed to -Verify...

Fixes #28925

Change-Id: I61afc02ca501d3d64ab4ad77bbb4cf10931e6f93
Reviewed-on: https://go-review.googlesource.com/c/151660
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>

Updates golang/go#27992

Change-Id: Ic327df7cc5002a3d537f9117559c25f30e1eab9c
Reviewed-on: https://go-review.googlesource.com/c/151799
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The call to os.Rename was failing on the darwin builders, despite having passed in the TryBots.
(I tested this change by running 'go test cmd/go' manually on a darwin gomote.)

This fixes the builder failures after CL 146382.

Updates #26794
Fixes #29030

Change-Id: I3644773421789f65e56f183d077b4e4fd17b8325
Reviewed-on: https://go-review.googlesource.com/c/151798Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I0390b382db0ca36de20af0ef15204c5bfc084d3d
Reviewed-on: https://go-review.googlesource.com/c/151937
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

We don't need a write barrier if:
1) The location we're writing to doesn't hold a heap pointer, and
2) The value we're writing isn't a heap pointer.

The freshly returned value from runtime.newobject satisfies (1).
Pointers to globals, and the contents of the read-only data section satisfy (2).

This is particularly helpful for code like:
p := []string{"abc", "def", "ghi"}

Where the compiler generates:
   a := new([3]string)
   move(a, statictmp_)  // eliminates write barriers here
   p := a[:]

For big slice literals, this makes the code a smaller and faster to
compile.

Update #13554. Reduces the compile time by ~10% and RSS by ~30%.

Change-Id: Icab81db7591c8777f68e5d528abd48c7e44c87eb
Reviewed-on: https://go-review.googlesource.com/c/151498
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

This adds several new checks to help debug #27993. It adds a mechanism
for freezing write barriers and gcWork puts during the mark completion
algorithm. This way, if we do detect mark completion, we can catch any
puts that happened during the completion algorithm. Based on build
dashboard failures, this seems to be the window of time when these are
happening.

This also double-checks that all work buffers are empty immediately
upon entering mark termination (much earlier than the current check).
This is unlikely to trigger based on the current failures, but is a
good safety net.

Change-Id: I03f56c48c4322069e28c50fbc3c15b2fee2130c2
Reviewed-on: https://go-review.googlesource.com/c/151797
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>

The old code ignored the field alignment, and only looked at the field
offset: if the field offset required padding, cgo added padding. But
while that approach works for Go (at least with the gc toolchain) it
doesn't work for C code using packed structs. With a packed struct the
added padding may leave the struct at a misaligned position, and the
inserted alignment, which cgo is not considering, may introduce
additional, unexpected, padding. Padding that ignores alignment is not
a good idea when the struct is not packed, and Go structs are never
packed. So don't ignore alignment.

Fixes #28896

Change-Id: Ie50ea15fa6dc35557497097be9fecfecb11efd8a
Reviewed-on: https://go-review.googlesource.com/c/150602
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

We refactor the conversion of quotes to their unicode equivalent
to a separate function so that it can be called from ToText and Synopsis.

And we introduce a temp buffer to write the escaped HTML and convert
the unicode quotes back to html escaped entities. This simplifies the logic
and gets rid of the need to track the index of the escaped text.

Fixes #27759

Change-Id: I71cf47ddcd4c6794ccdf2898ac25539388b393c1
Reviewed-on: https://go-review.googlesource.com/c/150377
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Adjust mTreap ordering logic to reflect the description of mTreap ordering.
Before it was using unsafe.Pointer in order to gather the base address of
a span. This has been changed to use base, which is the startAddress of a
span as the description is telling us in mgclarge.go.

Fixes: golang/go#28805

Change-Id: Ib3cd94a0757e23d135b5d41830f38fc08bcf16a3
GitHub-Last-Rev: 93f749b6700b1e179de16607a18395d5e162ecc1
GitHub-Pull-Request: golang/go#28973
Reviewed-on: https://go-review.googlesource.com/c/151499Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>

Previously, we were looking for the string go.mod specifically, but
the module-mode-outside-a-module logic added in CL 148517 sets GOMOD
to os.DevNull

Updates #28992

Change-Id: I62a4baaa911a495350294d78bae96be3fe4866cb
Reviewed-on: https://go-review.googlesource.com/c/151617
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: I4717964234fca0c8c5889ed710b66f39eb53a809
Reviewed-on: https://go-review.googlesource.com/c/151562
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

The test that used that module was removed in
https://golang.org/cl/128900.

Change-Id: Id96270a52398c8ccc09821efb2a6a6b4764f44d9
Reviewed-on: https://go-review.googlesource.com/c/151560
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

This assignment became a no-op in CL 36196, where both it and the
preceding assignment were changed to cfg.GOROOTbin (from gorootBin and
gobin respectively).

Change-Id: If74969c4cc3ffc5d8394ff9d8e8bcec9e0a4e3b0
Reviewed-on: https://go-review.googlesource.com/c/151561
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Also test that Download restores deleted files.

Updates #27783

Change-Id: If50074dbcffd74ff08fbaa9ad8c314cfdce0b02d
Reviewed-on: https://go-review.googlesource.com/c/151559
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

This is as minimal a change as I could comfortably make to enable 'go
get' outside of a module for 1.12.

In general, commands invoked in module mode while outside of a module
operate as though they are in a module with an initially-empty go.mod
file. ('go env GOMOD' reports os.DevNull.)

Commands that operate on the current directory (such as 'go list' and
'go get -u' without arguments) fail: without a module definition, we
don't know the package path. Likewise, commands whose sole purpose is
to write files within the main module (such as 'go mod edit' and 'go
mod vendor') fail, since we don't know where to write their output.

Since the go.sum file for the main module is authoritative, we do not
check go.sum files when operating outside of a module. I plan to
revisit that when the tree opens for 1.13.

We may also want to revisit the behavior of 'go list': it would be
useful to be able to query individual packages (and dependencies of
those packages) within versioned modules, but today we only allow
versioned paths in conjunction with the '-m' flag.

Fixes #24250

RELNOTE=yes

Change-Id: I028c323ddea27693a92ad0aa4a6a55d5e3f43f2c
Reviewed-on: https://go-review.googlesource.com/c/148517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

The lockfile guards calls that may change the repo's filesystem contents.

We don't know how robust VCS implementations are to running
simultaneous commands, and this way we don't need to care: only one
'go' command at a time will modify any given repository.

If we can guarantee that particular VCS implementations are robust
enough across all of the VCS tool versions we support, we may be able
to remove some of this locking to improve parallelism.

Updates #26794

Change-Id: I578524974f5015629239cef43d3793aee2b9075c
Reviewed-on: https://go-review.googlesource.com/c/146381
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>

Use an arbitrary lockfile to serialize edits, and use atomic renames
to actually write the go.mod file so that we never drop version
requirements due to a command failing partway through a write.

Multiple invocations of the 'go' command may read the go.mod file
concurrently, and will see some consistent version even if some other
invocation changes it concurrently.

Multiple commands may attempt to write the go.mod file concurrently.
One writer will succeed and write a consistent, complete go.mod file.
The others will detect the changed contents and fail explicitly: it is
not, in general, possible to resolve two conflicting changes to module
requirements, so we surface the problem to the user rather than trying
to solve the problem heuristically.

Updates #26794

Change-Id: Ia1a06a01ef93fa9be664f560eb83bb86b0207443
Reviewed-on: https://go-review.googlesource.com/c/146380
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

We employ the following new locking mechanisms:

• Zip files and list files within the module cache are written using
  atomic renames of temporary files, so that GOPROXY servers reading
  from the cache will never serve incomplete content.

• A lock file for each module version guards downloading and extraction of
  (immutable) module contents.

• A lock file alongside each version list (named 'list.lock')
  guards updates to the list.

• A single lock file in the module cache guards updates to all go.sum
  files. The go.sum files themselves are written using an atomic
  rename to ensure that we never accidentally discard existing sums.

Updates #26794

RELNOTE=yes

Change-Id: I16ef8b06ee4bd7b94d0c0a6f5d17e1cecc379076
Reviewed-on: https://go-review.googlesource.com/c/146382
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>

This will be used to eliminate a redundant copy in CL 145178.

(It also decouples two design points that were previously coupled: the
destination of the zip output and the program logic to write that
output.)

Updates #26794

Change-Id: I6cfd5a33c162c0016a1b83a278003684560a3772
Reviewed-on: https://go-review.googlesource.com/c/151341
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Also check to make sure we don't overwrite a newer timestamp with an
older one.

testexpire.txt may be written concurrently, and a partially-written
timestamp may appear much older than the actual intended one. We don't
want to re-run tests that should still be cached.

Updates #26794

Change-Id: If56348e799f0e7be3c5bc91b4a336e23ad99f791
Reviewed-on: https://go-review.googlesource.com/c/146379
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

lockedfile.File passes through to os.File, with Open, Create, and OpenFile
functions that mimic the corresponding os functions but acquire locks
automatically, releasing them when the file is closed.

lockedfile.Sentinel is a simplified wrapper around lockedfile.OpenFile for the
common use-case of files that signal the status of idempotent tasks.

lockedfile.Mutex is a Mutex-like synchronization primitive implemented in terms
of file locks.

lockedfile.Read is like ioutil.Read, but obtains a read-lock.

lockedfile.Write is like ioutil.Write, but obtains a write-lock and can be used
for read-only files with idempotent contents.

Updates #26794

Change-Id: I50f7132c71d2727862eed54411f3f27e1af55cad
Reviewed-on: https://go-review.googlesource.com/c/145178
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: Ief11612b67def93311707165910124d3ce28fb89
Reviewed-on: https://go-review.googlesource.com/c/151777Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Updates #26794

Change-Id: I2a50e3b756ff6a2bbaee4737ca7ed053b01c8d0e
Reviewed-on: https://go-review.googlesource.com/c/146378Reviewed-by: Russ Cox <rsc@golang.org>

renameio.WriteFile writes files atomically by renaming temporary files.

See the subsequent changes for usage examples.

Updates #26794
Updates #22397

Change-Id: I4bfe3125a53f58060587f98afbb4260bb1cc3d32
Reviewed-on: https://go-review.googlesource.com/c/146377
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Giovanni Bajo <rasky@develer.com>
Reviewed-by: Russ Cox <rsc@golang.org>

Packages in vendor/ directories have a "vendor/" path prefix in GOPATH
mode, but intentionally do not in module mode. Since the import path
is embedded in the compiled output, changing that path invalidates
cache entries and causes cmd/go to try to rebuild (and reinstall) the
vendored libraries, which will fail if the directory containing those
libraries is read-only.

If I understood correctly, this is the approach Russ suggested as an
alternative to https://golang.org/cl/136138.

Fixes #27285
Fixes #26988

Change-Id: I8a2507fa892b84cde0a803aaa79e460723da572b
Reviewed-on: https://go-review.googlesource.com/c/147443
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

This patch merges in support for reading the most recent
incarnation of V3 export data (initial inline function bodies),
from the importer portions of https://golang.org/cl/150061 and
https://golang.org/cl/150067.

Updates #28961.

Change-Id: I34e837acbf48b8fd1a4896a1a977d2241adfb28d
Reviewed-on: https://go-review.googlesource.com/c/151557
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

For the entry block, make the "first instruction" be truly
the first instruction.  This allows printing of incoming
parameters with Delve.

Also be sure Phis are marked as being at the start of their
block.  This is observed to move location list pointers,
and where moved, they become correct.

Leading zero-width instructions include LoweredGetClosurePtr.
Because this instruction is actually architecture-specific,
and it is now tested for in 3 different places, also created
Op.isLoweredGetClosurePtr() to reduce future surprises.

Change-Id: Ic043b7265835cf1790382a74334b5714ae4060af
Reviewed-on: https://go-review.googlesource.com/c/145179
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>

Apparently (see
https://github.com/golang/go/issues/27904#issuecomment-442140627
for details) kernel32.dll file is not present on windows/arm, so
use ws2_32.dll instead. ws2_32.dll imports table also lives in
'.rdata' section, so ws2_32.dll is as good as kernel32.dll for
testing issue #16103.

Updates #27904

Change-Id: Ibc72b24eea9a4d85abd371ffdcf00442e711b745
Reviewed-on: https://go-review.googlesource.com/c/151480Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>

Since they are sent after the handshake in TLS 1.3, the client was not
actually consuming them, as it doesn't make any Read calls. They were
then sitting in the kernel receive buffer when the client would call
Close. The kernel would see that and send a RST, which would race the
closeNotify, causing errors.

Also, we get to trim 600 lines of useless test data.

Fixes #28852

Change-Id: I7517feab77dabab7504bfc111098ba09ea07ae5e
Reviewed-on: https://go-review.googlesource.com/c/151659Reviewed-by: Ian Lance Taylor <iant@golang.org>

An untyped constant can be defined in any input file, we shouldn't
segregate them by file.

Updates #28772

Change-Id: I0347f15236833bb511eb49f86c449ee9241b0a25
Reviewed-on: https://go-review.googlesource.com/c/151600
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>

VPERMXOR is missing from the Go assembler for ppc64. It has the
same format as VPERM. It was requested by an external user so
they could write an optimized algorithm in asm.

Change-Id: Icf4c682f7f46716ccae64e6ae3d62e8cec67f6c1
Reviewed-on: https://go-review.googlesource.com/c/151578
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>

There are three functions that do Connection header write:
1. transport.go/ persistConn.roundTrip
2. transfer.go/ transferWriter.writeHeader
3. request.go/ Request.write

The root cause is roundTrip didn't lookup into request.Close and
transferWriter
didn't take care of extraHeaders.

Fixes #28886

Change-Id: I1d131019c7cd42eb1bcc972c631b7df7511c1f39
Reviewed-on: https://go-review.googlesource.com/c/150722
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Break ADDconst into ADD + MOVDconst, so that if the constant
is too big it won't overflow ADDconst's constant field.

For normal sizes, other rules will recombine into an ADDconst.

Fixes S390X breakage from CL 33909.

Change-Id: Id804ee052365527efb580f797688b0ce83c47915
Reviewed-on: https://go-review.googlesource.com/c/151597
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Michael Munday <mike.munday@ibm.com>

Change-Id: I956d6d1dbf8516cb65eb3a0686a3b0584b4a6840
GitHub-Last-Rev: 1c928f3c67eceae424cbcd6b0935605a78728604
GitHub-Pull-Request: golang/go#28991
Reviewed-on: https://go-review.googlesource.com/c/151324Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

CL 151478 was suppose to fix the build failure on linux/arm64 but the
build constraint didn't exclude linux/arm64 properly.

Fixes #28982

Change-Id: Ia80265b0adba0384cd28bc2deb1726418664975a
Reviewed-on: https://go-review.googlesource.com/c/151303
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>