We lazily map the bitmap and spans areas as the heap grows. However,
right now we're very slightly too lazy. Specifically, the following
can happen on 32-bit:

1. mallocinit fails to allocate any heap arena, so
   arena_used == arena_alloc == arena_end == bitmap.

2. There's less than 256MB between the end of the bitmap mapping and
   the next mapping.

3. On the first allocation, mheap.sysAlloc sees that there's not
   enough room in [arena_alloc, arena_end) because there's no room at
   all. It gets a 256MB mapping from somewhere *lower* in the address
   space than arena_used and sets arena_alloc and arena_end to this
   hole.

4. Since the new arena_alloc is lower than arena_used, mheap.sysAlloc
   doesn't bother to call mheap.setArenaUsed, so we still don't have a
   bitmap mapping or a spans array mapping.

5. mheap.grow, which called mheap.sysAlloc, attempts to fill in the
   spans array and crashes.

Fix this by mapping the metadata regions for the initial arena_used
when the heap is initialized, rather than trying to wait for an
allocation. This maintains the intended invariant that the structures
are always mapped for [arena_start, arena_used).

Fixes #21044.

Cherry-pick of CL 51714. Fixes #21234.

Change-Id: I4422375a6e234b9f979d22135fc63ae3395946b0
Reviewed-on: https://go-review.googlesource.com/52191
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Right now, if it's possible to grow the arena reservation but
mheap.sysAlloc fails to get 256MB more of memory, it simply fails.
However, on 32-bit we have a fallback path that uses much smaller
mmaps that could take in this situation, but fail to.

This commit fixes mheap.sysAlloc to use a common failure path in case
it can't grow the reservation. On 32-bit, this path includes the
fallback.

Ideally, mheap.sysAlloc would attempt smaller reservation growths
first, but taking the fallback path is a simple change for Go 1.9.

Updates #21044 (fixes one of two issues).

Cherry-pick of CL 51713. Updates #21234.

Change-Id: I1e0035ffba986c3551479d5742809e43da5e7c73
Reviewed-on: https://go-review.googlesource.com/52190
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I0bcae339624e7d61037d9ea0885b7bd07491bbb6
Reviewed-on: https://go-review.googlesource.com/51430
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
(cherry picked from commit 4833e920c1d7f6b23458e6ff3c73951fcf754219)
Reviewed-on: https://go-review.googlesource.com/51450Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I6251fc128990f9494a0b037cabb24ada5eabb6b5
Reviewed-on: https://go-review.googlesource.com/50951
Run-TryBot: Chris Broadfoot <cbro@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Chris Broadfoot <cbro@golang.org>

It is possible to have an unexported name with a nil package,
for an embedded field whose type is a pointer to an unexported type.
We must encode that fact in the type..namedata symbol name,
to avoid incorrectly merging an unexported name with an exported name.

Fixes #21120

Change-Id: I2e3879d77fa15c05ad92e0bf8e55f74082db5111
Reviewed-on: https://go-review.googlesource.com/50710
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-on: https://go-review.googlesource.com/50970Reviewed-by: Chris Broadfoot <cbro@golang.org>

This avoids an error from clang when using -nopie during compilation,
and permits us to check that the entire build succeeds.

Updates #21042

Change-Id: I2e6c7d5c97a85c223ed3288622bbb58ce33b8774
Reviewed-on: https://go-review.googlesource.com/50874
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

It was released with 1.8, so it should be widely available.

Change-Id: I7dcf205009b528071ea63f99cb5da0db183df341
Reviewed-on: https://go-review.googlesource.com/50932Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

GitHub recently added support for a SUPPORT file:

https://github.com/blog/2400-support-file-support

This SUPPORT file is a very lightly edited copy
of the wiki entry on asking questions:

https://github.com/golang/go/wiki/Questions

Change-Id: Ic1eb74d985ea30862defb99750fb42da84e492de
Reviewed-on: https://go-review.googlesource.com/50930Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

https://golang.org/cl/33773 fixes the JSON marshaler to avoid serializing
embedded fields on unexported types of non-struct types. However, Go allows
embedding pointer to types, so the check for whether the field is a non-struct
type must first dereference the pointer to get at the underlying type.

Furthermore, due to a edge-case in the behavior of StructField.PkgPath not
being a reliable indicator of whether the field is unexported (see #21122),
we use our own logic to determine whether the field is exported or not.

The logic in this CL may be simplified depending on what happens in #21122.

Fixes #21121
Updates #21122

Change-Id: I8dfd1cdfac8a87950df294a566fb96dfd04fd749
Reviewed-on: https://go-review.googlesource.com/50711Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

From Josh's comments on https://golang.org/cl/50310

Once I removed the "from the Go standard library" bit, the beginning
wasn't worth keeping. It also wasn't clear whether what it meant by
"cache contention". Processor caches, or user-level caches built with
sync.Map? It didn't seem worth clarifying and didn't convey any useful
information, so deleted.

Change-Id: Id1d76105a3081d0855f6a64540700932bb83d98e
Reviewed-on: https://go-review.googlesource.com/50632Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>

We can make it panic with a more explicit and readable error message
during Go 1.10, but document it for now. This has always been the
case; it's not a new rule.

Updates #20933

Change-Id: I53c1fefb47a8f4aae0bb32fa742afa3a2ed20e8a
Reviewed-on: https://go-review.googlesource.com/50634Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>

As per bcmills’s lightning talk at GopherCon 2017:
https://github.com/gophercon/2017-talks/tree/master/lightningtalks/BryanCMills-AnOverviewOfSyncMap

Change-Id: I12dd0daa608af175d110298780f32c6dc5e1e0a0
Reviewed-on: https://go-review.googlesource.com/50310Reviewed-by: Bryan Mills <bcmills@google.com>

Change-Id: I8f6bcfab27251ef6962306d56e40d306ef85fe60
Reviewed-on: https://go-review.googlesource.com/50472Reviewed-by: Rob Pike <r@golang.org>

Fixes #19514.

Change-Id: I93600d5c3d11ecab5a47dd4cd55ed3aea05e221e
Reviewed-on: https://go-review.googlesource.com/49611
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

In CL https://golang.org/cl/4893043 (6 years ago), a new package named
"url" was created (it is currently known as "net/url"). During that
change, some identifier name collisions were introduced, and two
parameters in net/http were renamed to "urlStr".

Since that time, Go has continued to put high emphasis on the quality
and readability of the documentation. Sometimes, that means making small
sacrifices in the implementation details of a package to ensure that
the godoc reads better, since that's what the majority of users interact
with. See https://golang.org/s/style#named-result-parameters:

> Clarity of docs is always more important than saving a line or two
> in your function.

I think the "urlStr" parameter name is suboptimal for godoc purposes,
and just "url" would be better.

During the review of https://golang.org/cl/4893043, it was also noted
by @rsc that having to rename parameters named "url" was suboptimal:

> It's unfortunate that naming the package url means
> you can't have a parameter or variable named url.

However, at the time, the name of the url package was still being
decided, and uri was an alternative name under consideration.
The reason urlStr was chosen is because it was a lesser evil
compared to naming the url package uri instead:

> Let's not get hung up on URI vs. URL, but I'd like s/uri/urlStr/ even for just
> that the "i" in "uri" looks very similar to the "l" in "url" in many fonts.

> Please let's go with urlStr instead of uri.

Now that we have the Go 1 compatibility guarantee, the name of the
net/url package is fixed. However, it's possible to improve the
signature of Redirect, NewRequest functions in net/http package
for godoc purposes by creating a package global alias to url.Parse,
and renaming urlStr parameter to url in the exported funcs. This CL
does so.

Updates #21077.

Change-Id: Ibcc10e3825863a663e6ad91b6eb47b1862a299a6
Reviewed-on: https://go-review.googlesource.com/49930
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

GCC says -no-pie, clang says -nopie.

Fixes #21042

Change-Id: Iadc83ea7a48ea0debc5064c1ee8da4ebff752044
Reviewed-on: https://go-review.googlesource.com/49710
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The mainline sync.Map has allowed mutations within Range callbacks
since https://golang.org/cl/37342. The reference implementations need
to do the same.

This change integrates https://go-review.googlesource.com/c/42956/
from x/sync.

Change-Id: I6b58cf874bb31cd4f6fdb8bfa8278888ed617a5a
Reviewed-on: https://go-review.googlesource.com/42957
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

http://golang.org/cl/50251 fixed a regression under TSAN.
This change adds a minimal reproducer for the observed symptom.

Change-Id: Ib9ad01b458b7fdec14d6c2fe3c243f9c64b3dcf2
Reviewed-on: https://go-review.googlesource.com/50371
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes TestGoBuildUmask when the user has a POSIX ACL on the Go source tree.

Fixes #17909.

Change-Id: I5bc19099af8353afd41071258f4f317612b4c8c1
Reviewed-on: https://go-review.googlesource.com/50370Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The runtime tests may be invoked from a parent that has SIGQUIT
blocked. For example, Java invokes subprocesses this way. In this
situation, TestCrashDumpsAllThreads and TestPanicSystemstack will fail
because they depend on SIGQUIT to get tracebacks, and any subprocess
test that times out will fail to kill the subprocess.

Fix this by detecting if SIGQUIT is blocked and, if so, skipping tests
that depend on it and using SIGKILL to kill timed-out subprocesses.

Based on a fix by Carl Henrik Lunde in
https://golang.org/issue/19196#issuecomment-316145733

Fixes #19196.

Change-Id: Ia20bf15b96086487d0ef6b75239dcc260c21714c
Reviewed-on: https://go-review.googlesource.com/50330
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

If we are using vfork, and if something (such as TSAN) is intercepting
the sigaction function, then we must call the system call, not the
libc function. Otherwise the intercepted sigaction call in the child
may trash the data structures in the parent.

Change-Id: Id9588bfeaa934f32c920bf829c5839be5cacf243
Reviewed-on: https://go-review.googlesource.com/50251Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>

Fixes #21075

Change-Id: Idfe5002dfe17943844d9427e27f82ce894b92e80
Reviewed-on: https://go-review.googlesource.com/50270Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #21089

Change-Id: Idd65c7185b3e19f33958eb165cb5b09c06db3d56
Reviewed-on: https://go-review.googlesource.com/50110Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The Go ecosystem provides many tools to make Go
development more productive and seamless. Document
the availability of the editor plugins and IDEs,
add an overview of feature support and screencasts.

Updates #20398.
Updates #20402.
Updates #20399.
Updates #20401.
Updates #20569.

Change-Id: I0c6cb48eb4e3848807aaad78390493e14f097916
Reviewed-on: https://go-review.googlesource.com/45772Reviewed-by: Steve Francia <spf@golang.org>

Currently we trace mark assists even if they're satisfied entirely by
stealing. This means even if background marking is keeping up with
allocation, we'll still emit a trace event every N bytes of
allocation. The event will be a few microseconds, if that, but they're
frequent enough that, when zoomed out in the trace view, it looks like
all of the time is spent in mark assists even if almost none is.

Change this so we only emit a trace event if the assist actually has
to do assisting. This makes the traces of these events far more
useful.

Change-Id: If4aed1c413b814341ef2fba61d2f10751d00451b
Reviewed-on: https://go-review.googlesource.com/50030
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>

Load/store-merging and move optimizations can result in unaligned
memory accesses. This is fine so long as the load/store instruction
used does not take a relative offset. In the SSA rules this means we
must not merge (MOVDaddr (SB)) ops into loads/stores unless we can
guarantee the alignment of the target.

Fixes #21048.

Change-Id: I70f13a62a148d5f0a56e704e8f76e36b4a4226d9
Reviewed-on: https://go-review.googlesource.com/49250
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

Fixes #21077

Change-Id: Ic61d7313907f58ff4027fd2eee1ddb8c1656304d
Reviewed-on: https://go-review.googlesource.com/49712Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com>

tSweepTerm and pauseStart are supposed to be when STW was triggered,
but right now they're captured a bit before STW. Move these down to
immediately before we trigger STW.

Fixes #19590.

Change-Id: Icd48a5c4d45c9b36187ff986e4f178b5064556c1
Reviewed-on: https://go-review.googlesource.com/49612
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Currently, Windows stacks are either 128kB or 2MB depending on whether
the binary uses cgo. This is because we assume that Go system stacks
and the small amount of C code invoked by the standard library can
operate within smaller stacks, but general Windows C code assumes
larger stacks.

However, it's easy to call into arbitrary C code using the syscall
package on Windows without ever importing cgo into a binary. Such
binaries need larger system stacks even though they don't use cgo.

Fix this on 64-bit by increasing the system stack size to 2MB always.
This only costs address space, which is free enough on 64-bit to not
worry about. We keep (for now) the existing heuristic on 32-bit, where
address space comes at more of a premium.

Updates #20975.

Change-Id: Iaaaa9a2fcbadc825cddc797aaaea8d34ef8debf2
Reviewed-on: https://go-review.googlesource.com/49331
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>

Change-Id: Ie6a88b70d7c45c59995ee2f57fb28f9a3cbb404d
Reviewed-on: https://go-review.googlesource.com/49470Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Add Adam Kisala (individual CLA)
Add Aditya Mukerjee (individual CLA)
Add Akhil Indurti (individual CLA)
Add André Carvalho (individual CLA)
Add Andy Walker (individual CLA)
Add Awn Umar (individual CLA)
Add Bastian Ike (individual CLA)
Add Brian Downs (individual CLA)
Add Cody Oss (individual CLA)
Add Costin Chirvasuta (corporate CLA for Google Inc.)
Add Dan Ballard (individual CLA)
Add Dong-hee Na (individual CLA)
Add Dylan Waits (individual CLA)
Add Evan Hicks (individual CLA)
Add Fannie Zhang (corporate CLA for ARM Ltd.)
Add Francisco Rojas (individual CLA)
Add Gabriel Nicolas Avellaneda (individual CLA)
Add Gabríel Arthúr Pétursson (individual CLA)
Add Greg Poirier (individual CLA)
Add Iccha Sethi (individual CLA)
Add Ivan Moscoso (individual CLA)
Add Jamie Kerr (individual CLA)
Add Joe Kyo (individual CLA)
Add Joey Geiger (individual CLA)
Add John R. Lenton (individual CLA)
Add Johnny Luo (individual CLA)
Add Josh Roppo (individual CLA)
Add Kate Manson (individual CLA)
Add Leo Rudberg (corporate CLA for Google Inc.)
Add Ma Peiqi (individual CLA)
Add Martynas Budriūnas (corporate CLA for Google Inc.)
Add Maryan Hratson (individual CLA)
Add Michael Edwards (individual CLA)
Add Michael Hendricks (individual CLA)
Add Pablo Santiago Blum de Aguiar (individual CLA)
Add Pat Moroney (individual CLA)
Add Shi Han Ng (individual CLA)
Add Steven Buss (corporate CLA for Google Inc.)
Add Suzy Mueller (corporate CLA for Google Inc.)
Add Taro Aoki (individual CLA)
Add Tim Heckman (individual CLA)
Add Tony Walker (individual CLA)
Add Yasha Bubnov (individual CLA)

Updates #12042

Change-Id: Iee063dd6c5a39de16907acfb5af87e81a05ab417
Reviewed-on: https://go-review.googlesource.com/49351Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #20796
Fixes #16283

Change-Id: Ib11992fbd2bc1fbb3b14ae5a6bf0da2e4c12f641
Reviewed-on: https://go-review.googlesource.com/49311Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Framepointer is the default now. Only print an X: list
if the settings are _not_ the default.

Before:

$ go tool compile -V
compile version devel +a5f30d9508 Sun Jul 16 14:43:48 2017 -0400 X:framepointer
$ go1.8 tool compile -V
compile version go1.8 X:framepointer
$

After:

$ go tool compile -V
compile version devel +a5f30d9508 Sun Jul 16 14:43:48 2017 -0400
$ go1.9 tool compile -V # imagined
compile version go1.9
$

Perpetuates #18317.

Change-Id: I981ba5c62be32e650a166fc9740703122595639b
Reviewed-on: https://go-review.googlesource.com/49252
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Missed in CL 49253 because I submitted from the Gerrit UI and
had not mailed the latest copy. Sigh.

Change-Id: I540f960278df43e2eaf1aac188eb124a1ff240dd
Reviewed-on: https://go-review.googlesource.com/49256
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

The init func was using testing.Short, but that's not available
until after flag parsing. Found by CL 49251.

Change-Id: Ia7b871043375260873fa2c7e81e1d43c1c83d33f
Reviewed-on: https://go-review.googlesource.com/49253
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #21043

Change-Id: I12db4f67ddee80500ff323f84e8fa34511650e63
Reviewed-on: https://go-review.googlesource.com/49230Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Per note at top of doc, we don't use fixed-width spaces
in fixed-width phrases like "go doc".

Also ASN.1 NULL is not code so it's not <code> at all.

Change-Id: I791e4e6030b8b8d42f4621d2f4bf32fef93cf343
Reviewed-on: https://go-review.googlesource.com/47693
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I55f50e45f8872f063c3b6c9e89261d14689e77ce
Reviewed-on: https://go-review.googlesource.com/47692
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Ia3e351169a4ebe6db5e5f37b668f23dc8c992c78
Reviewed-on: https://go-review.googlesource.com/48877
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I04849883dd2e1f6d083e9f57d2a8c1bd7d258953
Reviewed-on: https://go-review.googlesource.com/48878
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>