oneNewExtraM creates a spare M and G for use with cgo callbacks. The G
doesn't run right away, but goes directly into syscall status. For the
garbage collector, it's marked as "scan valid" and not on the rescan
list, but I forgot to also mark it as "scan done". As a result,
gcMarkRootCheck thinks that the goroutine hasn't been scanned and
panics.

This only affects GODEBUG=gccheckmark=1 mode, since we otherwise skip
the gcMarkRootCheck.

Fixes #17473.

Change-Id: I94f5671c42eb44bd5ea7dc68fbf85f0c19e2e52c
Reviewed-on: https://go-review.googlesource.com/31139
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

Change-Id: I79ebccdaefc434c47b77bd545cc3c50723c18b61
Reviewed-on: https://go-review.googlesource.com/31135
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

Updating the heap profile stats is one of the most expensive parts of
mark termination other than stack rescanning, but there's really no
need to do this with the world stopped. Move it to right after we've
started the world back up. This creates a *very* small window where
allocations from the next cycle can slip into the profile, but the
exact point where mark termination happens is so non-deterministic
already that a slight reordering here is unimportant.

Change-Id: I2f76f22c70329923ad6a594a2c26869f0736d34e
Reviewed-on: https://go-review.googlesource.com/31363
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

The only reason these flushes are still necessary at all is that
gcmarknewobject doesn't flush its gcWork stats like it's supposed to.
By changing gcmarknewobject to follow the standard protocol, the
flushes become completely unnecessary because mark 2 ensures caches
are flushed (and stay flushed) before we ever enter mark termination.

In the garbage benchmark, this takes roughly 50 µs, which is
surprisingly long for doing nothing. We still double-check after
draining that they are in fact empty.

Change-Id: Ia1c7cf98a53f72baa513792eb33eca6a0b4a7128
Reviewed-on: https://go-review.googlesource.com/31134
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

The pointer checking code needs to know the exact type of the parameter
expected by the C function, so that it can use a type assertion to
convert the empty interface returned by cgoCheckPointer to the correct
type. Previously this was done by using a type conversion, but that
meant that the code accepted arguments that were convertible to the
parameter type, rather than arguments that were assignable as in a
normal function call. In other words, some code that should not have
passed type checking was accepted.

This CL changes cgo to always use a function literal for pointer
checking. Now the argument is passed to the function literal, which has
the correct argument type, so type checking is performed just as for a
function call as it should be.

Since we now always use a function literal, simplify the checking code
to run as a statement by itself. It now no longer needs to return a
value, and we no longer need a type assertion.

This does have the cost of introducing another function call into any
call to a C function that requires pointer checking, but the cost of the
additional call should be minimal compared to the cost of pointer
checking.

Fixes #16591.

Change-Id: I220165564cf69db9fd5f746532d7f977a5b2c989
Reviewed-on: https://go-review.googlesource.com/31233
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Just happened to notice that these names (funcAlign and friends) are
never referenced outside their package, so no need to export them.

Change-Id: I4bbdaa4b0ef330c3c3ef50a2ca39593977a83545
Reviewed-on: https://go-review.googlesource.com/31496
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

mkbuiltin.go now generates builtin.go using go/ast instead of running
the compiler, so we don't need the -A flag anymore.

Passes toolstash -cmp.

Change-Id: Ifa70f4f3c9feae10c723cbec81a0a47c39610090
Reviewed-on: https://go-review.googlesource.com/31497Reviewed-by: Robert Griesemer <gri@golang.org>

Generating binary export data requires a working Go compiler. Even
trickier to change the export data format itself requires a careful
bootstrapping procedure.

Instead, simply generate normal Go code that lets us directly
construct the builtin runtime declarations.

Passes toolstash -cmp.

Fixes #17508.

Change-Id: I4f6078a3c7507ba40072580695d57c87a5604baf
Reviewed-on: https://go-review.googlesource.com/31493
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Fixes #16965

Change-Id: I3638fe280a5b1063ff589e6e1ff8a97c74b77c66
Reviewed-on: https://go-review.googlesource.com/30359Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Relevant PAX specification:
<<<
If the <value> field is zero length, it shall delete any header
block field, previously entered extended header value, or
global extended header value of the same name.
>>>

We don't delete global extender headers since the Reader doesn't
even support global headers (which the specification admits was
a controversial feature).

Change-Id: I2125a5c907b23a3dc439507ca90fa5dc47d474a9
Reviewed-on: https://go-review.googlesource.com/31440
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Took this opportunity to also embed tables in the functions
that they are actually used in and other stylistic cleanups.

There was no logical changes to the tests.

Change-Id: Ifa724060532175f6f4407d6cedc841891efd8f7b
Reviewed-on: https://go-review.googlesource.com/31436
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Add support for passing reflect.Values to and returning reflect.Values from
any registered functions in the FuncMap, much as if they were
interface{} values. Keeping the reflect.Value instead of round-tripping
to interface{} preserves addressability of the value, which is important
for method lookup.

Change index and a few other built-in functions to use reflect.Values,
making a loop using explicit indexing now match the semantics that
range has always had.

Fixes #14916.

Change-Id: Iae1a2fd9bb426886a7fcd9204f30a2d6ad4646ad
Reviewed-on: https://go-review.googlesource.com/31462Reviewed-by: Rob Pike <r@golang.org>

The GNU format does not have a prefix field, so we should make
no attempt to read it. It does however have atime and ctime fields.
Since Go previously placed incorrect values here, we liberally
read the atime and ctime fields and ignore errors so that old tar
files written by Go can at least be partially read.

This fixes half of #12594. The Writer is much harder to fix.

Updates #12594

Change-Id: Ia32845e2f262ee53366cf41dfa935f4d770c7a30
Reviewed-on: https://go-review.googlesource.com/31444
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The panic leaves the lock in an unusable state.
Trying to panic with a usable state makes the lock significantly
less efficient and scalable (see early CL patch sets and discussion).

Instead, use runtime.throw, which will crash the program directly.

In general throw is reserved for when the runtime detects truly
serious, unrecoverable problems. This problem is certainly serious,
and, without a significant performance hit, is unrecoverable.

Fixes #13879.

Change-Id: I41920d9e2317270c6f909957d195bd8b68177f8d
Reviewed-on: https://go-review.googlesource.com/31359Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Before: ... appears in an ambiguous URL context.
After:  ... appears in an ambiguous context within a URL.

It's a minor point, but it's confused multiple people.
Try to make clearer that the ambiguity is "where exactly inside the URL?"

Fixes #17319.

Change-Id: Id834868d1275578036c1b00c2bdfcd733d9d2b7b
Reviewed-on: https://go-review.googlesource.com/31465
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I4270bf81511a5bf80ed146f5e66e4f8aeede2aa2
Reviewed-on: https://go-review.googlesource.com/31463Reviewed-by: Ian Lance Taylor <iant@golang.org>

For #17428.

Change-Id: Ia902b50cf0c40e3c2167fb573a39d328331c38c7
Reviewed-on: https://go-review.googlesource.com/31449Reviewed-by: Ian Lance Taylor <iant@golang.org>

Updates #14892

Change-Id: I640c6e1635ccdf611f219521a7d297a9885c4cb3
Reviewed-on: https://go-review.googlesource.com/31446
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

This CL makes NewRequest set Body nil for known-zero bodies, and makes
the http1 Transport not peek-Read a byte to determine whether there's
a body.

Background:

Many fields of the Request struct have different meanings for whether
they're outgoing (via the Transport) or incoming (via the Server).

For outgoing requests, ContentLength and Body are documented as:

	// Body is the request's body.
	//
	// For client requests a nil body means the request has no
	// body, such as a GET request. The HTTP Client's Transport
	// is responsible for calling the Close method.
	Body io.ReadCloser

	// ContentLength records the length of the associated content.
	// The value -1 indicates that the length is unknown.
	// Values >= 0 indicate that the given number of bytes may
	// be read from Body.
	// For client requests, a value of 0 with a non-nil Body is
	// also treated as unknown.
	ContentLength int64

Because of the ambiguity of what ContentLength==0 means, the http1 and
http2 Transports previously Read the first byte of a non-nil Body when
the ContentLength was 0 to determine whether there was an actual body
(with a non-zero length) and ContentLength just wasn't populated, or
it was actually empty.

That byte-sniff has been problematic and gross (see #17480, #17071)
and was removed for http2 in a previous commit.

That means, however, that users doing:

    req, _ := http.NewRequest("POST", url, strings.NewReader(""))

... would not send a Content-Length header in their http2 request,
because the size of the reader (even though it was known, being one of
the three common recognized types from NewRequest) was zero, and so
the HTTP Transport thought it was simply unset.

To signal explicitly-zero vs unset-zero, this CL changes NewRequest to
signal explicitly-zero by setting the Body to nil, instead of the
strings.NewReader("") or other zero-byte reader.

This CL also removes the byte sniff from the http1 Transport, like
https://golang.org/cl/31326 did for http2.

Updates #17480
Updates #17071

Change-Id: I329f02f124659bf7d8bc01e2c9951ebdd236b52a
Reviewed-on: https://go-review.googlesource.com/31445
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #16531.

Change-Id: I41ec8123f2d3fbe063fd3b09a9366e69722793e5
Reviewed-on: https://go-review.googlesource.com/31355Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #17092.

Change-Id: If203d802a919e00594ddc1282782fc59a083fd63
Reviewed-on: https://go-review.googlesource.com/31458Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This was supposed to be in CL 31354
but was dropped due to a Git usage error.

For #16573.

Change-Id: I3d99087c8efc8cbc016c55e8365d0005f79d1b2f
Reviewed-on: https://go-review.googlesource.com/31461Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Make two important points clearer:

 - Giving a template definition containing
   nothing but spaces has no effect.
 - Giving a template definition containing
   non-spaces can only be done once per template.

Fixes #16912.
Fixes #16913.
Fixes #17360.

Change-Id: Ie3971b83ab148b7c8bb800fe4a21579566378e3e
Reviewed-on: https://go-review.googlesource.com/31459
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>

Noted in CL 31358 after submit.

Change-Id: I76ddad9b9e27dd6a03c1c4f49153213747fe0a61
Reviewed-on: https://go-review.googlesource.com/31365
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Many users quote literals in regular expressions just in case.
No need to allocate then.

Note: Also added benchmarks for quoting and not quoting.

	name             old time/op    new time/op     delta
	QuoteMetaAll-4      629ns ± 6%      654ns ± 5%    +4.01%        (p=0.001 n=20+19)
	QuoteMetaNone-4    1.02µs ± 6%     0.20µs ± 0%   -80.73%        (p=0.000 n=18+20)

	name             old speed      new speed       delta
	QuoteMetaAll-4   22.3MB/s ± 6%   21.4MB/s ± 5%    -3.94%        (p=0.001 n=20+19)
	QuoteMetaNone-4  25.3MB/s ± 3%  131.5MB/s ± 0%  +419.28%        (p=0.000 n=17+19)

	name             old alloc/op   new alloc/op    delta
	QuoteMetaAll-4      64.0B ± 0%      64.0B ± 0%      ~     (all samples are equal)
	QuoteMetaNone-4     96.0B ± 0%      0.0B ±NaN%  -100.00%        (p=0.000 n=20+20)

	name             old allocs/op  new allocs/op   delta
	QuoteMetaAll-4       2.00 ± 0%       2.00 ± 0%      ~     (all samples are equal)
	QuoteMetaNone-4      2.00 ± 0%      0.00 ±NaN%  -100.00%        (p=0.000 n=20+20)

Change-Id: I38d50f463cde463115d22534f8eb849e54d899af
Reviewed-on: https://go-review.googlesource.com/31395Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Implements Float.Scan which satisfies fmt.Scanner interface.
Also enforces docs' interface implementation claims with compile time
type assertions, that is:
+ Float always implements fmt.Formatter and fmt.Scanner
+ Int always implements fmt.Formatter and fmt.Scanner
+ Rat always implements fmt.Formatter
which will ensure that the API claims are strictly matched.

Also note that Float.Scan doesn't handle ±Inf.

Fixes #17391

Change-Id: I3d3dfbe7f602066975c7a7794fe25b4c645440ce
Reviewed-on: https://go-review.googlesource.com/30723Reviewed-by: Robert Griesemer <gri@golang.org>

For example, testing the current directory:

	$ go test -run XXX
	testing: warning: no tests to run
	PASS
	ok  	testing	0.013s
	$

And in a summary:

	$ go test -run XXX testing
	ok  	testing	0.013s [no tests to run]
	$

These make it easy to spot when the -run regexp hasn't matched anything
or there are no tests. Previously the message was printed in the "current directory"
case when there were no tests at all, but not for no matches, and either way
was not surfaced in the directory list summary form.

Fixes #15211.

Change-Id: I1c82a423d6bd429fb991c9ca964c9d26c96fd3c5
Reviewed-on: https://go-review.googlesource.com/22341Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>

Fixes #17078.

Change-Id: I1dfb71f64361b575ec461ed44b0779f2d5cf45fc
Reviewed-on: https://go-review.googlesource.com/31352Reviewed-by: Quentin Smith <quentin@golang.org>

Fixes #13453.
Fixes #16104.

Change-Id: I4e94f606df786af8143f8649c9afde570f346301
Reviewed-on: https://go-review.googlesource.com/31353Reviewed-by: Quentin Smith <quentin@golang.org>

Fixes #16573.

Change-Id: I5a26eaa8b258cb1861190f9690086725532b8a0d
Reviewed-on: https://go-review.googlesource.com/31354Reviewed-by: Ian Lance Taylor <iant@golang.org>

Current implementation of syscall.Readlink mistakenly calculates
the end offset of the PrintName field.
Also, there are some cases that the PrintName field is empty.
Instead, the CL uses SubstituteName with correct calculation.

Fixes #15978
Fixes #16145

Change-Id: If3257137141129ac1c552d003726d5b9c08bb754
Reviewed-on: https://go-review.googlesource.com/31118Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Requested in CL 31324 review.

Change-Id: Ic81410e07cce07c6f3727bc46d86b6c54c15eca0
Reviewed-on: https://go-review.googlesource.com/31410Reviewed-by: Rob Pike <r@golang.org>

Unix rejects this when new is a non-empty directory.
Other systems reject this when new is a directory, empty or not.
Make Unix reject empty directory too.

Fixes #14527.

Change-Id: Ice24b8065264c91c22cba24aa73e142386c29c87
Reviewed-on: https://go-review.googlesource.com/31358
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

A 16bit binary exponent permits a constant range covering roughly the range
from 7e-9865 to 7e9863 which is more than enough for any practical and
hypothetical constant arithmetic.

Furthermore, until recently cmd/compile could not handle very large exponents
correctly anyway; i.e., the chance that any real programs (but for tests that
explore corner cases) are affected are close to zero.

Finally, restricting the minimum supported range significantly reduces the
implementation complexity in an area that hardly matters in reality for new
or alternative spec-compliant implementations that don't or cannot rely on
pre-existing arbitratry precision arithmetic packages that support a 32bit
exponent range.

This is technically a language change but for the reasons mentioned above
this is unlikely to affect any real programs, and certainly not programs
compiled with the gc or gccgo compilers as they currently support up to
32bit exponents.

Fixes #13572.

Change-Id: I970f919c57fc82c0175844364cf48ea335f17d39
Reviewed-on: https://go-review.googlesource.com/17711Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Reuse the same mechanisms for handling universal builtins like len to
handle unsafe.Sizeof, etc. Allows us to drop package unsafe's export
data, and simplifies some code.

Updates #17508.

Change-Id: I620e0617c24e57e8a2d7cccd0e2de34608779656
Reviewed-on: https://go-review.googlesource.com/31433
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Add special case for Gamma(+∞) which speeds it up:

benchmark            old ns/op     new ns/op     delta
BenchmarkGamma-4     14.5          7.44          -48.69%

The documentation for math.Gamma already specifies it as a special
case:

        Gamma(+Inf) = +Inf

The original C code that has been used as the reference implementation
(as mentioned in the comments in gamma.go) also treats Gamma(+∞) as a
special case:

if( x == INFINITY )
        return(x);

Change-Id: Idac36e19192b440475aec0796faa2d2c7f8abe0b
Reviewed-on: https://go-review.googlesource.com/31370Reviewed-by: Robert Griesemer <gri@golang.org>

The way to send an explicitly-zero Content-Length is to set a nil Body.

Fix this test to do that, rather than relying on type sniffing.

Updates #17480
Updates #17071

Change-Id: I6a38e20f17013c88ec4ea69d73c507e4ed886947
Reviewed-on: https://go-review.googlesource.com/31434
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Chris Broadfoot <cbro@golang.org>

"go env" previously only printed a subset of the documented environment
variables; now it includes everything, such as GO386 and CGO_*.

This also fixes the CGO_CFLAGS environment variable to always have the
same default. According to iant@ and confirmed by testing, cgo can now
understand the default value of CGO_CFLAGS.

Fixes #17191.

Change-Id: Icf75055446dd250b6256ef1139e9ce848f4a9d3b
Reviewed-on: https://go-review.googlesource.com/31330
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: Quentin Smith <quentin@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

instrumentnode() accidentally copies parent's already-instrumented nodes
into child's Ninit block. This generates repeated code in race-instrumentation.
This case surfaces only when it duplicates inline-labels, because of
compile time error. In other cases, it silently generates incorrect
instrumented code. This change prevents it from doing so.

Fixes #17449.

Change-Id: Icddf2198990442166307e176b7e20aa0cf6c171c
Reviewed-on: https://go-review.googlesource.com/31317Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Interface methods can't have function bodies, so there's no need to
process their parameter lists as variable declarations. The only
possible reason would be to check for duplicate parameter names and/or
invalid types, but we do that anyway, and have regression tests for it
(test/funcdup.go).

Change-Id: Iedb15335467caa5d872dbab829bf32ab8cf6204d
Reviewed-on: https://go-review.googlesource.com/31430
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>