When the Transport was creating an bound HTTP connection (protocol
unknown initially) and then ends up deciding it doesn't need it, a
goroutine sits around to clean up whatever the result was. That
goroutine made the false assumption that the result was always an
HTTP/1 connection or an error. It may also be an alternate protocol
in which case the *persistConn.conn net.Conn field is nil, and the
alt field is non-nil.

Fixes #13839

Change-Id: Ia4972e5eb1ad53fa00410b3466d4129c753e0871
Reviewed-on: https://go-review.googlesource.com/18573Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: I9b4b76abfba66ff655aef55b43d9b4721aba604a
Reviewed-on: https://go-review.googlesource.com/18587Reviewed-by: Chris Broadfoot <cbro@golang.org>

Change-Id: If4a90c4017ef4b5c9f497cf117c8ad62b7e15c62
Reviewed-on: https://go-review.googlesource.com/18501
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The fucomi* opcodes were only introduced for the Pentium Pro.
They do not exist for an MMX Pentium.  Use the fucom* instructions
instead and move the condition codes from the fp flags register to
the integer flags register explicitly.

The use of fucomi* opcodes in ggen.go was introduced in 1.5 (CL 8738).
The bad ops were generated for 64-bit floating-point comparisons.

The use of fucomi* opcodes in gsubr.go dates back to at least 1.1.
The bad ops were generated for float{32,64} to uint64 conversions.

Fixes #13923

Change-Id: I5290599f5edea8abf8fb18036f44fa78bd1fc9e6
Reviewed-on: https://go-review.googlesource.com/18590Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Install pkg.h rather than libpkg.h.

Link against -lc.

Fixes #13860.

Change-Id: I4e429426f8363712a5dbbd2655b9aab802ab2888
Reviewed-on: https://go-review.googlesource.com/18592
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: Russ Cox <rsc@golang.org>

Add several instructions that were used via BYTE and use them.
Instructions added: PEXTRB, PEXTRD, PEXTRQ, PINSRB, XGETBV, POPCNT.

Change-Id: I5a80cd390dc01f3555dbbe856a475f74b5e6df65
Reviewed-on: https://go-review.googlesource.com/18593
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

This change applies the fix for #13564 to Plan 9 and Windows.
Also enables Lookup API test cases on builders.

Updates #13564.

Change-Id: I863f03c7cb6fbe58b3a55223bfa0ac5f9bf9c3df
Reviewed-on: https://go-review.googlesource.com/18559
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Update bundled http2 to git rev 76365a4 for https://golang.org/issue/18571

Fixes golang/go#13924

Change-Id: Ibb48cd6935b35d9965df70fb8761be5986d79ffc
Reviewed-on: https://go-review.googlesource.com/18591Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Conn.Close sends an encrypted "close notify" to signal secure EOF.
But writing that involves acquiring mutexes (handshake mutex + the
c.out mutex) and writing to the network. But if the reason we're
calling Conn.Close is because the network is already being
problematic, then Close might block, waiting for one of those mutexes.

Instead of blocking, and instead of introducing new API (at least for
now), distinguish between a normal Close (one that sends a secure EOF)
and a resource-releasing destructor-style Close based on whether there
are existing Write calls in-flight.

Because io.Writer and io.Closer aren't defined with respect to
concurrent usage, a Close with active Writes is already undefined, and
should only be used during teardown after failures (e.g. deadlines or
cancelations by HTTP users). A normal user will do a Write then
serially do a Close, and things are unchanged for that case.

This should fix the leaked goroutines and hung net/http.Transport
requests when there are network errors while making TLS requests.

Change-Id: If3f8c69d6fdcebf8c70227f41ad042ccc3f20ac9
Reviewed-on: https://go-review.googlesource.com/18572Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Brief background on "why heap allocate".  Things can be
forced to the heap for the following reasons:

1) address published, hence lifetime unknown.
2) size unknown/too large, cannot be stack allocated
3) multiplicity unknown/too large, cannot be stack allocated
4) reachable from heap (not necessarily published)

The bug here is a case of failing to enforce 4) when an
object Y was reachable from a heap allocation X forced
because of 3).  It was found in the case of a closure
allocated within a loop (X) and assigned to a variable
outside the loop (multiplicity unknown) where the closure
also captured a map (Y) declared outside the loop (reachable
from heap). Note the variable declared outside the loop (Y)
is not published, has known size, and known multiplicity
(one). The only reason for heap allocation is that it was
reached from a heap allocated item (X), but because that was
not forced by publication, it has to be tracked by loop
level, but escape-loop level was not tracked and thus a bug
results.

The fix is that when a heap allocation is newly discovered,
use its looplevel as the minimum loop level for downstream
escape flooding.

Every attempt to generalize this bug to X-in-loop-
references-Y-outside loop succeeded, so the fix was aimed
to be general.  Anywhere that loop level forces heap
allocation, the loop level is tracked.  This is not yet
tested for all possible X and Y, but it is correctness-
conservative and because it caused only one trivial
regression in the escape tests, it is probably also
performance-conservative.

The new test checks the following:
1) in the map case, that if fn escapes, so does the map.
2) in the map case, if fn does not escape, neither does the map.
3) in the &x case, that if fn escapes, so does &x.
4) in the &x case, if fn does not escape, neither does &x.

Fixes #13799.

Change-Id: Ie280bef2bb86ec869c7c206789d0b68f080c3fdb
Reviewed-on: https://go-review.googlesource.com/18234
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

A bit cleanuppy for 1.6 maybe, but something I happened to notice.

Change-Id: I70f3b48445f4f527d67f7b202b6171195440b09f
Reviewed-on: https://go-review.googlesource.com/18550Reviewed-by: Russ Cox <rsc@golang.org>

[Repeat of CL 18343 with build fixes.]

Before, NumGoroutine counted system goroutines and Stack (usually) didn't show them,
which was inconsistent and confusing.

To resolve which way they should be consistent, it seems like

	package main
	import "runtime"
	func main() { println(runtime.NumGoroutine()) }

should print 1 regardless of internal runtime details. Make it so.

Fixes #11706.

Change-Id: If26749fec06aa0ff84311f7941b88d140552e81d
Reviewed-on: https://go-review.googlesource.com/18432Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>

Fixes #13907.

Change-Id: Ieaa5183f399b12a9177372212adf481c8f0b4a0d
Reviewed-on: https://go-review.googlesource.com/18491Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Vlad Krasnov <vlad@cloudflare.com>
Reviewed-by: Adam Langley <agl@golang.org>

Update #9401.
Update #11925.
Update #13919.

Change-Id: I52c679353693e8165b2972d4d3974ee8bb1207ef
Reviewed-on: https://go-review.googlesource.com/18542
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Many browsers now support schemeless URLs in the Location headers
and also it is allowed in the draft HTTP/1.1 specification (see
http://stackoverflow.com/q/4831741#comment25926312_4831741), but
Go standard library lacks support for them.

This patch implements schemeless URLs support in http.Redirect().
Since url.Parse() correctly handles schemeless URLs, I've just added
an extra condition to verify URL's Host part in the absoulute/relative
check in the http.Redirect function.

Also I've moved oldpath variable initialization inside the block
of code where it is used.

Change-Id: Ib8a6347816a83e16576f00c4aa13224a89d610b5
Reviewed-on: https://go-review.googlesource.com/14172Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

It would certainly be a mistake to invoke a write barrier while
greying an object.

Change-Id: I34445a15ab09655ea8a3628a507df56aea61e618
Reviewed-on: https://go-review.googlesource.com/18533
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

It used to be the case that repeatedly getting one GC pointer and
enqueuing one GC pointer could cause contention on the work buffers as
each operation passed over the boundary of a work buffer. As of
b6c0934a, we use a two buffer cache that prevents this sort of
contention.

Change-Id: I4f1111623f76df9c5493dd9124dec1e0bfaf53b7
Reviewed-on: https://go-review.googlesource.com/18532
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

This comment is probably a hold-over from when the heap bitmap was
interleaved and the shift was 0, 2, 4, or 6. Now the shift is 0, 1, 2,
or 3.

Change-Id: I096ec729e1ca31b708455c98b573dd961d16aaee
Reviewed-on: https://go-review.googlesource.com/18531
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>

Fixes #9228

Change-Id: Ic4df4a39f6f363bdd6eb9228c8164e6e9dccee1b
Reviewed-on: https://go-review.googlesource.com/5561Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Austin Clements <austin@google.com>

Go fails to build on a system which has PIE enabled by default like this:

/usr/bin/ld: -r and -pie may not be used together
collect2: error: ld returned 1 exit status

The only system I know that has this property right now is Ubuntu Xenial
running on s390x, which is hardly the most accessible system, but it's planned
to enable this on amd64 soon too. The fix is to pass -no-pie along with -Wl,-r
to the compiler, but unfortunately that flag is very new as well. So this does
a test compile of a trivial file to see if the flag is supported.

Change-Id: I1345571142b7c3a96212e43297d19e84ec4a3d41
Reviewed-on: https://go-review.googlesource.com/18359Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

There are reports of corruption. Let's disable it for now (for Go 1.6,
especially) until we can investigate and fix properly.

Update #13892

Change-Id: I557275e5142fe616e8a4f89c00ffafb830eb3b78
Reviewed-on: https://go-review.googlesource.com/18540Reviewed-by: Dave Cheney <dave@cheney.net>

Change-Id: I5a4a842cab2173357e8d3e349011c0c2b63be4f9
Reviewed-on: https://go-review.googlesource.com/18512Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I5a0206e8074f3a2790954c45a217922b7b3fe851
Reviewed-on: https://go-review.googlesource.com/18487Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Per suggestion from adonovan.

Change-Id: Icbb4d2f201590bc94672b8d8141b6e7901e11dc5
Reviewed-on: https://go-review.googlesource.com/18510Reviewed-by: Robert Griesemer <gri@golang.org>

In the presence of vendored packages, the path found in a package
declaration may not be the path at which the package imported from
srcDir was found. Use the correct package path.

Change-Id: I74496c3cdf82a5dbd6a5bd189bb3cd0ca103fd52
Reviewed-on: https://go-review.googlesource.com/18460Reviewed-by: Alan Donovan <adonovan@google.com>

Fixes #13881.

Change-Id: Idff77db381640184ddd2b65022133bb226168800
Reviewed-on: https://go-review.googlesource.com/18449Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>

Currently, due to an oversight, we only balance work buffers
in background and idle workers and not in assists. As a
result, in assist-heavy workloads, assists are likely to tie
up large work buffers in per-P caches increasing the
likelihood that the global list will be empty. This increases
the likelihood that other GC workers will exit and assists
will block, slowing down the system as a whole. Fix this by
eagerly balancing work buffers as soon as the assists notice
that the global buffers are empty. This makes it much more
likely that work will be immediately available to other
workers and assists.

This change reduces the garbage benchmark time by 39% and
fixes the regresssion seen at CL 15893 golang.org/cl/15893.

Garbage benchmark times before and after this CL.
Before GOPERF-METRIC:time=4427020
After  GOPERF-METRIC:time=2721645

Fixes #13827

Change-Id: I9cb531fb873bab4b69ce9c1617e30df6c49cdcfe
Reviewed-on: https://go-review.googlesource.com/18341Reviewed-by: Austin Clements <austin@google.com>

This change documents the behaviour change caused by
https://go-review.googlesource.com/18480 in Go 1.6.

Updates #13886

Change-Id: I2daa08a62775bbc209f0f4cbeae21b8184ce7609
Reviewed-on: https://go-review.googlesource.com/18481Reviewed-by: Russ Cox <rsc@golang.org>

The AESNI GCM code decrypts and authenticates concurrently and so
overwrites the destination buffer even in the case of an authentication
failure.

This change updates the documentation to make that clear and also
mimics that behaviour in the generic code so that different platforms
act identically.

Fixes #13886

Change-Id: Idc54e51f01e27b0fc60c1745d50bb4c099d37e94
Reviewed-on: https://go-review.googlesource.com/18480Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

mips64 builder and one machine of the mips64le builder has small amount
of memory. Since CL 18199, they have been running slowly, as more
processes were launched in running 'test' directory, and a lot of swap
were used. This CL brings all.bash from 5h back to 3h on Loongson 2E
with 512 MB memory.

Change-Id: I4a22e239a542a99ba5986753205d8cd1f4b3d3c6
Reviewed-on: https://go-review.googlesource.com/18483Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Updates http2 to x/net git rev 0e6d34ef942 for https://golang.org/cl/18472
which means we'll get to delete a ton of grpc-go code and just use the
standard library's HTTP client instead.

Also, the comments in this CL aren't entirely accurate it turns out.
RFC 2616 says:

"The Trailer header field can be used to indicate which header fields
are included in a trailer (see section 14.40)."

And 14.40:

"  An HTTP/1.1 message SHOULD include a Trailer header field in a
   message using chunked transfer-coding with a non-empty trailer. Doing
   so allows the recipient to know which header fields to expect in the
   trailer.

   If no Trailer header field is present, the trailer SHOULD NOT include
   any header fields. See section 3.6.1 for restrictions on the use of
   trailer fields in a "chunked" transfer-coding."

So it's really a SHOULD more than a MUST.

And gRPC (at least Google's server) doesn't predeclare "grpc-status"
ahead of time in a Trailer Header, so we'll be lenient. We were too
strict anyway. It's also not a concern for the Go client we have a
different place to populate the Trailers, and it won't confuse clients
which aren't looking for them. The ResponseWriter server side is more
complicated (and strict), though, since we don't want to widen the
ResponseWriter interface. So the Go server still requires that you
predeclare Trailers.

Change-Id: Ia2defc11a2469fb8570ecfabb8453537121084eb
Reviewed-on: https://go-review.googlesource.com/18473
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #13893

Change-Id: I2577b38fdb19299227dc146f707cf9df663dcdfc
Reviewed-on: https://go-review.googlesource.com/18471
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I08d77d52b68b062c2eb1901fcfca34d45a210cce
Reviewed-on: https://go-review.googlesource.com/18142Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Change-Id: Ia6d40ead1e54dd0b8998370cbabc2d7cd8b7aa0b
Reviewed-on: https://go-review.googlesource.com/18470Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I95c6c85eecb53ffe52b64d521180148b624e9424
Reviewed-on: https://go-review.googlesource.com/18423Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The previous behaviour of installing the signal handlers in a separate
thread meant that Go initialization raced with non-Go initialization if
the non-Go initialization also wanted to install signal handlers.  Make
installing signal handlers synchronous so that the process-wide behavior
is predictable.

Update #9896.

Change-Id: Ice24299877ec46f8518b072a381932d273096a32
Reviewed-on: https://go-review.googlesource.com/18150
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

TBR adonovan

Fixes #13882.

Change-Id: I8664669f5d6adfec6f16e154263b1f0ea8988175
Reviewed-on: https://go-review.googlesource.com/18445Reviewed-by: Robert Griesemer <gri@golang.org>

Go 1.6 simplified the GC phases. The "synchronize Ps" phase no longer
exists and "root scan" and "mark" phases have been combined.

Update the gctrace line implementation and documentation to remove the
unused phases.

Fixes #13536.

Change-Id: I4fc37a3ce1ae3a99d48c0be2df64cbda3e05dee6
Reviewed-on: https://go-review.googlesource.com/18458
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #13688.

Change-Id: I53363aeeeba4560211d56d4571a8e058d5dbbd8a
Reviewed-on: https://go-review.googlesource.com/18308Reviewed-by: Alan Donovan <adonovan@google.com>

Sigh. Sleeps on FreeBSD also yield the rest of the time slice and
profiling signals are only delivered when a process completes a time
slice (worse, itimer time is only accounted to the process that
completes a time slice). It's less noticeable than the other BSDs
because the default tick rate is 1000Hz, but it's still failing
regularly.

Fixes #13846.

Change-Id: I41bf116bffe46682433b677183f86944d0944ed4
Reviewed-on: https://go-review.googlesource.com/18455Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Austin Clements <austin@google.com>