LGTM=bradfitz
R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/88000044

Make it clear that types that wrap another reader or writer delegate to the wrapped type.

Fixes #7667

LGTM=adg
R=golang-codereviews, adg
CC=golang-codereviews
https://golang.org/cl/85720044

Generated by addca.

R=gobot
CC=golang-codereviews
https://golang.org/cl/87830046

Fixes #7775

LGTM=rsc
R=agl, rsc
CC=golang-codereviews
https://golang.org/cl/88340043

so that the zlib.Reader returns nil error.

Fixes #7762.

LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/86750044

This code never got updated after the liblink shuffle.
Tested by hand that it works and respects GOROOT_FINAL.

The discussion in issue 6963 suggests that perhaps we should
just drop runtime-gdb.py entirely, but I am not convinced
that is true. It was in Go 1.2 and I don't see a reason not to
keep it in Go 1.3. The fact that binaries have not been emitting
the reference was just a missed detail in the liblink conversion,
not part of a grand plan.

Fixes #7506.
Fixes #6963.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews, iant, r
https://golang.org/cl/87870048

If we compile a generated file stored in a temporary
directory - let's say /tmp/12345/work/x.c - then by default
6c stores the full path and then the pcln table in the
final binary includes the full path. This makes repeated builds
(using different temporary directories) produce different
binaries, even if the inputs are the same.

In the old 'go tool pack', the P flag specified a prefix to remove
from all stored paths (if present), and cmd/go invoked
'go tool pack grcP $WORK' to remove references to the
temporary work directory.

We've changed the build to avoid pack as much as possible,
under the theory that instead of making pack convert from
.6 to .a, the tools should just write the .a directly and save a
round of I/O.

Instead of going back to invoking pack always, define a common
flag -trimpath in the assemblers, C compilers, and Go compilers,
implemented in liblink, and arrange for cmd/go to use the flag.
Then the object files being written out have the shortened paths
from the start.

While we are here, reimplement pcln support for GOROOT_FINAL.
A build in /tmp/go uses GOROOT=/tmp/go, but if GOROOT_FINAL=/usr/local/go
is set, then a source file named /tmp/go/x.go is recorded instead as
/usr/local/go/x.go. We use this so that we can prepare distributions
to be installed in /usr/local/go without actually working in that
directory. The conversion to liblink deleted all the old file name
handling code, including the GOROOT_FINAL translation.
Bring the GOROOT_FINAL translation back.

Before this CL, using GOROOT_FINAL=/goroot make.bash:

        g% strings $(which go) | grep -c $TMPDIR
        6
        g% strings $(which go) | grep -c $GOROOT
        793
        g%

After this CL:

        g% strings $(which go) | grep -c $TMPDIR
        0
        g% strings $(which go) | grep -c $GOROOT
        0
        g%

(The references to $TMPDIR tend to be cgo-generated source files.)

Adding the -trimpath flag to the assemblers required converting
them to the new Go-semantics flag parser. The text in go1.3.html
is copied and adjusted from go1.1.html, which is when we applied
that conversion to the compilers and linkers.

Fixes #6989.

LGTM=iant
R=r, iant
CC=golang-codereviews
https://golang.org/cl/88300045

Trying to understand the linux-386-387 failures:
http://build.golang.org/log/78a91da173c11e986b4e623527c2d0b746f4e814

Also modernize the closeOnce code with a method value, while I
was looking.

LGTM=adg
R=golang-codereviews, adg
CC=golang-codereviews, iant
https://golang.org/cl/87950044

Per golang-nuts question. Writing to p breaks
other writers (e.g. io.MultiWriter).

Make this explicit.

LGTM=gri, r, rsc
R=r, rsc, gri, joshlf13
CC=golang-codereviews
https://golang.org/cl/87780046

LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/88050046

My cmd/go got in a weird state where it started invoking pack grcP.
Change pack to print a 1-line explanation of the usage problem
before the generic usage message.

LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/87770047

Fixes #7188

LGTM=adg
R=golang-codereviews, adg
CC=golang-codereviews
https://golang.org/cl/88280044

Also make it clear this is not a complete description of all features.
Fixes #7790.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/88300044

LGTM=rsc
R=bradfitz, rsc
CC=golang-codereviews
https://golang.org/cl/87800044

Fixes #7768.

LGTM=iant, gri
R=golang-codereviews, iant, gri
CC=golang-codereviews
https://golang.org/cl/87260043

OpenBSD is excluded from all the usual thread-local storage
code, not just emitting the tbss section in the external link .o
but emitting a PT_TLS section in an internally-linked executable.
I assume it just has no proper TLS support. Exclude it here too.

TBR=iant
CC=golang-codereviews
https://golang.org/cl/87900045

We get
/usr/lib/libc.a(stack_protector.o): In function `__stack_chk_fail_local':
stack_protector.c:(.text+0x158): multiple definition of `__stack_chk_fail_local'
/var/tmp/go-link-04838a/000001.o:/tmp/gobuilder/netbsd-386-minux-c7a9e9243878/go/src/pkg/runtime/cgo/gcc_386.S:41: first defined here

I am assuming this has never worked and possibly is not intended to work.
(Some systems are vehemently against static linking.)

TBR=iant
CC=golang-codereviews
https://golang.org/cl/88130046

Fixes #7719.

LGTM=iant
R=iant
CC=golang-codereviews
https://golang.org/cl/87760050

When I did the original 386 ports on Linux and OS X, I chose to
define GS-relative expressions like 4(GS) as relative to the actual
thread-local storage base, which was usually GS but might not be
(it might be FS, or it might be a different constant offset from GS or FS).

The original scope was limited but since then the rewrites have
gotten out of control. Sometimes GS is rewritten, sometimes FS.
Some ports do other rewrites to enable shared libraries and
other linking. At no point in the code is it clear whether you are
looking at the real GS/FS or some synthesized thing that will be
rewritten. The code manipulating all these is duplicated in many
places.

The first step to fixing issue 7719 is to make the code intelligible
again.

This CL adds an explicit TLS pseudo-register to the 386 and amd64.
As a register, TLS refers to the thread-local storage base, and it
can only be loaded into another register:

        MOVQ TLS, AX

An offset from the thread-local storage base is written off(reg)(TLS*1).
Semantically it is off(reg), but the (TLS*1) annotation marks this as
indexing from the loaded TLS base. This emits a relocation so that
if the linker needs to adjust the offset, it can. For example:

        MOVQ TLS, AX
        MOVQ 8(AX)(TLS*1), CX // load m into CX

On systems that support direct access to the TLS memory, this
pair of instructions can be reduced to a direct TLS memory reference:

        MOVQ 8(TLS), CX // load m into CX

The 2-instruction and 1-instruction forms correspond roughly to
ELF TLS initial exec mode and ELF TLS local exec mode, respectively.

Liblink applies this rewrite on systems that support the 1-instruction form.
The decision is made using only the operating system (and probably
the -shared flag, eventually), not the link mode. If some link modes
on a particular operating system require the 2-instruction form,
then all builds for that operating system will use the 2-instruction
form, so that the link mode decision can be delayed to link time.

Obviously it is late to be making changes like this, but I despair
of correcting issue 7719 and issue 7164 without it. To make sure
I am not changing existing behavior, I built a "hello world" program
for every GOOS/GOARCH combination we have and then worked
to make sure that the rewrite generates exactly the same binaries,
byte for byte. There are a handful of TODOs in the code marking
kludges to get the byte-for-byte property, but at least now I can
explain exactly how each binary is handled.

The targets I tested this way are:

        darwin-386
        darwin-amd64
        dragonfly-386
        dragonfly-amd64
        freebsd-386
        freebsd-amd64
        freebsd-arm
        linux-386
        linux-amd64
        linux-arm
        nacl-386
        nacl-amd64p32
        netbsd-386
        netbsd-amd64
        openbsd-386
        openbsd-amd64
        plan9-386
        plan9-amd64
        solaris-amd64
        windows-386
        windows-amd64

There were four exceptions to the byte-for-byte goal:

windows-386 and windows-amd64 have a time stamp
at bytes 137 and 138 of the header.

darwin-386 and plan9-386 have five or six modified
bytes in the middle of the Go symbol table, caused by
editing comments in runtime/sys_{darwin,plan9}_386.s.

Fixes #7164.

LGTM=iant
R=iant, aram, minux.ma, dave
CC=golang-codereviews
https://golang.org/cl/87920043

It was said already but apparently not enough times.

Fixes #6985.

LGTM=crawshaw
R=golang-codereviews, crawshaw
CC=golang-codereviews
https://golang.org/cl/86300043

Do not consider idle finalizer/bgsweep/timer goroutines as doing something useful.
We can't simply set isbackground for the whole lifetime of the goroutines,
because when finalizer goroutine calls user function, we do want to consider it
as doing something useful.
This is borken due to timers for quite some time.
With background sweep is become even more broken.
Fixes #7784.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/87960044

Fixes #6559

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/81330045

TLS handshake failures didn't use to log, but do in Go 1.3.
Shut it up so the actual failure can be seen in e.g.
http://build.golang.org/log/ede7e12362a941d93bf1fe21db9208a3e298029e

LGTM=adg
R=adg
CC=golang-codereviews
https://golang.org/cl/87870043

This breaks "go get -d repo/path/...".

««« original CL description
cmd/go: do not miss an error if import path contains "cmd/something"

Fixes #7638

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/87300043
»»»

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/87890043

Per TODO email in my inbox.

LGTM=rsc
R=golang-codereviews, rsc
CC=adg, dsymonds, golang-codereviews, r
https://golang.org/cl/87550045

LGTM=r
R=rsc, r
CC=golang-codereviews
https://golang.org/cl/87750043

Windows is building a chain to the AddTrust root which is different
from the native Go code and causing a build failure.

This change alters the test so that both should build to the AddTrust
root.

R=bradfitz

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/87570044

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/87460043

Fixes #7638

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/87300043

Generated by addca.

R=gobot
CC=golang-codereviews
https://golang.org/cl/86960046

The relocation and automatic variable types were using
arch-specific numbers. Introduce portable enumerations
instead.

To the best of my knowledge, these are the only arch-specific
bits left in the new object file format.

Remove now, before Go 1.3, because file formats are forever.

LGTM=iant
R=iant
CC=golang-codereviews
https://golang.org/cl/87670044

Comodo are now using a SHA-384 signed intermediate. The crypto/x509
package seeks to import hash functions needed for typical operation
without needing to import every hash function possible. Since a SHA-384
certificate is being used by Comodo, crypto/sha512 now appears to fall
into the scope of "typical operation".

R=bradfitz

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/87670045

Update #7264

Races:
http://build.golang.org/log/a2e401fdcd4903a61a3375bff5da702a20ddafad
http://build.golang.org/log/ec4c69e92076a747ac6d5df7eb7b382b31ab3d43

I think this is the first time I've actually seen a manifestation
of Issue 7264, and one that I can reproduce.

I don't know why it triggers on this test and not any others
just like it, or why I can't reproduce Issue 7264
independently, even when Dmitry gives me minimal repros.

Work around it for now with some synchronization to make the
race detector happy.

The proper fix will probably be in net/http/httptest itself, not
in all hundred some tests.

LGTM=rsc
R=rsc
CC=dvyukov, golang-codereviews
https://golang.org/cl/87640043

Generated by addca.

R=gobot
CC=golang-codereviews
https://golang.org/cl/87650044

There are changes we know we want to make, but not before Go 1.3
Add a version number so that we can make them more easily later.

LGTM=iant
R=iant
CC=golang-codereviews
https://golang.org/cl/87670043

LGTM=rsc
R=rsc, r
CC=golang-codereviews
https://golang.org/cl/87620043

Currently Pool can cache up to 15 elements per P, and these elements are not accesible to other Ps.
If a Pool caches large objects, say 2MB, and GOMAXPROCS is set to a large value, say 32,
then the Pool can waste up to 960MB.
The new caching policy caches at most 1 per-P element, the rest is shared between Ps.

Get/Put performance is unchanged. Nested Get/Put performance is 57% worse.
However, overall scalability of nested Get/Put is significantly improved,
so the new policy starts winning under contention.

benchmark                     old ns/op     new ns/op     delta
BenchmarkPool                 27.4          26.7          -2.55%
BenchmarkPool-4               6.63          6.59          -0.60%
BenchmarkPool-16              1.98          1.87          -5.56%
BenchmarkPool-64              1.93          1.86          -3.63%
BenchmarkPoolOverlflow        3970          6235          +57.05%
BenchmarkPoolOverlflow-4      10935         1668          -84.75%
BenchmarkPoolOverlflow-16     13419         520           -96.12%
BenchmarkPoolOverlflow-64     10295         380           -96.31%

LGTM=rsc
R=rsc
CC=golang-codereviews, khr
https://golang.org/cl/86020043

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/87140044

These are not ready and will not be in Go 1.3.

Fixes #6932.

LGTM=bradfitz
R=golang-codereviews, bradfitz, minux.ma
CC=golang-codereviews, iant, r
https://golang.org/cl/87630043

We have never released cmd/prof and don't plan to.
Now that nm, addr2line, and objdump have been rewritten in Go,
cmd/prof is the only thing keeping us from deleting libmach.

Delete cmd/prof, and then since nothing is using libmach, delete libmach.

13,000 lines of C deleted.

LGTM=minux.ma
R=golang-codereviews, minux.ma
CC=golang-codereviews, iant, r
https://golang.org/cl/87020044