Variables declared with 'var' have no sym->def.

Fixes #7794.

LGTM=rsc
R=golang-codereviews, bradfitz, rsc
CC=golang-codereviews
https://golang.org/cl/88360043

Cgo writes C function declarations pretending every arg is a pointer.
If the C function is deferred, it does not inhibit stack copying on split.
The stack copying code believes the C declaration, possibly misinterpreting
integers as pointers.

Probably the right fix for Go 1.3 is to make deferred C functions inhibit
stack copying.

For Go 1.4 and beyond we probably need to make cgo generate Go code
for 6g here, not C code for 6c.

Update #7695

LGTM=khr
R=khr
CC=golang-codereviews
https://golang.org/cl/83820043

Fixes #6364.

LGTM=rsc
R=golang-codereviews, bradfitz, rsc, gobot
CC=golang-codereviews
https://golang.org/cl/13512052

Fixes #7767.

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/87420043

Fixes #7675

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/85040044

Before the switch to liblink, the linkers accepted the -c flag
to print the call graph. This change restores the functionality.

This came in handy when I was trying to audit the use of SSE
instructions inside the Plan 9 note handler.

LGTM=rsc
R=golang-codereviews, bradfitz, rsc
CC=golang-codereviews
https://golang.org/cl/73990043

https://golang.org/cl/60590044 edited
doc.go without editing the file it is generated from.
The edit was lost at the next mkdoc.sh.
Make the change in help.go and rerun mkdoc.sh.

Pointed out in the review of CL 68580043.

TBR=iant
CC=golang-codereviews
https://golang.org/cl/88760043

Fixes #6936

LGTM=rsc
R=golang-codereviews, bradfitz, rsc
CC=golang-codereviews
https://golang.org/cl/87770048

- output absolute addresses, not relative;
- accept negative section numbers.

Update #6936
Fixes #7738

LGTM=rsc
R=golang-codereviews, bradfitz, ruiu, rsc
CC=golang-codereviews
https://golang.org/cl/85240046

This information is required by cmd/nm
to calculate absolute symbol addresses.

Update #6936
Update #7738

LGTM=rsc
R=golang-codereviews, bradfitz, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/87500043

The new code is adapted from the Go 1.2 nosplit code,
but it does not have the bug reported in issue 7623:

g% go run nosplit.go
g% go1.2 run nosplit.go
BUG
rejected incorrectly:
        main 0 call f; f 120

        linker output:
        # _/tmp/go-test-nosplit021064539
        main.main: nosplit stack overflow
                120	guaranteed after split check in main.main
                112	on entry to main.f
                -8	after main.f uses 120

g%

Fixes #6931.
Fixes #7623.

LGTM=iant
R=golang-codereviews, iant, ality
CC=golang-codereviews, r
https://golang.org/cl/88190043

Fixes #7225

LGTM=r
R=golang-codereviews, r
CC=golang-codereviews, rsc
https://golang.org/cl/88710043

Fixes #7752.

LGTM=bradfitz, ruiu
R=golang-codereviews, bradfitz, ruiu
CC=golang-codereviews
https://golang.org/cl/88690043

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/88670043

The name linkwriteobj is misleading because it implies that
the function has something to do with the linker, which it
does not.  The name is historical: the function performs an
operation that was previously performed by the linker, but no
longer is.

LGTM=rsc
R=rsc, minux.ma
CC=golang-codereviews
https://golang.org/cl/88210045

Without the leaf bit, the linker cannot record
the correct frame size in the symbol table, and
then stack traces get mangled. (Only for ARM.)

Fixes #7338.
Fixes #7347.

LGTM=iant
R=iant
CC=golang-codereviews
https://golang.org/cl/88550043

A //line directive without a filename now denotes the empty
filename, not the current directory (the Go 1.2 behaviour) nor
the previous //line's filename (the behaviour since CL
86990044).

They should never appear (but they do, e.g. due to a bug in godoc).

Fixes #7765

LGTM=gri, rsc
R=rsc, gri
CC=golang-codereviews
https://golang.org/cl/88160050

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/88000047

A //line directive without a filename now denotes the same
filename as the previous line (as in C).
Previously it denoted the file's directory (!).

Fixes #7765

LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/86990044

Fixes #7733

LGTM=minux.ma
R=golang-codereviews, minux.ma
CC=golang-codereviews, nigeltao, r, rsc
https://golang.org/cl/88330044

Fixes #7264

LGTM=dvyukov
R=dvyukov
CC=golang-codereviews
https://golang.org/cl/87970045

In large functions with many variables, the register optimizer
may give up and choose not to track certain variables at all.
In this case, the "nextinnode" information linking together
all the words from a given variable will be incomplete, and
the result may be that only some of a multiword value is
preserved across a call. That confuses the garbage collector,
so don't do that. Instead, mark those variables as having
their address taken, so that they will be preserved at all
calls. It's overkill, but correct.

Tested by hand using the 6g -S output to see that it does fix
the buggy generated code leading to the issue 7726 failure.

There is no automated test because I managed to break the
compiler while writing a test (see issue 7727). I will check
in a test along with the fix to issue 7727.

Fixes #7726.

LGTM=khr
R=khr, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/85200043

Update #7056

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/88070045

This has typically crashed in the past, although usually with
an 'all goroutines are asleep - deadlock!' message that shows
no goroutines (because there aren't any).

Previous discussion at:
https://groups.google.com/d/msg/golang-nuts/uCT_7WxxopQ/BoSBlLFzUTkJ
https://groups.google.com/d/msg/golang-dev/KUojayEr20I/u4fp_Ej5PdUJ
http://golang.org/issue/7711

There is general agreement that runtime.Goexit terminates the
main goroutine, so that main cannot return, so the program does
not exit.

The interpretation that all other goroutines exiting causes an
exit(0) is relatively new and was not part of those discussions.
That is what this CL changes.

Thankfully, even though the exit(0) has been there for a while,
some other accounting bugs made it very difficult to trigger,
so it is reasonable to replace. In particular, see golang.org/issue/7711#c10
for an examination of the behavior across past releases.

Fixes #7711.

LGTM=iant, r
R=golang-codereviews, iant, dvyukov, r
CC=golang-codereviews
https://golang.org/cl/88210044

linklookup uses hash(name, v) as the hash table index but then
only compares name to find a symbol to return.
If hash(name, v1) == hash(name, v2) for v1 != v2, the lookup
for v2 will return the symbol with v1.

The input routines assume that each symbol is found only once,
and then each symbol is added to a linked list, with the list header
in the symbol. Adding a symbol to such a list multiple times
short-circuits the list the second time it is added, causing symbols
to be dropped.

The liblink rewrite introduced an elegant, if inefficient, handling
of duplicated symbols by creating a dummy symbol to read the
duplicate into. The dummy symbols are named .dup with
sequential version numbers. With many .dup symbols, eventually
there will be a conflict, causing a duplicate list add, causing elided
symbols, causing a crash when calling one of the elided symbols.

The bug is old (2011) but could not have manifested until the
liblink rewrite introduced this heavily duplicated symbol .dup.
(See History section below.)

1. Correct the lookup function.

2. Since we want all the .dup symbols to be different, there's no
point in inserting them into the table. Call linknewsym directly,
avoiding the lookup function entirely.

3. Since nothing can refer to the .dup symbols, do not bother
adding them to the list of functions (textp) at all.

4. In lieu of a unit test, introduce additional consistency checks to
detect adding a symbol to a list multiple times. This would have
caught the short-circuit more directly, and it will detect a variety
of double-use bugs, including the one arising from the bad lookup.

Fixes #7749.

History

On April 9, 2011, I submitted CL 4383047, making ld 25% faster.
Much of the focus was on the hash table lookup function, and
one of the changes was to remove the s->version == v comparison [1].

I don't know if this was a simple editing error or if I reasoned that
same name but different v would yield a different hash slot and
so the name test alone sufficed. It is tempting to claim the former,
but it was probably the latter.

Because the hash is an iterated multiply+add, the version ends up
adding v*3ⁿ to the hash, where n is the length of the name.
A collision would need x*3ⁿ ≡ y*3ⁿ (mod 2²⁴ mod 100003),
or equivalently x*3ⁿ ≡ x*3ⁿ + (y-x)*3ⁿ (mod 2²⁴ mod 100003),
so collisions will actually be periodic: versions x and y collide
when d = y-x satisfies d*3ⁿ ≡ 0 (mod 2²⁴ mod 100003).
Since we allocate version numbers sequentially, this is actually
about the best case one could imagine: the collision rate is
much lower than if the hash were more random.
http://play.golang.org/p/TScD41c_hA computes the collision
period for various name lengths.

The most common symbol in the new linker is .dup, and for n=4
the period is maximized: the 100004th symbol is the first collision.
Unfortunately, there are programs with more duplicated symbols
than that.

In Go 1.2 and before, duplicate symbols were handled without
creating a dummy symbol, so this particular case for generating
many duplicate symbols could not happen. Go does not use
versioned symbols. Only C does; each input file gives a different
version to its static declarations. There just aren't enough C files
for this to come up in that context.

So the bug is old but the realization of the bug is new.

[1] https://golang.org/cl/4383047/diff/5001/src/cmd/ld/lib.c

LGTM=minux.ma, iant, dave
R=golang-codereviews, minux.ma, bradfitz, iant, dave
CC=golang-codereviews, r
https://golang.org/cl/87910047

When preparing a call with an interface method, the argument
frame holds the receiver "iword", but funcLayout was being
asked to write a descriptor as if the receiver were a complete
interface value. This was originally caught by running a large
program with Debug=3 in runtime/mgc0.c, but the new panic
in funcLayout suffices to catch the mistake with the existing
tests.

Fixes #7748.

LGTM=bradfitz, iant
R=golang-codereviews, bradfitz, iant
CC=golang-codereviews, khr
https://golang.org/cl/88100048

Having the pointers means you can grub around in the
binary finding out more about them.

This helped with issue 7748.

LGTM=minux.ma, bradfitz
R=golang-codereviews, minux.ma, bradfitz
CC=golang-codereviews
https://golang.org/cl/88090045

Didn't manage to find a way to write test cases.

Fixes #7769.

LGTM=iant
R=iant
CC=golang-codereviews
https://golang.org/cl/88000045

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/88360044

Don't advertise -s anymore.
Fixes #7793.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/88030045

Fixes #7773.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/87400043

LGTM=bradfitz
R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/88000044

Make it clear that types that wrap another reader or writer delegate to the wrapped type.

Fixes #7667

LGTM=adg
R=golang-codereviews, adg
CC=golang-codereviews
https://golang.org/cl/85720044

Generated by addca.

R=gobot
CC=golang-codereviews
https://golang.org/cl/87830046

Fixes #7775

LGTM=rsc
R=agl, rsc
CC=golang-codereviews
https://golang.org/cl/88340043

so that the zlib.Reader returns nil error.

Fixes #7762.

LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/86750044

This code never got updated after the liblink shuffle.
Tested by hand that it works and respects GOROOT_FINAL.

The discussion in issue 6963 suggests that perhaps we should
just drop runtime-gdb.py entirely, but I am not convinced
that is true. It was in Go 1.2 and I don't see a reason not to
keep it in Go 1.3. The fact that binaries have not been emitting
the reference was just a missed detail in the liblink conversion,
not part of a grand plan.

Fixes #7506.
Fixes #6963.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews, iant, r
https://golang.org/cl/87870048

If we compile a generated file stored in a temporary
directory - let's say /tmp/12345/work/x.c - then by default
6c stores the full path and then the pcln table in the
final binary includes the full path. This makes repeated builds
(using different temporary directories) produce different
binaries, even if the inputs are the same.

In the old 'go tool pack', the P flag specified a prefix to remove
from all stored paths (if present), and cmd/go invoked
'go tool pack grcP $WORK' to remove references to the
temporary work directory.

We've changed the build to avoid pack as much as possible,
under the theory that instead of making pack convert from
.6 to .a, the tools should just write the .a directly and save a
round of I/O.

Instead of going back to invoking pack always, define a common
flag -trimpath in the assemblers, C compilers, and Go compilers,
implemented in liblink, and arrange for cmd/go to use the flag.
Then the object files being written out have the shortened paths
from the start.

While we are here, reimplement pcln support for GOROOT_FINAL.
A build in /tmp/go uses GOROOT=/tmp/go, but if GOROOT_FINAL=/usr/local/go
is set, then a source file named /tmp/go/x.go is recorded instead as
/usr/local/go/x.go. We use this so that we can prepare distributions
to be installed in /usr/local/go without actually working in that
directory. The conversion to liblink deleted all the old file name
handling code, including the GOROOT_FINAL translation.
Bring the GOROOT_FINAL translation back.

Before this CL, using GOROOT_FINAL=/goroot make.bash:

        g% strings $(which go) | grep -c $TMPDIR
        6
        g% strings $(which go) | grep -c $GOROOT
        793
        g%

After this CL:

        g% strings $(which go) | grep -c $TMPDIR
        0
        g% strings $(which go) | grep -c $GOROOT
        0
        g%

(The references to $TMPDIR tend to be cgo-generated source files.)

Adding the -trimpath flag to the assemblers required converting
them to the new Go-semantics flag parser. The text in go1.3.html
is copied and adjusted from go1.1.html, which is when we applied
that conversion to the compilers and linkers.

Fixes #6989.

LGTM=iant
R=r, iant
CC=golang-codereviews
https://golang.org/cl/88300045

Trying to understand the linux-386-387 failures:
http://build.golang.org/log/78a91da173c11e986b4e623527c2d0b746f4e814

Also modernize the closeOnce code with a method value, while I
was looking.

LGTM=adg
R=golang-codereviews, adg
CC=golang-codereviews, iant
https://golang.org/cl/87950044

Per golang-nuts question. Writing to p breaks
other writers (e.g. io.MultiWriter).

Make this explicit.

LGTM=gri, r, rsc
R=r, rsc, gri, joshlf13
CC=golang-codereviews
https://golang.org/cl/87780046