When building a program that links against Go shared libraries,
it needs to reference symbols defined in the shared library. At
compile time, we don't know where the shared library boundary is.
If we reference a symbol in package p by index, and package p is
actually part of a shared library, we cannot resolve the index at
link time, as the linker doesn't see the object file of p.

So when linking against Go shared libraries, always use named
reference for now.

To do this, the compiler needs to know whether we will be linking
against Go shared libraries. The -dynlink flag kind of indicates
that (as the document says), but currently it is actually
overloaded: it is also used when building a plugin or a shared
library, which is self-contained (if -linkshared is not otherwise
specified) and could use index for symbol reference. So we
introduce another compiler flag, -linkshared, specifically for
linking against Go shared libraries. The go command will pass
this flag if its -linkshared flag is specified
("go build -linkshared").

There may be better way to handle this. For example, we can
put the symbol indices in a special section in the shared library
that the linker can read. Or we can generate some per-package
description file to include the indices. (Currently we generate
a .shlibname file for each package that is included in a shared
library, which contains the path of the library. We could
consider extending this.) That said, this CL is a stop-gap
solution. And it is no worse than the old object files.

If we were to redesign the build system so that the shared
library boundary is known at compile time, we could use indices
for symbol references that do not cross shared library boundary,
as well as doing other things better.

Change-Id: I9c02aad36518051cc4785dbe25c4b4cef8f3faeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/201818
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>

If two symbols have the same name, the old code allows either one
being dupok (preferably both, but either is ok). Currently, the
new code only works when the new symbol being dupok (or both).
Allow only old symbol being dupok as well.

One example for this is the tls_g variable on ARM64 and PPC64
when the race detector is enabled.

Should fix Linux/ARM64 build.

Change-Id: I8dd21c017e826847f13471c30dfd71bf225d8076
Reviewed-on: https://go-review.googlesource.com/c/go/+/201642
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>

Clean merge.

Change-Id: I94d5e621b98cd5b3e1f2007db83d52293edbd9ec

Fixes #34749

Change-Id: Id97afc189ea387fc0fdd044140e30096594e185a
Reviewed-on: https://go-review.googlesource.com/c/go/+/202018Reviewed-by: Bryan C. Mills <bcmills@google.com>

This patch uses symbol NOOP to support arm64 instruction NOP. In
arm64, NOP stands for that No Operation does nothing, other than
advance the value of the program counter by 4. This instruction
can be used for instruction alignment purposes. This patch uses
NOOP to support arm64 instruction NOP, because we have a generic
"NOP" instruction, which is a zero-width pseudo-instruction.

In arm64, instruction NOP is an alias of HINT #0. This patch adds
test cases for instruction HINT #0.

Change-Id: I54e6854c46516eb652b412ef9e0f73ab7f171f8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/200578Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The Div functions in math/bits (Div, Div32, and Div64) compute both
quotients and remainders, but they panic if the quotients do not not
fit a 32/64 uint.

Since, on the other hand, the remainder will always fit the size of
the divisor, it is useful to have Div variants that only compute the
remainder, and don't panic on a quotient overflow.

This change adds to the math/bits package three new functions:

  Rem(hi, lo, y uint) uint
  Rem32(hi, lo, y uint32) uint32
  Rem64(hi, lo, y uint64) uint64

which can be used to compute (hi,lo)%y even when the quotient
overflows the uint size.

Fixes #28970

Change-Id: I119948429f737670c5e5ceb8756121e6a738dbdc
Reviewed-on: https://go-review.googlesource.com/c/go/+/197838
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Fixes #34869

Change-Id: I21bc60b9a5d1204dade1cceed6cddccf5b537b0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/200958
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Add test to make sure we get the right traceback when mid-stack inlining.

Update #33309

Change-Id: I23979cbe6b12fad105dbd26698243648aa86a354
Reviewed-on: https://go-review.googlesource.com/c/go/+/195984
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

Pulls in a new snapshot of the race detector, containing
a fix that lets it handle mid-stack inlining correctly.

Fixes #33309

Change-Id: I7551912a491f0615e77d069f198c1b8a6eead280
Reviewed-on: https://go-review.googlesource.com/c/go/+/201898
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Updating the underlying type of an imported type (even though
is was set to the same type again) leads to a race condition
if the imported package is imported by separate, concurrently
type-checked packages.

Fixes #31749.

Change-Id: Iabb8e8593eb067eb4816c1df81e545ff52d32c6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/201838
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #34968.

Change-Id: I538d653fab6cf7cf9b9b7022a1c2d4ae6ee497b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/201823
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The race detector C code expects the g register (aka R28) to be
preserved per the C calling convention. Make sure we save/restore it.

Once this is in we can revert the O3 -> O1 change to racebuild.

Change-Id: Ia785b2717c136f565d45bed283e87b744e35c62d
Reviewed-on: https://go-review.googlesource.com/c/go/+/201744Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Ia571b8aa791578a77ed5c2b8eaf45c9684eea1c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/201820Reviewed-by: Andrew Bonventre <andybons@golang.org>

We need to explicitly convert pointers to unsafe.Pointer before
passing to the runtime checkptr instrumentation in case the user
declared their own type with underlying type unsafe.Pointer.

Updates #22218.
Fixes #34966.

Change-Id: I3baa2809d77f8257167cd78f57156f819130baa8
Reviewed-on: https://go-review.googlesource.com/c/go/+/201782
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Change-Id: I832ba5f32d513b586bb0b02371231786b25631e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/201817Reviewed-by: Andrew Bonventre <andybons@golang.org>

Change-Id: I73f27924046a0a2493330ddc732d1a2fd3f730a5
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/575981Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/575983
Reviewed-on: https://go-review.googlesource.com/c/go/+/201785Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>

With the previous CL, the export data will not change whether it
is compiled with -dynlink flag or not. Restore the export data
hash, and reenable plugin version check.

TODO: it may be still better to just generate a fingerprint for
each package at compile time.

Change-Id: I1f298ac97c3ab9b8d05d1c95e8be74d10ca7cd0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/201720
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>

When we re-export an imported symbol that has an index, we should
pass the index through. Currently, if the symbol is not
referenced in the generated machine code, it does not get
assigned a package index, and the exporter will not export its
symbol index. Let the exporter handle this case -- if the symbol
has a symbol index but not a package index, still export its
symbol index. This is safe as referenced-by-name symbols always
have their package indices set to a special value.

This should reduce the number of referenced-by-name symbols, and
also make the export data more stable, less dependent on codegen
details.

Change-Id: Ic515a002ae84226e7fdbe68a53496c051b7badcc
Reviewed-on: https://go-review.googlesource.com/c/go/+/201719
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>

This CL tweaks escape analysis to treat unsafe.Pointer(ptr) as an
escaping operation when -d=checkptr is enabled. This allows better
detection of unsafe pointer arithmetic and conversions, because the
runtime checkptr instrumentation can currently only detect object
boundaries for heap objects, not stack objects.

Updates #22218.
Fixes #34959.

Change-Id: I856812cc23582fe4d0d401592583323e95919f28
Reviewed-on: https://go-review.googlesource.com/c/go/+/201781
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Add a new loader.Relocs method that reads all of the relocations for a
symbol into a slice. Handy in cases where the client knows in advance
that it wants to visit all the relocations on a symbol (as opposed to
just one or two).

Change-Id: I1a420513e160c8bb4b90c9824ae8d5b5de060c15
Reviewed-on: https://go-review.googlesource.com/c/go/+/201721
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

This CL helps race.bash finish in a reasonable amount of
time. Otherwise the Match/Hard1/32M benchmark takes over 1200 seconds
to finish on arm64, triggering a timeout.  With this change the regexp
benchmarks as a whole take only about a minute.

Change-Id: Ie2260ef9f5709e32a74bd76f135bc384b2d9853f
Reviewed-on: https://go-review.googlesource.com/c/go/+/201742
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

This CL extends the runtime instrumentation for (*T)(ptr) to also
check that the first and last bytes of *(*T)(ptr) are part of the same
heap object.

Updates #22218.
Updates #34959.

Change-Id: I2c8063fe1b7fe6e6145e41c5654cb64dd1c9dd41
Reviewed-on: https://go-review.googlesource.com/c/go/+/201778
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Updates #15581
Updates #34368

Change-Id: Ife3be7ed484cbe87960bf972ac701954d86127d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/201740Reviewed-by: Bryan C. Mills <bcmills@google.com>

Caught with -d=checkptr.

Updates #22218.

Change-Id: Ic0fcff4d2c8d83e4e7f5e0c6d01f03c9c7766c6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/201617
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #34799

Change-Id: I134b2717fa90c8955902e7eeaaf8510dcc28340e
Reviewed-on: https://go-review.googlesource.com/c/go/+/200760
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The issues associated with these skipped checks are closed.
If they are working around unfixed bugs, the issues should remain open.
If they are working around unfixable properties of the system, the skips
should refer to those properties rather than closed issues.

Updates #2603
Updates #3955
Updates #25628

Change-Id: I3491c69b2ef5bad0fb12001fe8f7e06b424883ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/201718
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Updates #34956

Change-Id: I35c39f3afda7226eeae0fd6936f7ee0d5d6c025b
Reviewed-on: https://go-review.googlesource.com/c/go/+/201737
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Skipping tests isn't great, but neither is a wall of red masking other
potential regressions.

Updates #34368

Change-Id: I5fdfa54846dd8d648001594c74f059af8af52247
Reviewed-on: https://go-review.googlesource.com/c/go/+/201482
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

Instructions added: CLDEMOTE, CLWB, TPAUSE, UMWAIT, UMONITOR.

Change-Id: I1ba550d4d5acc41a2fd97068ff5834e0412d3bcf
Reviewed-on: https://go-review.googlesource.com/c/go/+/183225
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

Also log errors from the lsof command on failure.
(That's how the missing environment was discovered.)

Updates #25628

Change-Id: I71594f60c15d0d254d5d4a86deec7431314c92ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/201717
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

These functions are not necessary and are not called anywhere.

Change-Id: I1c0d814ba3044c27e3626ac9e6052d8154140404
Reviewed-on: https://go-review.googlesource.com/c/go/+/201697
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This change removes the NewReader function (no longer used by objdump)
and prunes away the now unused code paths from Reader.BytesAt and
Reader.StringAt, which helps with performance. At the moment the
reader operates by always ingesting the entire object file (either via
direct read or by mmap), meaning that there will always be a slice
available for us to index into.

Change-Id: I3af7396effe19e50ed594fe8d82fd2d15465687c
Reviewed-on: https://go-review.googlesource.com/c/go/+/201437
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>

Convert the object file dumper to use NewReaderFromBytes when
reading new object files, as opposed to NewReader.

Change-Id: I9f5e0356bd21c16f545cdd70262e983a2ed38bfc
Reviewed-on: https://go-review.googlesource.com/c/go/+/201441
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

Change the new loader to keep a note of the set of "go.itablink.*"
symbols (using a small map), and add a method that clients can use to
query whether a given global index corresponds to a "go.itablink.*"
sym. This eliminates one instance of raw symbol name reading/matching
during new deadcode, which should produce a minor speedup.

Change-Id: I5915773a3f33c16099ccd68592dbba783d909bc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/201400
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

Add a flag bit to mark symbols in the new object file as containing Go
type information. The use of a flag eliminates the need to do symbol
name matching as part of the new dead code elimination pass, which
should produce a minor speedup.

Change-Id: Iec8700e1139e2c4e310644c0766379865d2d6f82
Reviewed-on: https://go-review.googlesource.com/c/go/+/201399
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

For the new object file format, don't tag individual symbols with a
"shared" flag, since that characteristic is better off as an attribute
of the containing object file as opposed to the individual symbol. Add
a new flags field in the object file header and put a bit in the flags
if the shared flags is in effect during compilation.

Change-Id: I2cf6d33bf7bf2fd8a7614ae0cd6ef03914777498
Reviewed-on: https://go-review.googlesource.com/c/go/+/201398
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

It seems that windowsZones.xml file has moved to Github. I opened

http://unicode.org/cldr/data/common/supplemental/windowsZones.xml

in my browser, and it redirected me to

https://github.com/unicode-org/cldr/blob/master/common/supplemental/windowsZones.xml

Very nice of them.

And we could see windowsZones.xml change history now. We could even
probably file issues against this file, if we find problems.

Anyway, this CL adjusts genzabbrs.go to use new GitHub location.

I also run 'go generate' command with updated genzabbrs.go to update
zoneinfo_abbrs_windows.go.

Fixes #34917

Change-Id: I69b71a4e02edd999435738ecb225a6f9793a66d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/201378
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

This CL adds -d=checkptr as a compile-time option for adding
instrumentation to check that Go code is following unsafe.Pointer
safety rules dynamically. In particular, it currently checks two
things:

1. When converting unsafe.Pointer to *T, make sure the resulting
pointer is aligned appropriately for T.

2. When performing pointer arithmetic, if the result points to a Go
heap object, make sure we can find an unsafe.Pointer-typed operand
that pointed into the same object.

These checks are currently disabled for the runtime, and can also be
disabled through a new //go:nocheckptr annotation. The latter is
necessary for functions like strings.noescape, which intentionally
violate safety rules to workaround escape analysis limitations.

Fixes #22218.

Change-Id: If5a51273881d93048f74bcff10a3275c9c91da6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/162237
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

This was spotted in CL 200767. This change just ensures internal
packages match their equivalents in x/mod.

Also pulled in test added in CL 201517.

Change-Id: I51d23d62697c256548f411930fcb6bccce51bf34
Reviewed-on: https://go-review.googlesource.com/c/go/+/201497
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>

1. Change mapencode.encode to use fmt.Error rather than MarshalerError.
MarshalerError refer to MarshalJSON, but mapencode.encode does not use that.

2. Add sourceFunc field to MarshalerError to record the name of the function
that creates the error, so that the Error method can report it correctly.

Fixes #29753

Change-Id: I186c2fac8470ae2f9e300501de3730face642230
Reviewed-on: https://go-review.googlesource.com/c/go/+/184119
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>