Move global state from obj.Link
to a new function-local state struct arm.ctxt5.

This ends up being cleaner than threading
all the state through as parameters; there's a lot of it.
While we're here, move newprog from a parameter to ctxt5.

We reserve the variable name c for ctxt5,
so a few local variables named c have been renamed.

Instead of lazily initializing deferreturn
and Sym_div and friends, initialize them up front.

Passes toolstash-check -all.

Updates #15756

Change-Id: Ifb4e4b9879e4e1f25e6168d8b7b2a25a3390dc11
Reviewed-on: https://go-review.googlesource.com/39922
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

This allows the go tool to run "go vet" with both the build flags
that make sense, such as -x and -tags, and vet with all its flags.

To do this, create a new package cmd/go/internal/cmdflag to
hold functionality common to flag handling for test and vet.

Fixes #19350

RELNOTES=yes

Change-Id: Ia1ae213bd3f6cab1c5e492501c8d43ce61a7ee89
Reviewed-on: https://go-review.googlesource.com/40112Reviewed-by: Russ Cox <rsc@golang.org>

Report an error if a predefined escaper (i.e. "html", "urlquery", or "js")
is found in a pipeline that will be rewritten by the contextual auto-escaper,
instead of trying to merge the escaper-inserted escaping directives
with these predefined escapers. This merging behavior is a source
of several security and correctness bugs (eee #19336, #19345, #19352,
and #19353.)

This merging logic was originally intended to ease migration of text/template
templates with user-defined escapers to html/template. Now that
migration is no longer an issue, this logic can be safely removed.

NOTE: this is a backward-incompatible change that fixes known security
bugs (see linked issues for more details). It will explicitly break users
that attempt to execute templates with pipelines containing predefined
escapers.

Fixes #19336, #19345, #19352, #19353

Change-Id: I46b0ca8a2809d179c13c0d4f42b63126ed1c3b49
Reviewed-on: https://go-review.googlesource.com/37880
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #19851.

Change-Id: I5ee9533406542be7d5418df154f6134139e75892
Reviewed-on: https://go-review.googlesource.com/39890
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Walk relative symlinks in windows os.Stat from
symlink path instead of from current directory.

Fixes #19870

Change-Id: I0a27473d11485f073084b1f19b30c5b3a2fbc0f7
Reviewed-on: https://go-review.googlesource.com/39932Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

The runtime.writeBarrier variable tries to be helpful by telling you
that the compiler also knows about this variable, which you could
probably guess, but doesn't say how the compiler knows about it. In
fact, the compiler has a complete copy in builtin/runtime.go that
needs to be kept in sync. Say so.

Change-Id: Ia7fb0c591cb6f9b8230decce01008b417dfcec89
Reviewed-on: https://go-review.googlesource.com/40150
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #19658

Change-Id: I41e46073b75c7674e2ed9d6a90ece367ce92166b
Reviewed-on: https://go-review.googlesource.com/39650
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

https://go-review.googlesource.com/37812 says fix panic parsing.
Actually, it doesn't. so fix it.

Fixes #19416

Change-Id: Ie0c4241f10e5ebcbac20e184c2a7b13b22632eab
Reviewed-on: https://go-review.googlesource.com/37912Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

They are dead code already, but the verifier is still not happy.
Don't assemble them at all.

Looks like it has been like that for long. I don't know why it
was ok. Maybe the verifier is now more picky?

Fixes #19884.

Change-Id: Ib806fb73ca469789dec56f52d484cf8baf7a245c
Reviewed-on: https://go-review.googlesource.com/40111
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Dave Cheney <dave@cheney.net>

Fixes #19891

Change-Id: I2f2684dd5fbfcf93fca0f2c3caf678f44605687f
Reviewed-on: https://go-review.googlesource.com/40090Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Reduces the number of cases that need to be tested and
reduces size of the evconst function by 101 bytes.

Change-Id: Ie56055a89d0dadd311fb940b51c488fc003694b9
Reviewed-on: https://go-review.googlesource.com/39950
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>

Change-Id: I3fe92d74ff259abdf5d1fd28cdc822db88aae191
Reviewed-on: https://go-review.googlesource.com/39993Reviewed-by: Ian Lance Taylor <iant@golang.org>

pkgByPath was added in d78c84c4 to eliminate the differences between the
export formats around the time of Go 1.7.

The last remnants of the textual export format was removed by Josh in
39850 making the pkgByPath sorting type unused.

Change-Id: I168816d6401f45119475a4fe5ada00d9ce571a9e
Reviewed-on: https://go-review.googlesource.com/40050Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

This is a re-roll of CL 39710,
which broke deterministic builds.

typenamesym is called from three places:
typename, ngotype, and Type.Symbol.
Only in typename do we actually need a Node.
ngotype and Type.Symbol require only a Sym.
And writing the newly created Node to
Sym.Def is unsafe in a concurrent backend.
Rather than use a mutex protect to Sym.Def,
make typenamesym not touch Sym.Def.

The assignment to Sym.Def was serving a second purpose,
namely to prevent duplicate entries on signatlist.
Preserve that functionality by switching signatlist to a map.
This in turn requires that we sort signatlist
when exporting it, to preserve reproducibility.

We sort using exactly the same mechanism
that the export code (dtypesym) uses.
Failure to do that led to non-deterministic builds (#19872).
Since we've already calculated the Type's export name,
we could pass it to dtypesym, sparing it a bit of work.
That can be done as a future optimization.

Updates #15756

name       old alloc/op      new alloc/op      delta
Template        39.2MB ± 0%       39.3MB ± 0%    ~     (p=0.075 n=10+10)
Unicode         29.8MB ± 0%       29.8MB ± 0%    ~     (p=0.393 n=10+10)
GoTypes          113MB ± 0%        113MB ± 0%  +0.06%  (p=0.027 n=10+8)
SSA             1.25GB ± 0%       1.25GB ± 0%  +0.05%  (p=0.000 n=8+10)
Flate           25.3MB ± 0%       25.3MB ± 0%    ~     (p=0.105 n=10+10)
GoParser        31.7MB ± 0%       31.8MB ± 0%    ~     (p=0.165 n=10+10)
Reflect         78.2MB ± 0%       78.2MB ± 0%    ~     (p=0.190 n=10+10)
Tar             26.6MB ± 0%       26.6MB ± 0%    ~     (p=0.481 n=10+10)
XML             42.2MB ± 0%       42.2MB ± 0%    ~     (p=0.968 n=10+9)

name       old allocs/op     new allocs/op     delta
Template          384k ± 1%         386k ± 1%  +0.43%  (p=0.019 n=10+10)
Unicode           320k ± 0%         321k ± 0%  +0.36%  (p=0.015 n=10+10)
GoTypes          1.14M ± 0%        1.14M ± 0%  +0.33%  (p=0.000 n=10+8)
SSA              9.69M ± 0%        9.71M ± 0%  +0.18%  (p=0.000 n=10+9)
Flate             233k ± 1%         233k ± 1%    ~     (p=0.481 n=10+10)
GoParser          315k ± 1%         316k ± 1%    ~     (p=0.113 n=9+10)
Reflect           979k ± 0%         979k ± 0%    ~     (p=0.971 n=10+10)
Tar               250k ± 1%         250k ± 1%    ~     (p=0.481 n=10+10)
XML               391k ± 1%         392k ± 0%    ~     (p=1.000 n=10+9)

Change-Id: Ia9f21cc29c047021fa8a18c2a3d861a5146aefac
Reviewed-on: https://go-review.googlesource.com/39915
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

No longer needed after previous CLs.

Change-Id: I9dd1040f3f28363c44434a810cbd7061cf6a028f
Reviewed-on: https://go-review.googlesource.com/39857
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Robert Griesemer <gri@golang.org>

It's unused and redundant with types.Type.Compare.

Change-Id: I8d93473e9cee61650cb0c19a402316a3d418160e
Reviewed-on: https://go-review.googlesource.com/40011
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Given code such as

type T struct {
  _ string
}

func f() {
  var x = T{"space"}
  // ...
}

the compiler rewrote the 'var x' line as

var x T
x._ = "space"

The compiler then rejected the assignment to
a blank field, thus rejecting valid code.

It also failed to catch a number of invalid assignments.
And there were insufficient checks for validity
when emitting static data, leading to ICEs.

To fix, check earlier for explicit blanks field names,
explicitly handle legit blanks in sinit,
and don't try to emit static data for nodes
for which typechecking has failed.

Fixes #19482

Change-Id: I594476171d15e6e8ecc6a1749e3859157fe2c929
Reviewed-on: https://go-review.googlesource.com/38006
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

This dowidth currently happens during AST to SSA conversion.
As such, it is a concurrency pinch point.
It's a bit silly, but do it here in walk instead.
This appears (fingers crossed) to be the last
unresolved dowidth concurrency problem.

Updates #15756

Change-Id: I87cbf718a14ad21aca74586003d79320cca75953
Reviewed-on: https://go-review.googlesource.com/39994
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

queuemethod was unused. As queuemethod is unused, nothing appends to the
methodqueue global. As methodqueue is always nil or empty, there are no
live callers of domethod, so it can be removed.

Change-Id: Ic7427ac4621bbf403947815e3988c3a1113487f2
Reviewed-on: https://go-review.googlesource.com/39931
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

I had too many failed attempts trying to remove iterFields that I
decided to overhaul this function. Much simpler and easier to
understand now (at least IMO).

Passes toolstash-check -all.

Change-Id: I41d00642a969698df3f4689e41a386346b966638
Reviewed-on: https://go-review.googlesource.com/39856
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

Change-Id: I9ec137502353e65325087dfb60ee9bd68ffd286d
Reviewed-on: https://go-review.googlesource.com/38447Reviewed-by: Russ Cox <rsc@golang.org>

Current code doesn't support floating point #define macros.
This CL compiles floats to a object file and retrive values from it.
That approach is the same work as we've already done for integers.

Updates #18720

Change-Id: I88b7ab174d0f73bda975cf90c5aeb797961fe034
Reviewed-on: https://go-review.googlesource.com/35511Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

There are some LSyms that are lazily initialized,
and which cannot be made eagerly initialized,
such as elements of a constant pool.

To avoid needing a mutex to protect the internals of
those LSyms, this CL introduces LookupInit,
which allows an LSym to be initialized only once.

By itself this is not fully concurrency-safe,
but Ctxt.Hash will need mutex protection anyway,
and that will be enough to support one-time LSym initialization.

Passes toolstash-check -all.

Updates #15756

Change-Id: Id7248dfdc4dfbdfe425fa31d0c0045018eeea1fa
Reviewed-on: https://go-review.googlesource.com/39990
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Ib55f7ea3f7dcd9d02f6027121663870a65cb886c
Reviewed-on: https://go-review.googlesource.com/39924Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Change-Id: I000bb530e00d0f0bc59e0f1366b5fb586adf4f37
Reviewed-on: https://go-review.googlesource.com/39912
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

This reverts commit c8b889cc.

Reason for revert: broke noopt build, compiler performance regression, new Curfn uses

Let's fix those and then try this again.

Change-Id: Icc3cad1365d04cac8fd09da9dbb0bbf55c13ef44
Reviewed-on: https://go-review.googlesource.com/39991Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Change compiler and linker to emit DWARF lexical blocks in debug_info.
Version of debug_info is updated from DWARF v.2 to DWARF v.3 since version 2
does not allow lexical blocks with discontinuous ranges.

Second attempt at https://go-review.googlesource.com/#/c/29591/

Remaining open problems:
- scope information is removed from inlined functions
- variables in debug_info do not have DW_AT_start_scope attributes so a
variable will shadow other variables with the same name as soon as its
containing scope begins, before its declaration.

Updates golang/go#12899, golang/go#6913

Change-Id: I0e260a45b564d14a87b88974eb16c5387cb410a5
Reviewed-on: https://go-review.googlesource.com/36879
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Updates #19865

Change-Id: I24fbf5d79b5e4cac09c14cfff678a8215397b670
Reviewed-on: https://go-review.googlesource.com/39914
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Ifdc41f6c77c83d22b9ad8811280f1e1db051c781
Reviewed-on: https://go-review.googlesource.com/39951
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>

CL 38662 changed the x86 assembler to be eagerly
initialized, for a concurrent backend.

This CL puts in place a proper mechanism for doing so,
and switches all architectures to use it.

Passes toolstash-check -all.

Updates #15756

Change-Id: Id2aa527d3a8259c95797d63a2f0d1123e3ca2a1c
Reviewed-on: https://go-review.googlesource.com/39917
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I9282c1907204ec5c6363de84faec222a38300c9f
Reviewed-on: https://go-review.googlesource.com/39919Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

CL 38776 was not updated to use the new types package.

Fixes build.

Change-Id: Ie80ff4837cac95bd628e0405a937045171d56e0c
Reviewed-on: https://go-review.googlesource.com/39918
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes vetall build broken by CL 38693.

Change-Id: I5c98f2164e0c7bb43f022c363b3b7c0650c4bcbe
Reviewed-on: https://go-review.googlesource.com/39916
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

Currently we expand comparison with small constant strings into len check
and a sequence of byte comparisons. Generate 16/32/64-bit comparisons,
instead of bytewise on 386 and amd64. Also increase limits on what is
considered small constant string.
Shaves ~30kb (0.5%) from go executable.

This also updates test/prove.go to keep test case valid.

Change-Id: I99ae8871a1d00c96363c6d03d0b890782fa7e1d9
Reviewed-on: https://go-review.googlesource.com/38776
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>

Move optimized Count implementation from bytes to runtime. Use in
both bytes and strings packages.
Add CountByte benchmark to strings.

Strings benchmarks:
name                       old time/op    new time/op    delta
CountHard1-4                 226µs ± 1%      226µs ± 2%      ~     (p=0.247 n=10+10)
CountHard2-4                 316µs ± 1%      315µs ± 0%      ~     (p=0.133 n=9+10)
CountHard3-4                 919µs ± 1%      920µs ± 1%      ~     (p=0.968 n=10+9)
CountTorture-4              15.4µs ± 1%     15.7µs ± 1%    +2.47%  (p=0.000 n=10+9)
CountTortureOverlapping-4   9.60ms ± 0%     9.65ms ± 1%      ~     (p=0.247 n=10+10)
CountByte/10-4              26.3ns ± 1%     10.9ns ± 1%   -58.71%  (p=0.000 n=9+9)
CountByte/32-4              42.7ns ± 0%     14.2ns ± 0%   -66.64%  (p=0.000 n=10+10)
CountByte/4096-4            3.07µs ± 0%     0.31µs ± 2%   -89.99%  (p=0.000 n=9+10)
CountByte/4194304-4         3.48ms ± 1%     0.34ms ± 1%   -90.09%  (p=0.000 n=10+9)
CountByte/67108864-4        55.6ms ± 1%      7.0ms ± 0%   -87.49%  (p=0.000 n=9+8)

name                      old speed      new speed       delta
CountByte/10-4             380MB/s ± 1%    919MB/s ± 1%  +142.21%  (p=0.000 n=9+9)
CountByte/32-4             750MB/s ± 0%   2247MB/s ± 0%  +199.62%  (p=0.000 n=10+10)
CountByte/4096-4          1.33GB/s ± 0%  13.32GB/s ± 2%  +898.13%  (p=0.000 n=9+10)
CountByte/4194304-4       1.21GB/s ± 1%  12.17GB/s ± 1%  +908.87%  (p=0.000 n=10+9)
CountByte/67108864-4      1.21GB/s ± 1%   9.65GB/s ± 0%  +699.29%  (p=0.000 n=9+8)

Fixes #19411

Change-Id: I8d2d409f0fa6df6d03b60790aa86e540b4a4e3b0
Reviewed-on: https://go-review.googlesource.com/38693Reviewed-by: Keith Randall <khr@golang.org>

Windows was missed in https://golang.org/cl/141600043.

Fixes #6163 (again).

Change-Id: I09076be80fb6b2148d3e5618461ebaa79f27d5b3
Reviewed-on: https://go-review.googlesource.com/39490Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

They are in the types package, no need to mention the Type suffix.

Change-Id: Ie4fe1e3c1793514145e33f9df373d715f63e1aad
Reviewed-on: https://go-review.googlesource.com/39911
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Change-Id: I04cd4dd0ed55b88247a056b429fc496539cd0985
Reviewed-on: https://go-review.googlesource.com/39910
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>

- created new package cmd/compile/internal/types
- moved Pkg, Sym, Type to new package
- to break cycles, for now we need the (ugly) types/utils.go
  file which contains a handful of functions that must be installed
  early by the gc frontend
- to break cycles, for now we need two functions to convert between
  *gc.Node and *types.Node (the latter is a dummy type)
- adjusted the gc's code to use the new package and the conversion
  functions as needed
- made several Pkg, Sym, and Type methods functions as needed
- renamed constructors typ, typPtr, typArray, etc. to types.New,
  types.NewPtr, types.NewArray, etc.

Passes toolstash-check -all.

Change-Id: I8adfa5e85c731645d0a7fd2030375ed6ebf54b72
Reviewed-on: https://go-review.googlesource.com/39855Reviewed-by: Matthew Dempsky <mdempsky@google.com>

This reverts commit 91433eb5.

Reason for revert: broke deterministic build.

Fixes #19872.

Change-Id: Ia1a0fc651b818bdf69454df43bd189689c0348a0
Reviewed-on: https://go-review.googlesource.com/39871
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>