MOVQ RARG0, 0(SP) smashes exactly what was saved by PUSHQ R15.
This code managed to work somehow with the current race runtime,
but corrupts caller arguments with new race runtime that I am testing.

Change-Id: I9ffe8b5eee86451db36e99dbf4d11f320192e576
Reviewed-on: https://go-review.googlesource.com/4810Reviewed-by: Keith Randall <khr@golang.org>

New race runtime is more scrupulous about env flags format.

Change-Id: I2828bc737a8be3feae5288ccf034c52883f224d8
Reviewed-on: https://go-review.googlesource.com/4811Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

drainworkbuf is now gcDrain, since it drains until there's
nothing left to drain.  drainobjects is now gcDrainN because it's
the bounded equivalent to gcDrain.

The new names use the Go camel case convention because we have to
start somewhere.  The "gc" prefix is because we don't have runtime
packages yet and just "drain" is too ambiguous.

Change-Id: I88dbdf32e8ce4ce6c3b7e1f234664be9b76cb8fd
Reviewed-on: https://go-review.googlesource.com/4785Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

All calls to drainworkbuf now pass true for this argument, so remove
the argument and update the documentation to reflect the simplified
interface.

At a higher level, there are no longer any situations where we drain
"one wbuf" (though drainworkbuf didn't guarantee this anyway).  We
either drain everything, or we drain a specific number of objects.

Change-Id: Ib7ee0fde56577eff64232ee1e711ec57c4361335
Reviewed-on: https://go-review.googlesource.com/4784Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

scanblock is only called during _GCscan and _GCmarktermination.
During _GCscan, scanblock didn't call drainworkbufs anyway.  During
_GCmarktermination, there's really no point in draining some (largely
arbitrary) amount of work during the scanblock, since the GC is about
to drain everything anyway, so simply eliminate this case.

Change-Id: I7f3c59ce9186a83037c6f9e9b143181acd04c597
Reviewed-on: https://go-review.googlesource.com/4783Reviewed-by: Russ Cox <rsc@golang.org>

We no longer ever call scanblock with b == 0.

Change-Id: I9b01da39595e0cc251668c24d58748d88f5f0792
Reviewed-on: https://go-review.googlesource.com/4782Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

scanblock(0, 0, nil, nil) was just a confusing way of saying

  wbuf = getpartialorempty()
  drainworkbuf(wbuf, true)

Make drainworkbuf accept a nil workbuf and perform the
getpartialorempty itself and replace all uses of scanblock(0, 0, nil,
nil) with direct calls to drainworkbuf(nil, true).

Change-Id: I7002a2f8f3eaf6aa85bbf17ccc81d7288acfef1c
Reviewed-on: https://go-review.googlesource.com/4781Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

Previously, scanblock called checknocurrentwbuf() after
drainworkbuf().  Move this call into drainworkbuf so that every return
path from drainworkbuf calls checknocurrentwbuf().  This is equivalent
to the previous code because scanblock was the only caller of
drainworkbuf.

Change-Id: I96ef2168c8aa169bfc4d368f296342fa0fbeafb4
Reviewed-on: https://go-review.googlesource.com/4780Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

Currently we always create context objects for closures that capture variables.
However, it is completely unnecessary for direct calls of closures
(whether it is func()(), defer func()() or go func()()).
This change transforms any OCALLFUNC(OCLOSURE) to normal function call.
Closed variables become function arguments.
This transformation is especially beneficial for go func(),
because we do not need to allocate context object on heap.
But it makes direct closure calls a bit faster as well (see BenchmarkClosureCall).

On implementation level it required to introduce yet another compiler pass.
However, the pass iterates only over xtop, so it should not be an issue.
Transformation consists of two parts: closure transformation and call site
transformation. We can't run these parts on different sides of escape analysis,
because tree state is inconsistent. We can do both parts during typecheck,
we don't know how to capture variables and don't have call site.
We can't do both parts during walk of OCALLFUNC, because we can walk
OCLOSURE body earlier.
So now capturevars pass only decides how to capture variables
(this info is required for escape analysis). New transformclosure
pass, that runs just before order/walk, does all transformations
of a closure. And later walk of OCALLFUNC(OCLOSURE) transforms call site.

benchmark                            old ns/op     new ns/op     delta
BenchmarkClosureCall                 4.89          3.09          -36.81%
BenchmarkCreateGoroutinesCapture     1634          1294          -20.81%

benchmark                            old allocs     new allocs     delta
BenchmarkCreateGoroutinesCapture     6              2              -66.67%

benchmark                            old bytes     new bytes     delta
BenchmarkCreateGoroutinesCapture     176           48            -72.73%

Change-Id: Ic85e1706e18c3235cc45b3c0c031a9c1cdb7a40e
Reviewed-on: https://go-review.googlesource.com/4050Reviewed-by: Russ Cox <rsc@golang.org>

The only remaining uses of four spaces instead of a tab is
when the line is too long (e.g. type Package).

Fixes #9809

Change-Id: Ifffd3639aa9264e795686ef1879a7686f182d2e5
Reviewed-on: https://go-review.googlesource.com/4182Reviewed-by: Andrew Gerrand <adg@golang.org>

This was accidentially removed in:
https://go-review.googlesource.com/#/c/3508/8/src/cmd/gc/reflect.c

Change-Id: I06dd5bb0cb3e2811bd4ef605d7a5225cfa033fe0
Reviewed-on: https://go-review.googlesource.com/4731Reviewed-by: Keith Randall <khr@golang.org>

Change-Id: I7c41cc6a5ab9fb3b0cc3812cf7e9776884658778
Reviewed-on: https://go-review.googlesource.com/4671Reviewed-by: Ian Lance Taylor <iant@golang.org>

Using a zero register results in shorter, faster code.
5g already did this. Bring 6g, 8g, and 9g up to speed.
Reduces godoc binary size by 0.29% using 6g.

This CL includes cosmetic changes to 5g and 8g.
With those cosmetic changes included, componentgen is now
character-for-character equivalent across the four architectures.

Change-Id: I0e13dd48374bad830c725b117a1c86d4197d390c
Reviewed-on: https://go-review.googlesource.com/2606Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>

Fix a flipped nil check.
The flipped check prevented componentgen
from zeroing a non-cadable nl.
This fix reduces the number of non-SB LEAQs
in godoc from 35323 to 34920 (-1.1%).

Update #1914

Change-Id: I15ea303068835f606f883ddf4a2bb4cb2287e9ae
Reviewed-on: https://go-review.googlesource.com/2605Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>

Consider an interface value i of type I and concrete value c of type C.

Prior to this CL, i==c was evaluated as
	I(c) == i

Evaluating I(c) can allocate.

This CL changes the evaluation of i==c to
	x, ok := i.(C); ok && x == c

The new generated code is shorter and does not allocate directly.

If C is small, as it is in every instance in the stdlib,
the new code also uses less stack space
and makes one runtime call instead of two.

If C is very large, the original implementation is used.
The cutoff for "very large" is 1<<16,
following the stack vs heap cutoff used elsewhere.

This kind of comparison occurs in 38 places in the stdlib,
mostly in the net and os packages.

benchmark                     old ns/op     new ns/op     delta
BenchmarkEqEfaceConcrete      29.5          7.92          -73.15%
BenchmarkEqIfaceConcrete      32.1          7.90          -75.39%
BenchmarkNeEfaceConcrete      29.9          7.90          -73.58%
BenchmarkNeIfaceConcrete      35.9          7.90          -77.99%

Fixes #9370.

Change-Id: I7c4555950bcd6406ee5c613be1f2128da2c9a2b7
Reviewed-on: https://go-review.googlesource.com/2096Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>

When compiling the stdlib most of the calls
to sgen are for exactly 2 or 3 words:
85% for 6g and 70% for 8g.
Special case them for performance.
This optimization is not relevant to 5g and 9g.

6g

benchmark                old ns/op     new ns/op     delta
BenchmarkCopyFat16       3.25          0.82          -74.77%
BenchmarkCopyFat24       5.47          0.95          -82.63%

8g

benchmark               old ns/op     new ns/op     delta
BenchmarkCopyFat8       3.84          2.42          -36.98%
BenchmarkCopyFat12      4.94          2.15          -56.48%

Change-Id: I8bc60b453f12597dfd916df2d072a7d5fc33ab85
Reviewed-on: https://go-review.googlesource.com/2607Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>

When possible, generate nodl/nodr directly into DI/SI
rather than going through a temporary register.

CX has already been saved; use it during trailing bytes cleanup.

Change-Id: I4ec6209bcc5d3bfdc927c5c132009bd8d791ada3
Reviewed-on: https://go-review.googlesource.com/2608Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>

Change-Id: Ic270ffa7ec6f6dd4b0a951c64ad965447cce1417
Reviewed-on: https://go-review.googlesource.com/4571Reviewed-by: Alan Donovan <adonovan@google.com>

No code modifications.

This is in preparation for improving the wbuf abstraction.

Change-Id: I719543a345c34d079b7e39b251eccd5dd8a07826
Reviewed-on: https://go-review.googlesource.com/4710Reviewed-by: Rick Hudson <rlh@golang.org>

Plan 9's sysFree has an optimization where if the object being freed
is the last object allocated, it will roll back the brk to allow the
memory to be reused by sysAlloc.  However, it does not zero this
"returned" memory, so as a result, sysAlloc can return non-zeroed
memory after a sysFree.  This leads to corruption because the runtime
assumes sysAlloc returns zeroed memory.

Fix this by zeroing the memory returned by sysFree.

Fixes #9846.

Change-Id: Id328c58236eb7c464b31ac1da376a0b757a5dc6a
Reviewed-on: https://go-review.googlesource.com/4700Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>

Change-Id: Id86994c8692e29f9d073b6322733ce9219887dc3
Reviewed-on: https://go-review.googlesource.com/4520
Run-TryBot: Andrew Gerrand <adg@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

Extend escape analysis to make(map[k]v).
If it does not escape, allocate temp buffer for hmap and one bucket on stack.

There are 75 cases of non-escaping maps in std lib.

benchmark                                    old allocs     new allocs     delta
BenchmarkConcurrentStmtQuery                 16161          15161          -6.19%
BenchmarkConcurrentTxQuery                   17658          16658          -5.66%
BenchmarkConcurrentTxStmtQuery               16157          15156          -6.20%
BenchmarkConcurrentRandom                    13637          13114          -3.84%
BenchmarkManyConcurrentQueries               22             20             -9.09%
BenchmarkDecodeComplex128Slice               250            188            -24.80%
BenchmarkDecodeFloat64Slice                  250            188            -24.80%
BenchmarkDecodeInt32Slice                    250            188            -24.80%
BenchmarkDecodeStringSlice                   2250           2188           -2.76%
BenchmarkNewEmptyMap                         1              0              -100.00%
BenchmarkNewSmallMap                         2              0              -100.00%

benchmark                old ns/op     new ns/op     delta
BenchmarkNewEmptyMap     124           55.7          -55.08%
BenchmarkNewSmallMap     317           148           -53.31%

benchmark                old allocs     new allocs     delta
BenchmarkNewEmptyMap     1              0              -100.00%
BenchmarkNewSmallMap     2              0              -100.00%

benchmark                old bytes     new bytes     delta
BenchmarkNewEmptyMap     48            0             -100.00%
BenchmarkNewSmallMap     192           0             -100.00%

Fixes #5449

Change-Id: I24fa66f949d2f138885d9e66a0d160240dc9e8fa
Reviewed-on: https://go-review.googlesource.com/3508Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>

Dump frames of functions.
Add function name and var width to output.

Change-Id: Ida06b8def96178fa550ca90836eb4a2509b9e13f
Reviewed-on: https://go-review.googlesource.com/3870Reviewed-by: Russ Cox <rsc@golang.org>

typedslicecopy is another write barrier that is not
understood by racewalk. It seems quite complex to handle it
in the compiler, so instead just instrument it in runtime.

Update #9796

Change-Id: I0eb6abf3a2cd2491a338fab5f7da22f01bf7e89b
Reviewed-on: https://go-review.googlesource.com/4370Reviewed-by: Russ Cox <rsc@golang.org>

Walk calls it outervalue, racewalk calls it basenod,
isstack does it manually and slightly differently.

Change-Id: Id5b5d32b8faf143fe9d34bd08457bfab6fb33daa
Reviewed-on: https://go-review.googlesource.com/3745Reviewed-by: Russ Cox <rsc@golang.org>

Support the following conversions in escape analysis:
[]rune("foo")
[]byte("foo")
string([]rune{})

If the result does not escape, allocate temp buffer on stack
and pass it to runtime functions.

Change-Id: I1d075907eab8b0109ad7ad1878104b02b3d5c690
Reviewed-on: https://go-review.googlesource.com/3590Reviewed-by: Russ Cox <rsc@golang.org>

We don't have executable memory anymore.

Change-Id: I9835f03a7bcd97d809841ecbed8718b3048bfb32
Reviewed-on: https://go-review.googlesource.com/4681Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Dave Cheney <dave@cheney.net>

Change-Id: I40291561a18bed3ca6be9dca12a664bdf28cb2f1
Reviewed-on: https://go-review.googlesource.com/4660Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I35fe76661c80ca808a711acf608a23c77aeb0608
Reviewed-on: https://go-review.googlesource.com/4651Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #9190

Change-Id: I22177687ed834feed165454019d28c11fcbf0fa2
Reviewed-on: https://go-review.googlesource.com/2307Reviewed-by: Alex Brainman <alex.brainman@gmail.com>

(In non-Go print formats, the 016 includes the leading 0x prefix.
No one noticed, but we were printing hex numbers with a minimum
of 30 digits, not 32.)

Change-Id: I10ff7a51a567ad7c8440418ac034be9e4b2d6bc1
Reviewed-on: https://go-review.googlesource.com/4592Reviewed-by: Austin Clements <austin@google.com>

This matches all the other pseudo-packages.
The line was simply forgotten.

Change-Id: I278f6cbcfc883ea7efad07f99fc8c853b9b5d274
Reviewed-on: https://go-review.googlesource.com/4591Reviewed-by: Austin Clements <austin@google.com>

Otherwise different qsort implementations might result
in different sort orders and therefore different compiled
object files.

Change-Id: Ie783ba55a55af06941307e150b0c406e0a8128b0
Reviewed-on: https://go-review.googlesource.com/4590Reviewed-by: Austin Clements <austin@google.com>

It does not convert to Go well.

Being able to do this just once, instead of 4 times, was the primary
motivation for all the recent refactoring (not that it wasn't overdue).

Still bit-for-bit identical.

Change-Id: Ia01f17948441bf64fa78ec4226f0bb40af0bbaab
Reviewed-on: https://go-review.googlesource.com/3962Reviewed-by: Austin Clements <austin@google.com>

Now there is only one registerizer shared among all the systems.
There are some unfortunate special cases based on arch.thechar
in reg.c, to preserve bit-for-bit compatibility during the refactoring.
Most are probably bugs one way or another and should be revisited.

Change-Id: I153b435c0eaa05bbbeaf8876822eeb6dedaae3cf
Reviewed-on: https://go-review.googlesource.com/3883Reviewed-by: Austin Clements <austin@google.com>

gc/order.c rewrites OASOP nodes into ordinary assignments.
The back ends never see them anymore.

Change-Id: I268ac8bdc92dccd7123110a21f99ada3ceeb2baa
Reviewed-on: https://go-review.googlesource.com/3882Reviewed-by: Austin Clements <austin@google.com>

This isn't everything, but it's a start.
Still producing bit-identical compiler output.

The semantics of the old back ends is preserved,
even when they are probably buggy.
There are some TODOs in gc/gsubr.c to
remove special cases to preserve bugs in 5g and 8g.

Change-Id: I28ae295fbfc94ef9df43e13ab96bd6fc2f194bc4
Reviewed-on: https://go-review.googlesource.com/3802Reviewed-by: Austin Clements <austin@google.com>

benchmark                               old ns/op     new ns/op     delta
BenchmarkFormatFloatDecimal             300           283           -5.67%
BenchmarkFormatFloat                    383           381           -0.52%
BenchmarkFormatFloatExp                 359           357           -0.56%
BenchmarkFormatFloatNegExp              357           358           +0.28%
BenchmarkFormatFloatBig                 468           430           -8.12%
BenchmarkAppendFloatDecimal             104           92.5          -11.06%
BenchmarkAppendFloat                    199           190           -4.52%
BenchmarkAppendFloatExp                 172           167           -2.91%
BenchmarkAppendFloatNegExp              172           169           -1.74%
BenchmarkAppendFloatBig                 280           235           -16.07%
BenchmarkAppendFloat32Integer           104           92.4          -11.15%
BenchmarkAppendFloat32ExactFraction     168           171           +1.79%
BenchmarkAppendFloat32Point             206           199           -3.40%
BenchmarkAppendFloat32Exp               167           167           +0.00%
BenchmarkAppendFloat32NegExp            167           166           -0.60%
BenchmarkAppendFloat64Fixed1            134           129           -3.73%
BenchmarkAppendFloat64Fixed2            144           136           -5.56%
BenchmarkAppendFloat64Fixed3            138           134           -2.90%
BenchmarkAppendFloat64Fixed4            145           138           -4.83%

Change-Id: Ia143840cb34cbd1cebd6b691dd0a45b7264b406c
Reviewed-on: https://go-review.googlesource.com/3920Reviewed-by: Alan Donovan <adonovan@google.com>

Change-Id: Ib3738492a2ec8fc99323e687168b17b7239db6ad
Reviewed-on: https://go-review.googlesource.com/4511Reviewed-by: Alan Donovan <adonovan@google.com>

Change-Id: Ic5f3864bc6d94d60b754e3ccf72b1d40c5c09117
Reviewed-on: https://go-review.googlesource.com/4510Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>