Problem was caused by use of Args[].Aux differences
in early partitioning.  This artificially separated
two equivalent expressions because sort ignores the
Aux field, hence things can end with equal things
separated by unequal things and thus the equal things
are split into more than one partition.  For example:
SliceLen(a), SliceLen(b), SliceLen(a).

Fix: don't use Args[].Aux in initial partitioning.

Left in a debugging flag and some debugging Fprintf's;
not sure if that is house style or not.  We'll probably
want to be more systematic in our naming conventions,
e.g. ssa.cse, ssa.scc, etc.

Change-Id: Ib1412539cc30d91ea542c0ac7b2f9b504108ca7f
Reviewed-on: https://go-review.googlesource.com/19316Reviewed-by: Keith Randall <khr@golang.org>

Examine both Aux and AuxInt to form more precise initial partitions.
Restructure loop to avoid repeated type.Equal() call.  Speeds up
compilation of testdata/gen/arithConst_ssa by 25%.

Change-Id: I3cfb1d254adf0601ee69239e1885b0cf2a23575b
Reviewed-on: https://go-review.googlesource.com/19313
Run-TryBot: Todd Neal <todd@tneal.org>
Reviewed-by: Keith Randall <khr@golang.org>

Panic doesn't return, so record that we immediately exit after a panic
call.  This will help code analysis.

Change-Id: I4d1f67494f97b6aee130c43ff4e44307b2b0f149
Reviewed-on: https://go-review.googlesource.com/19303
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>

The frontend does this for 32 bits and below, but SSA needs
to do it for 64 bits.  The algorithms are all copied from
cgen.go:cgen_div.

Speeds up TimeFormat substantially: ~40% slower to ~10% slower.

Change-Id: I023ea2eb6040df98ccd9105e15ca6ea695610a7a
Reviewed-on: https://go-review.googlesource.com/19302
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>

Use just a single write barrier flag test, even if there
are multiple pointer fields in a struct.

This helps move more of the wb-specific code (like the LEA
needed to materialize the write address) into the unlikely path.

Change-Id: Ic7a67145904369c4ff031e464d51267d71281c8f
Reviewed-on: https://go-review.googlesource.com/19085
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

Change-Id: Ifb8eba1929c79ee7a8cae2191613c55a3b8f74e5
Reviewed-on: https://go-review.googlesource.com/19236Reviewed-by: Todd Neal <todd@tneal.org>

If the output register is one of the input registers,
we can use a real add instead of LEA.

Change-Id: Ide58f1536afb077c0b939d3a8c7555807fd1c5e3
Reviewed-on: https://go-review.googlesource.com/19234Reviewed-by: Alexandru Moșoi <alexandru@mosoi.ro>

* Enclose each rule's code in a for with no condition
* The loop is ran at most once because it's always terminated by a return.
* Use break when matching condition fails
* Drop rule hashes
* Shaves about 3 lines of code per rule

The binary size is not afected.

Change-Id: I27c3e40dc8cae98dcd50739342dc38db2ef9c247
Reviewed-on: https://go-review.googlesource.com/19220Reviewed-by: Keith Randall <khr@golang.org>

Mostly indexed loads.  A few more LEA cases.

Change-Id: Idc1d447ed0dd6e906cd48e70307a95e77f61cf5f
Reviewed-on: https://go-review.googlesource.com/19172Reviewed-by: Todd Neal <todd@tneal.org>
Run-TryBot: Keith Randall <khr@golang.org>

LEAQ symbol+100(SB), AX

Under dynamic link, rewrites to

MOVQ symbol@GOT(SB), AX
ADDQ $100, AX

but ADDQ clobbers flags, whereas the original LEAQ (when not dynamic
linking) doesn't.

Use LEAQ instead of ADDQ to add that constant in so we preserve flags.

Change-Id: Ibb055403d94a4c5163e1c7d2f45da633ffd0b6a3
Reviewed-on: https://go-review.googlesource.com/19230Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Shaves about 3 lines per generated rule.

Change-Id: I94adc94ab79f90ac5fd033f896ece3b1eddf0f3d
Reviewed-on: https://go-review.googlesource.com/19197Reviewed-by: Keith Randall <khr@golang.org>

Add rewrite rules to optimize constant shifts.

Fixes #10637

Change-Id: I74b724d3e81aeb7098c696d02c050f7fdfd5b523
Reviewed-on: https://go-review.googlesource.com/19106Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Removes approx. one assignment per rule.

Change-Id: Ie9f0a7082ae12c4447ff6b4d40678cd92bdbb6f2
Reviewed-on: https://go-review.googlesource.com/19194Reviewed-by: Keith Randall <khr@golang.org>

* Simplify comparisons of form a + const1 == const2 or a + const1 != const2.
* Canonicalize Eq, Neq, Add, Sub to have a constant as first argument.
Needed for the above new rules and helps constant folding.

Change-Id: I8078702a5daa706da57106073a3e9f640a67f486
Reviewed-on: https://go-review.googlesource.com/19192Reviewed-by: Keith Randall <khr@golang.org>

Modify the simple domCheck to use the sparse tree code.  This
speeds up compilation of one of the generated test cases from
1m48s to 17s.

Change-Id: If577410ee77b54918147a66917a8e3721297ee0a
Reviewed-on: https://go-review.googlesource.com/19187
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Change-Id: I1738e3af7de0972c54d74325d80781059d0796d8
Reviewed-on: https://go-review.googlesource.com/19186
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Move the cached sparse sets to the Config.  I tested make.bash with
pre-allocating sets of size 150 and not caching very small sets, but the
difference between this implementation (no min size, no preallocation)
and a min size with preallocation was fairly negligible:

Number of sparse sets allocated:
Cached in Config w/none preallocated no min size    3684 *this CL*
Cached in Config w/three preallocated no min size   3370
Cached in Config w/three preallocated min size=150  3370
Cached in Config w/none preallocated min size=150  15947
Cached in Func,  w/no min                          96996 *previous code*

Change-Id: I7f9de8a7cae192648a7413bfb18a6690fad34375
Reviewed-on: https://go-review.googlesource.com/19152Reviewed-by: Keith Randall <khr@golang.org>

Add the aux type to opcodes.
Add rematerializeable as a flag.

Change-Id: I906e19281498f3ee51bb136299bf26e13a54b2ec
Reviewed-on: https://go-review.googlesource.com/19088
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>

Converted working slices of pointer into slices of pointer
index.  Half the size (on 64-bit machine) and no pointers
to trace if GC occurs while they're live.

TODO - could expose slice mapping ID->*Block; some dom
clients also construct these.

Minor optimization in regalloc that cuts allocation count.

Minor optimization in compile.go that cuts calls to Sprintf.

Change-Id: I28f0bfed422b7344af333dc52ea272441e28e463
Reviewed-on: https://go-review.googlesource.com/19104
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>

These changes were left out of the 1cc5789d commit.

Change-Id: Id7f49973da281a567b69228dbaea31846b82b4af
Reviewed-on: https://go-review.googlesource.com/19105Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>

Small optimizations I noticed while looking at Giovanni's test cases.

More shifts by constants.
Indexed stores for smaller types.
Fold LEA into loads/stores.
More extending loads.
CMP $0 of AND -> TEST

Fix order of TEST ops.

Giovanni's test cases at https://gist.github.com/rasky/62fba94e3a20d1b05b2a

Change-Id: I7077bc0b5319bf05767eeb39f401f4bb4b39f635
Reviewed-on: https://go-review.googlesource.com/19086
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
Reviewed-by: David Chase <drchase@google.com>

Cache sparse sets in the function so they can be reused by subsequent
compiler passes.

benchmark                        old ns/op     new ns/op     delta
BenchmarkDSEPass-8               206945        180022        -13.01%
BenchmarkDSEPassBlock-8          5286103       2614054       -50.55%
BenchmarkCSEPass-8               1790277       1790655       +0.02%
BenchmarkCSEPassBlock-8          18083588      18112771      +0.16%
BenchmarkDeadcodePass-8          59837         41375         -30.85%
BenchmarkDeadcodePassBlock-8     1651575       511169        -69.05%
BenchmarkMultiPass-8             531529        427506        -19.57%
BenchmarkMultiPassBlock-8        7033496       4487814       -36.19%

benchmark                        old allocs     new allocs     delta
BenchmarkDSEPass-8               11             4              -63.64%
BenchmarkDSEPassBlock-8          599            120            -79.97%
BenchmarkCSEPass-8               18             18             +0.00%
BenchmarkCSEPassBlock-8          2700           2700           +0.00%
BenchmarkDeadcodePass-8          4              3              -25.00%
BenchmarkDeadcodePassBlock-8     30             9              -70.00%
BenchmarkMultiPass-8             24             20             -16.67%
BenchmarkMultiPassBlock-8        1800           1000           -44.44%

benchmark                        old bytes     new bytes     delta
BenchmarkDSEPass-8               221367        142           -99.94%
BenchmarkDSEPassBlock-8          3695207       3846          -99.90%
BenchmarkCSEPass-8               303328        303328        +0.00%
BenchmarkCSEPassBlock-8          5006400       5006400       +0.00%
BenchmarkDeadcodePass-8          84232         10506         -87.53%
BenchmarkDeadcodePassBlock-8     1274940       163680        -87.16%
BenchmarkMultiPass-8             608674        313834        -48.44%
BenchmarkMultiPassBlock-8        9906001       5003450       -49.49%

Change-Id: Ib1fa58c7f494b374d1a4bb9cffbc2c48377b59d3
Reviewed-on: https://go-review.googlesource.com/19100Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>

From memory profiling, about 3% reduction in allocation count.

Change-Id: I4b662d55b8a94fe724759a2b22f05a08d0bf40f8
Reviewed-on: https://go-review.googlesource.com/19103Reviewed-by: Keith Randall <khr@golang.org>

We used to compare the init state with == to 0 and 2, which
requires 2 comparisons.  Instead, compare with 1 and use
<, ==.  That requires only one comparison.

This isn't a big deal performance-wise, as it is just init
code.  But there is a fair amount of init code, so this
should help a bit with code size.

Change-Id: I4a2765f1005776f0edce28ac143f4b7596d95a68
Reviewed-on: https://go-review.googlesource.com/18948Reviewed-by: David Chase <drchase@google.com>

Rename StoreConst to ValAndOff so we can use it for other ops.
Make ValAndOff print nicely.

Add some notes & checks related to my aborted attempt to
implement combined CMP+load ops.

Change-Id: I2f901d12d42bc5a82879af0334806aa184a97e27
Reviewed-on: https://go-review.googlesource.com/18947
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: David Chase <drchase@google.com>

The old write barriers used _nostore versions, which
don't work for Ian's cgo checker.  Instead, we adopt the
same write barrier pattern as the default compiler.

It's a bit trickier to code up but should be more efficient.

Change-Id: I6696c3656cf179e28f800b0e096b7259bd5f3bb7
Reviewed-on: https://go-review.googlesource.com/18941
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

Compiling && and || expressions often leads to control
flow of the following form:

p:
  If a goto b else c
b: <- p ...
  x = phi(a, ...)
  If x goto t else u

Note that if we take the edge p->b, then we are guaranteed
to take the edge b->t also.  So in this situation, we might
as well go directly from p to t.

Change-Id: I6974f1e6367119a2ddf2014f9741fdb490edcc12
Reviewed-on: https://go-review.googlesource.com/18910Reviewed-by: David Chase <drchase@google.com>

For each value that needs to be in a fixed register at the end of the
block, and try to pick that fixed register when the instruction
generating that value is scheduled (or restored from a spill).

Just used for end-of-block register requirements for now.
Fixed-register instruction requirements (e.g. shift in ecx) can be
added later.  Also two-instruction constraints (input reg == output
reg) might be recorded in a similar manner.

Change-Id: I59916e2e7f73657bb4fc3e3b65389749d7a23fa8
Reviewed-on: https://go-review.googlesource.com/18774
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>

The OpSB hack didn't quite work.  We need to really
CSE these ops to make regalloc happy.

Change-Id: I9f4d7bfb0929407c84ee60c9e25ff0c0fbea84af
Reviewed-on: https://go-review.googlesource.com/19083
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

Speeds up compilation by 6%.

Change-Id: Ibaad95710323ddbe13c1b0351843fe43a48d776e
Reviewed-on: https://go-review.googlesource.com/19080Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Some tests make multiple Funcs per Config at once.
With value & block caching, we can't do that any more.

Change-Id: Ibdb60aa2fcf478f1726b3be0fcaa06b04433eb67
Reviewed-on: https://go-review.googlesource.com/19081Reviewed-by: Keith Randall <khr@golang.org>

Speeds up the compiler ~5%.

Change-Id: Ia5cf0bcd58701fd14018ec77d01f03d5c7d6385b
Reviewed-on: https://go-review.googlesource.com/19060Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

It is one of the slowest compiler phases right now, and we
run two of them.

Instead of using a map to make the initial partition, use a sort.
It is much less memory intensive.

Do a few optimizations to avoid work for size-1 equivalence classes.

Implement -N.

Change-Id: I1d2d85d3771abc918db4dd7cc30b0b2d854b15e1
Reviewed-on: https://go-review.googlesource.com/19024Reviewed-by: David Chase <drchase@google.com>

The x86 backend automatically rewrites MOV $0, AX to
XOR AX, AX.  That rewrite isn't ok when the flags register
is live across the MOV.  Keep track of which moves care
about preserving flags, then disable this rewrite for them.

On x86, Prog.Mark was being used to hold the length of the
instruction.  We already store that in Prog.Isize, so no
need to store it in Prog.Mark also.  This frees up Prog.Mark
to hold a bitmask on x86 just like all the other architectures.

Update #12405

Change-Id: Ibad8a8f41fc6222bec1e4904221887d3cc3ca029
Reviewed-on: https://go-review.googlesource.com/18861Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>

The conversion from -0.0 to +0.0 happens inside mpgetflt now.
The SSA code doesn't need this fix any more.

Change-Id: I6cd4f4a4e75b13cf284ebbb95b08af050ed9891c
Reviewed-on: https://go-review.googlesource.com/18942Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>

Empty blocks are introduced to remove critical edges.
After regalloc, we can remove any of the added blocks
that are still empty.

Change-Id: I0b40e95ac3a6cc1e632a479443479532b6c5ccd9
Reviewed-on: https://go-review.googlesource.com/18833
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>

Change-Id: I78743987dcb45d821212caf95a00ae15b7a6cfd8
Reviewed-on: https://go-review.googlesource.com/18773Reviewed-by: Andrew Gerrand <adg@golang.org>

Break small structs up into their components so they
can be registerized.

Change StructSelect to use field indexes instead of
field offsets, as field offsets aren't unique in the
presence of zero-sized fields.

Change-Id: I2f1dc89f7fa58e1cf58aa1a32b238959d53f62e4
Reviewed-on: https://go-review.googlesource.com/18570
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>

Redo how we keep track of forward references when building SSA.
When the forward reference is resolved, update the Value node
in place.

Improve the phi elimination pass so it can simplify phis of phis.

Give SSA package access to decoded line numbers.  Fix line numbers
for constant booleans.

Change-Id: I3dc9896148d260be2f3dd14cbe5db639ec9fa6b7
Reviewed-on: https://go-review.googlesource.com/18674Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Keith Randall <khr@golang.org>

Semi-regular merge from tip to dev.ssa.

Conflicts:
	src/runtime/sys_windows_amd64.s

Change-Id: I5f733130049c810e6ceacd46dad85faebca52b29