Historically, yacc has supported various kinds of inspections
and manipulations of the parser state, exposed as global variables.
The Go implementation of yacc puts that state (properly) in local
stack variables, so it can only be exposed explicitly.

There is now an explicit parser type, yyParser, returned by a
constructor, yyNewParser.

	type yyParser interface {
		Parse(yyLexer) int
		Lookahead() int
	}

Parse runs a parse. A call to the top-level func Parse
is equivalent to calling yyNewParser().Parse, but constructing
the parser explicitly makes it possible to access additional
parser methods, such as Lookahead.

Lookahead can be called during grammar actions to read
(but not consume) the value of the current lookahead token,
as returned by yylex.Lex. If there is no current lookahead token,
Lookahead returns -1. Invoking Lookahead corresponds to
reading the global variable yychar in a traditional Unix yacc grammar.

To support Lookahead, the internal parsing code now separates
the return value from Lex (yychar) from the reencoding used
by the parsing tables (yytoken). This has the effect that grammars
that read yychar directly in the action (possible since the actions
are in the same function that declares yychar) now correctly see values
from the Lex return value space, not the internal reencoding space.
This can fix bugs in ported grammars not even using SetParse and Lookahead.
(The reencoding was added on Plan 9 for large character sets.
No Plan 9 programs using yacc looked at yychar.)

Other methods may be added to yyParser later as needed.
Obvious candidates include equivalents for the traditional
yyclearin and yyerrok macros.

Change-Id: Iaf7649efcf97e09f44d1f5bc74bb563a11f225de
Reviewed-on: https://go-review.googlesource.com/4850Reviewed-by: Rob Pike <r@golang.org>

Change-Id: If2aafc4dd3f91650fc7727ea7d534ad7aa627c8c
Reviewed-on: https://go-review.googlesource.com/5090Reviewed-by: Russ Cox <rsc@golang.org>

First draft of converted Go compiler, using rsc.io/c2go rev 83d795a.

Change-Id: I29f4c7010de07d2ff1947bbca9865879d83c32c3
Reviewed-on: https://go-review.googlesource.com/4851Reviewed-by: Rob Pike <r@golang.org>

Change-Id: I2853535ab6c79d14f430c780161e4c35c52d9fb3
Reviewed-on: https://go-review.googlesource.com/4839Reviewed-by: Rob Pike <r@golang.org>

Change-Id: I3be69a4ebf300ad24b55b5f43fd7ad1f001c762e
Reviewed-on: https://go-review.googlesource.com/4838Reviewed-by: Rob Pike <r@golang.org>

Change-Id: Ida60c30041505c321fbfc48b22b8ff5af1a3f474
Reviewed-on: https://go-review.googlesource.com/4837Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>

Set TYPE_BRANCH for x(PC) in the parser and the assembler has less work to do.
This also makes the operand test handle -4(PC) correctly.

Also add a special test case for AX:DX, which should be fixed in obj really.

Change-Id: If195e3a8cf3454a73508633e9b317d66030da826
Reviewed-on: https://go-review.googlesource.com/5071Reviewed-by: Russ Cox <rsc@golang.org>

Use R15.
May fix build.

Change-Id: Ia25b0936c5aab2a427f8e6531688c3e537fbfdd0
Reviewed-on: https://go-review.googlesource.com/5070Reviewed-by: Russ Cox <rsc@golang.org>

Generated by reducing all the amd64 operands in the core.
Will add 386 and ARM later; this is a trial balloon.

NOTE: There is at least one anomaly: AX:DX doesn't print correctly in this situation.

Change-Id: I9f327c1890b100e3edb7b1b2a1c01f3e4b798f43
Reviewed-on: https://go-review.googlesource.com/4967Reviewed-by: Russ Cox <rsc@golang.org>

R15 is the real register. PC is a pseudo-register that we are making
illegal in this context as part of the grand assembly unification.

Change-Id: Ie0ea38ce7ef4d2cf4fcbe23b851a570fd312ce8d
Reviewed-on: https://go-review.googlesource.com/4966Reviewed-by: Minux Ma <minux@golang.org>

Handle the special name of R10 on the ARM - it's g - when it appears
in a register list [R0, g, R3]. Also simplify the pseudo-register parsing
a little.

Should fix the ARM build.

Change-Id: Ifcafc8195dcd3622653b43663ced6e4a144a3e51
Reviewed-on: https://go-review.googlesource.com/4965Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: Ia47e1e387acd30f30559d766aa6fca18cbb098f9
Reviewed-on: https://go-review.googlesource.com/5010Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: I182ea770110255a5ac1c91cf30dd650696a8f1db
Reviewed-on: https://go-review.googlesource.com/4961Reviewed-by: Russ Cox <rsc@golang.org>

Mishandled the complex addressing mode in masks<>(SB)(CX*8)
as a casualty of the ARM work. Fix by backing all the flows up to
the state where registerIndirect is always called with the input
sitting on the opening paren.

With this, build passes for me with linux-arm, linux-386, and linux-amd64.

Change-Id: I7cae69a6fa9b635c79efd93850bd1e744b22bc79
Reviewed-on: https://go-review.googlesource.com/4964Reviewed-by: Russ Cox <rsc@golang.org>

A consequence of the ARM work overlooked that SP is a real register
on x86, so we need to detect it specially.

This will be done better soon, but this is a fast fix for the build.

Change-Id: Ia30d111c3f42a5f0b5f4eddd4cc4d8b10470c14f
Reviewed-on: https://go-review.googlesource.com/4963Reviewed-by: Russ Cox <rsc@golang.org>

The tools have been fixed to not do this, but verifyAsm depends on this
being fixed.

TBR=rsc

Change-Id: Ia8968cc803b3498dfa2f98188c6ed1cf2e11c66d
Reviewed-on: https://go-review.googlesource.com/4962Reviewed-by: Rob Pike <r@golang.org>

Change-Id: Ic33431cdcc93db300fc2c3467eafdb5340ee4896
Reviewed-on: https://go-review.googlesource.com/4924Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Russ Cox <rsc@golang.org>

There are many peculiarites of the ARM architecture that require work:
condition codes, new instructions, new instruction arg counts, and more.

Rewrite the parser to do a cleaner job, flowing left to right through the
sequence of elements of an operand.

Add ARM to arch.
Add ARM-specific details to the arch in a new file, internal/arch/arm.
These are probably better kept away from the "portable" asm. However
there are some pieces, like MRC, that are hard to disentangle. They
can be cleaned up later.

Change-Id: I8c06aedcf61f8a3960a406c094e168182d21b972
Reviewed-on: https://go-review.googlesource.com/4923Reviewed-by: Russ Cox <rsc@golang.org>

Because text/scanner hides the spaces, the lexer treated
	#define A(x)
and
	#define A (x)
the same, but they are not: the first is an argument with macros, the
second is a simple one-word macro whose definition contains parentheses.
Fix this by noticing the relative column number as we move from A to (.
Hacky but simple.

Also add a helper to recognize the peculiar ARM shifted register operators.

Change-Id: I2cad22f5f1e11d8dad40ad13955793d178afb3ae
Reviewed-on: https://go-review.googlesource.com/4872Reviewed-by: Russ Cox <rsc@golang.org>

The mechanical edit in the last round managed to miss ROUND1, among
other indgnities.

Change-Id: Ie3e19d00435a9e701b9872167e4bc7756a9fb5a5
Reviewed-on: https://go-review.googlesource.com/4870Reviewed-by: Minux Ma <minux@golang.org>

Change-Id: Id544d435620efffaf5757dd9d9ebbc6e969a052c
Reviewed-on: https://go-review.googlesource.com/4823Reviewed-by: Rob Pike <r@golang.org>

Several .s files for ARM had several properties the new assembler will not support.
These include:

- mentioning SP or PC as a hardware register
	These are always pseudo-registers except that in some contexts
	they're not, and it's confusing because the context should not affect
	which register you mean. Change the references to the hardware
	registers to be explicit: R13 for SP, R15 for PC.
- constant creation using assignment
	The files say a=b when they could instead say #define a b.
	There is no reason to have both mechanisms.
- R(0) to refer to R0.
	Some macros use this to a great extent. Again, it's easy just to
	use a #define to rename a register.

Change-Id: I002335ace8e876c5b63c71c2560533eb835346d2
Reviewed-on: https://go-review.googlesource.com/4822Reviewed-by: Dave Cheney <dave@cheney.net>

The point of GOOBJ=2 was to have an active test of the cmd/internal/obj code.
Now we have end-to-end tests of the assembler, and soon the compiler,
so we don't need this halfway test on by default anymore.
(It's still possible to enable during debugging with the
environment variable.)

The problem it causes on the builders is that this particular testing
mode ends up with both the C process and the Go objwriter subprocess
having the same very large Prog list in memory simultaneously,
which causes basically a 2x memory blowup. In large programs
(such as the one generated by test/rotate.go) this is significant.

Disabling GOOBJ=2 should help with the current dev.cc builder
failures.

Change-Id: I1b11e4f29ea575659f02d2234242a904f7c867e4
Reviewed-on: https://go-review.googlesource.com/4832
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Conflicts:
	src/cmd/dist/build.go

Change-Id: I98a4b5e010bee91507b85bb8efd9c74e1a1f649c

Make cmd/ld a real library invoked by the individual linkers.
There are no reverse symbol references anymore
(symbols referred to in cmd/ld but defined in cmd/5l etc).

This means that in principle we could do an automatic
conversion of these to Go, as a stopgap until cmd/link is done
or as a replacement for cmd/link.

Change-Id: I4a94570257a3a7acc31601bfe0fad9dea0aea054
Reviewed-on: https://go-review.googlesource.com/4649Reviewed-by: Rob Pike <r@golang.org>

Change-Id: I2db4db852492eaddaf09dd7bae2fbd49f916e78a
Reviewed-on: https://go-review.googlesource.com/4648Reviewed-by: Rob Pike <r@golang.org>

- remove a few uses of ? :
- rename variables named len
- rewrite a few gotos as nested switches
- move goto targets to scope allowed by Go
- use consistent return type of anyregalloc
  (was int or int32 in different places)
- remove unused nr variable in agen
- include proper headers in generated builtin1.c
- avoid strange sized %E formats (%-6E, %2E)
- change gengcmask argument from uint8[16] to uint8*
  (diagnosed by c2go; not an array in any real sense).
- replace #ifdef XXX with comment block in 5g/peep.c
- expand and remove FAIL macro from 5g
- expand and remove noimpl macro from 9g
- print regalloc errors to stdout in 8g
  (only use of fprint(2, ...) in all compilers)

Still producing bit-for-bit identical output.

Change-Id: Id46efcd2a89241082b234f63f375b66f2754d695
Reviewed-on: https://go-review.googlesource.com/4646Reviewed-by: Austin Clements <austin@google.com>

In mparith, all the a1-- are problematic. Rewrite it all without pointers.
It's clearer anyway.

In popt, v is problematic because it is used both as a fixed pointer
(v = byvar[i]) and as a moving pointer (v = var; v++) aka slice.
Eliminate pointer movement.

Tested that this still produces bit-for-bit output for 'go build -a std'
compared to d260756 (current master).

Change-Id: I1a1bed0f98b594c3864fe95075dd95f9b52113e0
Reviewed-on: https://go-review.googlesource.com/4645Reviewed-by: Austin Clements <austin@google.com>

Otherwise the exported variable collides with the type Arch.

While we're here, remove arch.dumpit (now in portable code)
and add arch.defframe (forgotten originally, somehow).

Change-Id: I1b3a7dd7e96c5f632dba7cd6c1217b42a2004d72
Reviewed-on: https://go-review.googlesource.com/4644Reviewed-by: Austin Clements <austin@google.com>

If the Go source says x.y, and x is undefined, today we get

	undefined: x

Change to:

	undefined: x in x.y

Change-Id: I8ea95503bd469ea933c6bcbd675b7122a5d454f3
Reviewed-on: https://go-review.googlesource.com/4643Reviewed-by: Austin Clements <austin@google.com>

Even with debugmerge = 1, the debugging output only happens
with the -v command-line flag. This is useful because it gets added
in automatically when debugging things like registerization with -R -v.

Change-Id: I9a5c7f562507b72e8e2fe2686fd07d069721345a
Reviewed-on: https://go-review.googlesource.com/4641Reviewed-by: Austin Clements <austin@google.com>

Change-Id: I36f77e7ac7f727d8f3b51133f4b3ef93c35b09f6
Reviewed-on: https://go-review.googlesource.com/4640Reviewed-by: Austin Clements <austin@google.com>

Noticed last week.
Just saw a strange build failure in the revised rcmp (called by qsort on region)
and this fixed it.

Submitting first to avoid finding out which of my pending CLs tickled the
problem.

Change-Id: I4cafd611e2bf8e813e57ad0025e48bde5ae54359
Reviewed-on: https://go-review.googlesource.com/4830Reviewed-by: Russ Cox <rsc@golang.org>

When the compiler echoes back an expression, it shows the
generated yacc expression. Change the generated code to
use a slice so that $3 shows up as yyDollar[3] in such messages.

Consider changing testdata/expr/expr.y to say:

	$$.Sub(float64($1), $3)

(The float64 conversion is incorrect.)

Before:
expr.y:70[expr.go:486]: cannot convert exprS[exprpt - 2].num (type *big.Rat) to type float64

After:
expr.y:70[expr.go:492]: cannot convert exprDollar[1].num (type *big.Rat) to type float64

Change-Id: I74e494069df588e62299d1fccb282f3658d8f8f4
Reviewed-on: https://go-review.googlesource.com/4630Reviewed-by: Rob Pike <r@golang.org>

The current XML printer does not understand the xmlns
attribute. This change changes it so that it interprets the
xmlns attributes in the tokens being printed, and uses
appropriate prefixes.

Fixes #7535.

Change-Id: I20fae291d20602d37deb41ed42fab4c9a50ec85d
Reviewed-on: https://go-review.googlesource.com/2660Reviewed-by: Nigel Tao <nigeltao@golang.org>

MOVQ RARG0, 0(SP) smashes exactly what was saved by PUSHQ R15.
This code managed to work somehow with the current race runtime,
but corrupts caller arguments with new race runtime that I am testing.

Change-Id: I9ffe8b5eee86451db36e99dbf4d11f320192e576
Reviewed-on: https://go-review.googlesource.com/4810Reviewed-by: Keith Randall <khr@golang.org>

New race runtime is more scrupulous about env flags format.

Change-Id: I2828bc737a8be3feae5288ccf034c52883f224d8
Reviewed-on: https://go-review.googlesource.com/4811Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

drainworkbuf is now gcDrain, since it drains until there's
nothing left to drain.  drainobjects is now gcDrainN because it's
the bounded equivalent to gcDrain.

The new names use the Go camel case convention because we have to
start somewhere.  The "gc" prefix is because we don't have runtime
packages yet and just "drain" is too ambiguous.

Change-Id: I88dbdf32e8ce4ce6c3b7e1f234664be9b76cb8fd
Reviewed-on: https://go-review.googlesource.com/4785Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

All calls to drainworkbuf now pass true for this argument, so remove
the argument and update the documentation to reflect the simplified
interface.

At a higher level, there are no longer any situations where we drain
"one wbuf" (though drainworkbuf didn't guarantee this anyway).  We
either drain everything, or we drain a specific number of objects.

Change-Id: Ib7ee0fde56577eff64232ee1e711ec57c4361335
Reviewed-on: https://go-review.googlesource.com/4784Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>

scanblock is only called during _GCscan and _GCmarktermination.
During _GCscan, scanblock didn't call drainworkbufs anyway.  During
_GCmarktermination, there's really no point in draining some (largely
arbitrary) amount of work during the scanblock, since the GC is about
to drain everything anyway, so simply eliminate this case.

Change-Id: I7f3c59ce9186a83037c6f9e9b143181acd04c597
Reviewed-on: https://go-review.googlesource.com/4783Reviewed-by: Russ Cox <rsc@golang.org>