R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5540056

R=golang-dev, bradfitz, r, rsc
CC=golang-dev
https://golang.org/cl/5539056

Fixes #2695.

R=golang-dev, mpimenov, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/5545052

Merge package files in the go/ast MergePackageFiles
function always	    in the same order (sorted by filename)
instead	 of map iteration order to obtain the same
package	 file each time.  This functionality is used
by godoc when displaying packages in ?m=src mode.

Also: minor cleanup in godoc.go.

R=rsc
CC=golang-dev
https://golang.org/cl/5540054

This CL improves the xml package in the following ways:

- makes its interface match established conventions
- brings Marshal and Unmarshal closer together
- fixes a large number of bugs and adds tests
- improves speed significantly
- organizes and simplifies the code

Fixes #2426.
Fixes #2406.
Fixes #1989.

What follows is a detailed list of those changes.

- All matching is case sensitive without special processing
  to the field name or xml tag in an attempt to match them.
  Customize the field tag as desired to match the correct XML
  elements.

- Flags are ",flag" rather than "flag". The names "attr",
  "chardata", etc, may be used to name actual XML elements.

- Overriding of attribute names is possible with "name,attr".

- Attribute fields are marshalled properly if they have
  non-string types. Previously they were unmarshalled, but were
  ignored at marshalling time.

- Comment fields tagged with ",comment" are marshalled properly,
  rather than being marshalled as normal fields.

- The handling of the Any field has been replaced by the ",any"
  flag to avoid unexpected results when using the field name for
  other purposes, and has also been fixed to interact properly
  with name paths. Previously the feature would not function
  if any field in the type had a name path in its tag.

- Embedded struct support fixed and cleaned so it works when
  marshalling and also when using field paths deeper than one level.

- Conflict reporting on field names have been expanded to cover
  all fields. Previously it'd catch only conflicts of paths
  deeper than one level. Also interacts correctly with embedded
  structs now.

- A trailing '>' is disallowed in xml tags. It used to be
  supported for removing the ambiguity between "attr" and "attr>",
  but the marshalling support for that was broken, and it's now
  unnecessary. Use "name" instead of "name>".

- Fixed docs to point out that a XMLName doesn't have to be
  an xml.Name (e.g. a struct{} is a good fit too). The code was
  already working like that.

- Fixed asymmetry in the precedence of XML element names between
  marshalling and unmarshalling. Marshal would consider the XMLName
  of the field type before the field tag, while unmarshalling would
  do the opposite. Now both respect the tag of the XMLName field
  first, and a nice error message is provided in case an attempt
  is made to name a field with its tag in a way that would
  conflict with the underlying type's XMLName field.

- Do not marshal broken "<???>" tags when in doubt. Use the type
  name, and error out if that's not possible.

- Do not break down unmarshalling if there's an interface{} field
  in a struct.

- Significant speed boost due to caching of type metadata and
  overall allocation clean ups. The following timings reflect
  processing of the the atom test data:

  Old:

  BenchmarkMarshal           50000             48798 ns/op
  BenchmarkUnmarshal          5000            357174 ns/op

  New:

  BenchmarkMarshal          100000             19799 ns/op
  BenchmarkUnmarshal         10000            128525 ns/op

R=cw, gustavo, kevlar, adg, rogpeppe, fullung, christoph, rsc
CC=golang-dev
https://golang.org/cl/5503078

Fixes godoc text mode (i.e., URL?m=text).

R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/5545043

- the main changes are removing the Doc suffix
  from the exported types, so instead of
  doc.TypeDoc one will have doc.Type, etc.

- All exported types now have a Name (or Names) field.
  For Values, the Names field lists all declared variables
  or constants.

- Methods have additional information about where they are
  coming from.

- There's a mode field instead of a bool to
  control the package's operation, which makes
  it easier to extend w/o API changes.

Except for the partially implemented new Method type,
this is based on existing code. A clean rewrite is in
progress based on this new API.

R=rsc, kevlar
CC=golang-dev
https://golang.org/cl/5528060

R=rsc, adg
CC=golang-dev
https://golang.org/cl/5533082

- remove wrapper.go from testing package (not needed anymore)

Fixes #1000.

R=rsc, golang-dev, n13m3y3r
CC=golang-dev
https://golang.org/cl/5502074

Only ParseFile, ParseDir, and ParseExpr are used in the tree.
If partial parsing of code is required, it is fairly simple
to wrap the relevant piece of code into a dummy package for
parsing (see parser.ParseExpr).

Also: minor cleanups.

R=rsc
CC=golang-dev
https://golang.org/cl/5535055

These functions are mostly of interest for debugging; the
number of bytes written is uninteresting.

R=r, bradfitz
CC=golang-dev
https://golang.org/cl/5540046

Fixes #2686.

R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5528089

Now it works to run 'go install' (no args) in cmd/go.

Fixes #2679.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5543046

TBR=gri
CC=golang-dev
https://golang.org/cl/5528090

Fixes #2681.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5540047

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5540048

Fixes #2688

R=golang-dev, gri, rsc
CC=golang-dev
https://golang.org/cl/5539050

Fixes #2331

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5544045

By the time a Unix linker gets to the end of the
command line it has forgotten what you told it
at the beginning of the command line, so you
have to put library arguments (like -lm) at the end.

R=golang-dev, r, bradfitz
CC=golang-dev
https://golang.org/cl/5541043

Pointed out by nekotaroh in issue 2645

R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5539045

benchmark                   old ns/op   new ns/op   delta
BenchmarkFormatFloatDecimal      3765        1386    -63%

R=rsc
CC=golang-dev, remy
https://golang.org/cl/5494060

Now only godoc imports go/doc.

R=gri
CC=golang-dev
https://golang.org/cl/5541045

Comment described the correct condition
but the code did not implement it.

Fixes #2625.

R=remyoudompheng
CC=golang-dev
https://golang.org/cl/5530082

Fixes #2622

R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/5533077

Also update default app and version to be correct.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5541044

This makes it possible to say -I c:/foo on Windows.

Fixes #2683 comment #3.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5540043

The domain returned by appengine.DefaultVersionHostname
isn't the one we want.
This change has been uploaded to build.golang.org

R=golang-dev, rsc, r
CC=golang-dev
https://golang.org/cl/5539043

This will let programs invoking ld prepare for it.
See issue 2676.

R=iant
CC=golang-dev
https://golang.org/cl/5535044

In a test that does

        func TestFoo(t *testing.T) {
                defer cleanup()
                t.Fatal("oops")
        }

it can be important that cleanup run as the test fails.
The old code did this in Fatal:

        t.signal <- t
        runtime.Goexit()

The runtime.Goexit would run the deferred cleanup
but the send on t.signal would cause the main test loop
to move on and possibly even exit the program before
the runtime.Goexit got a chance to run.

This CL changes tRunner (the top stack frame of a test
goroutine) to send on t.signal as part of a function
deferred by the top stack frame.  This delays the send
on t.signal until after runtime.Goexit has run functions
deferred by the test itself.

For the above TestFoo, this CL guarantees that cleanup
will run before the test binary exits.

This is particularly important when cleanup is doing
externally visible work, like removing temporary files
or unmounting file systems.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5532078

Especially affects tests, but not test-specific.
The linker was only being told where to find the
direct dependencies of package main.  Sometimes that
was sufficient to find the rest; sometimes not.

Fixes #2657.
Fixes #2666.
Fixes #2680.

R=golang-dev, adg, rogpeppe
CC=golang-dev
https://golang.org/cl/5528079

R=golang-dev, robert.hencke, r
CC=golang-dev
https://golang.org/cl/5538050

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5529080

This breakage is mainly due to API changes in pkg.
(e.g., package utf8 moved to unicode/utf8;
       remove of strconv.Atof64;
       change character type from int to rune.)
Also correct the usage comment.

This fixes issue 2646.
PS: I don't change the goyacc.go, because I think token type
    should not be force to rune.

R=golang-dev, adg, rogpeppe, r, r
CC=golang-dev
https://golang.org/cl/5502093

The escape analysis code does not make a distinction between
scalar and pointers fields in structs. Non-pointer fields
that escape should not make the whole struct escape.

R=lvd, rsc
CC=golang-dev, remy
https://golang.org/cl/5489128

Fixes #2621.

R=rsc, rsc
CC=golang-dev
https://golang.org/cl/5498104

It's hard enough to get right once.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5533073

R=lvd
CC=golang-dev
https://golang.org/cl/5533078

Tighter octal parsing broke some tests and were disabled in
https://golang.org/cl/5530051

Those tests were broken.  The CSS decoder was supposed to see CSS
hex escape sequences of the form '\' <hex>+, but those escape
sequences were instead being consumed by the template parser.

This change properly escapes those escape sequences, and uses
proper escaping for NULs.

R=golang-dev, rsc, nigeltao
CC=golang-dev
https://golang.org/cl/5529073

R=lvd
CC=golang-dev
https://golang.org/cl/5532077

- Changed the Scan API semantics slightly:
The token literal string is only returned
if the token is a literal, comment, semicolon,
or illegal character. In all other cases, the
token literal value is determined by the token
value.

Clients that care about the token literal value
when not present can always use the following
piece of code:

pos, tok, lit := scanner.Scan()
if lit == "" {
   lit = tok.String()
}

- Changed token.Lookup API to use a string instead
of a []byte argument.

- Both these changes were long-standing TODOs.

- Added BenchmarkScan.

This change permits a faster implementation of Scan
with much fewer string creations:

benchmark                old ns/op    new ns/op    delta
scanner.BenchmarkScan        74404        61457  -17.40%

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5532076