This was originally done in https://codereview.appspot.com/5690059
(Feb 2012) to deal with bad response headers coming back from webcams,
but it presents a potential security problem with HTTP request
smuggling for request headers containing "Content Length" instead of
"Content-Length".

Part of overall HTTP hardening for request smuggling. See RFC 7230.

Thanks to Régis Leroy for the report.

Change-Id: I92b17fb637c9171c5774ea1437979ae2c17ca88a
Reviewed-on: https://go-review.googlesource.com/11772Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

If you have more than 10 procs, then currently they are sorted alphabetically as
0, 10, 11, ..., 19, 2, 20, ...
Assign explicit order to procs so that they are sorted numerically.

Change-Id: I6d978d2cd439aa2fcbcf147842a643f9073eef75
Reviewed-on: https://go-review.googlesource.com/11750Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #9758

Change-Id: I3089ec06cddd74b547d8b10834d7478a04b02069
Reviewed-on: https://go-review.googlesource.com/11701Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

RawPath is a hint to the desired encoding of Path.
It is ignored when it is not a valid encoding of Path,
such as when Path has been changed but RawPath has not.
It is not ignored but also not useful when it matches
the url package's natural choice of encoding.
In this latter case, set it to the empty string.
This should help drive home the point that clients
cannot in general depend on it being present and
that they should use the EncodedPath method instead.

This also reduces the impact of the change on tests,
especially tests that use reflect.DeepEqual on parsed URLs.

Change-Id: I437c51a33b85439a31c307caf1436118508ea196
Reviewed-on: https://go-review.googlesource.com/11760Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Iacb84421215ca80c7add2818118b2af1a650fd58
Reviewed-on: https://go-review.googlesource.com/11639Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

When an xmlns="..." attribute was explicitly generated,
it was being ignored because the name space on the
attribute was assumed to have been explicitly set (to the empty
name space) and it's not possible to have an element in the
empty name space when there is a non-empty name space set.

We fix this by recording when a default name space has been
explicitly set and setting the name space of the element to that
so printer.defineNS can do its work correctly.

We do not attempt to add our own xmlns="..." attribute
when one is explicitly set.

We also add tests for EncodeElement, as that's the only way
to attain coverage of some of the changed behaviour.
Some other test coverage is also increased, although
more work remains to be done in this area.

This change was jointly developed with Martin Hilton (mhilton on github).

Fixes #11431.

Change-Id: I7b85e06eea5b18b2c15ec16dcbd92a8e1d6a9a4e
Reviewed-on: https://go-review.googlesource.com/11635Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #11438

Change-Id: Iedd31918aa5187fc3fdaed71c8dad82075559084
Reviewed-on: https://go-review.googlesource.com/11667Reviewed-by: Rob Pike <r@golang.org>

The spec says this is invalid, but it matches giflib's behavior.

Fixes #9856 (together with https://go-review.googlesource.com/11661).

Change-Id: I05701f62a9e5e724a2d85c6b87ae4111e537146b
Reviewed-on: https://go-review.googlesource.com/11663Reviewed-by: Rob Pike <r@golang.org>

This is needed by issue #9856.

Change-Id: Idad570a7e55ad903aab55372d390bc746c4e19cf
Reviewed-on: https://go-review.googlesource.com/11661Reviewed-by: Rob Pike <r@golang.org>

Fixes #11426

Change-Id: I77368b0e852149ed4533e139cc43887508ac7f78
Reviewed-on: https://go-review.googlesource.com/11662Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>

This allows for "race free" cancellation, in the sense discussed in
issue #11013: in contrast to Transport.CancelRequest, the cancellation
will not be lost if the user cancels before the request is put into the
transport's internal map.

Fixes #11013.

Change-Id: I0b5e7181231bdd65d900e343f764b4d1d7c422cd
Reviewed-on: https://go-review.googlesource.com/11601
Run-TryBot: David Symonds <dsymonds@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

For #11326 (but not a fix).

Change-Id: Ic51814f5cd7357427c3fd990a5522775d05e7987
Reviewed-on: https://go-review.googlesource.com/11673Reviewed-by: Robert Griesemer <gri@golang.org>

I think this has the same meaning as before,
but the text is tighter, and it makes some people happy.

Fixes #10182.

Change-Id: I7ee1eae4bcd6ee4a5898ea948648939e6bde5f01
Reviewed-on: https://go-review.googlesource.com/11674Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

This behavior is not what we might have designed from the start,
but it has been present since Go 1. Rather than make a visible
behavioral change that might cause programs to work differently
in Go ≤1.4 vs Go ≥1.5, document what SkipDir on a non-directory
has always meant. If code doesn't want this meaning, it is easy
enough not to return SkipDir on non-directories.

Fixes #10533.

Change-Id: Ic0612f032044bc7c69bf62583a02037e4b47530b
Reviewed-on: https://go-review.googlesource.com/11690Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rob Pike <r@golang.org>

Fixes #9621.

Change-Id: Ib9c6001378364af899f57fd4b89fb23af2042923
Reviewed-on: https://go-review.googlesource.com/11694Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>

Change-Id: I5791639e71874ce13fac836a0c6014e20ee7417e
Reviewed-on: https://go-review.googlesource.com/11664Reviewed-by: Russ Cox <rsc@golang.org>

Also: make (*Scope).Innermost work for Package scopes.

This change is identical to http://go-review.googlesource.com/#/c/11691/,
except for minor changes required by the use of testImporter.

Change-Id: Id07e66f78987f7242c2e642dfd6ee613676e10e5
Reviewed-on: https://go-review.googlesource.com/11714Reviewed-by: Robert Griesemer <gri@golang.org>

This code used to only be run for ELF, with the predictable
result that using -s with external linking broke on Windows and OS X.
Moving it here should fix Windows and does fix OS X.

CL 10835 also claims to fix the crash on Windows.
I don't know whether it does so correctly, but regardless,
this CL should make that one a no-op.

Fixes #10254.

Change-Id: I2e7b45ab0c28568ddbb1b50581dcc157ae0e7ffe
Reviewed-on: https://go-review.googlesource.com/11695Reviewed-by: David Crawshaw <crawshaw@golang.org>

Fixes #9865.

Change-Id: I8ce5b1708ed938910c59899706e470271c2e7e9d
Reviewed-on: https://go-review.googlesource.com/11699
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change 7c7126cf removed the primality
checking in Validate to save CPU time. That check happened to be
filtering out private keys with primes that were zero or one. Without
that filtering, such primes cause a panic when trying to use such a
private key.

This change specifically checks for and rejects primes ≤ 1 in Validate.

Fixes #11233.

Change-Id: Ie6537edb8250c07a45aaf50dab43227002ee7386
Reviewed-on: https://go-review.googlesource.com/11611Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Fixes #10725

Change-Id: Ic8685dc238a0ffc95fafb512a8587d2eb5fe9d5f
Reviewed-on: https://go-review.googlesource.com/11720Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixes #11410

Change-Id: I9d01be3e01f74f4b12a516aba8a5d20d9b277ec6
Reviewed-on: https://go-review.googlesource.com/11539Reviewed-by: Russ Cox <rsc@golang.org>

Update #11443

Change-Id: Icb7ea291a837dcf2799a791a2ba780fd2a5e712b
Reviewed-on: https://go-review.googlesource.com/11721Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>

Should fix arm64 build failure.

Change-Id: Ib35e4a69a1082e461a1eddf3265544a1d2ff98e5
Reviewed-on: https://go-review.googlesource.com/11710Reviewed-by: David Crawshaw <crawshaw@golang.org>

Fixes #8836.

Change-Id: Idda9f4a987e03b3bdf5e8fdb984fe56d6f84aa59
Reviewed-on: https://go-review.googlesource.com/11672Reviewed-by: David Chase <drchase@google.com>

Change-Id: I08de33aacb3fc932722286d69b1dd70ffe787c89
Reviewed-on: https://go-review.googlesource.com/11697Reviewed-by: Russ Cox <rsc@golang.org>

The old code was recording the current table output offset,
so the table from the next function would be used instead of
the runtime realizing that there was no table at all.

Add debug constant in runtime to check this for every function
at startup. It's too expensive to do that by default, but we can
do the last five functions. The end of the table is usually where
the C symbols end up, so that's where the problems typically are.

Fixes #10747.
Fixes #11396.

Change-Id: I13592e78017969fc22979fa902e19e1b151d41b1
Reviewed-on: https://go-review.googlesource.com/11657Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>

Currently we fail to reset the live heap accounting state before the
checkmark mark and before the gctrace=2 extra mark. As a result, if
either are enabled, at the end of GC it thinks there are 0 bytes of
live heap, which causes the GC controller to initiate a new GC
immediately, regardless of the true heap size.

Fix this by factoring this state reset into a function and calling it
before all three possible marks.

This function should be merged with gcResetGState, but doing so
requires some additional cleanup, so it will wait for after the
freeze. Filed #11427 for this cleanup.

Fixes #10492.

Change-Id: Ibe46348916fc8368fac6f086e142815c970a6f4d
Reviewed-on: https://go-review.googlesource.com/11561Reviewed-by: Russ Cox <rsc@golang.org>

Memory for stacks is manually managed by the runtime and, currently
(with one exception) we free stack spans immediately when the last
stack on a span is freed. However, the garbage collector assumes that
spans can never transition from non-free to free during scan or mark.
This disagreement makes it possible for the garbage collector to mark
uninitialized objects and is blocking us from re-enabling the bad
pointer test in the garbage collector (issue #9880).

For example, the following sequence will result in marking an
uninitialized object:

1. scanobject loads a pointer slot out of the object it's scanning.
   This happens to be one of the special pointers from the heap into a
   stack. Call the pointer p and suppose it points into X's stack.

2. X, running on another thread, grows its stack and frees its old
   stack.

3. The old stack happens to be large or was the last stack in its
   span, so X frees this span, setting it to state _MSpanFree.

4. The span gets reused as a heap span.

5. scanobject calls heapBitsForObject, which loads the span containing
   p, which is now in state _MSpanInUse, but doesn't necessarily have
   an object at p. The not-object at p gets marked, and at this point
   all sorts of things can go wrong.

We already have a partial solution to this. When shrinking a stack, we
put the old stack on a queue to be freed at the end of garbage
collection. This was done to address exactly this problem, but wasn't
a complete solution.

This commit generalizes this solution to both shrinking and growing
stacks. For stacks that fit in the stack pool, we simply don't free
the span, even if its reference count reaches zero. It's fine to reuse
the span for other stacks, and this enables that. At the end of GC, we
sweep for cached stack spans with a zero reference count and free
them. For larger stacks, we simply queue the stack span to be freed at
the end of GC. Ideally, we would reuse these large stack spans the way
we can small stack spans, but that's a more invasive change that will
have to wait until after the freeze.

Fixes #11267.

Change-Id: Ib7f2c5da4845cc0268e8dc098b08465116972a71
Reviewed-on: https://go-review.googlesource.com/11502Reviewed-by: Russ Cox <rsc@golang.org>

We don't use this state. _GCoff means we're sweeping in the
background. This makes it clear in the next commit that _GCoff and
only _GCoff means sweeping.

Change-Id: I416324a829ba0be3794a6cf3cf1655114cb6e47c
Reviewed-on: https://go-review.googlesource.com/11501Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Port of https://golang.org/cl/154360045 to Git.
Original author is Xia Bin <snyh@snyh.org> (already a contributor).

Fixes #8945.

Change-Id: I28bcaf3348794202ca59fbc3466bd7b9670030e4
Reviewed-on: https://go-review.googlesource.com/11658Reviewed-by: Ian Lance Taylor <iant@golang.org>

Currently the runtime fails to clear a G's stack barriers in gfput if
the G's stack allocation is _FixedStack bytes. This causes the runtime
to panic if the following sequence of events happens:

1) The runtime installs stack barriers on a G.

2) The G exits by calling runtime.Goexit. Since this does not
   necessarily return through the stack barriers installed on the G,
   there may still be untriggered stack barriers left on the G's stack
   in recorded in g.stkbar.

3) The runtime calls gfput to add the exiting G to the free pool. If
   the G's stack allocation is _FixedStack bytes, we fail to clear
   g.stkbar.

4) A new G starts and allocates the G that was just added to the free
   pool.

5) The new G begins to execute and overwrites the stack slots that had
   stack barriers in them.

6) The garbage collector enters mark termination, attempts to remove
   stack barriers from the new G, and finds that they've been
   overwritten.

Fix this by clearing the stack barriers in gfput in the case where it
reuses the stack.

Fixes #11256.

Change-Id: I377c44258900e6bcc2d4b3451845814a8eeb2bcf
Reviewed-on: https://go-review.googlesource.com/11461Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Russ Cox <rsc@golang.org>

Updated the protobuf documentation URL (code.google.com deprecated)
to avoid a redirect.

Change-Id: I134f6e4a2bf2bba699942883bf6347bc61700bcb
Reviewed-on: https://go-review.googlesource.com/11634Reviewed-by: Russ Cox <rsc@golang.org>

Also improve the documentation. A prior fix in this release
changed the properties for empty strings and slices, incorrectly.
Previous behavior is now restored and better documented.

Add lots of tests.

The behavior is that when using a string-like format (%s %q %x %X)
a byte slice is equivalent to a string, and printed as a unit. The padding
applies to the entire object. (The space and sharp flags apply
elementwise.)

Fixes #11422.
Fixes #10430.

Change-Id: I758f0521caf71630437e43990ec6d6c9a92655e3
Reviewed-on: https://go-review.googlesource.com/11600Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: Id2cfa63d4c749503f729097654d7cbd2b252f192
Reviewed-on: https://go-review.googlesource.com/11660Reviewed-by: Andrew Gerrand <adg@golang.org>

Change-Id: I4befb21d0811819ce0a5721421a2f6df7a9b62fa
Reviewed-on: https://go-review.googlesource.com/11605Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>

TestTransportAndServerSharedBodyRace got flaky after
issue #9662 was fixed by https://golang.org/cl/11412, which made
servers hang up on clients when a Handler stopped reading its body
early.

This test was affected by a race between the the two goroutines in the
test both only reading part of the request, which was an unnecessary
detail for what the test was trying to test (concurrent Read/Close
races on an *http.body)

Also remove an unused remnant from an old test from which this one was
derived. And make the test not deadlock when it fails. (which was why
the test was showing up as 2m timeouts on the dashboard)

Fixes #11418

Change-Id: Ic83d18aef7e09a9cd56ac15e22ebed75713026cb
Reviewed-on: https://go-review.googlesource.com/11610
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>

Fixes #11417

Change-Id: Iacea829a48b39df0a4f751b06b19e918fbb713d0
Reviewed-on: https://go-review.googlesource.com/11604Reviewed-by: Rob Pike <r@golang.org>

It looks like the test for whether symbols contain subsymbols is wrong.
In particular, symbols in C libraries are mistakenly considered container
symbols.

Fix the test so only symbols which actually have a subsymbol
are excluded from the symtab.  When linking cgo programs the list
of containers is small, something like:

container _/home/khr/sandbox/symtab/misc/cgo/test(.text)<74>
container _/home/khr/sandbox/symtab/misc/cgo/test/issue8828(.text)<75>
container _/home/khr/sandbox/symtab/misc/cgo/test/issue9026(.text)<76>
container runtime/cgo(.text)<77>

I'm not sure this is the right fix.  In particular I can't reproduce
the original problem.  Anyone have a repro they can try and see if
this fix works?

Fixes #10747
Fixes #11396

Change-Id: Id8b016389d33348b4a791fdcba0f9db8ae71ebf3
Reviewed-on: https://go-review.googlesource.com/11652Reviewed-by: Russ Cox <rsc@golang.org>

Invalid UTF-8 triggers an error when marshaling but, previously, not
when unmarshaling. This means that ASN.1 structures were not
round-tripping.

This change makes invalid UTF-8 in a string marked as UTF-8 to be an
error when Unmarshaling.

Fixes #11126.

Change-Id: Ic37be84d21dc5c03983525e244d955a8b1e1ff14
Reviewed-on: https://go-review.googlesource.com/11056
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>