GitLab

This CL reverts CL 76851 and takes a different approach to #21357.
The changes in encode.go and encode_test.go are reverts that
rolls back the changed behavior in CL 76851 where
embedded pointers to unexported struct types were
unilaterally ignored in both marshal and unmarshal.

Instead, these fields are handled as before with the exception that
it returns an error when Unmarshal is unable to set an unexported field.
The behavior of Marshal is now unchanged with regards to #21357.

This policy maintains the greatest degree of backwards compatibility
and avoids silently discarding data the user may have expected to be present.

Fixes #21357

Change-Id: I7dc753280c99f786ac51acf7e6c0246618c8b2b1
Reviewed-on: https://go-review.googlesource.com/82135


Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Update the tzdata database to version 2017c (released 2017-10-20).

Updates #22487

Change-Id: If58ff89173ef1f1044075505f42ff132b57795f0
Reviewed-on: https://go-review.googlesource.com/74230

Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Fixed by CL 76025 yesterday, without realizing it:
the testshared and testplugin builds of separate iface_i
packages were colliding incorrectly in the cache.
Including the build directory fixes that.

Fixes #22571.

Change-Id: Id8193781c67c3150823dc1f48eae781dfe3702fb
Reviewed-on: https://go-review.googlesource.com/76371


Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>

Fixes #22642.

Change-Id: I2ed6305555a0cf753b9cdce061463b1749d5e53e
Reviewed-on: https://go-review.googlesource.com/81975


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

For #23011.

Change-Id: Ibd934f32e41cb8dddbd3d68a324536cf29a97916
Reviewed-on: https://go-review.googlesource.com/82275


Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Change-Id: Ife60468d508f90321733ebdbdb0e6cf443ba1659
Reviewed-on: https://go-review.googlesource.com/82156

Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I8a9e4d00247fe9bda6e19551850bdec0367df2e1
Reviewed-on: https://go-review.googlesource.com/82157

Reviewed-by: Ian Lance Taylor <iant@golang.org>

Change-Id: I97a28379b1a9ca3daa875edbcd5213673ca138d0
Reviewed-on: https://go-review.googlesource.com/82115

Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: Ie246756452cd872961aeb659b35297c229504a3c
Reviewed-on: https://go-review.googlesource.com/81936

Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: Id6ced9a6c76131263041cdaf5ed17479be075fa6
Reviewed-on: https://go-review.googlesource.com/82075


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>

Change-Id: If16f069f410d043fd11864a4c1a7d54b5b4922d2
Reviewed-on: https://go-review.googlesource.com/82200

Reviewed-by: Ian Lance Taylor <iant@golang.org>

There are some basic tests in the packages implementing the hashes,
but this one is meant to be comprehensive for the standard library
as a whole.

Most importantly, it locks in the current representations and makes
sure that they do not change from release to release (and also, as a
result, that future releases can parse the representations generated
by older releases).

The crypto/* MarshalBinary implementations are being changed
in this CL to write only d.x[:d.nx] to the encoding, with zeros for
the remainder of the slice d.x[d.nx:]. The old encoding wrote the
whole d.x, but that exposed an internal detail: whether d.x is
cleared after a full buffer is accumulated, and also whether d.x was
used at all for previous blocks (consider 1-byte writes vs 1024-byte writes).
The new encoding writes only what the decoder needs to know,
nothing more.

In fact the old encodings were arguably also a security hole,
because they exposed data written even before the most recent
call to the Reset method, data that clearly has no impact on the
current hash and clearly should not be exposed. The leakage
is clearly visible in the old crypto/sha1 golden test tables also
being modified in this CL.

Change-Id: I4e9193a3ec5f91d27ce7d0aa24c19b3923741416
Reviewed-on: https://go-review.googlesource.com/82136


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>

String method comments should explain what they do,
not that they are attempting to implement fmt.Stringer.

Change-Id: If51dd1ff2f0c2f9ef9dca569bfa0c3914be2e8fe
Reviewed-on: https://go-review.googlesource.com/82081


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The docs make it seem like they are all things a single object
would implement. That's true of Driver and DriverContext,
but Connector is really something else. Attempt to clarify.

Change-Id: I8fdf1cff855a0fbe37ea22720c082045c719a267
Reviewed-on: https://go-review.googlesource.com/82082


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>

Updates #22961

Change-Id: Ib2f41aefb4f6470598d8637611da5491156ea840
Reviewed-on: https://go-review.googlesource.com/82015

Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Go 1.10 is adding new API MarshalPKCS1PublicKey and
ParsePKCS1PublicKey for converting rsa.PublicKeys.

Even though we'd prefer that users did not, check that
if users call asn1.Marshal and asn1.Unmarshal directly instead,
they get the same results. We know that code exists in the
wild that depends on this.

Change-Id: Ia385d6954fda2eba7da228dc42f229b6839ef11e
Reviewed-on: https://go-review.googlesource.com/82080


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Originally these routines could not fail except by
returning errors from the underlying writer.

Then we realized that header keys containing colons
needed to be rejected, and we started returning an error
from Encode. But that only happens after writing a
partial PEM block to the underlying writer, which is
unfortunate, but at least it was undocumented.

CL 77790 then documented this unfortunate behavior.

Instead of documenting unfortunate behavior, fix it.

Change-Id: Ic7467a576c4cecd16a99138571a1269cc4f96204
Reviewed-on: https://go-review.googlesource.com/82076


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

They do not convert a plural into a singular.
(Introduced recently, in CL 77050.)

Change-Id: I3b6c4d03b1866d4133e90b8ab05e8d4bfbd55125
Reviewed-on: https://go-review.googlesource.com/82078


Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The new example is shorter but illustrates the
interesting parts of the Unix function and methods.

Change-Id: Ief8ec38909d4ed7829e8d3da58e7b7f712537f99
Reviewed-on: https://go-review.googlesource.com/82079


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Historically, DetectContentType has returned "text/plain; charset=utf-8"
for an empty body, there was a test for this, and there should continue
to be one.

CL 46631 changed the content-serving handlers to avoid setting any
Content-Type header when serving empty content. Even if that change
in behavior is correct, the CL is explicitly not changing DetectContentType,
so it must also not change DetectContentType's tests.

Change-Id: I7a19c9fabb43be47e349b40e729e49fceb3f2894
Reviewed-on: https://go-review.googlesource.com/82077


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This fixes a regression where only CRLF was folded into LF at EOF.
Now, we also truncate trailing CR at EOF to preserve the old behavior.

Every one of the test cases added exactly matches the behavior
of Go1.9, even if the results are somewhat unexpected.

Fixes #22937

Change-Id: I1bc6550533163ae489ea77ec1e598163267b7eec
Reviewed-on: https://go-review.googlesource.com/81577


Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The previous implementation of net.Pipe was just a thin wrapper around
io.Pipe and did not wrap any of the io.Pipe errors as net.Errors.
As a result of Hyrum's law, users have come to depend on the fact that
net.Pipe returns io.ErrClosedPipe when the pipe is closed.
Thus, we preserve this behavior to avoid regressing such use cases.

Change-Id: I06b387877b944c1c08527601f58983872b7557b4
Reviewed-on: https://go-review.googlesource.com/81777


Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Sometimes people use run.bash repeatedly
or run go tool dist test by hand for cgo tests.
Avoid test caching in that case, by request.

Refactor code so that all go test commands
share a common prefix.

If not caching is problematic it will be a one-line
change to turn caching back on.

Fixes #22758.

Change-Id: I17d721b832d97bffe26629d21f85b05dbbf2b3ec
Reviewed-on: https://go-review.googlesource.com/80735


Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>

Fixes #23002

Change-Id: I87e72833757497aff49117dd40629cb7ec49e6e7
Reviewed-on: https://go-review.googlesource.com/81955

Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

There are, unfortunately, intermediate CA ceritificates in circulation
that contain the invalid character '&' in some PrintableString fields,
notably Organization Name. This patch allows for ampersand
to be parsed as though it is valid in an ASN.1 PrintableString.

Fixes #22970

Change-Id: Ifab1a10bbff1cdac68e843c6b857ff1a031051aa
Reviewed-on: https://go-review.googlesource.com/81635

Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

When the user context which passed in (*DB)BeginTx is canceled or
timeout, the current implementation could cause db transaction leak
in some extreme scenario.

Goroutine 1:
        Call (*DB) BeginTx begins a transaction with a userContext.
        In (*DB)BeginTx, a new goroutine (*Tx)awaitDone
        which monitor context and rollback tx if needed will be created

Goroutine 2(awaitDone):
        block on tx.ctx.Done()

Goroutine 1:
        Execute some insert or update sqls on the database

Goroutine 1:
        Commit the transaction, (*Tx)Commit set
        the atomic variable tx.done to 1

Goroutine 3(maybe global timer):
        Cancel userContext which be passed in Tx

Goroutine 1:
        (*Tx)Commit checks tx.ctx.Done().
        Due to the context has been canceled, it will return
        context.Canceled or context.DeadlineExceeded error immediately
        and abort the real COMMIT operation of transaction

Goroutine 2:
        Release with tx.ctx.Done() signal, execute (*Tx)rollback.
        However the atomic variable tx.done is 1 currently,
        it will return ErrTxDone error immediately and
        abort the real ROLLBACK operation of transaction

Fixes #22976

Change-Id: I3bc23adf25db823861d91e33d3cca6189fb1171d
Reviewed-on: https://go-review.googlesource.com/81736


Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>

Make sure that when we're assigning to a map, we evaluate the
right-hand side before we attempt to insert into the map.

We used to evaluate the left-hand side to a pointer-to-slot-in-bucket
(which as a side effect does len(m)++), then evaluate the right-hand side,
then do the assignment. That clearly isn't correct when the right-hand side
might panic.

Fixes #22881

Change-Id: I42a62870ff4bf480568c9bdbf0bb18958962bdf0
Reviewed-on: https://go-review.googlesource.com/81817

Reviewed-by: Matthew Dempsky <mdempsky@google.com>

Fixes #22981

Change-Id: I449eb7b5e022401e80a3ab138063e2f4499fbdf8
Reviewed-on: https://go-review.googlesource.com/81855

Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Turn off append-to-itself optimization if optimizations are turned off.

This optimization triggered a bug when doing
  s = append(s, s)
where we write to the leftmost s before reading the rightmost s.

Update #17039

Change-Id: I21996532d20a75db6ec8d49db50cb157a1360b80
Reviewed-on: https://go-review.googlesource.com/81816


Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>

Duplicate cmd/compile check into cmd/go. Manually tested that
"GOEXPERIMENT=fieldtrack make.bash" passes now.

Updates #22223.

Change-Id: I441970a8a5ad4aadf5bd4fbd4d6cc71847b43308
Reviewed-on: https://go-review.googlesource.com/81776


Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

This test was added recently as a regress test for the spec relaxation
in #9060, but doesn't work correctly yet. Disable for now to fix noopt
builders.

Updates #22444.

Change-Id: I45c521ae0da7ffb0c6859d6f7220c59828ac6149
Reviewed-on: https://go-review.googlesource.com/81775


Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

The DWARF inline info generation hooks weren't properly
handling unused auto vars in certain cases, triggering an assert (now
fixed). Also with this change, introduce a new autom "flavor" to
use for autom entries that are added to insure that a specific
auto type makes it into the linker (this is a follow-on to the fix
for 22941).

Fixes #22962.

Change-Id: I7a2d8caf47f6ca897b12acb6a6de0eb25f5cac8f
Reviewed-on: https://go-review.googlesource.com/81557


Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

The iOS test harness dumps the output of its lldb session to stdout,
but only if the lldb session was successfully started.
Make sure the log is always dumpede, so that lldb startup failures
such as

lldb setup error: exited (lldb start: exit status 253)

can be diagnosed.

For the iOS builders.

Change-Id: Ie0e3341dd8f84a88d26509c34816668d3ebbfaa0
Reviewed-on: https://go-review.googlesource.com/76195


Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>

The whole GOROOT/pkg tree is installed using the GOHOSTOS/GOHOSTARCH
toolchain (installed in GOROOT/pkg/tool/GOHOSTOS_GOHOSTARCH).
The testgo.exe we run during the cmd/go test will be built
for GOOS/GOARCH, which means it will use the GOOS/GOARCH toolchain
(installed in GOROOT/pkg/tool/GOOS_GOARCH).

If these are not the same toolchain, then the entire standard library
will look out of date to testgo.exe (the compilers in those two different
tool directories are built for different architectures and have different
buid IDs), which will cause many tests to do unnecessary rebuilds
and some tests to attempt to overwrite the installed standard library,
which will in turn make it look out of date to whatever runs after the
cmd/go test exits.

Bail out entirely in this case instead of destroying the world.

The changes outside TestMain are checks that might have caught
this a bit earlier and made it much less confusing to debug.

Fixes #22709.
Fixes #22965.

Change-Id: Ibf28fa19e29a1f1b8f17875f446d3474dd04a924
Reviewed-on: https://go-review.googlesource.com/81516


Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

If we're using -covermode=atomic with -coverpkg, to add coverage
to more than just the package being tested, then we need to make sure
to make sync/atomic available to the compiler for every package
being recompiled for coverage.

Fixes #22728.

Change-Id: I27f88f6a62e37d4a7455554cd03c8ca2b21f81a4
Reviewed-on: https://go-review.googlesource.com/81497


Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>

This change updates runtime.semasleep to no longer call
runtime.nanotime and instead calls lwp_park with a duration to sleep
relative to the monotonic clock, so the nanotime is never called.
(This requires updating to a newer version of the lwp_park system
call, which is safe, because Go 1.10 will require the unreleased
NetBSD 8+ anyway)

Additionally, this change makes the nanotime function use the
monotonic clock for netbsd/arm, which was forgotten from
https://golang.org/cl/81135 which updated netbsd/amd64 and netbsd/386.

Because semasleep previously depended on nanotime, the past few days
of netbsd have likely been unstable because lwp_park was then mixing
the monotonic and wall clocks. After this CL, lwp_park no longer
depends on nanotime.

Original patch submitted at:
https://www.netbsd.org/~christos/go-lwp-park-clock-monotonic.diff

This commit message (any any mistakes therein) were written by Brad
Fitzpatrick. (Brad migrated the patch to Gerrit and checked CLAs)

Updates #6007
Fixes #22968

Also updates netbsd/arm to use monotonic time for

Change-Id: If77ef7dc610b3025831d84cdfadfbbba2c52acb2
Reviewed-on: https://go-review.googlesource.com/81715


Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Note that Interrupt will compile but not work on Windows.

Fixes #22454

Change-Id: If011c32211f4bb45d458317e113b9794d5b4a4b1
Reviewed-on: https://go-review.googlesource.com/81035

Reviewed-by: Ian Lance Taylor <iant@golang.org>

The full truth seems too complicated to write in this method's doc, so
I'm going with a simple half truth.

The full truth is that Fd returns the descriptor in blocking mode,
because that is historically how it worked, and existing programs
would be surprised if the descriptor is suddenly non-blocking. On Unix
systems whether a file is non-blocking or not is a property of the
underlying file description, not of a particular file descriptor, so
changing the returned descriptor to blocking mode also changes the
existing File to blocking mode. Blocking mode works fine, althoug I/O
operations now take up a thread. SetDeadline and friends rely on the
runtime poller, and the runtime poller only works if the descriptor is
non-blocking. So it's correct that calling Fd disables SetDeadline.
The other half of the truth is that if the program is willing to work
with a non-blocking descriptor, it could call
syscall.SetNonblock(descriptor, true) to change the descriptor, and
the original File, to non-blocking mode. At that point SetDeadline
would start working again. I tried to write that in a way that is
short and comprehensible but failed. Since deadlines mostly work on
pipes, and there isn't much reason to call Fd on a pipe, and few
people use SetDeadline, I decided to punt.

Fixes #22934

Change-Id: I2e49e036f0bcf71f5365193831696f9e4120527c
Reviewed-on: https://go-review.googlesource.com/81636

Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>

Fixes #22958

Change-Id: Ib078a5f6e1105a2afca77c6d9a05f65ddf5d9010
Reviewed-on: https://go-review.googlesource.com/81435


Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>

That Parse doesn't parse ("foo.com/path" or "foo.com:443/path") has
become something of a FAQ.

Updates #19779
Updates #21415
Updates #22955

Change-Id: Ib68efddb67f59b1374e8ed94effd4a326988dee7
Reviewed-on: https://go-review.googlesource.com/81436

Reviewed-by: Ian Lance Taylor <iant@golang.org>