Skip to content

G
go
Switch branch/tag
  1. 04 Nov, 2019 12 commits
  3. 03 Nov, 2019 1 commit
    • Alex Brainman's avatar
      crypto/x509: make '-gcflags=all=-d=checkptr' flag work · d2c039fb
      Alex Brainman authored 
      Replace

buf := [HUGE_CONST]*T)(unsafe.Pointer(p))[:]

with

buf := [HUGE_CONST]*T)(unsafe.Pointer(p))[:n:n]

Pointer p points to n of T elements. New unsafe pointer conversion
logic verifies that both first and last elements point into the
same Go variable. And this change adjusts all code to comply with
this rule.

Verified by running

go test -a -short -gcflags=all=-d=checkptr crypto/x509

The test does not fail even with original version of this code. I
suspect it is because all variables I changed live outside of Go
memory. But I am just guessing, I don't really know how pointer
checker works.

Updates golang/go#34972

Change-Id: Ibc33fdc9e2023d9b14905c9badf2f0b683999ab8
Reviewed-on: https://go-review.googlesource.com/c/go/+/204621
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      d2c039fb
  5. 02 Nov, 2019 16 commits
    • Cherry Zhang's avatar
      runtime: don't async preempt NO_LOCAL_POINTERS assembly functions · 3873e549
      Cherry Zhang authored 
      We don't async preempt assembly functions. We do that by checking
whether the function has a local pointer map, and assume it is
an assembly (or, non-Go) function if there isn't one. However,
assembly functions marked with NO_LOCAL_POINTERS still have local
pointer maps, and we wouldn't identify them. For them, check for
the special pointer map runtime.no_pointers_stackmap as well, and
treat them as not async preemptible.

Change-Id: I1301e3b4d35893c31c4c5a5147a0d775987bd6f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/202337Reviewed-by: default avatarAustin Clements <austin@google.com>
      3873e549
    • Austin Clements's avatar
      runtime: add a test for asynchronous safe points · 7955eceb
      Austin Clements authored 
      This adds a test of preempting a loop containing no synchronous safe
points for STW and stack scanning.

We couldn't add this test earlier because it requires scheduler, STW,
and stack scanning preemption to all be working.

For #10958, #24543.

Change-Id: I73292db78ca3d14aab11bdafd26d03986920ef0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/201777
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      7955eceb
    • Austin Clements's avatar
      runtime: implement async scheduler preemption · 177a36a5
      Austin Clements authored 
      This adds signal-based preemption to preemptone.

Since STW and forEachP ultimately use preemptone, this also makes
these work with async preemption.

This also makes freezetheworld more robust so tracebacks from fatal
panics should be far less likely to report "goroutine running on other
thread; stack unavailable".

For #10958, #24543. (This doesn't fix it yet because asynchronous
preemption only works on POSIX platforms on 386 and amd64 right now.)

Change-Id: If776181dd5a9b3026a7b89a1b5266521b95a5f61
Reviewed-on: https://go-review.googlesource.com/c/go/+/201762
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      177a36a5
    • Austin Clements's avatar
      runtime: use signals to preempt Gs for suspendG · 62e53b79
      Austin Clements authored 
      This adds support for pausing a running G by sending a signal to its
M.

The main complication is that we want to target a G, but can only send
a signal to an M. Hence, the protocol we use is to simply mark the G
for preemption (which we already do) and send the M a "wake up and
look around" signal. The signal checks if it's running a G with a
preemption request and stops it if so in the same way that stack check
preemptions stop Gs. Since the preemption may fail (the G could be
moved or the signal could arrive at an unsafe point), we keep a count
of the number of received preemption signals. This lets stopG detect
if its request failed and should be retried without an explicit
channel back to suspendG.

For #10958, #24543.

Change-Id: I3e1538d5ea5200aeb434374abb5d5fdc56107e53
Reviewed-on: https://go-review.googlesource.com/c/go/+/201760
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      62e53b79
    • Austin Clements's avatar
      runtime: scan stacks conservatively at async safe points · d16ec137
      Austin Clements authored 
      This adds support for scanning the stack when a goroutine is stopped
at an async safe point. This is not yet lit up because asyncPreempt is
not yet injected, but prepares us for that.

This works by conservatively scanning the registers dumped in the
frame of asyncPreempt and its parent frame, which was stopped at an
asynchronous safe point.

Conservative scanning works by only marking words that are pointers to
valid, allocated heap objects. One complication is pointers to stack
objects. In this case, we can't determine if the stack object is still
"allocated" or if it was freed by an earlier GC. Hence, we need to
propagate the conservative-ness of scanning stack objects: if all
pointers found to a stack object were found via conservative scanning,
then the stack object itself needs to be scanned conservatively, since
its pointers may point to dead objects.

For #10958, #24543.

Change-Id: I7ff84b058c37cde3de8a982da07002eaba126fd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/201761
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      d16ec137
    • Austin Clements's avatar
      runtime: asynchronous preemption function for x86 · a3ffb0d9
      Austin Clements authored 
      This adds asynchronous preemption function for amd64 and 386. These
functions spill and restore all register state that can be used by
user Go code.

For the moment we stub out the other arches.

For #10958, #24543.

Change-Id: I6f93fabe9875f4834922a5712362e79045c00aca
Reviewed-on: https://go-review.googlesource.com/c/go/+/201759
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      a3ffb0d9
    • Austin Clements's avatar
      runtime: support for injecting calls at signals on x86 · 2d031dc5
      Austin Clements authored 
      This adds a sigctxt.pushCall method that pushes a call at the signaled
site. We'll use this to inject asynchronous preemptions and in some
places we use it to clean up preparePanic.

For the moment this only works on 386 and amd64. We stub it out on
other platforms and will avoid calling the stubbed version.

For #10958, #24543.

Change-Id: I49e0e853f935d32dd67a70c6cafbae44ee68af8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/201758
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      2d031dc5
    • Austin Clements's avatar
      cmd/compile: fix missing unsafe-points · 61fa7988
      Austin Clements authored 
      Currently, the compiler fails to mark any unsafe-points in the initial
instructions of a function as unsafe points. This happens because
unsafe points are encoded as a stack map index of -2 and the compiler
emits PCDATA instructions when there's a change in the stack map
index, but I had set the initial stack map index to -2. The actual
initial PCDATA value assumed by the PCDATA encoder and the runtime is
-1. Hence, if the first instructions had a stack map index of -2, no
PCDATA was emitted, which cause the runtime to assume the index was -1
instead.

This was particularly problematic in the runtime, where the compiler
was supposed to mark only calls as safe-points and everything else as
unsafe-points. Runtime leaf functions, for example, should have been
marked as entirely unsafe-points, but were instead marked entirely as
safe-points.

Fix this by making the PCDATA instruction generator assume the initial
PCDATA value is -1 instead of -2, so it will emit a PCDATA instruction
right away if the first real instruction is an unsafe-point.

This increases the size of the cmd/go binary by 0.02% since we now
emit slightly more PCDATA than before.

For #10958, #24543.

Change-Id: I92222107f799130072b36d49098d2686f1543699
Reviewed-on: https://go-review.googlesource.com/c/go/+/202084
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      61fa7988
    • Austin Clements's avatar
      runtime: add GODEBUG=asyncpreemptoff=1 · 40b74558
      Austin Clements authored 
      This doesn't do anything yet, but it will provide a way to disable
non-cooperative preemption.

For #10958, #24543.

Change-Id: Ifdef303f103eabd0922ced8d9bebbd5f0aa2cda4
Reviewed-on: https://go-review.googlesource.com/c/go/+/201757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      40b74558
    • Harshavardhana's avatar
      net/http/httputil: fix missing Transfer-Encoding header · bdb5e9d1
      Harshavardhana authored 
      Current implementation of httputil.DumpRequestOut
incorrectly resets the Request.Body prematurely
before Content-Length/Transfer-Encoding detection
in newTransferWriter()

This fix avoids resetting the Request.Body when
Request.ContentLength is set to '0' by the caller
and Request.Body is set to a custom reader. To allow
newTransferWriter() to treat this situation as
'Transfer-Encoding: chunked'.

Fixes #34504

Change-Id: Ieab6bf876ced28c32c084e0f4c8c4432964181f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/197898Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
      bdb5e9d1
    • Alberto Donizetti's avatar
      doc: update ports list description to reflect current status · afe50c11
      Alberto Donizetti authored 
      This change updates the GOARCH/GOOS discussion at the top of the
"Installing Go from source" document to better reflect the current
status. In particular:

- The GOARCH list now focuses on simply listing the supported
architectures, with no notes about their supposed "maturity", since
the same GOARCH can be mature on a GOOS and not so mature on another.

- Outdated notes about some archs being new and "not well-exercised"
have been removed in favour of a following list of which ports are
first class.

- The list of supported OS has been updated (added: AIX, Illumos),
and sorted in alphabetical order.

- A note about the runtime support being the same for all ARCHS,
"including garbage collection and efficient array slicing and" etc etc
has been removed, since it doesn't seem particularly relevant in a
"install from source" instruction page, and it's likely a leftover
from the time this doc page was the landing place for new people and
it felt the need to "sell" Go.

Updates #27689
Fixes #35009

Change-Id: Ic4eca91dca3135adc7bed4fe00b4f157768f0e81
Reviewed-on: https://go-review.googlesource.com/c/go/+/202197Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      afe50c11
    • Keith Randall's avatar
      hash/maphash: move bytes/hash to hash/maphash · 35cfe059
      Keith Randall authored 
      Fixes #34778

Change-Id: If8225a7c41cb2af3f67157fb9670eef86272e85e
Reviewed-on: https://go-review.googlesource.com/c/go/+/204997
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      35cfe059
    • Katie Hockman's avatar
      crypto/dsa: change bitwise checks to mod operations · dc0c23ec
      Katie Hockman authored 
      Even though bitwise operations may be slightly more
performant, the readability improvement of a mod
operation is worth the tradeoff.

Change-Id: I352c92ad355c6eb6ef99e3da00e1eff2d2ea5812
Reviewed-on: https://go-review.googlesource.com/c/go/+/204739Reviewed-by: default avatarFilippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
      dc0c23ec
    • Ian Lance Taylor's avatar
      runtime: clear preemptStop in dropm · 8de0bb77
      Ian Lance Taylor authored 
      Updates #10958
Updates #24543
Fixes #35294

Change-Id: I60f024d08451565df6d9751dab9832b50cbf637a
Reviewed-on: https://go-review.googlesource.com/c/go/+/204957
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      8de0bb77
    • Joshua M. Clulow's avatar
      runtime: check for events when port_getn fails with ETIME · 971ec872
      Joshua M. Clulow authored 
      On illumos systems, and at least historically on Solaris systems, it is
possible for port_getn(3C) calls to return some number of events and
then fail with error ETIME.

Generally we expect this to happen if the caller passes an nget value
larger than 1 and calls with a timeout; if less than the requested
number of events accumulate the system will still return them after
timeout failure so the caller must check the updated nget value in the
ETIME case.  Note that although less likely this can still happen even
when requesting just 1 event, especially with a short timeout value or
on a busy system.

Fixes #35261

Change-Id: I0d83251b69a2fadc64c4e8e280aa596e2e1548ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/204801Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      971ec872
    • Constantin Konstantinidis's avatar
      net/http: deflake TestCancelRequestWithChannelBeforeDo_Cancel · 1e4a3584
      Constantin Konstantinidis authored 
      Goroutines clean up takes longer when using deprecated CloseNotifier.

Fixes #35122

Change-Id: Id820a3012b5c781ddfb294b38ee3b009624e398c
Reviewed-on: https://go-review.googlesource.com/c/go/+/204661
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      1e4a3584
  7. 01 Nov, 2019 11 commits
    • Ian Lance Taylor's avatar
      runtime: use atomic.Cas to change timerRemoved to timerWaiting · dc39be8b
      Ian Lance Taylor authored 
      If multiple goroutines call time.(*Timer).Reset then the timer will go
from timerWaiting to timerDeleted to timerModifying to timerModifiedLater.
The timer can be on a different P, meaning that simultaneously cleantimers
could change it from timerDeleted to timerRemoving to timerRemoved.
If Reset sees timerRemoved, it was doing an atomic.Store of timerWaiting,
meaning that it did not necessarily see the other values set in the timer,
so the timer could appear to be in an inconsistent state. Use atomic.Cas
to avoid that possibility.

Updates #6239
Updates #27707
Fixes #35272

Change-Id: I1d59a13dc4f2ff4af110fc6e032c8c9d59cfc270
Reviewed-on: https://go-review.googlesource.com/c/go/+/204717
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarMichael Knyszek <mknyszek@google.com>
      dc39be8b
    • Cuong Manh Le's avatar
      cmd/compile: revert change to "bound" in typecheckarraylit · 130f3c06
      Cuong Manh Le authored 
      In CL 204617, I intend to make "bound" parameter to have special meaning
in typecheckarraylit, so we can distinguish between type-checks array
literal and slice literal. But we end up with other solution. The CL was
submitted without reverting the "bound" parameter in case of slice
literal.

Technically, it's not harmful, but causes the code harder to read and maintain.

Change-Id: Ia522ccc9a6b8e25d7eaad4aa4957cb4fa18edc60
Reviewed-on: https://go-review.googlesource.com/c/go/+/204618
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarMatthew Dempsky <mdempsky@google.com>
      130f3c06
    • Domas Tamašauskas's avatar
      time: document an alternative way of setting the system's time zone · 9bde9b4d
      Domas Tamašauskas authored 
      User's program was mutating time.Local variable and crashing
itself as a consequence. Instead of documenting that time.Local
variable should not be mutated, recommended way of setting the
system's time zone has been documented.

Fixes #34814

Change-Id: I7781189855c3bf2ea979dfa07f86c283eed27091
Reviewed-on: https://go-review.googlesource.com/c/go/+/200457Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      9bde9b4d
    • Ian Lance Taylor's avatar
      runtime: unlock sched lock when checkdead throws due to a deadlock · 4ff45821
      Ian Lance Taylor authored 
      I was doing some testing with GODEBUG=schedtrace=1,scheddetail=1 and I
noticed that the program hung after a throw with "all goroutines are
asleep". This is because when doing a throw or fatal panic with schedtrace
the panic code does a final schedtrace, which needs to acquire the
scheduler lock. The checkdead function is always called with the scheduler
lock held. So checkdead would throw with the scheduler lock held, then
the panic code would call schedtrace, which would block trying to acquire
the scheduler lock.

This problem will only happen for people debugging the runtime, but
it's easy to avoid by having checkdead unlock the scheduler lock before
it throws. I only did this for the throws that can happen for a normal
program, not for throws that indicate some corruption in the scheduler data.

Change-Id: Ic62277b3ca6bee6f0fca8d5eb516c59cb67855cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/204778
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      4ff45821
    • Michael Anthony Knyszek's avatar
      runtime: turn off scavenger when there's <1 physical page of work · 08a72c85
      Michael Anthony Knyszek authored 
      This change turns off the scavenger if there's less than one physical
page of work to do. If there's less than one phyiscal page of work
today, then the computed time for the work to be done will be zero,
resulting in a floating point division by zero.

This is bad on two accounts. On the one hand it could cause a fault on
some systems. On the other hand, it could cause the pacing computations
done by the scavenger to be nonsense. While this is generally harmless
in the case where there's a very small amount of work to do anyway (the
scavenger might just back off expontentially forever, or do some work
and immediately sleep, because there's not much of it to do), it causes
problems for the deadlock checker. On platforms with a larger physical
page size, such as 64 KiB, we might hit this path in a deadlock
scenario, in which case the deadlock checker will never fire and we'll
just hang.

Specifically, this happens on ppc64 trybot tests, which is where the
issue was discovered.

Fixes #34575.

Change-Id: I8677db539447b2f0e75b8cfcbe33932244e1508c
Reviewed-on: https://go-review.googlesource.com/c/go/+/203517
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarAustin Clements <austin@google.com>
      08a72c85
    • Brad Fitzpatrick's avatar
      make.bash: show bootstrap version · d0160bc3
      Brad Fitzpatrick authored 
      For debugging. (The "go1.4" can be misleading since it might actually
be go1.4.3 or go1.11 or go1.12 or master)

Change-Id: I27520b931a2be018de577a299592d082260aa467
Reviewed-on: https://go-review.googlesource.com/c/go/+/204757
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      d0160bc3
    • Bryan C. Mills's avatar
      cmd/go: adjust module-related logging · 617b4168
      Bryan C. Mills authored 
      Suppress “finding” messages unless they are unusually slow, and
“extracting” messages always (they almost always occur conjunction
with “downloading”, which is already logged).

Log “found” messages for module dependencies added to satisfy missing
import paths.

Log top-level version changes in 'go get' when the selected version
is not identical to the version requested on the command line.

Updates #26152
Updates #33284

Change-Id: I4d0de60fab58d7cc7df8a2aff05c8b5b2220e626
Reviewed-on: https://go-review.googlesource.com/c/go/+/204777
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarJay Conrod <jayconrod@google.com>
      617b4168
    • Lorenz Bauer's avatar
      encoding/binary: cache struct sizes to speed up Read and Write · c9d89f6b
      Lorenz Bauer authored 
      A majority of work is spent in dataSize when en/decoding the same
struct over and over again. This wastes a lot of work, since
the result doesn't change for a given reflect.Value.

Cache the result of the function for structs, so that subsequent
calls to dataSize can avoid doing work.

    name         old time/op    new time/op     delta
    ReadStruct     1.00µs ± 1%     0.37µs ± 1%   -62.99%  (p=0.029 n=4+4)
    WriteStruct    1.00µs ± 3%     0.37µs ± 1%   -62.69%  (p=0.008 n=5+5)

    name         old speed      new speed       delta
    ReadStruct   75.1MB/s ± 1%  202.9MB/s ± 1%  +170.16%  (p=0.029 n=4+4)
    WriteStruct  74.8MB/s ± 3%  200.4MB/s ± 1%  +167.96%  (p=0.008 n=5+5)

Fixes #34471

Change-Id: Ic5d987ca95f1197415ef93643a0af6fc1224fdf0
Reviewed-on: https://go-review.googlesource.com/c/go/+/199539Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
      c9d89f6b
    • Bryan C. Mills's avatar
      cmd/go: default to mod=readonly when the go.mod file is read-only · 2c8529cb
      Bryan C. Mills authored 
      Updates #30185
Updates #33326
Updates #34822

Change-Id: Ie13651585898d1bbbf4f779b97ee50b6c7e7ad50
Reviewed-on: https://go-review.googlesource.com/c/go/+/204521
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: default avatarJay Conrod <jayconrod@google.com>
      2c8529cb
    • Tobias Klauser's avatar
      syscall: add support for linux/riscv64 · 8d45e612
      Tobias Klauser authored 
      Based on the riscv-go port and the linux/riscv64 files in x/sys/unix.

Updates #27532

Change-Id: Ib33a59a61f6b2721b12292c18f1fc9f9d0509cd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/204659
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      8d45e612
    • empijei's avatar
      net/http: clarify Header.WriteSubset behavior · a570fcf7
      empijei authored 
      The current implementation performs a plain map lookup,
but other header methods canonicalize header keys before
using them.

Fixes #34918

Change-Id: Id4120488b8b39ecee97fa7a6ad8a34158687ffcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/201357Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      a570fcf7
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7