1. 05 Nov, 2019 6 commits
  2. 04 Nov, 2019 21 commits
    • Michael Anthony Knyszek's avatar
      runtime: clean up power-of-two rounding code with align functions · 383b447e
      Michael Anthony Knyszek authored
      This change renames the "round" function to the more appropriately named
      "alignUp" which rounds an integer up to the next multiple of a power of
      two.
      
      This change also adds the alignDown function, which is almost like
      alignUp but rounds down to the previous multiple of a power of two.
      
      With these two functions, we also go and replace manual rounding code
      with it where we can.
      
      Change-Id: Ie1487366280484dcb2662972b01b4f7135f72fec
      Reviewed-on: https://go-review.googlesource.com/c/go/+/190618Reviewed-by: default avatarAustin Clements <austin@google.com>
      Reviewed-by: default avatarKeith Randall <khr@golang.org>
      383b447e
    • Brad Fitzpatrick's avatar
      net/http: support disabling built-in HTTP/2 with a new build tag · 2566e21f
      Brad Fitzpatrick authored
      Fixes #35082
      Updates #6853
      
      Change-Id: I4eeb0e15f534cff57fefb6039cd33fadf15b946e
      Reviewed-on: https://go-review.googlesource.com/c/go/+/205139Reviewed-by: default avatarEmmanuel Odeke <emm.odeke@gmail.com>
      Reviewed-by: default avatarDavid Crawshaw <crawshaw@golang.org>
      2566e21f
    • Michael Knyszek's avatar
      runtime: place lower limit on trigger ratio · 74af7fc6
      Michael Knyszek authored
      This change makes it so that the GC pacer's trigger ratio can never fall
      below 0.6. Upcoming changes to the allocator make it significantly more
      scalable and thus much faster in certain cases, creating a large gap
      between the performance of allocation and scanning. The consequence of
      this is that the trigger ratio can drop very low (0.07 was observed) in
      order to drop GC utilization. A low trigger ratio like this results in a
      high amount of black allocations, which causes the live heap to appear
      larger, and thus the heap, and RSS, grows to a much higher stable point.
      
      This change alleviates the problem by placing a lower bound on the
      trigger ratio. The expected (and confirmed) effect of this is that
      utilization in certain scenarios will no longer converge to the expected
      25%, and may go higher. As a result of this artificially high trigger
      ratio, more time will also be spent doing GC assists compared to
      dedicated mark workers, since the GC will be on for an artifically short
      fraction of time (artificial with respect to the pacer). The biggest
      concern of this change is that allocation latency will suffer as a
      result, since there will now be more assists. But, upcoming changes to
      the allocator reduce the latency enough to outweigh the expected
      increase in latency from this change, without the blowup in RSS observed
      from the changes to the allocator.
      
      Updates #35112.
      
      Change-Id: Idd7c94fa974d0de673304c4397e716e89bfbf09b
      Reviewed-on: https://go-review.googlesource.com/c/go/+/200439Reviewed-by: default avatarAustin Clements <austin@google.com>
      74af7fc6
    • Richard Musiol's avatar
      syscall/js: garbage collect references to JavaScript values · 54e6ba67
      Richard Musiol authored
      The js.Value struct now contains a pointer, so a finalizer can
      determine if the value is not referenced by Go any more.
      
      Unfortunately this breaks Go's == operator with js.Value. This change
      adds a new Equal method to check for the equality of two Values.
      This is a breaking change. The == operator is now disallowed to
      not silently break code.
      
      Additionally the helper methods IsUndefined, IsNull and IsNaN got added.
      
      Fixes #35111
      
      Change-Id: I58a50ca18f477bf51a259c668a8ba15bfa76c955
      Reviewed-on: https://go-review.googlesource.com/c/go/+/203600
      Run-TryBot: Richard Musiol <neelance@gmail.com>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      54e6ba67
    • Matthew Dempsky's avatar
      cmd/compile: restore -m=2 diagnostics · 063d0f11
      Matthew Dempsky authored
      This is a rough attempt at restoring -m=2 escape analysis diagnostics
      on par with those that were available with esc.go. It's meant to be
      simple and non-invasive.
      
      For example, given this random example from bytes/reader.go:
      
      138	func (r *Reader) WriteTo(w io.Writer) (n int64, err error) {
      ...
      143	        b := r.s[r.i:]
      144	        m, err := w.Write(b)
      
      esc.go used to report:
      
      bytes/reader.go:138:7: leaking param content: r
      bytes/reader.go:138:7:       from r.s (dot of pointer) at bytes/reader.go:143:8
      bytes/reader.go:138:7:       from b (assigned) at bytes/reader.go:143:4
      bytes/reader.go:138:7:       from w.Write(b) (parameter to indirect call) at bytes/reader.go:144:19
      
      With this CL, escape.go now reports:
      
      bytes/reader.go:138:7: parameter r leaks to {heap} with derefs=1:
      bytes/reader.go:138:7:   flow: b = *r:
      bytes/reader.go:138:7:     from r.s (dot of pointer) at bytes/reader.go:143:8
      bytes/reader.go:138:7:     from r.s[r.i:] (slice) at bytes/reader.go:143:10
      bytes/reader.go:138:7:     from b := r.s[r.i:] (assign) at bytes/reader.go:143:4
      bytes/reader.go:138:7:   flow: {heap} = b:
      bytes/reader.go:138:7:     from w.Write(b) (call parameter) at bytes/reader.go:144:19
      
      Updates #31489.
      
      Change-Id: I0c2b943a0f9ce6345bfff61e1c635172a9290cbb
      Reviewed-on: https://go-review.googlesource.com/c/go/+/196959
      Run-TryBot: Matthew Dempsky <mdempsky@google.com>
      Reviewed-by: default avatarDavid Chase <drchase@google.com>
      063d0f11
    • Michael Munday's avatar
      internal/bytealg: add SIMD byte count implementation for s390x · b6245cef
      Michael Munday authored
      Add a 'single lane' SIMD implemementation of the single byte count
      function for use on machines that support the vector facility. This
      allows up to 16 bytes to be counted per loop iteration.
      
      We can probably improve performance further by adding more 'lanes'
      (i.e. counting more bytes in parallel) however this will increase
      the complexity of the function so I'm not sure it is worth doing
      yet.
      
      name                old speed      new speed       delta
      pkg:strings goos:linux goarch:s390x
      CountByte/10         789MB/s ± 0%   1131MB/s ± 0%    +43.44%  (p=0.000 n=9+9)
      CountByte/32         936MB/s ± 0%   3236MB/s ± 0%   +245.87%  (p=0.000 n=8+9)
      CountByte/4096      1.06GB/s ± 0%  21.26GB/s ± 0%  +1907.07%  (p=0.000 n=10+10)
      CountByte/4194304   1.06GB/s ± 0%  20.54GB/s ± 0%  +1838.50%  (p=0.000 n=10+10)
      CountByte/67108864  1.06GB/s ± 0%  18.31GB/s ± 0%  +1629.51%  (p=0.000 n=10+10)
      pkg:bytes goos:linux goarch:s390x
      CountSingle/10       800MB/s ± 0%    986MB/s ± 0%    +23.21%  (p=0.000 n=9+10)
      CountSingle/32       925MB/s ± 0%   2744MB/s ± 0%   +196.55%  (p=0.000 n=9+10)
      CountSingle/4K      1.26GB/s ± 0%  19.44GB/s ± 0%  +1445.59%  (p=0.000 n=10+10)
      CountSingle/4M      1.26GB/s ± 0%  20.28GB/s ± 0%  +1510.26%  (p=0.000 n=8+10)
      CountSingle/64M     1.23GB/s ± 0%  17.78GB/s ± 0%  +1350.67%  (p=0.000 n=9+10)
      
      Change-Id: I230d57905db92a8fdfc50b1d5be338941ae3a7a1
      Reviewed-on: https://go-review.googlesource.com/c/go/+/199979
      Run-TryBot: Michael Munday <mike.munday@ibm.com>
      Reviewed-by: default avatarKeith Randall <khr@golang.org>
      b6245cef
    • Cherry Zhang's avatar
      cmd/link/internal/ld: fix TestArchiveBuildInvokeWithExec · d9ee4b28
      Cherry Zhang authored
      TestArchiveBuildInvokeWithExec is failing on darwin due to
      duplicated symbols, because the C definition (int fortytwo;) is
      copied to two generated cgo sources. In fact, this test is about
      building c-archive, but doesn't need to import "C". Removed the
      "C" import.
      
      Change-Id: I3a17546e01272a7ae37e6417791ab949fb44597e
      Reviewed-on: https://go-review.googlesource.com/c/go/+/205278
      Run-TryBot: Cherry Zhang <cherryyz@google.com>
      Reviewed-by: default avatarThan McIntosh <thanm@google.com>
      d9ee4b28
    • Ian Lance Taylor's avatar
      runtime: wake netpoller when dropping P, don't sleep too long in sysmon · d80ab3e8
      Ian Lance Taylor authored
      When dropping a P, if it has any timers, and if some thread is
      sleeping in the netpoller, wake the netpoller to run the P's timers.
      This mitigates races between the netpoller deciding how long to sleep
      and a new timer being added.
      
      In sysmon, if all P's are idle, check the timers to decide how long to sleep.
      This avoids oversleeping if no thread is using the netpoller.
      This can happen in particular if some threads use runtime.LockOSThread,
      as those threads do not block in the netpoller.
      
      Also, print the number of timers per P for GODEBUG=scheddetail=1.
      
      Before this CL, TestLockedDeadlock2 would fail about 1% of the time.
      With this CL, I ran it 150,000 times with no failures.
      
      Updates #6239
      Updates #27707
      Fixes #35274
      Fixes #35288
      
      Change-Id: I7e5193e6c885e567f0b1ee023664aa3e2902fcd1
      Reviewed-on: https://go-review.googlesource.com/c/go/+/204800
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarMichael Knyszek <mknyszek@google.com>
      d80ab3e8
    • Russ Cox's avatar
      hash/maphash: revise API to be more idiomatic · 5a7c571e
      Russ Cox authored
      This CL makes these changes to the hash/maphash API to make it fit a bit
      more into the standard library:
      
       - Move some of the package doc onto type Hash, so that `go doc maphash.Hash` shows it.
      
       - Instead of having identical AddBytes and Write methods,
         standardize on Write, the usual name for this function.
         Similarly, AddString -> WriteString, AddByte -> WriteByte.
      
       - Instead of having identical Hash and Sum64 methods,
         standardize on Sum64 (for hash.Hash64). Dropping the "Hash" method
         also helps because Hash is usually reserved to mean the state of a
         hash function (hash.Hash etc), not the hash value itself.
      
       - Make an uninitialized hash.Hash auto-seed with a random seed.
         It is critical that users not use the same seed for all hash functions
         in their program, at least not accidentally. So the Hash implementation
         must either panic if uninitialized or initialize itself.
         Initializing itself is less work for users and can be done lazily.
      
       - Now that the zero hash.Hash is useful, drop maphash.New in favor of
         new(maphash.Hash) or simply declaring a maphash.Hash.
      
       - Add a [0]func()-typed field to the Hash so that Hashes cannot be compared.
         (I considered doing the same for Seed but comparing seeds seems OK.)
      
       - Drop the integer argument from MakeSeed, to match the original design
         in golang.org/issue/28322. There is no point to giving users control
         over the specific seed bits, since we want the interpretation of those
         bits to be different in every different process. The only thing users
         need is to be able to create a new random seed at each call.
         (Fixes a TODO in MakeSeed's public doc comment.)
      
      This API is new in Go 1.14, so these changes do not violate the compatibility promise.
      
      Fixes #35060.
      Fixes #35348.
      
      Change-Id: Ie6fecc441f3f5ef66388c6ead92e875c0871f805
      Reviewed-on: https://go-review.googlesource.com/c/go/+/205069
      Run-TryBot: Russ Cox <rsc@golang.org>
      Reviewed-by: default avatarAlan Donovan <adonovan@google.com>
      Reviewed-by: default avatarKeith Randall <khr@golang.org>
      5a7c571e
    • Filippo Valsorda's avatar
      crypto/tls: make -update only regenerate transcripts for failing tests · 03aca99f
      Filippo Valsorda authored
      Change-Id: Ie68fd4fe2879e6b5417a1a4240971e3d837bf115
      Reviewed-on: https://go-review.googlesource.com/c/go/+/204377
      Run-TryBot: Filippo Valsorda <filippo@golang.org>
      Run-TryBot: Katie Hockman <katie@golang.org>
      Reviewed-by: default avatarKatie Hockman <katie@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      03aca99f
    • Michael Munday's avatar
      cmd/compile: absorb more Not ops into Neq* and Eq* ops · bb7890b8
      Michael Munday authored
      We absorbed Not into most integer comparisons but not into pointer
      and floating point equality checks.
      
      The new cases trigger more than 300 times during make.bash.
      
      Change-Id: I77c6b31fcacde10da5470b73fc001a19521ce78d
      Reviewed-on: https://go-review.googlesource.com/c/go/+/200618
      Run-TryBot: Michael Munday <mike.munday@ibm.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarKeith Randall <khr@golang.org>
      bb7890b8
    • Cuong Manh Le's avatar
      cmd/compile: add test for skipping empty init functions · 26d5f032
      Cuong Manh Le authored
      CL 200958 adds skipping empty init function feature without any tests
      for it. A codegen test sounds ideal, but it's unlikely that we can make
      one for now, so use a program to manipulate runtime/proc.go:initTask
      directly.
      
      Updates #34869
      
      Change-Id: I2683b9a1ace36af6861af02a3a9fb18b3110b282
      Reviewed-on: https://go-review.googlesource.com/c/go/+/204217
      Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
      Reviewed-by: default avatarMatthew Dempsky <mdempsky@google.com>
      Reviewed-by: default avatarKeith Randall <khr@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      26d5f032
    • Wang Xuerui's avatar
      runtime: use vDSO clock_gettime on linux/mips64x · 210e3677
      Wang Xuerui authored
      Speed up nanotime1 and walltime1 on MIPS64 with vDSO, just like the
      other vDSO-enabled targets.
      
      Benchmark numbers on Loongson 3A3000 (GOARCH=mips64le, 1.4GHz) against
      current master:
      
      benchmark                old ns/op     new ns/op     delta
      BenchmarkNow             868           293           -66.24%
      BenchmarkNowUnixNano     851           296           -65.22%
      
      Performance hit on fallback case, tested by using a wrong vDSO symbol name:
      
      benchmark                old ns/op     new ns/op     delta
      BenchmarkNow             868           889           +2.42%
      BenchmarkNowUnixNano     851           893           +4.94%
      
      Change-Id: Ibfb48893cd060536359863ffee7624c00def646b
      GitHub-Last-Rev: 03a58ac2e4e036a4f61227cfd013082871e92863
      GitHub-Pull-Request: golang/go#35181
      Reviewed-on: https://go-review.googlesource.com/c/go/+/203578
      Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      210e3677
    • Than McIntosh's avatar
      cmd/link: directly exec archive command if external tmpdir · bf7e55b6
      Than McIntosh authored
      When linking a Go archive, if the archiver invocation is the very last
      thing that needs to happen in the link (no "atexit" cleanups required
      remove the locally created tmpdir) then call syscall.Exec to invoke
      the archiver command instead of the usual exec.Command. This has the
      effect of reducing peak memory use for the linker overall, since we
      don't be holding onto all of the linker's live memory while the
      archiver is running.
      
      Change-Id: Ibbe22d8d67a70cc2a4f91c68aab56d19fb77c393
      Reviewed-on: https://go-review.googlesource.com/c/go/+/203821
      Run-TryBot: Than McIntosh <thanm@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      bf7e55b6
    • Cherry Zhang's avatar
      cmd/internal/obj/s390x: add support of SPM instruction · 1da575a7
      Cherry Zhang authored
      For restoring condition code (we already support IPM instruction
      for saving condition code).
      
      Change-Id: I56d376df44a5f831134a130d052521cec6b5b781
      Reviewed-on: https://go-review.googlesource.com/c/go/+/204104Reviewed-by: default avatarMichael Munday <mike.munday@ibm.com>
      1da575a7
    • Dan Scales's avatar
      runtime: ensure that Goexit cannot be aborted by a recursive panic/recover · 7dcd343e
      Dan Scales authored
      When we do a successful recover of a panic, we resume normal execution by
      returning from the frame that had the deferred call that did the recover (after
      executing any remaining deferred calls in that frame).
      
      However, suppose we have called runtime.Goexit and there is a panic during one of the
      deferred calls run by the Goexit. Further assume that there is a deferred call in
      the frame of the Goexit or a parent frame that does a recover. Then the recovery
      process will actually resume normal execution above the Goexit frame and hence
      abort the Goexit.  We will not terminate the thread as expected, but continue
      running in the frame above the Goexit.
      
      To fix this, we explicitly create a _panic object for a Goexit call. We then
      change the "abort" behavior for Goexits, but not panics. After a recovery, if the
      top-level panic is actually a Goexit that is marked to be aborted, then we return
      to the Goexit defer-processing loop, so that the Goexit is not actually aborted.
      
      Actual code changes are just panic.go, runtime2.go, and funcid.go. Adjusted the
      test related to the new Goexit behavior (TestRecoverBeforePanicAfterGoexit) and
      added several new tests of aborted panics (whose behavior has not changed).
      
      Fixes #29226
      
      Change-Id: Ib13cb0074f5acc2567a28db7ca6912cfc47eecb5
      Reviewed-on: https://go-review.googlesource.com/c/go/+/200081
      Run-TryBot: Dan Scales <danscales@google.com>
      Reviewed-by: default avatarKeith Randall <khr@golang.org>
      7dcd343e
    • Ruixin(Peter) Bao's avatar
      cmd/asm/internal/asm/testdata/s390x: add test cases for some assembly instructions · a8fc82f7
      Ruixin(Peter) Bao authored
      From CL 199979, I noticed that there were some
      instructions not covered by the test cases. Added those in this CL.
      
      Additional tests for assembly instructions are also added
      based on suggestions made during the review of this CL.
      
      Previously, VSB and VSH are not included in asmz.go, they were also
      added in this patch.
      
      Change-Id: I6060a9813b483a161d61ad2240c30eec6de61536
      Reviewed-on: https://go-review.googlesource.com/c/go/+/203721Reviewed-by: default avatarMichael Munday <mike.munday@ibm.com>
      a8fc82f7
    • Cherry Zhang's avatar
      cmd/link: enable PIE on darwin/arm · d3660e8e
      Cherry Zhang authored
      We used to pass -no_pie to external linker on darwin/arm, which
      is incompatible with -fembed-bitcode. CL 201358 attempted to
      remove the -no_pie flag, but it resulted the darwin linker to
      complain about absolute addressing in TEXT segment.
      
      On darwin/arm, we already get away from absolute addressing in
      the TEXT section. The complained absolute addressing is in
      RODATA, which was embedded in the TEXT segment. This CL moves
      RODATA to the DATA segment, like what we already did on ARM64
      and on AMD64 in c-archive/c-shared buildmodes for the same reason.
      So there is no absolute addressing in the TEXT segment, which
      allows us to remove -no_pie flag.
      
      Fixes #35252.
      Updates #32963.
      
      Change-Id: Id6e3a594cb066d257d4f58fadb4a3ee4672529f7
      Reviewed-on: https://go-review.googlesource.com/c/go/+/205060Reviewed-by: default avatarElias Naur <mail@eliasnaur.com>
      d3660e8e
    • Roger Peppe's avatar
      testing: implement Cleanup method · 1b2ff101
      Roger Peppe authored
      Fixes #32111
      
      Change-Id: I7078947889d1e126d9679fb28f27b3fa6ce133ef
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201359Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      1b2ff101
    • Ian Lance Taylor's avatar
      misc/cgo/testcarchive: add missing exit(0) in main7.c · fb29e225
      Ian Lance Taylor authored
      Fixes #35327
      
      Change-Id: I3726bfad24851a0bef8891014f7c5a7c48352307
      Reviewed-on: https://go-review.googlesource.com/c/go/+/205077
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      fb29e225
    • Carlos Eduardo Seo's avatar
      cmd/internal/obj/ppc64: add support for DQ-form instructions · 6b67f7d6
      Carlos Eduardo Seo authored
      POWER9 (ISA 3.0) introduced a new format of load/store instructions to
      implement indexed load/store quadword, using an immediate value instead
      of a register index.
      
      This change adds support for this new instruction encoding and adds the
      new load/store quadword instructions (lxv/stxv) to the assembler.
      
      This change also adds the missing XX1-form loads/stores (halfword and byte)
      included in ISA 3.0.
      
      Change-Id: Ibcdf53c342d7a352d64a9403c2fe7b25be9c3b24
      Reviewed-on: https://go-review.googlesource.com/c/go/+/200399
      Run-TryBot: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarLynn Boger <laboger@linux.vnet.ibm.com>
      6b67f7d6
  3. 03 Nov, 2019 1 commit
    • Alex Brainman's avatar
      crypto/x509: make '-gcflags=all=-d=checkptr' flag work · d2c039fb
      Alex Brainman authored
      Replace
      
      buf := [HUGE_CONST]*T)(unsafe.Pointer(p))[:]
      
      with
      
      buf := [HUGE_CONST]*T)(unsafe.Pointer(p))[:n:n]
      
      Pointer p points to n of T elements. New unsafe pointer conversion
      logic verifies that both first and last elements point into the
      same Go variable. And this change adjusts all code to comply with
      this rule.
      
      Verified by running
      
      go test -a -short -gcflags=all=-d=checkptr crypto/x509
      
      The test does not fail even with original version of this code. I
      suspect it is because all variables I changed live outside of Go
      memory. But I am just guessing, I don't really know how pointer
      checker works.
      
      Updates golang/go#34972
      
      Change-Id: Ibc33fdc9e2023d9b14905c9badf2f0b683999ab8
      Reviewed-on: https://go-review.googlesource.com/c/go/+/204621
      Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
      Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      d2c039fb
  4. 02 Nov, 2019 12 commits
    • Cherry Zhang's avatar
      runtime: don't async preempt NO_LOCAL_POINTERS assembly functions · 3873e549
      Cherry Zhang authored
      We don't async preempt assembly functions. We do that by checking
      whether the function has a local pointer map, and assume it is
      an assembly (or, non-Go) function if there isn't one. However,
      assembly functions marked with NO_LOCAL_POINTERS still have local
      pointer maps, and we wouldn't identify them. For them, check for
      the special pointer map runtime.no_pointers_stackmap as well, and
      treat them as not async preemptible.
      
      Change-Id: I1301e3b4d35893c31c4c5a5147a0d775987bd6f4
      Reviewed-on: https://go-review.googlesource.com/c/go/+/202337Reviewed-by: default avatarAustin Clements <austin@google.com>
      3873e549
    • Austin Clements's avatar
      runtime: add a test for asynchronous safe points · 7955eceb
      Austin Clements authored
      This adds a test of preempting a loop containing no synchronous safe
      points for STW and stack scanning.
      
      We couldn't add this test earlier because it requires scheduler, STW,
      and stack scanning preemption to all be working.
      
      For #10958, #24543.
      
      Change-Id: I73292db78ca3d14aab11bdafd26d03986920ef0a
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201777
      Run-TryBot: Austin Clements <austin@google.com>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      7955eceb
    • Austin Clements's avatar
      runtime: implement async scheduler preemption · 177a36a5
      Austin Clements authored
      This adds signal-based preemption to preemptone.
      
      Since STW and forEachP ultimately use preemptone, this also makes
      these work with async preemption.
      
      This also makes freezetheworld more robust so tracebacks from fatal
      panics should be far less likely to report "goroutine running on other
      thread; stack unavailable".
      
      For #10958, #24543. (This doesn't fix it yet because asynchronous
      preemption only works on POSIX platforms on 386 and amd64 right now.)
      
      Change-Id: If776181dd5a9b3026a7b89a1b5266521b95a5f61
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201762
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      177a36a5
    • Austin Clements's avatar
      runtime: use signals to preempt Gs for suspendG · 62e53b79
      Austin Clements authored
      This adds support for pausing a running G by sending a signal to its
      M.
      
      The main complication is that we want to target a G, but can only send
      a signal to an M. Hence, the protocol we use is to simply mark the G
      for preemption (which we already do) and send the M a "wake up and
      look around" signal. The signal checks if it's running a G with a
      preemption request and stops it if so in the same way that stack check
      preemptions stop Gs. Since the preemption may fail (the G could be
      moved or the signal could arrive at an unsafe point), we keep a count
      of the number of received preemption signals. This lets stopG detect
      if its request failed and should be retried without an explicit
      channel back to suspendG.
      
      For #10958, #24543.
      
      Change-Id: I3e1538d5ea5200aeb434374abb5d5fdc56107e53
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201760
      Run-TryBot: Austin Clements <austin@google.com>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      62e53b79
    • Austin Clements's avatar
      runtime: scan stacks conservatively at async safe points · d16ec137
      Austin Clements authored
      This adds support for scanning the stack when a goroutine is stopped
      at an async safe point. This is not yet lit up because asyncPreempt is
      not yet injected, but prepares us for that.
      
      This works by conservatively scanning the registers dumped in the
      frame of asyncPreempt and its parent frame, which was stopped at an
      asynchronous safe point.
      
      Conservative scanning works by only marking words that are pointers to
      valid, allocated heap objects. One complication is pointers to stack
      objects. In this case, we can't determine if the stack object is still
      "allocated" or if it was freed by an earlier GC. Hence, we need to
      propagate the conservative-ness of scanning stack objects: if all
      pointers found to a stack object were found via conservative scanning,
      then the stack object itself needs to be scanned conservatively, since
      its pointers may point to dead objects.
      
      For #10958, #24543.
      
      Change-Id: I7ff84b058c37cde3de8a982da07002eaba126fd6
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201761
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      d16ec137
    • Austin Clements's avatar
      runtime: asynchronous preemption function for x86 · a3ffb0d9
      Austin Clements authored
      This adds asynchronous preemption function for amd64 and 386. These
      functions spill and restore all register state that can be used by
      user Go code.
      
      For the moment we stub out the other arches.
      
      For #10958, #24543.
      
      Change-Id: I6f93fabe9875f4834922a5712362e79045c00aca
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201759
      Run-TryBot: Austin Clements <austin@google.com>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      a3ffb0d9
    • Austin Clements's avatar
      runtime: support for injecting calls at signals on x86 · 2d031dc5
      Austin Clements authored
      This adds a sigctxt.pushCall method that pushes a call at the signaled
      site. We'll use this to inject asynchronous preemptions and in some
      places we use it to clean up preparePanic.
      
      For the moment this only works on 386 and amd64. We stub it out on
      other platforms and will avoid calling the stubbed version.
      
      For #10958, #24543.
      
      Change-Id: I49e0e853f935d32dd67a70c6cafbae44ee68af8e
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201758
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      2d031dc5
    • Austin Clements's avatar
      cmd/compile: fix missing unsafe-points · 61fa7988
      Austin Clements authored
      Currently, the compiler fails to mark any unsafe-points in the initial
      instructions of a function as unsafe points. This happens because
      unsafe points are encoded as a stack map index of -2 and the compiler
      emits PCDATA instructions when there's a change in the stack map
      index, but I had set the initial stack map index to -2. The actual
      initial PCDATA value assumed by the PCDATA encoder and the runtime is
      -1. Hence, if the first instructions had a stack map index of -2, no
      PCDATA was emitted, which cause the runtime to assume the index was -1
      instead.
      
      This was particularly problematic in the runtime, where the compiler
      was supposed to mark only calls as safe-points and everything else as
      unsafe-points. Runtime leaf functions, for example, should have been
      marked as entirely unsafe-points, but were instead marked entirely as
      safe-points.
      
      Fix this by making the PCDATA instruction generator assume the initial
      PCDATA value is -1 instead of -2, so it will emit a PCDATA instruction
      right away if the first real instruction is an unsafe-point.
      
      This increases the size of the cmd/go binary by 0.02% since we now
      emit slightly more PCDATA than before.
      
      For #10958, #24543.
      
      Change-Id: I92222107f799130072b36d49098d2686f1543699
      Reviewed-on: https://go-review.googlesource.com/c/go/+/202084
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarCherry Zhang <cherryyz@google.com>
      61fa7988
    • Austin Clements's avatar
      runtime: add GODEBUG=asyncpreemptoff=1 · 40b74558
      Austin Clements authored
      This doesn't do anything yet, but it will provide a way to disable
      non-cooperative preemption.
      
      For #10958, #24543.
      
      Change-Id: Ifdef303f103eabd0922ced8d9bebbd5f0aa2cda4
      Reviewed-on: https://go-review.googlesource.com/c/go/+/201757
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      40b74558
    • Harshavardhana's avatar
      net/http/httputil: fix missing Transfer-Encoding header · bdb5e9d1
      Harshavardhana authored
      Current implementation of httputil.DumpRequestOut
      incorrectly resets the Request.Body prematurely
      before Content-Length/Transfer-Encoding detection
      in newTransferWriter()
      
      This fix avoids resetting the Request.Body when
      Request.ContentLength is set to '0' by the caller
      and Request.Body is set to a custom reader. To allow
      newTransferWriter() to treat this situation as
      'Transfer-Encoding: chunked'.
      
      Fixes #34504
      
      Change-Id: Ieab6bf876ced28c32c084e0f4c8c4432964181f5
      Reviewed-on: https://go-review.googlesource.com/c/go/+/197898Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      bdb5e9d1
    • Alberto Donizetti's avatar
      doc: update ports list description to reflect current status · afe50c11
      Alberto Donizetti authored
      This change updates the GOARCH/GOOS discussion at the top of the
      "Installing Go from source" document to better reflect the current
      status. In particular:
      
      - The GOARCH list now focuses on simply listing the supported
      architectures, with no notes about their supposed "maturity", since
      the same GOARCH can be mature on a GOOS and not so mature on another.
      
      - Outdated notes about some archs being new and "not well-exercised"
      have been removed in favour of a following list of which ports are
      first class.
      
      - The list of supported OS has been updated (added: AIX, Illumos),
      and sorted in alphabetical order.
      
      - A note about the runtime support being the same for all ARCHS,
      "including garbage collection and efficient array slicing and" etc etc
      has been removed, since it doesn't seem particularly relevant in a
      "install from source" instruction page, and it's likely a leftover
      from the time this doc page was the landing place for new people and
      it felt the need to "sell" Go.
      
      Updates #27689
      Fixes #35009
      
      Change-Id: Ic4eca91dca3135adc7bed4fe00b4f157768f0e81
      Reviewed-on: https://go-review.googlesource.com/c/go/+/202197Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      afe50c11
    • Keith Randall's avatar
      hash/maphash: move bytes/hash to hash/maphash · 35cfe059
      Keith Randall authored
      Fixes #34778
      
      Change-Id: If8225a7c41cb2af3f67157fb9670eef86272e85e
      Reviewed-on: https://go-review.googlesource.com/c/go/+/204997
      Run-TryBot: Keith Randall <khr@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      35cfe059