1. 27 May, 2016 15 commits
    • Austin Clements's avatar
      runtime: always call stackfree on the system stack · 6a86dbe7
      Austin Clements authored
      Currently when the garbage collector frees stacks of dead goroutines
      in markrootFreeGStacks, it calls stackfree on a regular user stack.
      This is a problem, since stackfree manipulates the stack cache in the
      per-P mcache, so if it grows the stack or gets preempted in the middle
      of manipulating the stack cache (which are both possible since it's on
      a user stack), it can easily corrupt the stack cache.
      
      Fix this by calling markrootFreeGStacks on the system stack, so that
      all calls to stackfree happen on the system stack. To prevent this bug
      in the future, mark stack functions that manipulate the mcache as
      go:systemstack.
      
      Fixes #15853.
      
      Change-Id: Ic0d1c181efb342f134285a152560c3a074f14a3d
      Reviewed-on: https://go-review.googlesource.com/23511
      Run-TryBot: Austin Clements <austin@google.com>
      Reviewed-by: default avatarKeith Randall <khr@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      6a86dbe7
    • Austin Clements's avatar
      runtime: record Python stack on TestGdbPython failure · 966baedf
      Austin Clements authored
      For #15599.
      
      Change-Id: Icc2e58a3f314b7a098d78fe164ba36f5b2897de6
      Reviewed-on: https://go-review.googlesource.com/23481
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      966baedf
    • Russ Cox's avatar
      crypto/tls: adjust dynamic record sizes to grow arithmetically · fa3543e3
      Russ Cox authored
      The current code, introduced after Go 1.6 to improve latency on
      low-bandwidth connections, sends 1 kB packets until 1 MB has been sent,
      and then sends 16 kB packets (the maximum record size).
      
      Unfortunately this decreases throughput for 1-16 MB responses by 20% or so.
      
      Following discussion on #15713, change cutoff to 128 kB sent
      and also grow the size allowed for successive packets:
      1 kB, 2 kB, 3 kB, ..., 15 kB, 16 kB.
      This fixes the throughput problems: the overhead is now closer to 2%.
      
      I hope this still helps with latency but I don't have a great way to test it.
      At the least, it's not worse than Go 1.6.
      
      Comparing MaxPacket vs DynamicPacket benchmarks:
      
      name              maxpkt time/op  dyn. time/op delta
      Throughput/1MB-8    5.07ms ± 7%   5.21ms ± 7%  +2.73%  (p=0.023 n=16+16)
      Throughput/2MB-8   15.7ms ±201%    8.4ms ± 5%    ~     (p=0.604 n=20+16)
      Throughput/4MB-8    14.3ms ± 1%   14.5ms ± 1%  +1.53%  (p=0.000 n=16+16)
      Throughput/8MB-8    26.6ms ± 1%   26.8ms ± 1%  +0.47%  (p=0.003 n=19+18)
      Throughput/16MB-8   51.0ms ± 1%   51.3ms ± 1%  +0.47%  (p=0.000 n=20+20)
      Throughput/32MB-8    100ms ± 1%    100ms ± 1%  +0.24%  (p=0.033 n=20+20)
      Throughput/64MB-8    197ms ± 0%    198ms ± 0%  +0.56%   (p=0.000 n=18+7)
      
      The small MB runs are bimodal in both cases, probably GC pauses.
      But there's clearly no general slowdown anymore.
      
      Fixes #15713.
      
      Change-Id: I5fc44680ba71812d24baac142bceee0e23f2e382
      Reviewed-on: https://go-review.googlesource.com/23487Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      fa3543e3
    • Russ Cox's avatar
      doc/go1.7.html: fix broken sentence · 52c2db7e
      Russ Cox authored
      Change-Id: Ia540c890767dcb001d3b3b55d98d9517b13b21da
      Reviewed-on: https://go-review.googlesource.com/23510Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      52c2db7e
    • Russ Cox's avatar
      net/http: change Transport.Dialer to Transport.DialContext · 605e751b
      Russ Cox authored
      New in Go 1.7 so still possible to change.
      This allows implementations not tied to *net.Dialer.
      
      Fixes #15748.
      
      Change-Id: I5fabbf13c7f1951c06587a4ccd120def488267ce
      Reviewed-on: https://go-review.googlesource.com/23489Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      605e751b
    • Russ Cox's avatar
      cmd/compile: clean up, document Node closure fields · 36a80c59
      Russ Cox authored
      Requested during CL 23431.
      
      Change-Id: I513ae42166b3a9fcfe51231ff55c163ab672e7d2
      Reviewed-on: https://go-review.googlesource.com/23485Reviewed-by: default avatarDavid Chase <drchase@google.com>
      36a80c59
    • Russ Cox's avatar
      cmd/compile: delete Func.Outer · 93369001
      Russ Cox authored
      This was just storage for a linked list.
      
      Change-Id: I850e8db1e1f5e72410f5c904be9409179b56a94a
      Reviewed-on: https://go-review.googlesource.com/23484Reviewed-by: default avatarDavid Chase <drchase@google.com>
      93369001
    • Russ Cox's avatar
      doc/go1.7.html: incorporate Rob's comments from CL 23379 · cedc7c8f
      Russ Cox authored
      For #15810.
      
      Change-Id: Ib529808f664392feb9b36770f3d3d875fcb54528
      Reviewed-on: https://go-review.googlesource.com/23488
      Run-TryBot: Russ Cox <rsc@golang.org>
      Reviewed-by: default avatarRob Pike <r@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      cedc7c8f
    • Emmanuel Odeke's avatar
      doc/go1.7: document signal name printing during panics · 65dd0819
      Emmanuel Odeke authored
      Document new behavior about signal name printing
      in panics as per CL golang.org/cl/22753.
      
      For #15810
      
      Change-Id: I9c677d5dd779b41e82afa25e3c797d8e739600d3
      Reviewed-on: https://go-review.googlesource.com/23493Reviewed-by: default avatarRuss Cox <rsc@golang.org>
      65dd0819
    • Russ Cox's avatar
      cmd/compile: additional paranoia and checking in plive.go · dec1bae9
      Russ Cox authored
      The main check here is that liveness now crashes if it finds an instruction
      using a variable that should be tracked but is not.
      
      Comments and adjustments in nodarg to explain what's going on and
      to remove the "-1" argument added a few months ago, plus a sketch
      of a future simplification.
      
      The need for n.Orig in the earlier CL seems to have been an intermediate
      problem rather than fundamental: the new explanations in nodarg make
      clear that nodarg is not causing the problem I thought, and in fact now
      using n instead of n.Orig works fine in plive.go.
      
      Change-Id: I3f5cf9f6e4438a6d27abac7d490e7521545cd552
      Reviewed-on: https://go-review.googlesource.com/23450
      Run-TryBot: Russ Cox <rsc@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarDavid Chase <drchase@google.com>
      dec1bae9
    • Dmitri Shuralyov's avatar
      cmd/go: fixup for parsing SCP-like addresses · e9228dd9
      Dmitri Shuralyov authored
      This is a fixup change for commit 5cd29448
      that added parsing of SCP-like addresses. To get the expected output
      from (*url.URL).String(), Path needs to be set, not RawPath.
      
      Add a test for this, since it has already regressed multiple times.
      
      Updates #11457.
      
      Change-Id: I806f5abbd3cf65e5bdcef01aab872caa8a5b8891
      Reviewed-on: https://go-review.googlesource.com/23447
      Run-TryBot: Andrew Gerrand <adg@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarAndrew Gerrand <adg@golang.org>
      e9228dd9
    • Russ Cox's avatar
      cmd/compile: eliminate PPARAMREF · 20803b84
      Russ Cox authored
      As in the elimination of PHEAP|PPARAM in CL 23393,
      this is something the front end can trivially take care of
      and then not bother the back ends with.
      It also eliminates some suspect (and only lightly exercised)
      code paths in the back ends.
      
      I don't have a smoking gun for this one but it seems
      more clearly correct.
      
      Change-Id: I3b3f5e669b3b81d091ff1e2fb13226a6f14c69d5
      Reviewed-on: https://go-review.googlesource.com/23431Reviewed-by: default avatarKeith Randall <khr@golang.org>
      Run-TryBot: Russ Cox <rsc@golang.org>
      20803b84
    • Russ Cox's avatar
      cmd/compile: fix liveness computation for heap-escaped parameters · b6dc3e6f
      Russ Cox authored
      The liveness computation of parameters generally was never
      correct, but forcing all parameters to be live throughout the
      function covered up that problem. The new SSA back end is
      too clever: even though it currently keeps the parameter values live
      throughout the function, it may find optimizations that mean
      the current values are not written back to the original parameter
      stack slots immediately or ever (for example if a parameter is set
      to nil, SSA constant propagation may replace all later uses of the
      parameter with a constant nil, eliminating the need to write the nil
      value back to the stack slot), so the liveness code must now
      track the actual operations on the stack slots, exposing these
      problems.
      
      One small problem in the handling of arguments is that nodarg
      can return ONAME PPARAM nodes with adjusted offsets, so that
      there are actually multiple *Node pointers for the same parameter
      in the instruction stream. This might be possible to correct, but
      not in this CL. For now, we fix this by using n.Orig instead of n
      when considering PPARAM and PPARAMOUT nodes.
      
      The major problem in the handling of arguments is general
      confusion in the liveness code about the meaning of PPARAM|PHEAP
      and PPARAMOUT|PHEAP nodes, especially as contrasted with PAUTO|PHEAP.
      The difference between these two is that when a local variable "moves"
      to the heap, it's really just allocated there to start with; in contrast,
      when an argument moves to the heap, the actual data has to be copied
      there from the stack at the beginning of the function, and when a
      result "moves" to the heap the value in the heap has to be copied
      back to the stack when the function returns
      This general confusion is also present in the SSA back end.
      
      The PHEAP bit worked decently when I first introduced it 7 years ago (!)
      in 391425ae. The back end did nothing sophisticated, and in particular
      there was no analysis at all: no escape analysis, no liveness analysis,
      and certainly no SSA back end. But the complications caused in the
      various downstream consumers suggest that this should be a detail
      kept mainly in the front end.
      
      This CL therefore eliminates both the PHEAP bit and even the idea of
      "heap variables" from the back ends.
      
      First, it replaces the PPARAM|PHEAP, PPARAMOUT|PHEAP, and PAUTO|PHEAP
      variable classes with the single PAUTOHEAP, a pseudo-class indicating
      a variable maintained on the heap and available by indirecting a
      local variable kept on the stack (a plain PAUTO).
      
      Second, walkexpr replaces all references to PAUTOHEAP variables
      with indirections of the corresponding PAUTO variable.
      The back ends and the liveness code now just see plain indirected
      variables. This may actually produce better code, but the real goal
      here is to eliminate these little-used and somewhat suspect code
      paths in the back end analyses.
      
      The OPARAM node type goes away too.
      
      A followup CL will do the same to PPARAMREF. I'm not sure that
      the back ends (SSA in particular) are handling those right either,
      and with the framework established in this CL that change is trivial
      and the result clearly more correct.
      
      Fixes #15747.
      
      Change-Id: I2770b1ce3cbc93981bfc7166be66a9da12013d74
      Reviewed-on: https://go-review.googlesource.com/23393Reviewed-by: default avatarKeith Randall <khr@golang.org>
      Run-TryBot: Russ Cox <rsc@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      b6dc3e6f
    • Alex Brainman's avatar
      path/filepath: fix globbing of c:\*dir\... pattern · 99d29d5a
      Alex Brainman authored
      The problem was introduced by the recent filepath.Join change.
      
      Fixes #14949
      
      Change-Id: I7ee52f210e12bbb1369e308e584ddb2c7766e095
      Reviewed-on: https://go-review.googlesource.com/23240
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRuss Cox <rsc@golang.org>
      99d29d5a
    • Ian Lance Taylor's avatar
      cmd/cgo: remove -O options when generating compiler errors · b5d18b50
      Ian Lance Taylor authored
      The cgo tool generates compiler errors to find out what kind of name it
      is using.  Turning on optimization can confuse that process by producing
      new unexpected messages.
      
      Fixes #14669.
      
      Change-Id: Idc8e35fd259711ecc9638566b691c11d17140325
      Reviewed-on: https://go-review.googlesource.com/23231
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRuss Cox <rsc@golang.org>
      b5d18b50
  2. 26 May, 2016 17 commits
  3. 25 May, 2016 8 commits
    • Seth Vargo's avatar
      net/http: add missing HTTP status codes · b9ec0024
      Seth Vargo authored
      This commit adds missing status codes:
      
      * 102 - Processing
      * 207 - Multi-Status
      * 208 - Already Reported
      * 226 - IM Used
      * 308 - Permanent Redirect
      * 422 - Unprocessable Entity
      * 423 - Locked
      * 424 - Failed Dependency
      * 426 - Upgrade Required
      * 506 - Variant Also Negotiates
      * 507 - Insufficient Storage
      * 508 - Loop Detected
      * 510 - Not Extended
      * 511 - Network Authentication Required
      
      Change-Id: Ife0e5b064f4b1e3542d2fd41abc9e7b1e410b644
      Reviewed-on: https://go-review.googlesource.com/23090Reviewed-by: default avatarAndrew Gerrand <adg@golang.org>
      Run-TryBot: Andrew Gerrand <adg@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      b9ec0024
    • Ian Lance Taylor's avatar
      cmd/cgo, runtime, runtime/cgo: TSAN support for malloc · a5d1a72a
      Ian Lance Taylor authored
      Acquire and release the TSAN synchronization point when calling malloc,
      just as we do when calling any other C function. If we don't do this,
      TSAN will report false positive errors about races calling malloc and
      free.
      
      We used to have a special code path for malloc and free, going through
      the runtime functions cmalloc and cfree. The special code path for cfree
      was no longer used even before this CL. This CL stops using the special
      code path for malloc, because there is no place along that path where we
      could conditionally insert the TSAN synchronization. This CL removes
      the support for the special code path for both functions.
      
      Instead, cgo now automatically generates the malloc function as though
      it were referenced as C.malloc.  We need to automatically generate it
      even if C.malloc is not called, even if malloc and size_t are not
      declared, to support cgo-provided functions like C.CString.
      
      Change-Id: I829854ec0787a80f33fa0a8a0dc2ee1d617830e2
      Reviewed-on: https://go-review.googlesource.com/23260Reviewed-by: default avatarDmitry Vyukov <dvyukov@google.com>
      Run-TryBot: Ian Lance Taylor <iant@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarAustin Clements <austin@google.com>
      a5d1a72a
    • Russ Cox's avatar
      runtime: align C library startup calls on amd64 · 10c8b237
      Russ Cox authored
      This makes GOEXPERIMENT=framepointer, GOOS=darwin, and buildmode=carchive coexist.
      
      Change-Id: I9f6fb2f0f06f27df683e5b51f2fa55cd21872453
      Reviewed-on: https://go-review.googlesource.com/23454Reviewed-by: default avatarAustin Clements <austin@google.com>
      10c8b237
    • Austin Clements's avatar
      runtime: pass gcWork to scanstack · 3be48b4d
      Austin Clements authored
      Currently scanstack obtains its own gcWork from the P for the duration
      of the stack scan and then, if called during mark termination,
      disposes the gcWork.
      
      However, this means that the number of workbufs allocated will be at
      least the number of stacks scanned during mark termination, which may
      be very high (especially during a STW GC). This happens because, in
      steady state, each scanstack will obtain a fresh workbuf (either from
      the empty list or by allocating it), fill it with the scan results,
      and then dispose it to the full list. Nothing is consuming from the
      full list during this (and hence nothing is recycling them to the
      empty list), so the length of the full list by the time mark
      termination starts draining it is at least the number of stacks
      scanned.
      
      Fix this by pushing the gcWork acquisition up the stack to either the
      gcDrain that calls markroot that calls scanstack (which batches across
      many stack scans and is the path taken during STW GC) or to newstack
      (which is still a single scanstack call, but this is roughly bounded
      by the number of Ps).
      
      This fix reduces the workbuf allocation for the test program from
      issue #15319 from 213 MB (roughly 2KB * 1e5 goroutines) to 10 MB.
      
      Fixes #15319.
      
      Note that there's potentially a similar issue in write barriers during
      mark 2. Fixing that will be more difficult since there's no broader
      non-preemptible context, but it should also be less of a problem since
      the full list is being drained during mark 2.
      
      Some overall improvements in the go1 benchmarks, plus the usual noise.
      No significant change in the garbage benchmark (time/op or GC memory).
      
      name                      old time/op    new time/op    delta
      BinaryTree17-12              2.54s ± 1%     2.51s ± 1%  -1.09%  (p=0.000 n=20+19)
      Fannkuch11-12                2.12s ± 0%     2.17s ± 0%  +2.18%  (p=0.000 n=19+18)
      FmtFprintfEmpty-12          45.1ns ± 1%    45.2ns ± 0%    ~     (p=0.078 n=19+18)
      FmtFprintfString-12          127ns ± 0%     128ns ± 0%  +1.08%  (p=0.000 n=19+16)
      FmtFprintfInt-12             125ns ± 0%     122ns ± 1%  -2.71%  (p=0.000 n=14+18)
      FmtFprintfIntInt-12          196ns ± 0%     190ns ± 1%  -2.91%  (p=0.000 n=12+20)
      FmtFprintfPrefixedInt-12     196ns ± 0%     194ns ± 1%  -0.94%  (p=0.000 n=13+18)
      FmtFprintfFloat-12           253ns ± 1%     251ns ± 1%  -0.86%  (p=0.000 n=19+20)
      FmtManyArgs-12               807ns ± 1%     784ns ± 1%  -2.85%  (p=0.000 n=20+20)
      GobDecode-12                7.13ms ± 1%    7.12ms ± 1%    ~     (p=0.351 n=19+20)
      GobEncode-12                5.89ms ± 0%    5.95ms ± 0%  +0.94%  (p=0.000 n=19+19)
      Gzip-12                      219ms ± 1%     221ms ± 1%  +1.35%  (p=0.000 n=18+20)
      Gunzip-12                   37.5ms ± 1%    37.4ms ± 0%    ~     (p=0.057 n=20+19)
      HTTPClientServer-12         81.4µs ± 4%    81.9µs ± 3%    ~     (p=0.118 n=17+18)
      JSONEncode-12               15.7ms ± 1%    15.8ms ± 1%  +0.73%  (p=0.000 n=17+18)
      JSONDecode-12               57.9ms ± 1%    57.2ms ± 1%  -1.34%  (p=0.000 n=19+19)
      Mandelbrot200-12            4.12ms ± 1%    4.10ms ± 0%  -0.33%  (p=0.000 n=19+17)
      GoParse-12                  3.22ms ± 2%    3.25ms ± 1%  +0.72%  (p=0.000 n=18+20)
      RegexpMatchEasy0_32-12      70.6ns ± 1%    71.1ns ± 2%  +0.63%  (p=0.005 n=19+20)
      RegexpMatchEasy0_1K-12       240ns ± 0%     239ns ± 1%  -0.59%  (p=0.000 n=19+20)
      RegexpMatchEasy1_32-12      71.3ns ± 1%    71.3ns ± 1%    ~     (p=0.844 n=17+17)
      RegexpMatchEasy1_1K-12       384ns ± 2%     371ns ± 1%  -3.45%  (p=0.000 n=19+20)
      RegexpMatchMedium_32-12      109ns ± 1%     108ns ± 2%  -0.48%  (p=0.029 n=19+19)
      RegexpMatchMedium_1K-12     34.3µs ± 1%    34.5µs ± 2%    ~     (p=0.160 n=18+20)
      RegexpMatchHard_32-12       1.79µs ± 9%    1.72µs ± 2%  -3.83%  (p=0.000 n=19+19)
      RegexpMatchHard_1K-12       53.3µs ± 4%    51.8µs ± 1%  -2.82%  (p=0.000 n=19+20)
      Revcomp-12                   386ms ± 0%     388ms ± 0%  +0.72%  (p=0.000 n=17+20)
      Template-12                 62.9ms ± 1%    62.5ms ± 1%  -0.57%  (p=0.010 n=18+19)
      TimeParse-12                 325ns ± 0%     331ns ± 0%  +1.84%  (p=0.000 n=18+19)
      TimeFormat-12                338ns ± 0%     343ns ± 0%  +1.34%  (p=0.000 n=18+20)
      [Geo mean]                  52.7µs         52.5µs       -0.42%
      
      Change-Id: Ib2d34736c4ae2ec329605b0fbc44636038d8d018
      Reviewed-on: https://go-review.googlesource.com/23391
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      3be48b4d
    • Austin Clements's avatar
      runtime: document scanstack · a1f7db88
      Austin Clements authored
      Also mark it go:systemstack and explain why.
      
      Change-Id: I88baf22741c04012ba2588d8e03dd3801d19b5c0
      Reviewed-on: https://go-review.googlesource.com/23390
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      a1f7db88
    • Robert Griesemer's avatar
      cmd/compile: document how to update builtin.go · a689f6b8
      Robert Griesemer authored
      No code changes.
      
      Fixes #15835.
      
      Change-Id: Ibae3f20882f976babc4093df5e9fea0b2cf0e9d9
      Reviewed-on: https://go-review.googlesource.com/23443Reviewed-by: default avatarAlan Donovan <adonovan@google.com>
      a689f6b8
    • David Crawshaw's avatar
      doc: reflect {Num,}Method skips unexported methods · b7d96b8e
      David Crawshaw authored
      For #15673
      
      Change-Id: I3ce8d4016854d41860c5a9f05a54cda3de49f337
      Reviewed-on: https://go-review.googlesource.com/23430Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      b7d96b8e
    • Marcel van Lohuizen's avatar
      math/big: use run for benchmarks · 07f0c19a
      Marcel van Lohuizen authored
      shortens code and gives an example of the use of Run.
      
      Change-Id: I75ffaf762218a589274b4b62e19022e31e805d1b
      Reviewed-on: https://go-review.googlesource.com/23424Reviewed-by: default avatarRuss Cox <rsc@golang.org>
      Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      07f0c19a