1. 03 May, 2016 4 commits
    • Dmitry Vyukov's avatar
      runtime: per-P contexts for race detector · caa21475
      Dmitry Vyukov authored
      Race runtime also needs local malloc caches and currently uses
      a mix of per-OS-thread and per-goroutine caches. This leads to
      increased memory consumption. But more importantly cache of
      synchronization objects is per-goroutine and we don't always
      have goroutine context when feeing memory in GC. As the result
      synchronization object descriptors leak (more precisely, they
      can be reused if another synchronization object is recreated
      at the same address, but it does not always help). For example,
      the added BenchmarkSyncLeak has effectively runaway memory
      consumption (based on a real long running server).
      
      This change updates race runtime with support for per-P contexts.
      BenchmarkSyncLeak now stabilizes at ~1GB memory consumption.
      
      Long term, this will allow us to remove race runtime dependency
      on glibc (as malloc is the main cornerstone).
      
      I've also implemented a different scheme to pass P context to
      race runtime: scheduler notified race runtime about association
      between G and P by calling procwire(g, p)/procunwire(g, p).
      But it turned out to be very messy as we have lots of places
      where the association changes (e.g. syscalls). So I dropped it
      in favor of the current scheme: race runtime asks scheduler
      about the current P.
      
      Fixes #14533
      
      Change-Id: Iad10d2f816a44affae1b9fed446b3580eafd8c69
      Reviewed-on: https://go-review.googlesource.com/19970Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      caa21475
    • Dmitry Vyukov's avatar
      runtime: fix CPU underutilization · fcd7c02c
      Dmitry Vyukov authored
      Runqempty is a critical predicate for scheduler. If runqempty spuriously
      returns true, then scheduler can fail to schedule arbitrary number of
      runnable goroutines on idle Ps for arbitrary long time. With the addition
      of runnext runqempty predicate become broken (can spuriously return true).
      Consider that runnext is not nil and the main array is empty. Runqempty
      observes that the array is empty, then it is descheduled for some time.
      Then queue owner pushes another element to the queue evicting runnext
      into the array. Then queue owner pops runnext. Then runqempty resumes
      and observes runnext is nil and returns true. But there were no point
      in time when the queue was empty.
      
      Fix runqempty predicate to not return true spuriously.
      
      Change-Id: Ifb7d75a699101f3ff753c4ce7c983cf08befd31e
      Reviewed-on: https://go-review.googlesource.com/20858Reviewed-by: default avatarAustin Clements <austin@google.com>
      Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      fcd7c02c
    • Michael Hudson-Doyle's avatar
      cmd/cgo: an approach to tsan that works with gcc · 499cd337
      Michael Hudson-Doyle authored
      GCC, unlike clang, does not provide any way for code being compiled to tell if
      -fsanitize-thread was passed. But cgo can look to see if that flag is being
      passed and generate different code in that case.
      
      Fixes #14602
      
      Change-Id: I86cb5318c2e35501ae399618c05af461d1252d2d
      Reviewed-on: https://go-review.googlesource.com/22688
      Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      499cd337
    • Brad Fitzpatrick's avatar
      net/http: keep idle conns sorted by usage · f459660c
      Brad Fitzpatrick authored
      Addressing feedback from Alan Su in https://golang.org/cl/22655
      
      Change-Id: Ie0724efea2b4da67503c074e265ec7f8d7de7791
      Reviewed-on: https://go-review.googlesource.com/22709Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      f459660c
  2. 02 May, 2016 8 commits
  3. 01 May, 2016 19 commits
  4. 30 Apr, 2016 7 commits
    • Brad Fitzpatrick's avatar
      net/http: expand documentation of Server.MaxHeaderBytes · 38cfaa5f
      Brad Fitzpatrick authored
      Clarify that it includes the RFC 7230 "request-line".
      
      Fixes #15494
      
      Change-Id: I9cc5dd5f2d85ebf903229539208cec4da5c38d04
      Reviewed-on: https://go-review.googlesource.com/22656Reviewed-by: default avatarAndrew Gerrand <adg@golang.org>
      38cfaa5f
    • Kevin Burke's avatar
      database/sql: clone data for named []byte types · 4e0cd1ee
      Kevin Burke authored
      Previously named byte types like json.RawMessage could get dirty
      database memory from a call to Scan. These types would activate a
      code path that didn't clone the byte data coming from the database
      before assigning it. Another thread could then overwrite the byte
      array in src, which has unexpected consequences.
      
      Originally reported by Jason Moiron; the patch and test are his
      suggestions. Fixes #13905.
      
      Change-Id: Iacfef61cbc9dd51c8fccef9b2b9d9544c77dd0e0
      Reviewed-on: https://go-review.googlesource.com/22393Reviewed-by: default avatarBrad Fitzpatrick <bradfitz@golang.org>
      4e0cd1ee
    • Austin Clements's avatar
      runtime: reclaim scan/dead bit in first word · a20fd1f6
      Austin Clements authored
      With the switch to separate mark bitmaps, the scan/dead bit for the
      first word of each object is now unused. Reclaim this bit and use it
      as a scan/dead bit, just like words three and on. The second word is
      still used for checkmark.
      
      This dramatically simplifies heapBitsSetTypeNoScan and hasPointers,
      since they no longer need different cases for 1, 2, and 3+ word
      objects. They can instead just manipulate the heap bitmap for the
      first word and be done with it.
      
      In order to enable this, we change heapBitsSetType and runGCProg to
      always set the scan/dead bit to scan for the first word on every code
      path. Since these functions only apply to types that have pointers,
      there's no need to do this conditionally: it's *always* necessary to
      set the scan bit in the first word.
      
      We also change every place that scans an object and checks if there
      are more pointers. Rather than only checking morePointers if the word
      is >= 2, we now check morePointers if word != 1 (since that's the
      checkmark word).
      
      Looking forward, we should probably reclaim the checkmark bit, too,
      but that's going to be quite a bit more work.
      
      Tested by setting doubleCheck in heapBitsSetType and running all.bash
      on both linux/amd64 and linux/386, and by running GOGC=10 all.bash.
      
      This particularly improves the FmtFprintf* go1 benchmarks, since they
      do a large amount of noscan allocation.
      
      name                      old time/op    new time/op    delta
      BinaryTree17-12              2.34s ± 1%     2.38s ± 1%  +1.70%  (p=0.000 n=17+19)
      Fannkuch11-12                2.09s ± 0%     2.09s ± 1%    ~     (p=0.276 n=17+16)
      FmtFprintfEmpty-12          44.9ns ± 2%    44.8ns ± 2%    ~     (p=0.340 n=19+18)
      FmtFprintfString-12          127ns ± 0%     125ns ± 0%  -1.57%  (p=0.000 n=16+15)
      FmtFprintfInt-12             128ns ± 0%     122ns ± 1%  -4.45%  (p=0.000 n=15+20)
      FmtFprintfIntInt-12          207ns ± 1%     193ns ± 0%  -6.55%  (p=0.000 n=19+14)
      FmtFprintfPrefixedInt-12     197ns ± 1%     191ns ± 0%  -2.93%  (p=0.000 n=17+18)
      FmtFprintfFloat-12           263ns ± 0%     248ns ± 1%  -5.88%  (p=0.000 n=15+19)
      FmtManyArgs-12               794ns ± 0%     779ns ± 1%  -1.90%  (p=0.000 n=18+18)
      GobDecode-12                7.14ms ± 2%    7.11ms ± 1%    ~     (p=0.072 n=20+20)
      GobEncode-12                5.85ms ± 1%    5.82ms ± 1%  -0.49%  (p=0.000 n=20+20)
      Gzip-12                      218ms ± 1%     215ms ± 1%  -1.22%  (p=0.000 n=19+19)
      Gunzip-12                   36.8ms ± 0%    36.7ms ± 0%  -0.18%  (p=0.006 n=18+20)
      HTTPClientServer-12         77.1µs ± 4%    77.1µs ± 3%    ~     (p=0.945 n=19+20)
      JSONEncode-12               15.6ms ± 1%    15.9ms ± 1%  +1.68%  (p=0.000 n=18+20)
      JSONDecode-12               55.2ms ± 1%    53.6ms ± 1%  -2.93%  (p=0.000 n=17+19)
      Mandelbrot200-12            4.05ms ± 1%    4.05ms ± 0%    ~     (p=0.306 n=17+17)
      GoParse-12                  3.14ms ± 1%    3.10ms ± 1%  -1.31%  (p=0.000 n=19+18)
      RegexpMatchEasy0_32-12      69.3ns ± 1%    70.0ns ± 0%  +0.89%  (p=0.000 n=19+17)
      RegexpMatchEasy0_1K-12       237ns ± 1%     236ns ± 0%  -0.62%  (p=0.000 n=19+16)
      RegexpMatchEasy1_32-12      69.5ns ± 1%    70.3ns ± 1%  +1.14%  (p=0.000 n=18+17)
      RegexpMatchEasy1_1K-12       377ns ± 1%     366ns ± 1%  -3.03%  (p=0.000 n=15+19)
      RegexpMatchMedium_32-12      107ns ± 1%     107ns ± 2%    ~     (p=0.318 n=20+19)
      RegexpMatchMedium_1K-12     33.8µs ± 3%    33.5µs ± 1%  -1.04%  (p=0.001 n=20+19)
      RegexpMatchHard_32-12       1.68µs ± 1%    1.73µs ± 0%  +2.50%  (p=0.000 n=20+18)
      RegexpMatchHard_1K-12       50.8µs ± 1%    52.0µs ± 1%  +2.50%  (p=0.000 n=19+18)
      Revcomp-12                   381ms ± 1%     385ms ± 1%  +1.00%  (p=0.000 n=17+18)
      Template-12                 64.9ms ± 3%    62.6ms ± 1%  -3.55%  (p=0.000 n=19+18)
      TimeParse-12                 324ns ± 0%     328ns ± 1%  +1.25%  (p=0.000 n=18+18)
      TimeFormat-12                345ns ± 0%     334ns ± 0%  -3.31%  (p=0.000 n=15+17)
      [Geo mean]                  52.1µs         51.5µs       -1.00%
      
      Change-Id: I13e74da3193a7f80794c654f944d1f0d60817049
      Reviewed-on: https://go-review.googlesource.com/22632Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      a20fd1f6
    • Austin Clements's avatar
      runtime: use morePointers and isPointer in more places · d5e3d08b
      Austin Clements authored
      This makes this code better self-documenting and makes it easier to
      find these places in the future.
      
      Change-Id: I31dc5598ae67f937fb9ef26df92fd41d01e983c3
      Reviewed-on: https://go-review.googlesource.com/22631Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      d5e3d08b
    • Austin Clements's avatar
      runtime: avoid conditional execution in morePointers and isPointer · a5d3f7ec
      Austin Clements authored
      heapBits.bits is carefully written to produce good machine code. Use
      it in heapBits.morePointers and heapBits.isPointer to get good machine
      code there, too.
      
      Change-Id: I208c7d0d38697e7a22cad67f692162589b75f1e2
      Reviewed-on: https://go-review.googlesource.com/22630Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      a5d3f7ec
    • Keith Randall's avatar
      cmd/compile: ecx is reserved for PIC, don't let peep work on it · 7a60a962
      Keith Randall authored
      Fixes #15496
      
      Change-Id: Ieb5be1caa4b1c23e23b20d56c1a0a619032a9f5d
      Reviewed-on: https://go-review.googlesource.com/22652Reviewed-by: default avatarJosh Bleecher Snyder <josharian@gmail.com>
      7a60a962
    • Michael Munday's avatar
      runtime: fix cgocallback_gofunc on ppc64x · 58f52cbb
      Michael Munday authored
      Fix issues introduced in 5f9a870b.
      
      Change-Id: Ia75945ef563956613bf88bbe57800a96455c265d
      Reviewed-on: https://go-review.googlesource.com/22661Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      58f52cbb
  5. 29 Apr, 2016 2 commits