1. 31 Mar, 2017 16 commits
    • Brad Fitzpatrick's avatar
      syscall: skip test on TestUnshareMountNameSpace permission error · 3e4afe23
      Brad Fitzpatrick authored
      TestUnshareMountNameSpace fails on arm64 due to permission problems.
      
      Skip that test for now when permission problems are encountered, so we
      don't regress elsewhere in the meantime.
      
      Updates #19698
      
      Change-Id: I9058928afa474b813652c9489f343b8957160a6c
      Reviewed-on: https://go-review.googlesource.com/39052
      Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarIan Lance Taylor <iant@golang.org>
      Reviewed-by: default avatarJosh Bleecher Snyder <josharian@gmail.com>
      3e4afe23
    • Austin Clements's avatar
      runtime: make runtime.GC() trigger a concurrent GC · 9ffbdabd
      Austin Clements authored
      Currently runtime.GC() triggers a STW GC. For common uses in tests and
      benchmarks, it doesn't matter whether it's STW or concurrent, but for
      uses in servers for things like collecting heap profiles and
      controlling memory footprint, this pause can be a bit problem for
      latency.
      
      This changes runtime.GC() to trigger a concurrent GC. In order to
      remain as close as possible to its current meaning, we define it to
      always perform a full mark/sweep GC cycle before returning (even if
      that means it has to finish up a cycle we're in the middle of first)
      and to publish the heap profile as of the triggered mark termination.
      While it must perform a full cycle, simultaneous runtime.GC() calls
      can be consolidated into a single full cycle.
      
      Fixes #18216.
      
      Change-Id: I9088cc5deef4ab6bcf0245ed1982a852a01c44b5
      Reviewed-on: https://go-review.googlesource.com/37520
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      9ffbdabd
    • Austin Clements's avatar
      runtime: track the number of active sweepone calls · 44ed88a5
      Austin Clements authored
      sweepone returns ^uintptr(0) when there are no more spans to *start*
      sweeping, but there may be spans being swept concurrently at the time
      and there's currently no efficient way to tell when the sweeper is
      done sweeping all the spans.
      
      We'll need this for concurrent runtime.GC(), so add a count of the
      number of active sweepone calls to make it possible to block until
      sweeping is truly done.
      
      This is also useful for more accurately printing the gcpacertrace,
      since that should be printed after all of the sweeping stats are in
      (currently we can print it slightly too early).
      
      For #18216.
      
      Change-Id: I06e6240c9e7b40aca6fd7b788bb6962107c10a0f
      Reviewed-on: https://go-review.googlesource.com/37716
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      44ed88a5
    • Austin Clements's avatar
      runtime: don't adjust GC trigger on forced GC · 2919132e
      Austin Clements authored
      Forced GCs don't provide good information about how to adjust the GC
      trigger. Currently we avoid adjusting the trigger on forced GC because
      forced GC is also STW and we don't adjust the trigger on STW GC.
      However, this will become a problem when forced GC is concurrent.
      
      Fix this by skipping trigger adjustment if the GC was user-forced.
      
      For #18216.
      
      Change-Id: I03dfdad12ecd3cfeca4573140a0768abb29aac5e
      Reviewed-on: https://go-review.googlesource.com/38951
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      2919132e
    • Austin Clements's avatar
      runtime: track forced GCs independent of gcMode · 29fdbcfe
      Austin Clements authored
      Currently gcMode != gcBackgroundMode implies this was a user-forced GC
      cycle. This is no longer going to be true when we make runtime.GC()
      trigger a concurrent GC, so replace this with an explicit
      work.userForced bit.
      
      For #18216.
      
      Change-Id: If7d71bbca78b5f0b35641b070f9d457f5c9a52bd
      Reviewed-on: https://go-review.googlesource.com/37519
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      29fdbcfe
    • Austin Clements's avatar
      runtime: make debug.FreeOSMemory call runtime.GC() · 786eb5b7
      Austin Clements authored
      Currently freeOSMemory calls gcStart directly, but we really just want
      it to behave like runtime.GC() and then perform a scavenge, so make it
      call runtime.GC() rather than gcStart.
      
      For #18216.
      
      Change-Id: I548ec007afc788e87d383532a443a10d92105937
      Reviewed-on: https://go-review.googlesource.com/37518
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      786eb5b7
    • Austin Clements's avatar
      runtime: simplify forced GC triggering · 3d58498f
      Austin Clements authored
      Now that the gcMode is no longer involved in the GC trigger condition,
      we can simplify the triggering of forced GCs. By making the trigger
      condition for forced GCs true even if gcphase is not _GCoff, we don't
      need any special case path in gcStart to ensure that forced GCs don't
      get consolidated.
      
      Change-Id: I6067a13d76e40ff2eef8fade6fc14adb0cb58ee5
      Reviewed-on: https://go-review.googlesource.com/37517
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      3d58498f
    • Austin Clements's avatar
      runtime: generalize GC trigger · 29be3f19
      Austin Clements authored
      Currently the GC triggering condition is an awkward combination of the
      gcMode (whether or not it's gcBackgroundMode) and a boolean
      "forceTrigger" flag.
      
      Replace this with a new gcTrigger type that represents the range of
      transition predicates we need. This has several advantages:
      
      1. We can remove the awkward logic that affects the trigger behavior
         based on the gcMode. Now gcMode purely controls whether to run a
         STW GC or not and the gcTrigger controls whether this is a forced
         GC that cannot be consolidated with other GC cycles.
      
      2. We can lift the time-based triggering logic in sysmon to just
         another type of GC trigger and move the logic to the trigger test.
      
      3. This sets us up to have a cycle count-based trigger, which we'll
         use to make runtime.GC trigger concurrent GC with the desired
         consolidation properties.
      
      For #18216.
      
      Change-Id: If9cd49349579a548800f5022ae47b8128004bbfc
      Reviewed-on: https://go-review.googlesource.com/37516
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      29be3f19
    • Austin Clements's avatar
      runtime: check transition condition before triggering periodic GC · 640cd3b3
      Austin Clements authored
      Currently sysmon triggers periodic GC if GC is not currently running
      and it's been long enough since the last GC. This misses some
      important conditions; for example, whether GC is enabled at all by
      GOGC. As a result, if GOGC is off, once we pass the timeout for
      periodic GC, sysmon will attempt to trigger a GC every 10ms. This GC
      will be a no-op because gcStart will check all of the appropriate
      conditions and do nothing, but it still goes through the motions of
      waking the forcegc goroutine and printing a gctrace line.
      
      Fix this by making sysmon call gcShouldStart to check *all* of the
      appropriate transition conditions before attempting to trigger a
      periodic GC.
      
      Fixes #19247.
      
      Change-Id: Icee5521ce175e8419f934723849853d53773af31
      Reviewed-on: https://go-review.googlesource.com/37515
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      640cd3b3
    • Austin Clements's avatar
      runtime: simplify heap profile flushing · 1be3e76e
      Austin Clements authored
      Currently the heap profile is flushed by *either* gcSweep in STW mode
      or by gcMarkTermination in concurrent mode. Simplify this by making
      gcMarkTermination always flush the heap profile and by making gcSweep
      do one extra flush (instead of two) in STW mode.
      
      Change-Id: I62147afb2a128e1f3d92ef4bb8144c8a345f53c4
      Reviewed-on: https://go-review.googlesource.com/37715
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      1be3e76e
    • Austin Clements's avatar
      runtime: snapshot heap profile during mark termination · eee85fc5
      Austin Clements authored
      Currently we snapshot the heap profile just *after* mark termination
      starts the world because it's a relatively expensive operation.
      However, this means any alloc or free events that happen between
      starting the world and snapshotting the heap profile can be accounted
      to the wrong cycle. In the worst case, a free can be accounted to the
      cycle before the alloc; if the heap is small, this can result
      temporarily in a negative "in use" count in the profile.
      
      Fix this without making STW more expensive by using a global heap
      profile cycle counter. This lets us split up the operation into a two
      parts: 1) a super-cheap snapshot operation that simply increments the
      global cycle counter during STW, and 2) a more expensive cleanup
      operation we can do after starting the world that frees up a slot in
      all buckets for use by the next heap profile cycle.
      
      Fixes #19311.
      
      Change-Id: I6bdafabf111c48b3d26fe2d91267f7bef0bd4270
      Reviewed-on: https://go-review.googlesource.com/37714
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      eee85fc5
    • Austin Clements's avatar
      runtime: pull heap profile cycle into a type · 3ebe7d7d
      Austin Clements authored
      Currently memRecord has the same set of four fields repeated three
      times. Pull these into a type and use this type three times. This
      cleans up and simplifies the code a bit and will make it easier to
      switch to a globally tracked heap profile cycle for #19311.
      
      Change-Id: I414d15673feaa406a8366b48784437c642997cf2
      Reviewed-on: https://go-review.googlesource.com/37713
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      3ebe7d7d
    • Robert Griesemer's avatar
      cmd/compile: remove confusing comment, fix comment for symExport · 42aa608f
      Robert Griesemer authored
      The symExport flag tells whether a symbol is in the export list
      already or not (and it's also used to avoid being added to that
      list). Exporting is based on that export list - no need to check
      again.
      
      Change-Id: I6056f97aa5c24a19376957da29199135c8da35f9
      Reviewed-on: https://go-review.googlesource.com/39033Reviewed-by: default avatarDave Cheney <dave@cheney.net>
      42aa608f
    • Austin Clements's avatar
      runtime: diagram flow of stats through heap profile · 673a8fdf
      Austin Clements authored
      Every time I modify heap profiling, I find myself redrawing this
      diagram, so add it to the comments. This shows how allocations and
      frees are accounted, how we arrive at consistent profile snapshots,
      and when those snapshots are published to the user.
      
      Change-Id: I106aba1200af3c773b46e24e5f50205e808e2c69
      Reviewed-on: https://go-review.googlesource.com/37514
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      673a8fdf
    • Austin Clements's avatar
      runtime: improve TestMemStats checks · ef1829d1
      Austin Clements authored
      Now that we have a nice predicate system, improve the tests performed
      by TestMemStats. We add some more non-zero checks (now that we force a
      GC, things like NumGC must be non-zero), checks for trivial boolean
      fields, and a few more range checks.
      
      Change-Id: I6da46d33fa0ce5738407ee57d587825479413171
      Reviewed-on: https://go-review.googlesource.com/37513
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      ef1829d1
    • Austin Clements's avatar
      runtime: make TestMemStats failure messages useful · bda74b0e
      Austin Clements authored
      Currently most TestMemStats failures dump the whole MemStats object if
      anything is amiss without telling you what is amiss, or even which
      field is wrong. This makes it hard to figure out what the actual
      problem is.
      
      Replace this with a reflection walk over MemStats and a map of
      predicates to check. If one fails, we can construct a detailed and
      descriptive error message. The predicates are a direct translation of
      the current tests.
      
      Change-Id: I5a7cafb8e6a1eeab653d2e18bb74e2245eaa5444
      Reviewed-on: https://go-review.googlesource.com/37512
      Run-TryBot: Austin Clements <austin@google.com>
      TryBot-Result: Gobot Gobot <gobot@golang.org>
      Reviewed-by: default avatarRick Hudson <rlh@golang.org>
      bda74b0e
  2. 30 Mar, 2017 24 commits