runtime: close window that hides GC work from concurrent mark

Currently we enter mark 2 by first flushing all existing gcWork caches
and then setting gcBlackenPromptly, which disables further gcWork
caching. However, if a worker or assist pulls a work buffer in to its
gcWork cache after that cache has been flushed but before caching is
disabled, that work may remain in that cache until mark termination.
If that work represents a heap bottleneck (e.g., a single pointer that
is the only way to reach a large amount of the heap), this can force
mark termination to do a large amount of work, resulting in a long
STW.

Fix this by reversing the order of these steps: first disable caching,
then flush all existing caches.

Rick Hudson <rlh> did the hard work of tracking this down. This CL
combined with CL 12672 and CL 12646 distills the critical parts of his
fix from CL 12539.

Fixes #11694.

Change-Id: Ib10d0a21e3f6170a80727d0286f9990df049fed2
Reviewed-on: https://go-review.googlesource.com/12688Reviewed-by: Rick Hudson <rlh@golang.org>

src/runtime/mgc.go

@@ -1003,12 +1003,18 @@ func gc(mode int) {
 			// rescan global data and bss.
 			markroot(nil, _RootData)
 			markroot(nil, _RootBss)
+
+			// Disallow caching workbufs.
+			gcBlackenPromptly = true
+
+			// Flush all currently cached workbufs. This
+			// also forces any remaining background
+			// workers out of their loop.
 			forEachP(func(_p_ *p) {
 				_p_.gcw.dispose()
 			})
 		})
 
-		gcBlackenPromptly = true
 		// Wait for this more aggressive background mark to complete.
 		work.bgMark2.clear()
 		work.bgMark2.wait()