record finalizers and heap profile info.  Enables
removing the special bit from the heap bitmap.  Also
provides a generic mechanism for annotating occasional
heap objects.

finalizers
        overhead      per obj
old	680 B	      80 B avg
new	16 B/span     48 B

profile
        overhead      per obj
old	32KB	      24 B + hash tables
new	16 B/span     24 B

R=cshapiro, khr, dvyukov, gobot
CC=golang-codereviews
https://golang.org/cl/13314053

Currently lots of sys allocations are not accounted in any of XxxSys,
including GC bitmap, spans table, GC roots blocks, GC finalizer blocks,
iface table, netpoll descriptors and more. Up to ~20% can unaccounted.
This change introduces 2 new stats: GCSys and OtherSys for GC metadata
and all other misc allocations, respectively.
Also ensures that all XxxSys indeed sum up to Sys. All sys memory allocation
functions require the stat for accounting, so that it's impossible to miss something.
Also fix updating of mcache_sys/inuse, they were not updated after deallocation.

test/bench/garbage/parser before:
Sys		670064344
HeapSys		610271232
StackSys	65536
MSpanSys	14204928
MCacheSys	16384
BuckHashSys	1439992

after:
Sys		670064344
HeapSys		610271232
StackSys	65536
MSpanSys	14188544
MCacheSys	16384
BuckHashSys	3194304
GCSys		39198688
OtherSys	3129656

Fixes #5799.

R=rsc, dave, alex.brainman
CC=golang-dev
https://golang.org/cl/12946043

Fixes #5584.

R=golang-dev, chaishushan, alex.brainman
CC=golang-dev
https://golang.org/cl/12720043

Fixes #5900.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12669043

Remove dead code related to allocation of type metadata with SysAlloc.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12311045

notetsleepg is the same as notetsleep, but is called on user g.
It includes entersyscall/exitsyscall and will help to avoid
split stack functions in syscall status.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/11681043

Currently it replaces GOGCTRACE env var (GODEBUG=gctrace=1).
The plan is to extend it with other type of debug tracing,
e.g. GODEBUG=gctrace=1,schedtrace=100.

R=rsc
CC=bradfitz, daniel.morsing, gobot, golang-dev
https://golang.org/cl/10026045

Also reduce FixAlloc allocation granulatiry from 128k to 16k,
small programs do not need that much memory for MCache's and MSpan's.

R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/10140044

mheap.map become a pointer, so nelem(h->map) returns 1 rather than the map size.
As the result coalescing with subsequent spans does not happen.

R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9649046

Then use the limit to make sure MHeap_LookupMaybe & inlined
copies don't return a span if the pointer is beyond the limit.
Use this fact to optimize all call sites.

R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/9869045

as was dicussed in cl/9791044

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9853046

This depends on: 9791044: runtime: allocate page table lazily
Once page table is moved out of heap, the heap becomes small.
This removes unnecessary dereferences during heap access.
No logical changes.

R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9802043

This removes the 256MB memory allocation at startup,
which conflicts with ulimit.
Also will allow to eliminate an unnecessary memory dereference in GC,
because the page table is usually mapped at known address.
Update #5049.
Update #5236.

R=golang-dev, khr, r, khr, rsc
CC=golang-dev
https://golang.org/cl/9791044

Currently per-sizeclass stats are lost for destroyed MCache's. This patch fixes this.
Also, only update mstats.heap_alloc on heap operations, because that's the only
stat that needs to be promptly updated. Everything else needs to be up-to-date only in ReadMemStats().

R=golang-dev, remyoudompheng, dave, iant
CC=golang-dev
https://golang.org/cl/9207047

Update #4979.

R=dvyukov, r, bradfitz
CC=golang-dev
https://golang.org/cl/8697045

The issue was that scvg is assigned *after* the scavenger goroutine is started,
so when the scavenger calls entersyscall() the g==scvg check can fail.
Fixes #5025.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7629045

R=golang-dev, rsc
CC=golang-dev, minux.ma
https://golang.org/cl/7424047

Step 1 of http://golang.org/s/go11func.

R=golang-dev, r, daniel.morsing, remyoudompheng
CC=golang-dev
https://golang.org/cl/7393045

In preparation for the new scheduler.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7386044

Before, the mheap structure was in the bss,
but it's quite large (today, 256 MB, much of
which is never actually paged in), and it makes
Go binaries run afoul of exec-time bss size
limits on some BSD systems.

Fixes #4447.

R=golang-dev, dave, minux.ma, remyoudompheng, iant
CC=golang-dev
https://golang.org/cl/7307122

Fixes #4090.

R=golang-dev, iant, bradfitz, dsymonds
CC=golang-dev
https://golang.org/cl/7229070

Mark candidate spans one GC pass earlier.

Move scavenger's code out from mgc0 and constrain it into mheap (where it belongs).

R=rsc, dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/7002049

Fixes #4243.

R=golang-dev, iant
CC=golang-dev, sebastien.paolacci
https://golang.org/cl/6682050

R=rsc
CC=golang-dev
https://golang.org/cl/6554060

It will be required for scheduler that maintains
GOMAXPROCS MCache's.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/6350062

+move zeroization out of the heap mutex

R=golang-dev, iant, rsc
CC=golang-dev
https://golang.org/cl/6094050

make MHeap.allspans an array instead on a linked-list,
it's required for parallel for

benchmark                              old ns/op    new ns/op    delta

garbage.BenchmarkTree                  494435529    487962705   -1.31%
garbage.BenchmarkTree-2                499652705    485358000   -2.86%
garbage.BenchmarkTree-4                468482117    454093117   -3.07%
garbage.BenchmarkTree-8                488533235    471872470   -3.41%
garbage.BenchmarkTree-16               507835176    492558470   -3.01%

garbage.BenchmarkTree2                  31453900     31404300   -0.16%
garbage.BenchmarkTree2-2                21440600     21477000   +0.17%
garbage.BenchmarkTree2-4                10982000     11117400   +1.23%
garbage.BenchmarkTree2-8                 7544700      7456700   -1.17%
garbage.BenchmarkTree2-16                7049500      6805700   -3.46%

garbage.BenchmarkParser               4448988000   4453264000   +0.10%
garbage.BenchmarkParser-2             4086045000   4057948000   -0.69%
garbage.BenchmarkParser-4             3677365000   3661246000   -0.44%
garbage.BenchmarkParser-8             3517253000   3540190000   +0.65%
garbage.BenchmarkParser-16            3506562000   3463478000   -1.23%

garbage.BenchmarkTreePause              20969784     21100238   +0.62%
garbage.BenchmarkTreePause-2            20215875     20139572   -0.38%
garbage.BenchmarkTreePause-4            17240709     16683624   -3.23%
garbage.BenchmarkTreePause-8            18196386     17639306   -3.06%
garbage.BenchmarkTreePause-16           20621158     20215056   -1.97%

garbage.BenchmarkTree2Pause            173992142    173872380   -0.07%
garbage.BenchmarkTree2Pause-2          131281904    131366666   +0.06%
garbage.BenchmarkTree2Pause-4           93484952     95109619   +1.74%
garbage.BenchmarkTree2Pause-8           88950523     86533333   -2.72%
garbage.BenchmarkTree2Pause-16          86071238     84089190   -2.30%

garbage.BenchmarkParserPause           135815000    135255952   -0.41%
garbage.BenchmarkParserPause-2          92691523     91451428   -1.34%
garbage.BenchmarkParserPause-4          53392190     51611904   -3.33%
garbage.BenchmarkParserPause-8          36059523     35116666   -2.61%
garbage.BenchmarkParserPause-16         30174300     27340600   -9.39%

garbage.BenchmarkTreeLastPause          28420000     29142000   +2.54%
garbage.BenchmarkTreeLastPause-2        23514000     26779000  +13.89%
garbage.BenchmarkTreeLastPause-4        21773000     18660000  -14.30%
garbage.BenchmarkTreeLastPause-8        24072000     21276000  -11.62%
garbage.BenchmarkTreeLastPause-16       25149000     28541000  +13.49%

garbage.BenchmarkTree2LastPause        314491000    313982000   -0.16%
garbage.BenchmarkTree2LastPause-2      214363000    214715000   +0.16%
garbage.BenchmarkTree2LastPause-4      109778000    111115000   +1.22%
garbage.BenchmarkTree2LastPause-8       75390000     74522000   -1.15%
garbage.BenchmarkTree2LastPause-16      70333000     67880000   -3.49%

garbage.BenchmarkParserLastPause       327247000    326815000   -0.13%
garbage.BenchmarkParserLastPause-2     217039000    212529000   -2.08%
garbage.BenchmarkParserLastPause-4     119722000    111535000   -6.84%
garbage.BenchmarkParserLastPause-8      70806000     69613000   -1.68%
garbage.BenchmarkParserLastPause-16     62813000     48009000  -23.57%

R=rsc, r
CC=golang-dev
https://golang.org/cl/5992055

Breaks closure test when GOMAXPROCS=2 or more.

««« original CL description
runtime: restore deadlock detection in the simplest case.

Fixes #3342.

R=iant, r, dave, rsc
CC=golang-dev, remy
https://golang.org/cl/5844051

»»»

R=rsc
CC=golang-dev
https://golang.org/cl/5924045

Fixes #3342.

R=iant, r, dave, rsc
CC=golang-dev, remy
https://golang.org/cl/5844051

Fixes #3125.

R=golang-dev, r, minux.ma
CC=golang-dev
https://golang.org/cl/5714049

Fixes #3061.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5656089

Periodically browse MHeap's freelists for long unused spans and release them if any.

Current hardcoded settings:
        - GC is forced if none occured over the last 2 minutes.
        - spans are handed back after 5 minutes of uselessness.

SysUnused (for Unix) is a wrapper on madvise MADV_DONTNEED on Linux and MADV_FREE on BSDs.

R=rsc, dvyukov, remyoudompheng
CC=golang-dev
https://golang.org/cl/5451057

Collapse the arch,os-specific directories into the main directory
by renaming xxx/foo.c to foo_xxx.c, and so on.

There are no substantial edits here, except to the Makefile.
The assumption is that the Go tool will #define GOOS_darwin
and GOARCH_amd64 and will make any file named something
like signals_darwin.h available as signals_GOOS.h during the
build.  This replaces what used to be done with -I$(GOOS).

There is still work to be done to make runtime build with
standard tools, but this is a big step.  After this we will have
to write a script to generate all the generated files so they
can be checked in (instead of generated during the build).

R=r, iant, r, lucio.dere
CC=golang-dev
https://golang.org/cl/5490053

Linux/amd64, 2 x Intel Xeon E5620, 8 HT cores, 2.40GHz
benchmark                    old ns/op    new ns/op    delta
BenchmarkFinalizer              420.00       261.00  -37.86%
BenchmarkFinalizer-2            985.00       201.00  -79.59%
BenchmarkFinalizer-4           1077.00       244.00  -77.34%
BenchmarkFinalizer-8           1155.00       180.00  -84.42%
BenchmarkFinalizer-16          1182.00       184.00  -84.43%

BenchmarkFinalizerRun          2128.00      1378.00  -35.24%
BenchmarkFinalizerRun-2        1655.00      1418.00  -14.32%
BenchmarkFinalizerRun-4        1634.00      1522.00   -6.85%
BenchmarkFinalizerRun-8        2213.00      1581.00  -28.56%
BenchmarkFinalizerRun-16       2424.00      1599.00  -34.03%

Darwin/amd64, Intel L9600, 2 cores, 2.13GHz
benchmark                    old ns/op    new ns/op    delta
BenchmarkChanCreation          1451.00       926.00  -36.18%
BenchmarkChanCreation-2        3124.00      1412.00  -54.80%
BenchmarkChanCreation-4        6121.00      2628.00  -57.07%

BenchmarkFinalizer              684.00       420.00  -38.60%
BenchmarkFinalizer-2          11195.00       398.00  -96.44%
BenchmarkFinalizer-4          15862.00       654.00  -95.88%

BenchmarkFinalizerRun          2025.00      1397.00  -31.01%
BenchmarkFinalizerRun-2        3920.00      1447.00  -63.09%
BenchmarkFinalizerRun-4        9471.00      1545.00  -83.69%

R=golang-dev, cw, rsc
CC=golang-dev
https://golang.org/cl/4963057

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4995045

Avoid touching centralized state during
memory manager opreations.

R=rsc
CC=golang-dev
https://golang.org/cl/4766042

runtime: memory allocated by OS not in usable range
runtime: out of memory: cannot allocate 1114112-byte block (2138832896 in use)
throw: out of memory

runtime.throw+0x40 /Users/rsc/g/go/src/pkg/runtime/runtime.c:102
        runtime.throw(0x1fffd, 0x101)
runtime.mallocgc+0x2af /Users/rsc/g/go/src/pkg/runtime/malloc.c:60
        runtime.mallocgc(0x100004, 0x0, 0x1, 0x1, 0xc093, ...)
runtime.mal+0x40 /Users/rsc/g/go/src/pkg/runtime/malloc.c:289
        runtime.mal(0x100004, 0x20bc4)
runtime.new+0x26 /Users/rsc/g/go/src/pkg/runtime/malloc.c:296
        runtime.new(0x100004, 0x8fe84000, 0x20bc4)
main.main+0x29 /Users/rsc/x.go:11
        main.main()
runtime.mainstart+0xf /Users/rsc/g/go/src/pkg/runtime/386/asm.s:93
        runtime.mainstart()
runtime.goexit /Users/rsc/g/go/src/pkg/runtime/proc.c:178
        runtime.goexit()
----- goroutine created by -----
_rt0_386+0xbf /Users/rsc/g/go/src/pkg/runtime/386/asm.s:80

R=iant, r
CC=golang-dev
https://golang.org/cl/4444073

Used to fault trying to access l->list->next
when l->list == nil after MCentral_AllocList.
Now prints

runtime: out of memory: no room in arena for 65536-byte allocation (536870912 in use)
throw: out of memory

followed by stack trace.

Fixes #1650.

R=r, dfc
CC=golang-dev
https://golang.org/cl/4446062

GC is still single-threaded.
Multiple threads will happen in another CL.

Garbage collection pauses are typically
about half as long as they were before this CL.

R=brainman, iant, r
CC=golang-dev
https://golang.org/cl/3975046

The old heap maps used a multilevel table, but that
was overkill: there are only 1M entries on a 32-bit
machine and we can arrange to use a dense address
range on a 64-bit machine.

The heap map is in bss.  The assumption is that if
we don't touch the pages they won't be mapped in.

Also moved some duplicated memory allocation
code out of the OS-specific files.

R=r
CC=golang-dev
https://golang.org/cl/4118042