Add third GC arena

Previously it was:
tier 0: ast interpreter
tier 1: llvm, no speculations, no llvm opts
tier 2: llvm, w/ speculations, no llvm opts
tier 3: llvm, w/ speculations, w/ llvm opts

tier 2 seemed pretty useless, and very little would stay in it.  Also,
OSR would always skip from tier 1 to tier 3.

Separately, add configurable OSR/reopt thresholds.  This is mostly for the
sake of tests, where we can set lower limits and force OSR/reopts to happen.

Generator tracebacks

add reversed() builtin function, as well as compatible __reversed__ methods on list/xrange.

Fix: GC must visit the 'iter' field of every BoxedIterWrapper

(previously we would just throw an assert)

add a comment about further work for realloc(), and factor out the duplicate list.forEach in large/huge arenas

don't use toggle, since our assert ensures not free only in debug builds.  also, more valgrind error disabling for access to the GCAllocation data

port over sgen's idea of LOSSections as a mid-sized arena, so that we now have:

SmallArena original non-large allocator, free bitmaps, segregated-fit allocator.
  handles objects where size <= 3584 bytes
LargeArena (new code, size-specific free lists.)
  handles object  where 3584 < size <= ~1 meg
HugeArena (original large allocator, 1 mmap per object.
  handles objects where size > ~1meg

Implement str.join(iterable)

We were having problems with spawning subprocesses from threads,
since the children would inherit the "wait for another thread to acquire
the gil" flag, but would not inherit the thread that would actually
try to acquire the gil; this would make the child hang.

before we stopped at the generatorEntry func and did not unwind
into the caller.

Also, register string memory as additional GC pressure.
The subprocess module repeatedly allocates a 1MB str as a buffer
for reading from file descriptors.  The first issue was that we would
never free this memory; but even once we would, we wouldn't trigger a GC
since there were only small GC allocations (sizeof(BoxedString) at a time)
but with large other memory pressure, so add a hook for adding GC pressure.

We don't call pthread_join on the threads we create, so apparently
unless we call pthread_detach we just don't get the resources back
(ex stack memory).

This doesn't handle all the cases that CPython does.

And fix a bunch of places that they weren't.

This might introduce a perf regression (ICs get slightly slower) but cleans
things up a lot and fixes a few bugs.

We clear a patchpoint by writing a jmp-to-slowpath at the beginning.
But there could be a stack frame currently inside the IC, and we
need to be able to invalidate the IC while letting the other frame
return successfully.  So make sure that we don't overwrite
any potential return address.

Before, we tried to do it based on whether the rewrite we were
adding was "compatible" with whatever was already in there.  But we
weren't really doing this and there were a lot of limitations with this
method anyway.

This is kind of hacky due to our method of first doing the rewrite,
and then picking the slot to rewrite to.  This means that at the time
that we create the rewrite, we don't know the exact location of the
"num_inside" counter, so we have to go back at the end and rewrite it.

We could also switch things to pick the rewrite slot first, but that
is also complicated but perhaps more subtly, since during the course
of the rewrite the chosen slot could have gotten rewritten!

This changeset adds this functionality but doesn't use it for anything.

Thanks to @denji for the report!

We weren't scanning it as part of the gc handler, so eventually it would
get collected and then __name__ would break.  (un)fortunately, in the
cases that would use it, it looks like it would always get replaced by
another string (the only thing being allocated out of that size bin?),
so it would look like the __name__ was just wrong.

Anyway, this should fix interp2.py

I'm surprised we got away so long without handling them!

Also, it looks like del statements go through full tuple unpacking --
so you can do something like
del (l[1], [c.b])

So that we have a whole number of pointers to scan.  I think there's a bunch
more we can do to optimize this situation (do we even need to scan these at
all?) but this seems like the simplest way to make this work for now.

Need to unbox-rebox bools like we do for ints and floats.

Also, improve rewriting to still be able to rewrite object construction.
For now, be able to rewrite the case that a function takes kwargs but
the kwargs is empty.

Also, add an even faster path to typeCallInternal.  This is partially
obviated by the improved rewriting, but we might as well keep it.