And add some extra ref checking.  The rewriter didn't verify that it didn't
over-decref which made this hard to find.