• Kevin Modzelewski's avatar
    Add "log hashing" to see new failures · eb3edfd6
    Kevin Modzelewski authored
    A common issue is that we see that a test goes from N failures to N+1,
    but it's not clear at all which of the failures is new.
    
    This adds a simple hashing mechanism to try to help us in these cases:
    we hash each line in the log, and construct a small bloom filter of
    which lines were seen.  We can then put this into the test file, and
    when the test fails we can run the received output against the expected
    bloom filter.
    
    I tried testing it and it seems ok, but we'll have to see how well it
    works in practice.
    eb3edfd6
sass_test.py 1011 Bytes