• Kirill Smelkov's avatar
    Verify tag/tree/blob encoding is consistent and always the same · 5aac4734
    Kirill Smelkov authored
    In upcoming patch we are going to switch xcommit_tree() to our own
    implementation, and since this can potentially change how commits are
    represented, for backward compatibility reason we need to make sure
    objects encoded as commits stay the same.
    
    So for all kind of objects (they are present in testdata/ repositories)
    add checks that:
    
        - encode/decode is idempotent
        - encoding and decoding produces exactly expected sha1
    
    One nice side effect of this is that we can now remove runtime
    consistency check from tail of decoding. That check was there from the
    beginning - from 6f237f22 (git-backup: Initial draft) mainly present
    because there was no testsuite at that time. That check place is however
    even not completely right - in case we somehow wrongly pulled an object
    it has to be detected at pull time, not restore time. So that check was
    checking only 1/2 of implementation - and not the main one - that
    decoding does not mess up.
    
    Since now we have proper testsuite and add encode/decode tests in this
    patch, we can remove that partial runtime check. And even if decoding
    messes something up, despite having it testsuited, it will be 100%
    caught by restore process, because for an extracted repository, if
    there is no some object which needs to be present in it, pack generation
    for that repository will fail. So we can be safe with the removal.
    
    Time for restoring kirr/slapos.git from lab.nexedi.com backup
    
    before: 5.5s
    after:  3.5s
    
    ( so much because there are ~ 500 tags in slapos.git and currently tag
      encoding is done with spawning separate subprocess per tag )
    5aac4734
git-backup.go 33.4 KB