1. 29 Jul, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 37563 from 3.4 branch. · f47937f6
      Tim Peters authored
      Removed the `key2size` OIBTree.
      
      Maintaining this is a significant expense; the code didn't
      make real use of it; and every time I've tried to exploit it
      I've ended up with grossly too-large simulated hit rates.
      Would probably take another day to figure out exactly why that
      is, and the simulated rates are "good enough" now without it.
      f47937f6
  2. 26 Jul, 2005 4 commits
    • Tim Peters's avatar
      Merge 3.4.1b1 news. · df4c8dfd
      Tim Peters authored
      df4c8dfd
    • Tim Peters's avatar
      Merge rev 37435 from 3.4 branch. · 7bcf73f8
      Tim Peters authored
      More words; update examples w/ new output.
      7bcf73f8
    • Tim Peters's avatar
      Merge rev 37432 from 3.4 branch. · a138c76e
      Tim Peters authored
      The cache simulation seems good enough to be useful now,
      although the toy app I wrote to generate a 500MB trace
      file doesn't use MVCC in an essential way (some of the
      MVCC simulation code is nevertheless exercised, since
      an invalidation of current data in the presence of MVCC
      always creates a cache record for the newly non-current
      revision).
      
      Still puzzling over what to do when the trace file records
      a load hit but the simulated cache gets a miss.  The
      old simulation code seemed to assume that a store for the same
      oid would show up in the trace file next, and it could get
      the info it needed about the missing object from the store
      trace.  But that isn't true:  precisely because the load was
      a hit in the trace file, the object isn't going to be stored
      again "soon" in the trace file.
      
      Here are some actual-vs-simulated hit rate results, for a
      20MB cache, with a trace file covering about 9 million
      loads, over 3 ZEO client (re)starts:
      
      actual   simulated
      ------   ---------
        93.1        92.7
        79.8        79.0
        68.0        69.1
            
        81.4        81.1  overall
      
      Since the simulated hit rates are both higher and lower
      than the actual hit rates, that argues against a gross
      systematic bias in the simulation (although there may
      be several systematic biases in opposite directions).
      a138c76e
    • Tim Peters's avatar
      Merge rev 37422 from 3.4 branch. · 67d48555
      Tim Peters authored
      Massive rewrite to make simul.py's circular cache aware of
      MVCC.  It isn't blowing up, but it thinks the hit rate is
      100% -- might have missed something there <wink>.
      67d48555
  3. 25 Jul, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 37411 from 3.4 branch. · f46415f5
      Tim Peters authored
      More steps on the way toward having useful cache simulation.
      
      I think the primary hangup now is that simul.py doesn't know
      anything about MVCC.  As a result, it thinks there's significatly
      more free space in the cache than there really is, and that
      probably accounts for it predicting significantly higher hit
      rates than I actually see.
      f46415f5
  4. 23 Jul, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 37389 from 3.4 branch. · 27b61d35
      Tim Peters authored
      Lots of fiddling to work w/ current trace file format.
      
      Output isn't obviously wholly insane, but isn't a reasonably
      close enough match to stats.py output either yet.
      27b61d35
  5. 22 Jul, 2005 5 commits
    • Tim Peters's avatar
      Merge rev 37387 from 3.4 branch. · fb8f5ad2
      Tim Peters authored
      stats.py:  cut the # of file reads by 1/3, and the # of
      struct.unpack() calls by 1/2.
      
      Various bugfixes elsewhere.
      fb8f5ad2
    • Tim Peters's avatar
      Merge rev 37385 from 3.4 branch. · 437ec8a6
      Tim Peters authored
      Many little bugfixes and improvements in stats.py.
      This has survived several 100 MB of trace files
      I generated over the last few days, so it's solid
      now if not necessarily perfect.
      
      Replaced simul.py with the much broader-ranging code
      Jeremy and I were working on a couple years ago,
      although it can't work with the current trace file
      format (no real loss there -- the simul.py it's
      replacing can't work with the current format either).
      437ec8a6
    • Tim Peters's avatar
      Merge rev 37382 from 3.4 branch. · 6b4c92e0
      Tim Peters authored
      Stitch in ZConfig's docs from ZConfig.
      6b4c92e0
    • Tim Peters's avatar
      Merge rev 37380 from 3.4 branch. · ae9265b2
      Tim Peters authored
      Remove this -- it's a way out-of-date copy of the ZConfig
      docs.  I'll stitch them back in from the ZConfig repo next,
      via svn:externals.  Ideally, ZODB shouldn't have this as
      part of its checkout tree at all, but that also has implications
      for the way ZRS gets packaged, etc.
      ae9265b2
    • Tim Peters's avatar
      Merge rev 37375 from 3.4 branch. · 5335331a
      Tim Peters authored
      A large number of changes so that cache tracing isn't
      100% broken -- I can get a trace file now, and stats.py
      doesn't blow up anymore.  Don't yet know whether it's
      reporting sensible things, but it's not obviously insane.
      simul.py doesn't know about the current cache implementation
      either yet.
      5335331a
  6. 21 Jul, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 37372 from 3.4 branch. · 2158579f
      Tim Peters authored
      ClientCache:  made it a new-style class.
      
      FileCache:  Removed the reuse= argument.  IIRC, it made some sense
      when different cache schemes were first being tried, but complicated
      the logic now to no good end.
      
      Nuked the maddening log warnings about reuse=True.  They were never
      helpful and were often confusing.
      
      Added info-level log messages to record the path of the client cache
      file, whether it's persistent or temporary, and if it's persistent
      whether we're creating or reusing it.
      
      Minor changes to comments.
      2158579f
  7. 20 Jul, 2005 1 commit
  8. 19 Jul, 2005 2 commits
  9. 18 Jul, 2005 2 commits
    • Tim Peters's avatar
      Merge rev 37244 from 3.4 branch. · b795c11a
      Tim Peters authored
      Collector 1843:  IISet.keys: bad exception handling.
      
      Bucket_rangeSearch():  De-obfuscated the calls to
      Bucket_findRangeEnd(), so that they stop ignoring
      the latter's error returns (a mind-bending combination
      of embedded assignment nested in an UNLESS macro,
      seemingly copy+paste'd so that the error occurred twice).
      b795c11a
    • Tim Peters's avatar
      Merge rev 37239 from 3.4 branch. · 98ff971b
      Tim Peters authored
      Port from ZODB 3.2.
      
      Collector #1846:  If an uncommitted transaction was found, fsrecover.py
      fell into an infinite loop.  Fixed that, and added a new test
      (testUncommittedAtEnd) to ensure this stays fixed.
      98ff971b
  10. 14 Jul, 2005 3 commits
  11. 12 Jul, 2005 3 commits
    • Tim Peters's avatar
      Merge news from 3.4.1a5. · ac37da01
      Tim Peters authored
      ac37da01
    • Tim Peters's avatar
      Merge rev 33285 from 3.4 branch. · 6ad07564
      Tim Peters authored
      Convert internal uses of subtxns to use savepoints instead.
      
      I suspect BTrees/convert.py should be removed instead.
      6ad07564
    • Tim Peters's avatar
      Merge rev 33276 from 3.4 branch. · 22d59055
      Tim Peters authored
      Revert extra copying in Connection._rollback().
      
      Since TmpStore.reset() had to change too to copy the index anyway,
      it no longer hurts that _rollback() clears the index (as a side
      effect of self._cache.invalidate(src.index)).
      22d59055
  12. 11 Jul, 2005 2 commits
    • Tim Peters's avatar
      Merge rev 33273 from 3.4 branch. · ce0290a6
      Tim Peters authored
      Merge tim-savepoint branch.
      
      Allow rollback to a given savepoint multiple times.
      
      transaction/_transaction.py
          _remove_and_invalidate_after():  Don't remove the
          argument savepoint from the savepoint stack anymore.
      
          abort():  abort(True) relied on the subtxn savepoint
          being invalidated as a side effect of rolling back
          to it.  Invalidate it explicitly now.
      
          Savepoint.rollback():  Don't invalidate self.
      
      ZODB/Connection.py
          Connection._rollback():  Pass a temp container to
          _cache.invalidate(), because that method clears the
          container as a side effect.  Clearing the index
          in a savepoint was a disaster if the savepoint was
          used again.
      
          TmpStore.reset():  Make a copy of the argument index.
          An alternative would be to make all callers make
          copies, but the only caller now thinks of its `state`
          argument as being opaque.
      
      transaction/savepoint.txt
      ZODB/tests/{testSubTransaction.py, testConnectionSavepoint.txt}
          Test the new semantics, and beef up the tests.
      
      transaction/tests/savepointsample.py
          _rollback_savepoint():  Needed to copy the data.
      ce0290a6
    • Tim Peters's avatar
      Merge rev 33265 from 3.4 branch. · 28dd6d89
      Tim Peters authored
      English repairs.
      28dd6d89
  13. 08 Jul, 2005 1 commit
  14. 05 Jul, 2005 3 commits
    • Tim Peters's avatar
      Merge rev 31016 from 3.4 branch. · 3b8d1eb9
      Tim Peters authored
      Simplify.
      
      Primarily, _tindex should have been a dict all along, pack()
      worked too hard at the end, and loadEx() is a minor variant
      of load().
      3b8d1eb9
    • Tim Peters's avatar
      Merge rev 31012 from 3.4 branch. · a58e7181
      Tim Peters authored
      load() and loadEx():  eliminate gratuitous differences.
      
      Ideally, load() should call loadEx() instead, but we really
      don't want "an extra" Python-level call here (heavily used).
      
      loadEx():  deleted pointless call of self._read_txn_header().
      
      ServerStub.loadEx() comments:  these were obviously wrong in
      several ways, but I don't know the full truth.  Better to
      say so up front than to leave them clearly wrong, though.
      a58e7181
    • Tim Peters's avatar
      Merge rev 31010 from 3.4 branch. · 729faf25
      Tim Peters authored
      Collector 1831.
      
      The BTree minKey() and maxKey() methods gave a misleading message if no key
      satisfying the constraints existed in a non-empty tree.
      729faf25
  15. 02 Jul, 2005 3 commits
    • Tim Peters's avatar
      Merge rev 30988 from 3.4 branch. · 15537907
      Tim Peters authored
      Collector #1830.
      
      In some error cases when reading a FileStorage index, the
      code referenced an undefined global.
      15537907
    • Tim Peters's avatar
      Merge rev 30984 from 3.4 branch. · b1427270
      Tim Peters authored
      Get rid of all code defining or referencing t32.
      
      Such code never made sense; it was all of the form
      
          t32 = 1L << 32
          if something < 0:
              something = t32 - something
      
      which is a way to change negative integers into
      gigantic positive integers greater than 4 billion, and
      never made sense in context.  Possibly
      
              something = t32 + something
      
      was intended, but that never made sense in context either.
      
      In any case, `something` is invariably obtained from
      struct.unpack using the "H" format code, and that never
      returns a negative integer to begin with.
      b1427270
    • Tim Peters's avatar
      Merge rev 30984 from 3.4 branch. · 844a079a
      Tim Peters authored
      read_index():  Mechanical reformat for easier reading.
      844a079a
  16. 01 Jul, 2005 2 commits
  17. 29 Jun, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 30944 from 3.4 branch. · 45814f90
      Tim Peters authored
      Collector 1822.
      
      Make undo{Log,Info} arguments act like Python slice indices when
      both are non-negative.  The code used to do that before ZODB 3.4a9,
      but changed to match ZODB's UML documentation.  Alas, some
      (untested) code in Zope relied on the actual behavior (see the
      collector report).  Changed code, docs, and tests to bless the
      old behavior in these cases.
      
      DemoStorage.UndoLog:  this was wrong in several ways.  I'm still
      unsure about why it skips "packed" transactions.  That doesn't seem
      right, but I don't have time to wonder about that now.
      45814f90
  18. 27 Jun, 2005 1 commit
  19. 24 Jun, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 30900 from 3.4 branch. · 35136be9
      Tim Peters authored
      _triggerbase:  Make new-style class, for better debugability.
      
      Windows trigger.__init__:  Simplify more.  The trigger can connect
      after the other end has done  bind() and listen(); no need to wait
      for an accept() too, and so no need to fiddle with blocking/
      non-blocking or "expected" socket.error's either.
      35136be9
  20. 22 Jun, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 30894 from 3.4 branch. · c43d7912
      Tim Peters authored
      Largely rewritten.
      
      remove_loop_callback():  There were two definitions of this function.
      Deleted one, changed the other to use enumerate().
      
      poll():  This was trying to repair a bug in Python 2.2's asyncore.poll().
      2.2 is no longer supported, so removed this poll().
      
      _start_loop(), _stop_loop():  Removed; functionality folded into
      reworked loop().
      
      loop():  The signature of asyncore.loop() changed in 2.4, but this
      still had "the old" signature.  Instead of sniffing the signature
      of the Python in use, this does a saner thing:  the original
      asyncore.loop is captured, and called from the body of this loop().
      That way, (A) we don't care what asyncore.loop's signature is; and,
      (B) we still call the original Python code, so debugging prints and
      breakpoints (etc) stuffed into Python's asyncore no longer "vanish
      by magic" when ZEO is used.
      c43d7912
  21. 21 Jun, 2005 1 commit
    • Tim Peters's avatar
      Merge rev 30877 from 3.4 branch. · f0bb9db5
      Tim Peters authored
      Massive refactoring, to move the bulk of the trigger code into
      an OS-indepedent base class.
      
      __repr__:  Use the postive_id function to embed the machine address.
      Addresses with the high bit set trigger warnings before Python 2.4,
      and come out as negative numbers in 2.4+.
      
      Windows trigger.__init__:  Don't make 50 guesses at a port number
      to use, let Windows pick an available port for us.  Also documented
      the baffling single-thread socket setup dance, which took an hour to
      reverse-engineer (in large part because it used a bare "except" w/ no
      clue as to why).
      f0bb9db5