Commit 6e696faa authored by Kirill Smelkov's avatar Kirill Smelkov

v↑ tempstorage 2.12.2 -> 5.2(ZODB5) 3-nxd(ZODB4-wc2)

tempstorage 2.12.2 is from 2012 and misses loadBefore fixes which are
important when ERP5 is run with ZODB5 or ZODB4-wc2:

https://github.com/zopefoundation/tempstorage/commit/d187a86d
https://github.com/zopefoundation/tempstorage/pull/16

When the first loadBefore fix is not there, the problem manifests itself as
Zope not starting correctly with the following error in its log:

    2021-07-02 08:37:25,579 ERROR Zope.ZODBMountPoint Failed to mount database. <class 'ZODB.POSException.ReadConflictError'> (^@^@^@^@^@^@^@^@)
    Traceback (most recent call last):
     File ".../eggs/Zope2-2.13.30-py2.7.egg/Products/ZODBMountPoint/MountedObject.py", line 248, in _getOrOpenObject
       root = conn.root()
     File ".../parts/ZODB4-wc2/src/ZODB/Connection.py", line 405, in root
       return RootConvenience(self.get(z64))
     File ".../parts/ZODB4-wc2/src/ZODB/Connection.py", line 276, in get
       p, serial = self._load(oid)
     File ".../parts/ZODB4-wc2/src/ZODB/Connection.py", line 246, in _load
       return self._loadFrom(oid, self._storage)
     File ".../parts/ZODB4-wc2/src/ZODB/Connection.py", line 255, in _loadFrom
       raise ReadConflictError(oid)
    ReadConflictError: ^@^@^@^@^@^@^@^@

When the second loadBefore fix is not there, the problem manifests
itself as data corruption and sporadic POSKeyErrors
(https://github.com/zopefoundation/tempstorage/issues/8).

-> Fix both problem by using recent tempstorage with ZODB5 and ZODB4:

- ZODB5: just use tempstorage 5.2 egg
- ZODB4-wc2: use tempstorage 3 + loadBefore backports:
  https://lab.nexedi.com/nexedi/tempstorage/commits/3-nxd
- ZODB4: stays at 2.12.2 for now for this change to be as risk-free as
  possible. However, due to the fixes and e.g. https://github.com/zopefoundation/tempstorage/issues/8#issuecomment-873894941
  it makes sense to switch ZODB4 to tempstorage 3-nxd as well.

/cc @tomo, @romain, @rafael
parent 3e8439b3
# SlapOS component for tempstorage.
# https://github.com/zopefoundation/tempstorage
[buildout]
extends =
../ZODB/buildout.cfg
../git/buildout.cfg
# tempstorage provides tempstorage<X> depending on ZODB major version.
#
# - tempstorage >= 4 works only with ZODB5 because ZODB commit protocol was changed
# https://github.com/zopefoundation/tempstorage/commit/5cc223ea
# - tempstorage <= 5.2 is vulnerable to data corruption in loadBefore
# https://github.com/zopefoundation/tempstorage/issues/8
# https://github.com/zopefoundation/tempstorage/pull/16
# - tempstorage 3-nxd provides loadBefore backports to tempstorage 3.
[tempstorage]
recipe = slapos.recipe.build
depends = ${ZODB:egg}
init =
# link/depend to tempstorage<ZODB.major>
zodb = self.buildout['ZODB']
zmajor = zodb['major']
tempstorage_x = self.buildout['tempstorage'+zmajor]
options['depends'] += '$${%s:egg}' % tempstorage_x.name
options['egg'] = tempstorage_x['egg']
# update [versions] from what is needed by tempstorage<X>
self.buildout.parse('[_tempstorage-versions]\n' + tempstorage_x['egg-versions'])
versions = self.buildout['versions']
versions.update(self.buildout['_tempstorage-versions'])
# propagate updated [versions] -> easy_install
# (buildout does this in Buildout constructor)
import zc.buildout.easy_install
zc.buildout.easy_install.default_versions(versions)
# tempstorage5 is plain upstream egg
[tempstorage5]
recipe = zc.recipe.egg:eggs
egg = tempstorage
eggs = ${:egg}
egg-versions =
tempstorage = 5.2
# tempstorage4 is plain upstream egg, but better be changed to be the same as
# tempstorage4-wc2 because of loadBefore fixes.
[tempstorage4]
<= tempstorage5
egg-versions =
tempstorage = 2.12.2
# tempstorage4-wc2 is tempstorage 3 + backports for loadBefore fixes
[tempstorage4-wc2]
recipe = zc.recipe.egg:develop
setup = ${tempstorage4-wc2-repository:location}
egg = tempstorage
egg-versions =
tempstorage =
[tempstorage4-wc2-repository]
recipe = slapos.recipe.build:gitclone
repository = https://lab.nexedi.com/nexedi/tempstorage.git
branch = 3-nxd
revision= 77b49295db78
location = ${buildout:parts-directory}/tempstorage
git-executable = ${git:location}/bin/git
......@@ -37,6 +37,7 @@ extends =
../../component/scikit-image/buildout.cfg
../../component/PyWavelets/buildout.cfg
../../component/subversion/buildout.cfg
../../component/tempstorage/buildout.cfg
../../component/tesseract/buildout.cfg
../../component/w3m/buildout.cfg
../../component/poppler/buildout.cfg
......@@ -497,6 +498,7 @@ eggs = ${neoppod:eggs}
# Zope
Zope2
${tempstorage:egg}
# Zope acquisition patch
Acquisition
# for runzeo
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment