Skip to content

N
nxd-bom
Commit 950b65ce authored by Kirill Smelkov's avatar Kirill Smelkov
Browse files
Options

Also detect eggs that are used indirectly 

@jerome reports that BOM(ERP5) does not contain `certifi` even though it
is installed and used:

kirr/nxd-bom!1 (comment 185323)

The problem is: we were emitting information only about eggs that are
explicitly listed in the recipes, but dependent eggs were not reported
at all.

-> Fix this by using scripts, that buildout installs, as another root to
scan for eggs from. This works because zc.recipe.eggs puts all installed
eggs into sys.path adjustement inside those scripts.

On kirr/nxd-bom!1 (comment 185331) I
made wrong statement that .installed.cfg does not contain information
about which scripts were installed. This is not true: zc.recipe.egg puts
generated scripts into __buildout_installed__. This makes those scripts
to be reliable source of truth.

The diff for BOM(ERP5) before and after this patch is as follows:

--- d	2023-06-06 09:51:35.800234166 +0200
+++ e	2023-06-07 14:11:54.646892623 +0200
@@ -189,42 +189,91 @@
 zstd                         1.4.9      https://github.com/facebook/zstd/releases/download/v1.4.9/zstd-1.4.9.tar.gz

 >>> eggs:
+AccessControl                4.3        https://pypi.org/project/AccessControl/4.3/
 Acquisition                  4.7+slapospatched001 https://pypi.org/project/Acquisition/4.7/
+alabaster                    0.7.12     https://pypi.org/project/alabaster/0.7.12/
 APacheDEX                    1.8        https://pypi.org/project/APacheDEX/1.8/
 astor                        0.5        https://pypi.org/project/astor/0.5/
 astroid                      1.3.8+slapospatched001 https://pypi.org/project/astroid/1.3.8/
+atomize                      0.2.0      https://pypi.org/project/atomize/0.2.0/
+attrs                        18.2.0     https://pypi.org/project/attrs/18.2.0/
+AuthEncoding                 4.3        https://pypi.org/project/AuthEncoding/4.3/
+backports.functools_lru_cache 1.6.1      https://pypi.org/project/backports.functools_lru_cache/1.6.1/
 backports.lzma               0.0.14     https://pypi.org/project/backports.lzma/0.0.14/
+backports.shutil_get_terminal_size 1.0.0      https://pypi.org/project/backports.shutil_get_terminal_size/1.0.0/
+backports_abc                0.5        https://pypi.org/project/backports_abc/0.5/
 bcrypt                       3.1.4      https://pypi.org/project/bcrypt/3.1.4/
+Beaker                       1.11.0     https://pypi.org/project/Beaker/1.11.0/
+beautifulsoup4               4.8.2      https://pypi.org/project/beautifulsoup4/4.8.2/
 BTrees                       4.11.3     https://pypi.org/project/BTrees/4.11.3/
+CacheControl                 0.12.6     https://pypi.org/project/CacheControl/0.12.6/
 caucase                      0.9.14     https://pypi.org/project/caucase/0.9.14/
+certifi                      2020.4.5.1 https://pypi.org/project/certifi/2020.4.5.1/
 cffi                         1.15.0     https://pypi.org/project/cffi/1.15.0/
+Chameleon                    3.9.1      https://pypi.org/project/Chameleon/3.9.1/
 chardet                      3.0.4      https://pypi.org/project/chardet/3.0.4/
+click                        6.7        https://pypi.org/project/click/6.7/
+cliff                        2.8.3      https://pypi.org/project/cliff/2.8.3/
+cloudpickle                  0.5.3      https://pypi.org/project/cloudpickle/0.5.3/
+cmd2                         0.7.0      https://pypi.org/project/cmd2/0.7.0/
 cns.recipe.symlink           0.2.3      https://pypi.org/project/cns.recipe.symlink/0.2.3/
 collective.recipe.template   2.0        https://pypi.org/project/collective.recipe.template/2.0/
+cookies                      2.2.1      https://pypi.org/project/cookies/2.2.1/
 coverage                     5.5        https://pypi.org/project/coverage/5.5/
+croniter                     0.3.25     https://pypi.org/project/croniter/0.3.25/
 cryptography                 3.3.2      https://pypi.org/project/cryptography/3.3.2/
+cycler                       0.10.0     https://pypi.org/project/cycler/0.10.0/
 Cython                       0.29.24    https://pypi.org/project/Cython/0.29.24/
 cython_zstd                  0.2        https://pypi.org/project/cython_zstd/0.2/
+dask                         0.18.1     https://pypi.org/project/dask/0.18.1/
+dateparser                   0.7.6      https://pypi.org/project/dateparser/0.7.6/
+DateTime                     4.9        https://pypi.org/project/DateTime/4.9/
 decorator                    4.3.0      https://pypi.org/project/decorator/4.3.0/
 deepdiff                     3.3.0      https://pypi.org/project/deepdiff/3.3.0/
+distro                       1.6.0      https://pypi.org/project/distro/1.6.0/
+dnspython                    1.16.0     https://pypi.org/project/dnspython/1.16.0/
+DocumentTemplate             3.4        https://pypi.org/project/DocumentTemplate/3.4/
 docutils                     0.17.1     https://pypi.org/project/docutils/0.17.1/
+enum34                       1.1.10     https://pypi.org/project/enum34/1.1.10/
 erp5_coverage_plugin         0.0.1      https://pypi.org/project/erp5_coverage_plugin/0.0.1/
 erp5diff                     0.8.1.8    https://pypi.org/project/erp5diff/0.8.1.8/
+et_xmlfile                   1.0.1      https://pypi.org/project/et_xmlfile/1.0.1/
+ExtensionClass               4.9        https://pypi.org/project/ExtensionClass/4.9/
 facebook_sdk                 2.0.0      https://pypi.org/project/facebook_sdk/2.0.0/
 feedparser                   5.2.1      https://pypi.org/project/feedparser/5.2.1/
+five.localsitemanager        3.4        https://pypi.org/project/five.localsitemanager/3.4/
+Flask                        1.1.2      https://pypi.org/project/Flask/1.1.2/
+fpconst                      0.7.2      https://pypi.org/project/fpconst/0.7.2/
+funcsigs                     1.0.2      https://pypi.org/project/funcsigs/1.0.2/
+functools32                  3.2.3.post2 https://pypi.org/project/functools32/3.2.3.post2/
+future                       0.18.2     https://pypi.org/project/future/0.18.2/
+futures                      3.3.0      https://pypi.org/project/futures/3.3.0/
+gevent                       20.9.0     https://pypi.org/project/gevent/20.9.0/
+gitdb2                       2.0.5      https://pypi.org/project/gitdb2/2.0.5/
+GitPython                    2.1.11     https://pypi.org/project/GitPython/2.1.11/
 google_api_python_client     1.6.1      https://pypi.org/project/google_api_python_client/1.6.1/
+graphviz                     0.5.2      https://pypi.org/project/graphviz/0.5.2/
+greenlet                     0.4.17     https://pypi.org/project/greenlet/0.4.17/
 h5py                         2.7.1      https://pypi.org/project/h5py/2.7.1/
 haufe.requestmonitoring      0.6.0      https://pypi.org/project/haufe.requestmonitoring/0.6.0/
+html5lib                     1.1        https://pypi.org/project/html5lib/1.1/
 httplib2                     0.10.3     https://pypi.org/project/httplib2/0.10.3/
 huBarcode                    1.0.0      https://pypi.org/project/huBarcode/1.0.0/
+idna                         2.9        https://pypi.org/project/idna/2.9/
+Importing                    1.10       https://pypi.org/project/Importing/1.10/
 interval                     1.0.0      https://pypi.org/project/interval/1.0.0/
+ipaddress                    1.0.23     https://pypi.org/project/ipaddress/1.0.23/
 ipdb                         0.10.2     https://pypi.org/project/ipdb/0.10.2/
 ipykernel                    4.5.2      https://pypi.org/project/ipykernel/4.5.2/
 ipython                      5.3.0      https://pypi.org/project/ipython/5.3.0/
 ipython_genutils             0.1.0      https://pypi.org/project/ipython_genutils/0.1.0/
 ipywidgets                   6.0.0      https://pypi.org/project/ipywidgets/6.0.0/
+itsdangerous                 0.24       https://pypi.org/project/itsdangerous/0.24/
+jdcal                        1.3        https://pypi.org/project/jdcal/1.3/
 jedi                         0.15.1     https://pypi.org/project/jedi/0.15.1/
 Jinja2                       2.11.3     https://pypi.org/project/Jinja2/2.11.3/
+jsonpickle                   0.9.6      https://pypi.org/project/jsonpickle/0.9.6/
+jsonpointer                  2.2        https://pypi.org/project/jsonpointer/2.2/
 jsonschema                   3.0.2      https://pypi.org/project/jsonschema/3.0.2/
 jupyter                      1.0.0      https://pypi.org/project/jupyter/1.0.0/
 jupyter_client               5.0.0      https://pypi.org/project/jupyter_client/5.0.0/
@@ -233,19 +282,31 @@
 jupyterlab                   0.26.3     https://pypi.org/project/jupyterlab/0.26.3/
 jupyterlab_launcher          0.3.1      https://pypi.org/project/jupyterlab_launcher/0.3.1/
 lock_file                    2.0        https://pypi.org/project/lock_file/2.0/
+lockfile                     0.12.2     https://pypi.org/project/lockfile/0.12.2/
+logilab_common               1.3.0      https://pypi.org/project/logilab_common/1.3.0/
 lxml                         4.9.1      https://pypi.org/project/lxml/4.9.1/
+Mako                         1.1.4      https://pypi.org/project/Mako/1.1.4/
+MarkupSafe                   1.0        https://pypi.org/project/MarkupSafe/1.0/
 matplotlib                   2.1.2      https://pypi.org/project/matplotlib/2.1.2/
 mechanize                    0.4.8      https://pypi.org/project/mechanize/0.4.8/
+Missing                      4.2        https://pypi.org/project/Missing/4.2/
+mistune                      0.7.3      https://pypi.org/project/mistune/0.7.3/
 mock                         3.0.5      https://pypi.org/project/mock/3.0.5/
 more_itertools               5.0.0      https://pypi.org/project/more_itertools/5.0.0/
+mpmath                       0.19       https://pypi.org/project/mpmath/0.19/
 msgpack                      0.6.2      https://pypi.org/project/msgpack/0.6.2/
+MultiMapping                 4.1        https://pypi.org/project/MultiMapping/4.1/
+multipart                    0.1.1      https://pypi.org/project/multipart/0.1.1/
 munnel                       0.3        https://pypi.org/project/munnel/0.3/
 mysqlclient                  1.3.12     https://pypi.org/project/mysqlclient/1.3.12/
 nbconvert                    4.1.0      https://pypi.org/project/nbconvert/4.1.0/
 nbformat                     4.3.0      https://pypi.org/project/nbformat/4.3.0/
+netaddr                      0.7.19     https://pypi.org/project/netaddr/0.7.19/
+netifaces                    0.10.7     https://pypi.org/project/netifaces/0.10.7/
 networkx                     2.1        https://pypi.org/project/networkx/2.1/
 notebook                     4.4.1      https://pypi.org/project/notebook/4.4.1/
 numpy                        1.13.1     https://pypi.org/project/numpy/1.13.1/
+oauth2client                 4.0.0      https://pypi.org/project/oauth2client/4.0.0/
 oauthlib                     3.1.0      https://pypi.org/project/oauthlib/3.1.0/
 objgraph                     3.1.0      https://pypi.org/project/objgraph/3.1.0/
 ocropy                       1.0+slapospatched001 https://pypi.org/project/ocropy/1.0/
@@ -253,40 +314,72 @@
 openpyxl                     2.4.8      https://pypi.org/project/openpyxl/2.4.8/
 pandas                       0.24.2     https://pypi.org/project/pandas/0.24.2/
 paramiko                     2.11.0     https://pypi.org/project/paramiko/2.11.0/
+parso                        0.5.1      https://pypi.org/project/parso/0.5.1/
+passlib                      1.7.4      https://pypi.org/project/passlib/1.7.4/
+PasteDeploy                  2.1.1      https://pypi.org/project/PasteDeploy/2.1.1/
+pathlib2                     2.2.1      https://pypi.org/project/pathlib2/2.2.1/
 patsy                        0.4.1      https://pypi.org/project/patsy/0.4.1/
+pbr                          5.9.0      https://pypi.org/project/pbr/5.9.0/
+pem                          21.1.0     https://pypi.org/project/pem/21.1.0/
+Persistence                  3.6        https://pypi.org/project/Persistence/3.6/
 persistent                   4.9.3      https://pypi.org/project/persistent/4.9.3/
+pexpect                      4.2.1      https://pypi.org/project/pexpect/4.2.1/
+pickleshare                  0.7.4      https://pypi.org/project/pickleshare/0.7.4/
 Pillow                       6.2.2      https://pypi.org/project/Pillow/6.2.2/
 plone.recipe.command         1.1        https://pypi.org/project/plone.recipe.command/1.1/
 ply                          3.11       https://pypi.org/project/ply/3.11/
 polib                        1.0.8      https://pypi.org/project/polib/1.0.8/
 pprofile                     2.0.4      https://pypi.org/project/pprofile/2.0.4/
+prettytable                  0.7.2      https://pypi.org/project/prettytable/0.7.2/
+Products.BTreeFolder2        4.4+slapospatched001 https://pypi.org/project/Products.BTreeFolder2/4.4/
+Products.CMFCore             2.7.0+slapospatched001 https://pypi.org/project/Products.CMFCore/2.7.0/
 Products.DCWorkflow          2.4.1+slapospatched001 https://pypi.org/project/Products.DCWorkflow/2.4.1/
 Products.ExternalMethod      4.7        https://pypi.org/project/Products.ExternalMethod/4.7/
+Products.GenericSetup        2.3.0      https://pypi.org/project/Products.GenericSetup/2.3.0/
+Products.MailHost            4.13       https://pypi.org/project/Products.MailHost/4.13/
 Products.MimetypesRegistry   2.1.8      https://pypi.org/project/Products.MimetypesRegistry/2.1.8/
 Products.PluggableAuthService 2.8.1      https://pypi.org/project/Products.PluggableAuthService/2.8.1/
+Products.PluginRegistry      1.6        https://pypi.org/project/Products.PluginRegistry/1.6/
+Products.PythonScripts       4.15       https://pypi.org/project/Products.PythonScripts/4.15/
 Products.Sessions            4.15       https://pypi.org/project/Products.Sessions/4.15/
 Products.SiteErrorLog        5.7        https://pypi.org/project/Products.SiteErrorLog/5.7/
+Products.StandardCacheManagers 4.2        https://pypi.org/project/Products.StandardCacheManagers/4.2/
 Products.TIDStorage          5.5.0      https://pypi.org/project/Products.TIDStorage/5.5.0/
+Products.ZCatalog            5.4        https://pypi.org/project/Products.ZCatalog/5.4/
 Products.ZODBMountPoint      1.3        https://pypi.org/project/Products.ZODBMountPoint/1.3/
 Products.ZSQLMethods         3.16       https://pypi.org/project/Products.ZSQLMethods/3.16/
+prompt_toolkit               1.0.13     https://pypi.org/project/prompt_toolkit/1.0.13/
 psutil                       5.8.0      https://pypi.org/project/psutil/5.8.0/
+ptyprocess                   0.5.1      https://pypi.org/project/ptyprocess/0.5.1/
+pyasn1                       0.4.5      https://pypi.org/project/pyasn1/0.4.5/
+pyasn1_modules               0.0.8      https://pypi.org/project/pyasn1_modules/0.0.8/
 pycountry                    17.1.8     https://pypi.org/project/pycountry/17.1.8/
+pycparser                    2.20       https://pypi.org/project/pycparser/2.20/
 pycrypto                     2.6.1      https://pypi.org/project/pycrypto/2.6.1/
+pycryptodomex                3.10.1     https://pypi.org/project/pycryptodomex/3.10.1/
 pycurl                       7.43.0     https://pypi.org/project/pycurl/7.43.0/
 pydot                        1.4.2      https://pypi.org/project/pydot/1.4.2/
 pyflakes                     1.5.0      https://pypi.org/project/pyflakes/1.5.0/
+Pygments                     2.2.0      https://pypi.org/project/Pygments/2.2.0/
 pygolang                     0.1        https://pypi.org/project/pygolang/0.1/
+pyjwkest                     1.4.2      https://pypi.org/project/pyjwkest/1.4.2/
+PyJWT                        1.7.1      https://pypi.org/project/PyJWT/1.7.1/
 pylint                       1.4.4+slapospatched002 https://pypi.org/project/pylint/1.4.4/
 Pympler                      0.4.3      https://pypi.org/project/Pympler/0.4.3/
 PyMySQL                      0.10.1     https://pypi.org/project/PyMySQL/0.10.1/
 PyNaCl                       1.3.0      https://pypi.org/project/PyNaCl/1.3.0/
+pyOpenSSL                    19.1.0     https://pypi.org/project/pyOpenSSL/19.1.0/
+pyparsing                    2.2.0      https://pypi.org/project/pyparsing/2.2.0/
 PyPDF2                       1.26.0+slapospatched001 https://pypi.org/project/PyPDF2/1.26.0/
 pyrsistent                   0.16.1     https://pypi.org/project/pyrsistent/0.16.1/
 PyRSS2Gen                    1.1        https://pypi.org/project/PyRSS2Gen/1.1/
 PyStemmer                    1.3.0      https://pypi.org/project/PyStemmer/1.3.0/
 pysvn                        1.9.15+slapospatched001 https://pypi.org/project/pysvn/1.9.15/
 pytesseract                  0.2.2      https://pypi.org/project/pytesseract/0.2.2/
+python_dateutil              2.8.2      https://pypi.org/project/python_dateutil/2.8.2/
+python_gettext               4.1        https://pypi.org/project/python_gettext/4.1/
 python_ldap                  2.4.32+slapospatched001 https://pypi.org/project/python_ldap/2.4.32/
+python_libmilter             1.0.3      https://pypi.org/project/python_libmilter/1.0.3/
 python_magic                 0.4.12+slapospatched001 https://pypi.org/project/python_magic/0.4.12/
 python_memcached             1.58       https://pypi.org/project/python_memcached/1.58/
 pytracemalloc                1.2        https://pypi.org/project/pytracemalloc/1.2/
@@ -295,32 +388,68 @@
 PyYAML                       5.4.1      https://pypi.org/project/PyYAML/5.4.1/
 pyzmq                        16.0.2     https://pypi.org/project/pyzmq/16.0.2/
 qrcode                       5.3        https://pypi.org/project/qrcode/5.3/
+qtconsole                    4.3.0      https://pypi.org/project/qtconsole/4.3.0/
 Record                       3.6        https://pypi.org/project/Record/3.6/
+regex                        2020.9.27  https://pypi.org/project/regex/2020.9.27/
 requests                     2.27.1     https://pypi.org/project/requests/2.27.1/
 responses                    0.10.6     https://pypi.org/project/responses/0.10.6/
+RestrictedPython             5.2        https://pypi.org/project/RestrictedPython/5.2/
+rfc3987                      1.3.8      https://pypi.org/project/rfc3987/1.3.8/
+roman                        3.3        https://pypi.org/project/roman/3.3/
+rsa                          3.4.2      https://pypi.org/project/rsa/3.4.2/
+scandir                      1.5        https://pypi.org/project/scandir/1.5/
 scikit_image                 0.14.0     https://pypi.org/project/scikit_image/0.14.0/
 scikit_learn                 0.18.1     https://pypi.org/project/scikit_learn/0.18.1/
 scipy                        0.19.0     https://pypi.org/project/scipy/0.19.0/
 seaborn                      0.7.1      https://pypi.org/project/seaborn/0.7.1/
+selectors34                  1.2        https://pypi.org/project/selectors34/1.2/
 selenium                     3.141.0+slapospatched001 https://pypi.org/project/selenium/3.141.0/
 setproctitle                 1.1.10     https://pypi.org/project/setproctitle/1.1.10/
+setuptools                   44.1.1     https://pypi.org/project/setuptools/44.1.1/
+simplegeneric                0.8.1      https://pypi.org/project/simplegeneric/0.8.1/
+singledispatch               3.4.0.3    https://pypi.org/project/singledispatch/3.4.0.3/
+six                          1.16.0     https://pypi.org/project/six/1.16.0/
 slapos.cookbook              1.0.305    https://pypi.org/project/slapos.cookbook/1.0.305/
+slapos.core                  1.9.3      https://pypi.org/project/slapos.core/1.9.3/
 slapos.recipe.template       5.1        https://pypi.org/project/slapos.recipe.template/5.1/
 slapos.toolbox               0.128      https://pypi.org/project/slapos.toolbox/0.128/
+smmap2                       2.0.5      https://pypi.org/project/smmap2/2.0.5/
 SOAPpy                       0.12.0nxd001 http://www.nexedi.org/static/packages/source/SOAPpy-0.12.0nxd001.tar.gz
+soupsieve                    1.9.5      https://pypi.org/project/soupsieve/1.9.5/
 spyne                        2.12.14    https://pypi.org/project/spyne/2.12.14/
 statsmodels                  0.8.0      https://pypi.org/project/statsmodels/0.8.0/
+stevedore                    1.21.0     https://pypi.org/project/stevedore/1.21.0/
 strict_rfc3339               0.7        https://pypi.org/project/strict_rfc3339/0.7/
 StructuredText               2.11.1     https://pypi.org/project/StructuredText/2.11.1/
+subprocess32                 3.5.4      https://pypi.org/project/subprocess32/3.5.4/
 suds                         0.4        https://pypi.org/project/suds/0.4/
+supervisor                   4.1.0      https://pypi.org/project/supervisor/4.1.0/
 sympy                        1.1.1      https://pypi.org/project/sympy/1.1.1/
 tempstorage                  5.2        https://pypi.org/project/tempstorage/5.2/
+terminado                    0.6        https://pypi.org/project/terminado/0.6/
+toolz                        0.9.0      https://pypi.org/project/toolz/0.9.0/
+tornado                      4.4.2      https://pypi.org/project/tornado/4.4.2/
+traitlets                    4.3.3      https://pypi.org/project/traitlets/4.3.3/
+transaction                  3.0.1      https://pypi.org/project/transaction/3.0.1/
+trollius                     2.2.1      https://pypi.org/project/trollius/2.2.1/
 typing                       3.10.0.0   https://pypi.org/project/typing/3.10.0.0/
+tzlocal                      1.5.1      https://pypi.org/project/tzlocal/1.5.1/
+unicodecsv                   0.14.1     https://pypi.org/project/unicodecsv/0.14.1/
 unidiff                      0.5.5      https://pypi.org/project/unidiff/0.5.5/
+uritemplate                  3.0.0      https://pypi.org/project/uritemplate/3.0.0/
+urllib3                      1.26.12    https://pypi.org/project/urllib3/1.26.12/
 urlnorm                      1.1.4      https://pypi.org/project/urlnorm/1.1.4/
 uuid                         1.30       https://pypi.org/project/uuid/1.30/
 validictory                  1.1.0      https://pypi.org/project/validictory/1.1.0/
 waitress                     1.4.4+slapospatched006 https://pypi.org/project/waitress/1.4.4/
+wcwidth                      0.1.7      https://pypi.org/project/wcwidth/0.1.7/
+webcolors                    1.10       https://pypi.org/project/webcolors/1.10/
+webencodings                 0.5.1      https://pypi.org/project/webencodings/0.5.1/
+WebOb                        1.8.5      https://pypi.org/project/WebOb/1.8.5/
+WebTest                      2.0.33     https://pypi.org/project/WebTest/2.0.33/
+Werkzeug                     1.0.1      https://pypi.org/project/Werkzeug/1.0.1/
+widgetsnbextension           2.0.0      https://pypi.org/project/widgetsnbextension/2.0.0/
+WSGIProxy2                   0.4.6      https://pypi.org/project/WSGIProxy2/0.4.6/
 xfw                          0.10       https://pypi.org/project/xfw/0.10/
 xlrd                         1.1.0      https://pypi.org/project/xlrd/1.1.0/
 xml_marshaller               1.0.2      https://pypi.org/project/xml_marshaller/1.0.2/
@@ -328,16 +457,70 @@
 xupdate_processor            0.5        https://pypi.org/project/xupdate_processor/0.5/
 yapf                         0.28.0     https://pypi.org/project/yapf/0.28.0/
 z3c.etestbrowser             3.0.1      https://pypi.org/project/z3c.etestbrowser/3.0.1/
+z3c.pt                       3.3.1      https://pypi.org/project/z3c.pt/3.3.1/
 zbarlight                    2.3        https://pypi.org/project/zbarlight/2.3/
+zc.buildout                  2.7.1+slapos019 http://www.nexedi.org/static/packages/source/slapos.buildout/zc.buildout-2.7.1+slapos019.tar.gz
+zc.lockfile                  1.4        https://pypi.org/project/zc.lockfile/1.4/
+ZConfig                      3.6.1      https://pypi.org/project/ZConfig/3.6.1/
+zdaemon                      4.2.0      https://pypi.org/project/zdaemon/4.2.0/
 ZEO                          5.4.0      https://pypi.org/project/ZEO/5.4.0/
+zExceptions                  4.3        https://pypi.org/project/zExceptions/4.3/
 zLOG                         3.1        https://pypi.org/project/zLOG/3.1/
 ZODB                         5.8.0      https://pypi.org/project/ZODB/5.8.0/
+ZODB3                        3.11.0     https://pypi.org/project/ZODB3/3.11.0/
+zodbpickle                   2.6        https://pypi.org/project/zodbpickle/2.6/
 zodbtools                    0.0.0.dev8 https://pypi.org/project/zodbtools/0.0.0.dev8/
+zodburi                      2.5.0      https://pypi.org/project/zodburi/2.5.0/
 Zope                         4.8.7      https://pypi.org/project/Zope/4.8.7/
+zope.annotation              4.8        https://pypi.org/project/zope.annotation/4.8/
+zope.app.appsetup            4.2.0      https://pypi.org/project/zope.app.appsetup/4.2.0/
+zope.app.publication         4.5        https://pypi.org/project/zope.app.publication/4.5/
+zope.authentication          5.0        https://pypi.org/project/zope.authentication/5.0/
+zope.browser                 2.4        https://pypi.org/project/zope.browser/2.4/
+zope.browsermenu             4.4        https://pypi.org/project/zope.browsermenu/4.4/
+zope.browserpage             4.4.0      https://pypi.org/project/zope.browserpage/4.4.0/
+zope.browserresource         4.4        https://pypi.org/project/zope.browserresource/4.4/
+zope.cachedescriptors        4.4        https://pypi.org/project/zope.cachedescriptors/4.4/
+zope.component               5.0.1      https://pypi.org/project/zope.component/5.0.1/
+zope.configuration           4.4.1      https://pypi.org/project/zope.configuration/4.4.1/
 zope.container               4.10       https://pypi.org/project/zope.container/4.10/
+zope.contentprovider         4.2.1      https://pypi.org/project/zope.contentprovider/4.2.1/
+zope.contenttype             4.6        https://pypi.org/project/zope.contenttype/4.6/
+zope.datetime                4.3.0      https://pypi.org/project/zope.datetime/4.3.0/
+zope.deferredimport          4.4        https://pypi.org/project/zope.deferredimport/4.4/
+zope.deprecation             4.4.0      https://pypi.org/project/zope.deprecation/4.4.0/
+zope.dottedname              4.3        https://pypi.org/project/zope.dottedname/4.3/
+zope.error                   4.6        https://pypi.org/project/zope.error/4.6/
+zope.event                   4.6        https://pypi.org/project/zope.event/4.6/
+zope.exceptions              4.6        https://pypi.org/project/zope.exceptions/4.6/
+zope.filerepresentation      5.0.0      https://pypi.org/project/zope.filerepresentation/5.0.0/
 zope.globalrequest           1.6        https://pypi.org/project/zope.globalrequest/1.6/
+zope.hookable                5.4        https://pypi.org/project/zope.hookable/5.4/
+zope.i18n                    4.9.0      https://pypi.org/project/zope.i18n/4.9.0/
+zope.i18nmessageid           5.1.1      https://pypi.org/project/zope.i18nmessageid/5.1.1/
+zope.interface               5.4.0      https://pypi.org/project/zope.interface/5.4.0/
+zope.lifecycleevent          4.4        https://pypi.org/project/zope.lifecycleevent/4.4/
+zope.location                4.3        https://pypi.org/project/zope.location/4.3/
+zope.minmax                  2.3        https://pypi.org/project/zope.minmax/2.3/
+zope.pagetemplate            4.6.0      https://pypi.org/project/zope.pagetemplate/4.6.0/
+zope.processlifetime         2.4        https://pypi.org/project/zope.processlifetime/2.4/
+zope.proxy                   4.6.1      https://pypi.org/project/zope.proxy/4.6.1/
+zope.ptresource              4.3.0      https://pypi.org/project/zope.ptresource/4.3.0/
+zope.publisher               6.1.0      https://pypi.org/project/zope.publisher/6.1.0/
+zope.schema                  6.2.1      https://pypi.org/project/zope.schema/6.2.1/
 zope.security                5.8        https://pypi.org/project/zope.security/5.8/
+zope.sendmail                5.3        https://pypi.org/project/zope.sendmail/5.3/
+zope.sequencesort            4.2        https://pypi.org/project/zope.sequencesort/4.2/
+zope.session                 4.5        https://pypi.org/project/zope.session/4.5/
+zope.site                    4.6.1      https://pypi.org/project/zope.site/4.6.1/
+zope.size                    4.4        https://pypi.org/project/zope.size/4.4/
+zope.structuredtext          4.4        https://pypi.org/project/zope.structuredtext/4.4/
+zope.tal                     4.5        https://pypi.org/project/zope.tal/4.5/
+zope.tales                   5.2        https://pypi.org/project/zope.tales/5.2/
+zope.testbrowser             5.5.1      https://pypi.org/project/zope.testbrowser/5.5.1/
 zope.testing                 4.7        https://pypi.org/project/zope.testing/4.7/
+zope.traversing              4.4.1      https://pypi.org/project/zope.traversing/4.4.1/
+zope.viewlet                 4.3        https://pypi.org/project/zope.viewlet/4.3/
 Zope2                        4.0        https://pypi.org/project/Zope2/4.0/
 ZServer                      4.0.2      https://pypi.org/project/ZServer/4.0.2/
parent 6bcdd408
Hide whitespace changes
Inline Side-by-side
Showing with 87 additions and 3 deletions
nxdbom/__init__.py
View file @ 950b65ce
...@@ -28,7 +28,7 @@ An example of generated bill of material is provided in example/ors-bom.txt . ...@@ -28,7 +28,7 @@ An example of generated bill of material is provided in example/ors-bom.txt .
from __future__ import print_function from __future__ import print_function
import sys, configparser, re import sys, configparser, re, codecs
from os.path import basename from os.path import basename
from glob import glob from glob import glob
from collections import namedtuple from collections import namedtuple
...@@ -188,6 +188,30 @@ def bom_software(installed_software_path): # -> {} (name,kind) -> PkgInfo ...@@ -188,6 +188,30 @@ def bom_software(installed_software_path): # -> {} (name,kind) -> PkgInfo
addbom(eggpath, 'egg') # from where destination is downloaded from addbom(eggpath, 'egg') # from where destination is downloaded from
elif recipe in ('zc.recipe.egg', 'zc.recipe.egg:eggs', 'zc.recipe.egg:scripts'): elif recipe in ('zc.recipe.egg', 'zc.recipe.egg:eggs', 'zc.recipe.egg:scripts'):
# zc.recipe.egg:* installs
# 1) eggs, that are explicitly specified,
# 2) indirect dependencies of eggs from "1", and
# 3) scripts that use all those eggs from "1" and "2".
# go through installed scripts and retrieve the eggs we can
# retrieve to cover both "1" and "2". This does not cover the case
# when no scripts are installed at all, but is better than nothing.
# An alternative is to fix zc.recipe.egg itself to emit information
# about all eggs it installs:
xeggs = {} # xeggname -> xeggname-ver-....egg on the filesystem
installedv = part['__buildout_installed__'].split()
for f in installedv:
for xeggpath in eggscript_imports(f):
if not xeggpath.endswith('.egg'):
continue # e.g. neoppod-repository, *.egg-link
xeggfile = basename(xeggpath)
xegg, _ = namever(xeggfile)
if xegg in xeggs:
assert xeggs[xegg] == xeggfile, (xeggs[xegg], xeggfile)
else:
xeggs[xegg] = xeggfile
addbom(xeggfile, 'egg')
# now go through explicitly listed eggs and also add them
# XXX sadly zc.recipe.egg neither saves in .installed.cfg information about where the eggs are installed, # XXX sadly zc.recipe.egg neither saves in .installed.cfg information about where the eggs are installed,
# nor there are [versions] save. # nor there are [versions] save.
# -> look the egg on the filesystem # -> look the egg on the filesystem
...@@ -201,14 +225,17 @@ def bom_software(installed_software_path): # -> {} (name,kind) -> PkgInfo ...@@ -201,14 +225,17 @@ def bom_software(installed_software_path): # -> {} (name,kind) -> PkgInfo
m = _egg_re.match(eggname) m = _egg_re.match(eggname)
assert m is not None, eggname assert m is not None, eggname
eggname = m.group('name') # neoppod[admin, ctl, master] -> neoppod eggname = m.group('name') # neoppod[admin, ctl, master] -> neoppod
eggname = eggcanon(eggname) # cython-zstd -> cython_zstd
if eggname in xeggs:
continue # we already reported this egg via scripts above
if glob('%s/%s.egg-link' % (eggdev, eggname)): if glob('%s/%s.egg-link' % (eggdev, eggname)):
continue # e.g. wendelin.core.egg-link points to wendelin.core cloned via git and already reported as git continue # e.g. wendelin.core.egg-link points to wendelin.core cloned via git and already reported as git
installed_eggs = [basename(_) for _ in (glob('%s/*.egg' % eggdir) + installed_eggs = [basename(_) for _ in (glob('%s/*.egg' % eggdir) +
glob('%s/*.egg' % eggdev))] glob('%s/*.egg' % eggdev))]
eggv = [_ for _ in installed_eggs if _.lower().startswith('%s-' % eggcanon(eggname))] eggv = [_ for _ in installed_eggs if _.lower().startswith('%s-' % eggname)]
if len(eggv) == 0: if len(eggv) == 0:
raise ValueError('egg %s not found' % eggname) raise ValueError('egg %s not found' % eggname)
if len(eggv) > 1: if len(eggv) > 1:
...@@ -254,6 +281,28 @@ def eggcanon(eggname): ...@@ -254,6 +281,28 @@ def eggcanon(eggname):
return eggname.lower().replace('-', '_') return eggname.lower().replace('-', '_')
# eggscript_imports returns list of eggs put into python path in python script
# generated by zc.recipe.egg at path.
_re_importv = re.compile(r"\nsys\.path\[0:0\] = \[\n(?P<importv>( *'[^'\n]+',\n)+) *]", re.MULTILINE)
def eggscript_imports(path):
with open(path, 'r') as f:
data = f.read()
m = _re_importv.search(data)
if m is None:
raise RuntimeError("%s does not look like zc.recipe.egg script" % path)
importv = []
for x in m.group('importv').splitlines():
x = x.strip()
x = removeprefix(x, "'")
x = removesuffix(x, "',")
x = codecs.escape_decode(x)[0].decode('utf-8')
importv.append(x)
return importv
def bom_node(XXX): def bom_node(XXX):
1/0 1/0
# TODO bom_node should: # TODO bom_node should:
......
nxdbom/nxdbom_test.py
View file @ 950b65ce
...@@ -177,6 +177,7 @@ case1("""\ ...@@ -177,6 +177,7 @@ case1("""\
recipe = zc.recipe.egg recipe = zc.recipe.egg
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = lxml eggs = lxml
pycurl pycurl
Mako Mako
...@@ -195,6 +196,7 @@ case1("""\ ...@@ -195,6 +196,7 @@ case1("""\
recipe = zc.recipe.egg:eggs recipe = zc.recipe.egg:eggs
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = ZODB eggs = ZODB
BTrees BTrees
-- /ROOT/eggs/ZODB-5.8.0-py2.7.egg/x -- -- /ROOT/eggs/ZODB-5.8.0-py2.7.egg/x --
...@@ -210,6 +212,7 @@ case1("""\ ...@@ -210,6 +212,7 @@ case1("""\
recipe = zc.recipe.egg:scripts recipe = zc.recipe.egg:scripts
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = pygolang eggs = pygolang
neoppod[admin, ctl, master] neoppod[admin, ctl, master]
-- /ROOT/develop-eggs/pygolang-0.1-py2.7-linux-x86_64.egg/x -- -- /ROOT/develop-eggs/pygolang-0.1-py2.7-linux-x86_64.egg/x --
...@@ -225,6 +228,7 @@ case1("""\ ...@@ -225,6 +228,7 @@ case1("""\
recipe = zc.recipe.egg recipe = zc.recipe.egg
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
-- /ROOT/eggs/selenium-3.141.0-py2.7.egg/x -- -- /ROOT/eggs/selenium-3.141.0-py2.7.egg/x --
""", """ """, """
>>> eggs: >>> eggs:
...@@ -256,6 +260,7 @@ case1("""\ ...@@ -256,6 +260,7 @@ case1("""\
recipe = zc.recipe.egg recipe = zc.recipe.egg
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = eggs =
aaa aaa
bbb%(__buildout_space_n__)s bbb%(__buildout_space_n__)s
...@@ -283,6 +288,7 @@ case1("""\ ...@@ -283,6 +288,7 @@ case1("""\
recipe = zc.recipe.egg recipe = zc.recipe.egg
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = SOAPpy eggs = SOAPpy
zc.recipe.egg zc.recipe.egg
zc.buildout zc.buildout
...@@ -360,6 +366,7 @@ __buildout_installed__ = /Root/develop-eggs/Cython-0.29.24-py2.7-linux-x86_64.eg ...@@ -360,6 +366,7 @@ __buildout_installed__ = /Root/develop-eggs/Cython-0.29.24-py2.7-linux-x86_64.eg
recipe = zc.recipe.egg recipe = zc.recipe.egg
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = cython-zstd eggs = cython-zstd
cython cython
-- /ROOT/develop-eggs/cython_zstd-0.2-py2.7-linux-x86_64.egg/x -- -- /ROOT/develop-eggs/cython_zstd-0.2-py2.7-linux-x86_64.egg/x --
...@@ -376,6 +383,7 @@ case1("""\ ...@@ -376,6 +383,7 @@ case1("""\
recipe = zc.recipe.egg recipe = zc.recipe.egg
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = aaa eggs = aaa
bbb bbb
ccc ccc
...@@ -396,12 +404,39 @@ case1(""" ...@@ -396,12 +404,39 @@ case1("""
recipe = zc.recipe.egg recipe = zc.recipe.egg
_d = /ROOT/develop-eggs _d = /ROOT/develop-eggs
_e = /ROOT/eggs _e = /ROOT/eggs
__buildout_installed__ =
eggs = setuptools eggs = setuptools
-- /ROOT/eggs/setuptools-44.1.1-py3.7.egg -- -- /ROOT/eggs/setuptools-44.1.1-py3.7.egg --
-- /ROOT/eggs/setuptools-44.1.1-py3.9.egg -- -- /ROOT/eggs/setuptools-44.1.1-py3.9.egg --
""", """,
ValueError("egg setuptools is present multiple times: ['setuptools-44.1.1-py3.7.egg', 'setuptools-44.1.1-py3.9.egg']")) ValueError("egg setuptools is present multiple times: ['setuptools-44.1.1-py3.7.egg', 'setuptools-44.1.1-py3.9.egg']"))
# eggs installed indirectly are also reported
case1("""
[eggs]
recipe = zc.recipe.egg
_d = /ROOT/develop-eggs
_e = /ROOT/eggs
eggs = Zope
__buildout_installed__ = /ROOT/bin/runwsgi
-- /ROOT/eggs/Zope-4.8.7-py2.7.egg/x --
-- /ROOT/bin/runwsgi --
#!/ROOT/bin/python2.7
import sys
sys.path[0:0] = [
'/ROOT/part/neoppod-repository',
'/ROOT/eggs/Zope-4.8.7-py2.7.egg',
'/ROOT/eggs/certifi-2020.4.5.1-py2.7.egg',
]
...
""", """
>>> eggs:
certifi 2020.4.5.1 https://pypi.org/project/certifi/2020.4.5.1/
Zope 4.8.7 https://pypi.org/project/Zope/4.8.7/
""")
# libreoffice # libreoffice
case1("""\ case1("""\
[libreoffice-bin] [libreoffice-bin]
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7