Skip to content

erp5
erp5
Commit 23e06437 authored by Douglas's avatar Douglas
Browse files
Options

erp5 kernel: improved detection of variables from user context that cannot be put in the zodb 

Variables are investigated, recursively in case of container objects (like lists, for example),
to detect if they can be stored in the ZODB.
In this investigation persistent objects are identified by being an instance of the object
class and implementing a `__getstate__` method that raises no exception. If the variable is
not a Persistent object then we try to pickle and load it.

While developing the pickleable object identification a complication was found. It seems that
the code cannot capture cPickle.PicklingError in the usual way, `except cPickle.PicklingError`.
It's consequence of some weirdness with regards to pickle/cPickle modules exceptions classes and
more about it can be read at http://bugs.python.org/issue1457119. So, the workaround for this complication
was to catch all exceptions and check the exception class name as string.

The whole check for zodb persistence was moved into an utility function for the sake of readability
and code maintenance.

The Base_executeJupyter script object was transformed into an extension to be able to properly handle
transaction errors and render them correctly inside Jupyter.
parent 11eb5168
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 182 additions and 196 deletions
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/Base_executeJupyter.py deleted 100644 → 0
View file @ 11eb5168
"""
Python script to create Data Notebook or update existing Data Notebooks
identifying notebook by reference from user.
Expected behaviour from this script:-
1. Return unauthorized message for non-developer user.
2. Create new 'Data Notebook' for new reference.
3. Add new 'Data Notebook Line'to the existing Data Notebook on basis of reference.
4. Return python dictionary containing list of all notebooks for 'request_reference=True'
"""
portal = context.getPortalObject()
# Check permissions for current user and display message to non-authorized user
if not portal.Base_checkPermission('portal_components', 'Manage Portal'):
return "You are not authorized to access the script"
import json
# Convert the request_reference argument string to their respeced boolean values
request_reference = {'True': True, 'False': False}.get(request_reference, False)
# Return python dictionary with title and reference of all notebooks
# for request_reference=True
if request_reference:
data_notebook_list = portal.portal_catalog(portal_type='Data Notebook')
notebook_detail_list = [{'reference': obj.getReference(), 'title': obj.getTitle()} for obj in data_notebook_list]
return notebook_detail_list
if not reference:
message = "Please set or use reference for the notebook you want to use"
return message
# Take python_expression as '' for empty code from jupyter frontend
if not python_expression:
python_expression = ''
# Get Data Notebook with the specific reference
data_notebook = portal.portal_catalog.getResultValue(portal_type='Data Notebook',
reference=reference)
# Create new Data Notebook if reference doesn't match with any from existing ones
if not data_notebook:
notebook_module = portal.getDefaultModule(portal_type='Data Notebook')
data_notebook = notebook_module.DataNotebookModule_addDataNotebook(
title=title,
reference=reference,
batch_mode=True
)
# Add new Data Notebook Line to the Data Notebook
data_notebook_line = data_notebook.DataNotebook_addDataNotebookLine(
notebook_code=python_expression,
batch_mode=True
)
# Gets the context associated to the data notebook being used
#
old_notebook_context = data_notebook.getNotebookContext()
if not old_notebook_context:
old_notebook_context = portal.Base_createNotebookContext()
# Pass all to code Base_runJupyter external function which would execute the code
# and returns a dict of result
final_result = context.Base_runJupyter(python_expression, old_notebook_context)
code_result = final_result['result_string']
new_local_variable_dict = final_result['notebook_context']
ename = final_result['ename']
evalue = final_result['evalue']
traceback = final_result['traceback']
status = final_result['status']
mime_type = final_result['mime_type']
# Updates the context in the notebook with the resulting context of code
# execution.
#
try:
data_notebook.setNotebookContext(new_local_variable_dict)
except Exception as e:
return context.Base_getErrorMessageForException(e, new_local_variable_dict)
result = {
u'code_result': code_result,
u'ename': ename,
u'evalue': evalue,
u'traceback': traceback,
u'status': status,
u'mime_type': mime_type
}
# Catch exception while seriaizing the result to be passed to jupyter frontend
# and in case of error put code_result as None and status as 'error' which would
# be shown by Jupyter frontend
try:
serialized_result = json.dumps(result)
except UnicodeDecodeError:
result = {
u'code_result': None,
u'ename': u'UnicodeDecodeError',
u'evalue': None,
u'traceback': None,
u'status': u'error',
u'mime_type': mime_type
}
serialized_result = json.dumps(result)
data_notebook_line.edit(notebook_code_result=code_result, mime_type=mime_type)
return serialized_result
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/Base_executeJupyter.xml
View file @ 23e06437
......@@ -2,71 +2,26 @@
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_Access_contents_information_Permission</string> </key>
<value>
<tuple>
<string>Authenticated</string>
<string>Author</string>
<string>Manager</string>
<string>Owner</string>
</tuple>
</value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
<key> <string>_function</string> </key>
<value> <string>Base_executeJupyter</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>python_expression=None, reference=None, title=None, request_reference=False, **kw</string> </value>
<key> <string>_module</string> </key>
<value> <string>JupyterCompile</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Base_executeJupyter</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
......
bt5/erp5_data_notebook/TestTemplateItem/portal_components/test.erp5.testExecuteJupyter.py
View file @ 23e06437
......@@ -150,9 +150,9 @@ portal.%s()
portal = self.portal
self.login('member_user')
result = portal.Base_executeJupyter.Base_checkPermission('portal_components', 'Manage Portal')
result = portal.Base_executeJupyter(title='Any title', reference='Any reference')
self.assertFalse(result)
self.assertEquals(result, 'You are not authorized to access the script')
def testUserCanCreateNotebookWithoutCode(self):
"""
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7