Skip to content

erp5
erp5
Commit b88ef920 authored by Łukasz Nowak's avatar Łukasz Nowak
Browse files
Options

erp5_data_notebook: Implement basic execution 

By providing simple scripts and following the principle of storing code
locally on the CDNL it is easy to postExecution from CDN and simply
return URL to the document which will contain the result and then fetch
the result.

Use External Method, as more analysis of executed code will be needed.

Tests proves, that the call is in the end executed in restricted
environment, thanks to using PyData Script internally.

Accompanying test exposes API.

By using frame information, the internals of the called script are available
to outer caller.
parent 5c93d189
Hide whitespace changes
Inline Side-by-side
Showing with 3138 additions and 5 deletions
bt5/erp5_data_notebook/ActionTemplateItem/portal_types/Cluster%20Data%20Notebook%20Line/view.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ActionInformation" module="Products.CMFCore.ActionInformation"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>action</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>action_type/object_view</string>
</tuple>
</value>
</item>
<item>
<key> <string>category</string> </key>
<value> <string>object_view</string> </value>
</item>
<item>
<key> <string>condition</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>view</string> </value>
</item>
<item>
<key> <string>permissions</string> </key>
<value>
<tuple>
<string>View</string>
</tuple>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Action Information</string> </value>
</item>
<item>
<key> <string>priority</string> </key>
<value> <float>1.0</float> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>View</string> </value>
</item>
<item>
<key> <string>visible</string> </key>
<value> <int>1</int> </value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="Expression" module="Products.CMFCore.Expression"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>text</string> </key>
<value> <string>string:${object_url}/ClusterDataNotebookLine_view</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/ActionTemplateItem/portal_types/Cluster%20Data%20Script/proxy_role_view.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ActionInformation" module="Products.CMFCore.ActionInformation"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>action</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>action_type/object_view</string>
</tuple>
</value>
</item>
<item>
<key> <string>category</string> </key>
<value> <string>object_view</string> </value>
</item>
<item>
<key> <string>condition</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>proxy_role_view</string> </value>
</item>
<item>
<key> <string>permissions</string> </key>
<value>
<tuple>
<string>View</string>
</tuple>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Action Information</string> </value>
</item>
<item>
<key> <string>priority</string> </key>
<value> <float>3.0</float> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Proxy Roles</string> </value>
</item>
<item>
<key> <string>visible</string> </key>
<value> <int>1</int> </value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="Expression" module="Products.CMFCore.Expression"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>text</string> </key>
<value> <string>string:${object_url}/PythonScript_viewProxyRole</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/ActionTemplateItem/portal_types/Cluster%20Data%20Script/repository_history_view.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ActionInformation" module="Products.CMFCore.ActionInformation"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>action</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>action_type/object_view</string>
</tuple>
</value>
</item>
<item>
<key> <string>category</string> </key>
<value> <string>object_view</string> </value>
</item>
<item>
<key> <string>condition</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>repository_history_view</string> </value>
</item>
<item>
<key> <string>permissions</string> </key>
<value>
<tuple>
<string>View</string>
</tuple>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Action Information</string> </value>
</item>
<item>
<key> <string>priority</string> </key>
<value> <float>10.0</float> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Repository History</string> </value>
</item>
<item>
<key> <string>visible</string> </key>
<value> <int>1</int> </value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="Expression" module="Products.CMFCore.Expression"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>text</string> </key>
<value> <string>string:${object_url}/Base_viewRepositoryHistory</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/ActionTemplateItem/portal_types/Cluster%20Data%20Script/run.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ActionInformation" module="Products.CMFCore.ActionInformation"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>action</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>action_type/object_view</string>
</tuple>
</value>
</item>
<item>
<key> <string>category</string> </key>
<value> <string>object_view</string> </value>
</item>
<item>
<key> <string>condition</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>run</string> </value>
</item>
<item>
<key> <string>permissions</string> </key>
<value>
<tuple>
<string>View</string>
</tuple>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Action Information</string> </value>
</item>
<item>
<key> <string>priority</string> </key>
<value> <float>2.0</float> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Run</string> </value>
</item>
<item>
<key> <string>visible</string> </key>
<value> <int>1</int> </value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="Expression" module="Products.CMFCore.Expression"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>text</string> </key>
<value> <string>string:${object_url}/PythonScript_viewRunScript</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/ActionTemplateItem/portal_types/Cluster%20Data%20Script/view.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ActionInformation" module="Products.CMFCore.ActionInformation"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>action</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>action_type/object_view</string>
</tuple>
</value>
</item>
<item>
<key> <string>category</string> </key>
<value> <string>object_view</string> </value>
</item>
<item>
<key> <string>condition</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>icon</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>view</string> </value>
</item>
<item>
<key> <string>permissions</string> </key>
<value>
<tuple>
<string>View</string>
</tuple>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Action Information</string> </value>
</item>
<item>
<key> <string>priority</string> </key>
<value> <float>1.0</float> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>View</string> </value>
</item>
<item>
<key> <string>visible</string> </key>
<value> <int>1</int> </value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="Expression" module="Products.CMFCore.Expression"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>text</string> </key>
<value> <string>string:${object_url}/PythonScript_view</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/DocumentTemplateItem/portal_components/document.erp5.ClusterDataScript.py 0 → 100644
View file @ b88ef920
from AccessControl import ClassSecurityInfo
#
from Products.ERP5Type import Permissions, PropertySheet
#from App.special_dtml import HTMLFile
#from Products.ERP5Type.XMLObject import XMLObject
#from Products.PythonScripts.PythonScript import \
# PythonScript as ZopePythonScript
#from Products.ERP5.mixin.expression import ExpressionMixin
from Products.ERP5.Document.PythonScript import PythonScript
from AccessControl.SecurityManagement import getSecurityManager
from Products.PythonScripts.PythonScript import PythonScriptTracebackSupplement
import new
#from Products.ERP5Type.Log import log
import sys
# securite enhacements for Cluster Data Script
# XXX: It impacts or guards, not only Cluster Data Script
# XXX: Is that good place or shall it go to product/ERP5Type/patches/Restricted.py ?
from AccessControl import (
allow_module,
allow_class,
allow_type,
# ModuleSecurityInfo,
)
allow_module('h5py')
allow_module('math')
allow_module('matplotlib.pyplot')
allow_module('numpy.random.mtrand')
allow_module('openpyxl')
from pandas.core.frame import DataFrame
allow_class(DataFrame)
DataFrame.__guarded_setitem__ = DataFrame.__setitem__.__func__
DataFrame.__guarded_delitem__ = DataFrame.__delitem__.__func__
allow_module('pandas.core.frame')
allow_module('pandas.core.groupby')
from pandas.core.groupby import DataFrameGroupBy
allow_class(DataFrameGroupBy)
allow_module('pandas.core.series')
from pandas.core.series import Series
allow_class(Series)
allow_type(type(Series.dt))
allow_module('pandas.core.strings')
from pandas.core.strings import StringMethods
allow_class(StringMethods)
allow_module('pandas.tseries.common')
allow_module('pylab')
allow_module('scipy')
allow_module('seaborn')
allow_module('sklearn')
allow_module('statsmodels')
allow_module('sympy')
allow_module('xlrd')
_marker = []
IGNORE_LOCAL_KEY_LIST = ['_print']
IGNORE_GLOBAL_KEY_LIST = [
'_getiter_', '__metaclass__', 'context', 'sequence', '_print_', '__traceback_supplement__',
'__file__', '_apply_', '_write_', '_getattr_', 'traverse_subpath', '__builtins__',
'script', 'container', '_inplacevar_', '__name__', '__debug__', '_getitem_', '__package__']
class ClusterDataScript(PythonScript):
""" Cluster Data Script for ERP5"""
meta_type = 'ERP5 Cluster Data Script'
portal_type = 'Cluster Data Script'
add_permission = Permissions.AddPortalContent
# Declarative security
security = ClassSecurityInfo()
security.declareObjectProtected(Permissions.AccessContentsInformation)
#
# #View content list, Force /view, Standart option in python scripts
# manage_options = ( XMLObject.manage_options[0],
# {'icon':'', 'label':'View','action':'view'}) \
# + ZopePythonScript.manage_options
#
# Declarative properties
property_sheets = ( PropertySheet.Base
, PropertySheet.XMLObject
, PropertySheet.CategoryCore
, PropertySheet.DublinCore
, PropertySheet.PythonScript
# CatalogFilter property_sheet needed for bootstrapping
# some Python Script during ERP5 Catalog installation
, PropertySheet.CatalogFilter
)
def _exec_with_fill(self, filler, *args, **kw):
bindcode = getattr(self, '_v_bindcode', _marker)
if bindcode is _marker:
bindcode = self._prepareBindCode()
security = getSecurityManager()
security.addContext(self)
try:
if bindcode is None:
bound_data = {}
else:
bound_data = []
exec bindcode # pylint: disable=W0122
bound_data = bound_data[0]
bound_data.update(filler)
return self._exec(bound_data, (), {})
finally:
security.removeContext(self)
def _exec(self, bound_names, args, kw):
"""Call a Python Script
Calling a Python Script is an actual function invocation.
"""
ft = self._v_ft
if ft is None:
__traceback_supplement__ = ( # pylint: disable=W0612
PythonScriptTracebackSupplement, self)
raise RuntimeError, '%s %s has errors.' % (self.meta_type, self.id)
fcode, g, fadefs = ft # pylint: disable=W0633
g = g.copy()
if bound_names is not None:
g.update(bound_names)
g['__traceback_supplement__'] = (
PythonScriptTracebackSupplement, self, -1)
g['__file__'] = getattr(self, '_filepath', None) or self.get_filepath()
f = new.function(fcode, g, None, fadefs)
class Executor(object):
func_namespace = {}
def execute(self, method, *args, **kwargs):
def tracer(frame, event, arg):
if event=='return':
f_locals = frame.f_locals.copy()
for k in IGNORE_LOCAL_KEY_LIST:
f_locals.pop(k, None)
f_globals = frame.f_globals.copy()
for k in IGNORE_GLOBAL_KEY_LIST:
f_globals.pop(k, None)
self.func_namespace = f_locals
self.func_namespace.update(f_globals)
try:
sys.setprofile(tracer)
return method(*args, **kw)
finally:
sys.setprofile(None)
executor = Executor()
try:
result = executor.execute(f, *args, **kw)
except SystemExit:
raise ValueError('SystemExit cannot be raised within a PythonScript')
return result, executor.func_namespace
\ No newline at end of file
bt5/erp5_data_notebook/DocumentTemplateItem/portal_components/document.erp5.ClusterDataScript.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Document Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>ClusterDataScript</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>document.erp5.ClusterDataScript</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Document Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.patches.WorkflowTool"/>
</pickle>
<pickle>
<tuple>
<none/>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</tuple>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/ExtensionTemplateItem/portal_components/extension.erp5.ClusterDataNotebookExtension.py 0 → 100644
View file @ b88ef920
#from Products.ERP5Type.Log import log
import traceback
import cPickle
import types
import importlib
class ImportInfo(object):
def __init__(self, module_name):
self.module_name = module_name
def ClusterDataNotebookLine_execute(self):
self.ERP5Site_assertClusterDataNotebookEnabled()
data_notebook = self.getParentValue()
# restore previous call information
pickle = data_notebook.getPickle()
if pickle is not None:
unpickled = cPickle.loads(pickle)
else:
unpickled = {}
filler = {}
for k, v in unpickled.items():
if isinstance(v, ImportInfo):
filler[k] = importlib.import_module(v.module_name)
else:
filler[k] = v
new_pickle = {}
try:
result, func_locals = self.getDefaultCodeValue()._exec_with_fill(filler)
except Exception:
result = traceback.format_exc()
else:
# store in pickle for next run
for k,v in func_locals.items():
if isinstance(v, types.ModuleType):
new_pickle[k] = ImportInfo(v.__name__)
else:
new_pickle[k] = v
data_notebook.edit(pickle=cPickle.dumps(new_pickle))
if result is not None:
result = str(result)
# store textual representation to send it over the wire
self.edit(result=result)
bt5/erp5_data_notebook/ExtensionTemplateItem/portal_components/extension.erp5.ClusterDataNotebookExtension.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Extension Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>ClusterDataNotebookExtension</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>extension.erp5.ClusterDataNotebookExtension</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Extension Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.patches.WorkflowTool"/>
</pickle>
<pickle>
<tuple>
<none/>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</tuple>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PortalTypeAllowedContentTypeTemplateItem/allowed_content_types.xml
View file @ b88ef920
<allowed_content_type_list> <allowed_content_type_list>
<portal_type id="Cluster Data Notebook">
<item>Cluster Data Notebook Line</item>
</portal_type>
<portal_type id="Cluster Data Notebook Line">
<item>Cluster Data Script</item>
</portal_type>
<portal_type id="Cluster Data Notebook Module"> <portal_type id="Cluster Data Notebook Module">
<item>Cluster Data Notebook</item> <item>Cluster Data Notebook</item>
</portal_type> </portal_type>
......
bt5/erp5_data_notebook/PortalTypeHiddenContentTypeTemplateItem/hidden_content_type_list.xml 0 → 100644
View file @ b88ef920
<hidden_content_type_list>
<portal_type id="Cluster Data Notebook Line">
<item>Cluster Data Script</item>
</portal_type>
</hidden_content_type_list>
\ No newline at end of file
bt5/erp5_data_notebook/PortalTypePropertySheetTemplateItem/property_sheet_list.xml
View file @ b88ef920
<property_sheet_list> <property_sheet_list>
<portal_type id="Cluster Data Notebook">
<item>ClusterDataNotebook</item>
</portal_type>
<portal_type id="Cluster Data Notebook Line">
<item>ClusterDataNotebookLine</item>
</portal_type>
<portal_type id="Data Notebook"> <portal_type id="Data Notebook">
<item>JupyterNotebook</item> <item>JupyterNotebook</item>
<item>Reference</item> <item>Reference</item>
......
bt5/erp5_data_notebook/PortalTypeTemplateItem/portal_types/Cluster%20Data%20Notebook%20Line.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Base Type" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>content_icon</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>Runtime execution representation of Jupyter Notebook cell.</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Cluster Data Notebook Line</string> </value>
</item>
<item>
<key> <string>init_script</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>permission</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Base Type</string> </value>
</item>
<item>
<key> <string>type_class</string> </key>
<value> <string>XMLObject</string> </value>
</item>
<item>
<key> <string>type_interface</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>type_mixin</string> </key>
<value>
<tuple/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PortalTypeTemplateItem/portal_types/Cluster%20Data%20Script.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Base Type" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>content_icon</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Cluster Data Script</string> </value>
</item>
<item>
<key> <string>init_script</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>permission</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Base Type</string> </value>
</item>
<item>
<key> <string>type_class</string> </key>
<value> <string>ClusterDataScript</string> </value>
</item>
<item>
<key> <string>type_interface</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>type_mixin</string> </key>
<value>
<tuple/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PortalTypeWorkflowChainTemplateItem/workflow_chain_type.xml
View file @ b88ef920
...@@ -3,6 +3,10 @@ ...@@ -3,6 +3,10 @@
<type>Cluster Data Notebook</type> <type>Cluster Data Notebook</type>
<workflow>edit_workflow</workflow> <workflow>edit_workflow</workflow>
</chain> </chain>
<chain>
<type>Cluster Data Notebook Line</type>
<workflow>edit_workflow</workflow>
</chain>
<chain> <chain>
<type>Data Notebook</type> <type>Data Notebook</type>
<workflow>edit_workflow</workflow> <workflow>edit_workflow</workflow>
......
bt5/erp5_data_notebook/PropertySheetTemplateItem/portal_property_sheets/ClusterDataNotebook.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Property Sheet" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_count</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>_mt_index</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
<item>
<key> <string>_tree</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ClusterDataNotebook</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Property Sheet</string> </value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="Length" module="BTrees.Length"/>
</pickle>
<pickle> <int>0</int> </pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="OOBTree" module="BTrees.OOBTree"/>
</pickle>
<pickle>
<none/>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="OOBTree" module="BTrees.OOBTree"/>
</pickle>
<pickle>
<none/>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PropertySheetTemplateItem/portal_property_sheets/ClusterDataNotebook/pickle_property.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Standard Property" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>elementary_type/string</string>
</tuple>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>pickle_property</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Standard Property</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PropertySheetTemplateItem/portal_property_sheets/ClusterDataNotebookLine.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Property Sheet" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_count</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>_mt_index</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
<item>
<key> <string>_tree</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ClusterDataNotebookLine</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Property Sheet</string> </value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="Length" module="BTrees.Length"/>
</pickle>
<pickle> <int>0</int> </pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="OOBTree" module="BTrees.OOBTree"/>
</pickle>
<pickle>
<none/>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="OOBTree" module="BTrees.OOBTree"/>
</pickle>
<pickle>
<none/>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PropertySheetTemplateItem/portal_property_sheets/ClusterDataNotebookLine/code_property.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Acquired Property" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>acquisition_accessor_id</string> </key>
<value> <string>getDefaultCodeValue</string> </value>
</item>
<item>
<key> <string>acquisition_base_category</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>acquisition_object_id</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>alt_accessor_id</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>elementary_type/content</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_acquired_property_id</string> </key>
<value>
<tuple>
<string>body</string>
</tuple>
</value>
</item>
<item>
<key> <string>content_portal_type</string> </key>
<value> <string>python: (\'Cluster Data Script\',)</string> </value>
</item>
<item>
<key> <string>content_translation_acquired_property_id</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>code_property</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Acquired Property</string> </value>
</item>
<item>
<key> <string>storage_id</string> </key>
<value> <string>default_code</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PropertySheetTemplateItem/portal_property_sheets/ClusterDataNotebookLine/result_property.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Standard Property" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>categories</string> </key>
<value>
<tuple>
<string>elementary_type/string</string>
</tuple>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>result_property</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Standard Property</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/PropertySheetTemplateItem/portal_property_sheets/DataNotebookPreference/preferred_cluster_data_notebook_enabled_property.xml
View file @ b88ef920
...@@ -18,7 +18,7 @@ ...@@ -18,7 +18,7 @@
<key> <string>description</string> </key> <key> <string>description</string> </key>
<value> <string>When enabled will allow to execute restricted and asynchronous Data Notebook code.\n <value> <string>When enabled will allow to execute restricted and asynchronous Data Notebook code.\n
\n \n
Note: This is safe implementation, which restricts execution using PyData Script.</string> </value> Note: This is safe implementation, which restricts execution using Cluster Data Script.</string> </value>
</item> </item>
<item> <item>
<key> <string>id</string> </key> <key> <string>id</string> </key>
......
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_execute.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>ClusterDataNotebookLine_execute</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>ClusterDataNotebookExtension</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ClusterDataNotebookLine_execute</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_getState.py 0 → 100644
View file @ b88ef920
context.ERP5Site_assertClusterDataNotebookEnabled()
if context.hasErrorActivity():
return 'Error'
elif context.hasActivity():
return 'Running'
else:
return 'Ready'
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_getState.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ClusterDataNotebookLine_getState</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_postExecution.py 0 → 100644
View file @ b88ef920
context.ERP5Site_assertClusterDataNotebookEnabled()
context.activate().ClusterDataNotebookLine_execute()
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_postExecution.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ClusterDataNotebookLine_postExecution</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_view.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ERP5 Form" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_objects</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>action</string> </key>
<value> <string>Base_edit</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>edit_order</string> </key>
<value>
<list/>
</value>
</item>
<item>
<key> <string>encoding</string> </key>
<value> <string>UTF-8</string> </value>
</item>
<item>
<key> <string>enctype</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>group_list</string> </key>
<value>
<list>
<string>left</string>
<string>right</string>
<string>center</string>
<string>bottom</string>
<string>hidden</string>
</list>
</value>
</item>
<item>
<key> <string>groups</string> </key>
<value>
<dictionary>
<item>
<key> <string>bottom</string> </key>
<value>
<list/>
</value>
</item>
<item>
<key> <string>center</string> </key>
<value>
<list>
<string>my_default_code_body</string>
<string>my_result</string>
</list>
</value>
</item>
<item>
<key> <string>hidden</string> </key>
<value>
<list/>
</value>
</item>
<item>
<key> <string>left</string> </key>
<value>
<list>
<string>my_title</string>
</list>
</value>
</item>
<item>
<key> <string>right</string> </key>
<value>
<list>
<string>your_state</string>
</list>
</value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ClusterDataNotebookLine_view</string> </value>
</item>
<item>
<key> <string>method</string> </key>
<value> <string>POST</string> </value>
</item>
<item>
<key> <string>name</string> </key>
<value> <string>ClusterDataNotebookLine_view</string> </value>
</item>
<item>
<key> <string>pt</string> </key>
<value> <string>form_view</string> </value>
</item>
<item>
<key> <string>row_length</string> </key>
<value> <int>4</int> </value>
</item>
<item>
<key> <string>stored_encoding</string> </key>
<value> <string>UTF-8</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Cluster Data Notebook Line</string> </value>
</item>
<item>
<key> <string>unicode_mode</string> </key>
<value> <int>0</int> </value>
</item>
<item>
<key> <string>update_action</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>update_action_title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_view/my_default_code_body.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ProxyField" module="Products.ERP5Form.ProxyField"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>delegated_list</string> </key>
<value>
<list>
<string>editable</string>
<string>title</string>
</list>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>my_default_code_body</string> </value>
</item>
<item>
<key> <string>message_values</string> </key>
<value>
<dictionary>
<item>
<key> <string>external_validator_failed</string> </key>
<value> <string>The input failed the external validator.</string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>overrides</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>tales</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>values</string> </key>
<value>
<dictionary>
<item>
<key> <string>editable</string> </key>
<value> <int>0</int> </value>
</item>
<item>
<key> <string>field_id</string> </key>
<value> <string>my_text_area_field</string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string>Base_viewFieldLibrary</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Code</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_view/my_result.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ProxyField" module="Products.ERP5Form.ProxyField"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>delegated_list</string> </key>
<value>
<list>
<string>editable</string>
<string>title</string>
</list>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>my_result</string> </value>
</item>
<item>
<key> <string>message_values</string> </key>
<value>
<dictionary>
<item>
<key> <string>external_validator_failed</string> </key>
<value> <string>The input failed the external validator.</string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>overrides</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>tales</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>values</string> </key>
<value>
<dictionary>
<item>
<key> <string>editable</string> </key>
<value> <int>0</int> </value>
</item>
<item>
<key> <string>field_id</string> </key>
<value> <string>my_text_area_field</string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string>Base_viewFieldLibrary</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Result</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_view/my_title.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ProxyField" module="Products.ERP5Form.ProxyField"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>delegated_list</string> </key>
<value>
<list/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>my_title</string> </value>
</item>
<item>
<key> <string>message_values</string> </key>
<value>
<dictionary>
<item>
<key> <string>external_validator_failed</string> </key>
<value> <string>The input failed the external validator.</string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>overrides</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>tales</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>values</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string>my_view_mode_title</string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string>Base_viewFieldLibrary</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebookLine_view/your_state.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ProxyField" module="Products.ERP5Form.ProxyField"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>delegated_list</string> </key>
<value>
<list>
<string>default</string>
</list>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>your_state</string> </value>
</item>
<item>
<key> <string>message_values</string> </key>
<value>
<dictionary>
<item>
<key> <string>external_validator_failed</string> </key>
<value> <string>The input failed the external validator.</string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>overrides</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>tales</string> </key>
<value>
<dictionary>
<item>
<key> <string>default</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>values</string> </key>
<value>
<dictionary>
<item>
<key> <string>default</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>field_id</string> </key>
<value> <string>my_view_mode_translated_validation_state_title</string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string>Base_viewConfiguratorFieldLibrary</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="TALESMethod" module="Products.Formulator.TALESField"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_text</string> </key>
<value> <string>here/ClusterDataNotebookLine_getState</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebook_postExecution.py 0 → 100644
View file @ b88ef920
context.ERP5Site_assertClusterDataNotebookEnabled()
cluster_data_notebook_line = context.newContent(
portal_type='Cluster Data Notebook Line',
default_code_body=code)
cluster_data_notebook_line.ClusterDataNotebookLine_postExecution()
# XXX: how to return HALish 201 with X-Location?!
return cluster_data_notebook_line.getRelativeUrl()
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebook_postExecution.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>code, dialog_id=\'\', dialog_method=\'\'</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ClusterDataNotebook_postExecution</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebook_view.xml
View file @ b88ef920
...@@ -53,7 +53,9 @@ ...@@ -53,7 +53,9 @@
<item> <item>
<key> <string>bottom</string> </key> <key> <string>bottom</string> </key>
<value> <value>
<list/> <list>
<string>listbox</string>
</list>
</value> </value>
</item> </item>
<item> <item>
......
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ClusterDataNotebook_view/listbox.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ProxyField" module="Products.ERP5Form.ProxyField"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>delegated_list</string> </key>
<value>
<list>
<string>columns</string>
<string>portal_types</string>
<string>title</string>
</list>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>listbox</string> </value>
</item>
<item>
<key> <string>message_values</string> </key>
<value>
<dictionary>
<item>
<key> <string>external_validator_failed</string> </key>
<value> <string>The input failed the external validator.</string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>overrides</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>tales</string> </key>
<value>
<dictionary>
<item>
<key> <string>field_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
<item>
<key> <string>values</string> </key>
<value>
<dictionary>
<item>
<key> <string>columns</string> </key>
<value>
<list>
<tuple>
<string>title</string>
<string>Title</string>
</tuple>
<tuple>
<string>default_code_body</string>
<string>Code</string>
</tuple>
<tuple>
<string>result</string>
<string>Result</string>
</tuple>
<tuple>
<string>ClusterDataNotebookLine_getState</string>
<string>State</string>
</tuple>
</list>
</value>
</item>
<item>
<key> <string>field_id</string> </key>
<value> <string>my_view_mode_listbox</string> </value>
</item>
<item>
<key> <string>form_id</string> </key>
<value> <string>Base_viewFieldLibrary</string> </value>
</item>
<item>
<key> <string>portal_types</string> </key>
<value>
<list>
<tuple>
<string>Cluster Data Notebook Line</string>
<string>Cluster Data Notebook Line</string>
</tuple>
</list>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Cluster Data Notebook Lines</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ERP5Site_assertClusterDataNotebookEnabled.py 0 → 100644
View file @ b88ef920
from zExceptions import Unauthorized
portal_preferences = context.getPortalObject().portal_preferences
if portal_preferences.isPreferredClusterDataNotebookEnabled():
return True
raise Unauthorized('The asynchronous restricted implementation is not enabled on the server.')
bt5/erp5_data_notebook/SkinTemplateItem/portal_skins/erp5_data_notebook/ERP5Site_assertClusterDataNotebookEnabled.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ERP5Site_assertClusterDataNotebookEnabled</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/TestTemplateItem/portal_components/test.erp5.testClusterDataNotebook.py 0 → 100644
View file @ b88ef920
# -*- coding: utf-8 -*-
from Products.ERP5Type.tests.ERP5TypeTestCase import ERP5TypeTestCase
from zExceptions import Unauthorized
import transaction
class ClusterDataNotebookMixin(object):
def setupPreference(self):
self.preference = self.portal.portal_preferences.newContent(
portal_type='System Preference',
title=self.id(),
priority=1,
preferred_cluster_data_notebook_enabled=True
)
self.preference.enable()
self.tic()
def disablePreference(self):
self.preference.disable()
self.tic()
def afterSetUp(self):
self.setupPreference()
# Create user to be used in tests
user_folder = self.getPortal().acl_users
user_folder._doAddUser('dev_user', '', ['Manager',], [])
self.tic()
self.login('dev_user')
def beforeTearDown(self):
self.disablePreference()
def newNotebook(self):
return self.portal.cluster_data_notebook_module.newContent(
portal_type='Cluster Data Notebook')
def executeCell(self, notebook, code):
cluster_data_notebook_line_url = notebook.ClusterDataNotebook_postExecution(code)
cluster_data_notebook_line = self.portal.restrictedTraverse(cluster_data_notebook_line_url)
self.tic()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
return cluster_data_notebook_line.getResult()
class TestClusterDataNotebook(ClusterDataNotebookMixin, ERP5TypeTestCase):
def test_ClusterDataNotebook_postExecution(self):
code = 'return "tested"'
cluster_data_notebook = self.portal.cluster_data_notebook_module.newContent(
portal_type='Cluster Data Notebook')
cluster_data_notebook_line_url = cluster_data_notebook.ClusterDataNotebook_postExecution(code)
cluster_data_notebook_line = self.portal.restrictedTraverse(cluster_data_notebook_line_url)
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
transaction.commit()
self.assertEqual('Running', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.tic()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.assertEqual(
'tested',
cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebook_postExecution_disabled(self):
self.preference.edit(preferred_cluster_data_notebook_enabled=False)
self.tic()
code = 'return "tested"'
cluster_data_notebook = self.portal.cluster_data_notebook_module.newContent(
portal_type='Cluster Data Notebook')
self.assertRaises(Unauthorized, cluster_data_notebook.ClusterDataNotebook_postExecution, code)
class TestClusterDataNotebookLine(ClusterDataNotebookMixin, ERP5TypeTestCase):
def newClusterDataNotebookLine(self):
return self.portal.cluster_data_notebook_module.newContent(
portal_type='Cluster Data Notebook').newContent(portal_type='Cluster Data Notebook Line')
def test_code_storage(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
code = 'return 1'
cluster_data_notebook_line.edit(
default_code_body=code
)
self.tic()
self.assertEqual(
['default_code'],
list(cluster_data_notebook_line.objectIds())
)
self.assertEqual(
code,
cluster_data_notebook_line.getDefaultCodeBody().strip()
)
def test_ClusterDataNotebookLine_execute(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='return "tested"')
cluster_data_notebook_line.ClusterDataNotebookLine_execute()
self.assertEqual(
'tested',
cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebookLine_executeEmpty(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='')
cluster_data_notebook_line.ClusterDataNotebookLine_execute()
self.assertEqual(
None,
cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebookLine_execute_disabled(self):
self.preference.edit(preferred_cluster_data_notebook_enabled=False)
self.tic()
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='return "tested"')
self.assertRaises(Unauthorized, cluster_data_notebook_line.ClusterDataNotebookLine_execute)
def test_ClusterDataNotebookLine_execute_nonAllowedModule(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='import pickle')
cluster_data_notebook_line.ClusterDataNotebookLine_execute()
self.assertTrue(
'Unauthorized: import of \'pickle\' is unauthorized' in cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebookLine_execute_withException(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='raise ValueError("Stored value error")')
cluster_data_notebook_line.ClusterDataNotebookLine_execute()
self.assertTrue(
'ValueError: Stored value error' in cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebookLine_execute_withSyntaxError(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='ret urn "Stored problem"')
cluster_data_notebook_line.ClusterDataNotebookLine_execute()
self.assertTrue(
'RuntimeError: ERP5 Cluster Data Script default_code has errors.' in cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebookLine_postExecution_disabled(self):
self.preference.edit(preferred_cluster_data_notebook_enabled=False)
self.tic()
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='return "tested"')
self.assertRaises(Unauthorized, cluster_data_notebook_line.ClusterDataNotebookLine_postExecution)
def test_ClusterDataNotebookLine_postExecution(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='return "tested"')
cluster_data_notebook_line.ClusterDataNotebookLine_postExecution()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
transaction.commit()
self.assertEqual(
None,
cluster_data_notebook_line.getResult()
)
self.assertEqual('Running', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.tic()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.assertEqual(
'tested',
cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebookLine_postExecution_withException(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='raise ValueError("Stored value error")')
cluster_data_notebook_line.ClusterDataNotebookLine_postExecution()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
transaction.commit()
self.assertEqual(
None,
cluster_data_notebook_line.getResult()
)
self.assertEqual('Running', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.tic()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.assertTrue(
"ValueError: Stored value error" in cluster_data_notebook_line.getResult()
)
def test_ClusterDataNotebookLine_postExecution_withSyntaxError(self):
cluster_data_notebook_line = self.newClusterDataNotebookLine()
cluster_data_notebook_line.edit(default_code_body='ret urn "Stored problem"')
cluster_data_notebook_line.ClusterDataNotebookLine_postExecution()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
transaction.commit()
self.assertEqual(
None,
cluster_data_notebook_line.getResult()
)
self.assertEqual('Running', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.tic()
self.assertEqual('Ready', cluster_data_notebook_line.ClusterDataNotebookLine_getState())
self.assertTrue(
"RuntimeError: ERP5 Cluster Data Script default_code has errors." in cluster_data_notebook_line.getResult()
)
class TestClusterDataNotebookPostExecution(ClusterDataNotebookMixin, ERP5TypeTestCase):
"""Top level tests for asserting that code is called as expected"""
def testVariablesBetweenCells(self):
notebook = self.newNotebook()
# Prove that directly using variables works
result = self.executeCell(notebook, """
some_int = 1
some_string = "string"
print some_int
print some_string
return printed
""")
self.assertEqual(
result,
"1\nstring\n")
# Prove that variables from previous cell are available
result = self.executeCell(notebook, """
print some_int
print some_string
return printed
""")
self.assertEqual(
result,
"1\nstring\n"
)
def testImportModule(self):
notebook = self.newNotebook()
# Prove that directly using variables works
result = self.executeCell(notebook, """
import pprint
print pprint.pformat('1')
return printed
""")
self.assertEqual(
result,
"'1'\n")
# Prove that variables from previous cell are available
result = self.executeCell(notebook, """
print pprint.pformat('1')
return printed
""")
self.assertEqual(
result,
"'1'\n"
)
def testImportModuleAs(self):
notebook = self.newNotebook()
# Prove that directly using variables works
result = self.executeCell(notebook, """
import pprint as pp
print pp.pformat('1')
return printed
""")
self.assertEqual(
result,
"'1'\n")
# Prove that variables from previous cell are available
result = self.executeCell(notebook, """
print pp.pformat('1')
return printed
""")
self.assertEqual(
result,
"'1'\n"
)
def testImportModuleFrom(self):
notebook = self.newNotebook()
# Prove that directly using variables works
result = self.executeCell(notebook, """
from pprint import pformat
print pformat('1')
return printed
""")
self.assertEqual(
result,
"'1'\n")
# Prove that variables from previous cell are available
result = self.executeCell(notebook, """
print pformat('1')
return printed
""")
self.assertEqual(
result,
"'1'\n"
)
def testVariable_context(self):
notebook = self.newNotebook()
# Prove that directly using variables works
result = self.executeCell(notebook, """
print context.getParentValue().getRelativeUrl()
return printed
""")
self.assertEqual(
result,
"%s\n" % (notebook.getRelativeUrl(),))
def testVariable_script(self):
notebook = self.newNotebook()
# Prove that directly using variables works
result = self.executeCell(notebook, """
print script.id
return printed
""")
self.assertEqual(
result,
"default_code\n")
def test_returnLast(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
a = 1
a
""")
self.assertEqual(result, "1")
def test_delWorks(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
a = 1
return a
""")
self.assertEqual(result, "1")
result = self.executeCell(notebook, """
print a ; return printed
#del a""")
self.assertEqual(result, "1\n")
result = self.executeCell(notebook, """
del a
""")
self.assertEqual(result, None)
result = self.executeCell(notebook, """
return a
""")
self.assertEqual(result, "check")
self.assertTrue(
'UnboundLocalError: local variable \'a\' referenced before assignment' in result,
result)
def test_exceptionDoesNotResetVariables(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
a = 1
return a
""")
self.assertEqual(
result,
"1")
result = self.executeCell(notebook, """
raise ValueError
""")
self.assertTrue(
"ValueError" in result,
result)
result = self.executeCell(notebook, """
return a
""")
self.assertEqual(
result,
"1")
def test_variablesMerged(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
a = 1
return a
""")
self.assertEqual(
result,
"1")
result = self.executeCell(notebook, """
b = 2
return b
""")
self.assertEqual(
result,
"2")
result = self.executeCell(notebook, """
return (a, b)
""")
self.assertEqual(
result,
"(1, 2)")
def testPandasDataFrame(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
import pandas as pd
dataframe = pd.DataFrame({
'dates' : pd.date_range('20130101', periods=6)
})
return dataframe[['dates']]
""")
self.assertEqual(
result,
""" dates
0 2013-01-01
1 2013-01-02
2 2013-01-03
3 2013-01-04
4 2013-01-05
5 2013-01-06""")
def testNumpySum(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
import numpy
return numpy.sum([1,2,3])
""")
self.assertEqual(
result,
"6")
def testPandasDatetimeProperties(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
import pandas as pd
dataframe = pd.DataFrame({
'dates' : pd.date_range('20130101', periods=6)
})
return dataframe['dates'].dt.month
""")
self.assertEqual(
result,
"""0 1
1 1
2 1
3 1
4 1
5 1
Name: dates, dtype: int64""")
def testNumpyRandomRandn(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
# import numpy # <-- this is not enough, some kind of side effect while import numpy.random is called
# # which result in access to numpy.random.randn
import numpy.random
numpy.random.randn(6,4)
""")
self.assertEqual(
result,
None)
def testPandasDateRange(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
import pandas
return pandas.date_range('20130101', periods=6)
""")
self.assertEqual(
result,
"""DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')""")
def testPandasSeries(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
import pandas as pd
rng = pd.date_range('1/1/2012', periods=5, freq='S')
ts = pd.Series([9,8,7,6,5], index=rng)
return ts.resample('5Min').sum()
""")
self.assertEqual(
result,
"2012-01-01 35\nFreq: 5T, dtype: int64")
def testPandasStringMethods(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
import pandas as pd
df = pd.DataFrame({
'idx': pd.Series(1,index=list(range(1)),dtype='float32'),
'text': 'some/nice/text'
})
return df['text'].str.split('/', expand=True)
""")
self.assertEqual(
result,
' 0 1 2\n0 some nice text')
def testPandasDataFrameGroupBy(self):
notebook = self.newNotebook()
result = self.executeCell(notebook, """
import pandas as pd
df = pd.DataFrame({
'idx': pd.Series(1,index=list(range(1)),dtype='float32'),
'text': 'some/nice/text'
})
return df.groupby(['text']).agg({'idx': lambda x: x.sum()})
""")
self.assertEqual(
result,
" idx\ntext \nsome/nice/text 1.0")
def testExternalModule(self):
def checkModule(module):
if self.executeCell(self.newNotebook(), 'import %s as _' % module) is None:
return True
else:
return False
module_list = [
'datetime',
'h5py',
'math',
'matplotlib',
'matplotlib.pyplot',
'numpy',
'numpy.random',
'openpyxl',
'pandas',
'pylab',
'scipy',
'seaborn',
'sklearn',
'statsmodels',
'sympy',
'xlrd',
]
imported_module_list = [module for module in module_list if checkModule(module)]
self.assertEqual(
set(module_list),
set(imported_module_list),
'Missing modules: %s' % ' '.join(set(module_list)-set(imported_module_list)))
class TestClusterDataNotebookScenarios(ClusterDataNotebookMixin, ERP5TypeTestCase):
def test_10_minutes_to_pandas(self):
# Checks the most important aspects of https://pandas.pydata.org/pandas-docs/stable/10min.html
notebook = self.newNotebook()
def executeCell(code):
return self.executeCell(notebook, code)
def executeAssertNone(code):
result = executeCell(code)
self.assertEqual(None, result)
executeAssertNone('import pandas as pd')
executeAssertNone('import numpy as np')
executeAssertNone('import matplotlib.pyplot as plt')
executeAssertNone('s = pd.Series([1,3,5,np.nan,6,8])')
result = executeCell('return s') # XXX: shall be without return
self.assertEqual(
result,
"""0 1.0
1 3.0
2 5.0
3 NaN
4 6.0
5 8.0
dtype: float64""")
executeAssertNone("dates = pd.date_range('20130101', periods=6)")
result = executeCell('return dates') # XXX: shall be without return
self.assertEqual(
result,
"""DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')""")
executeAssertNone("""
df = pd.DataFrame(
[
[1, 2, 3, 4],
[17, 18, 19, 20],
[9, 10, 11, 12],
[13, 14, 15, 16],
[21, 22, 23, 24],
[5, 6, 7, 8],
], index=dates, columns=list('ABCD'))""")
result = executeCell("return df")
self.assertEqual(result, """ A B C D
2013-01-01 1 2 3 4
2013-01-02 17 18 19 20
2013-01-03 9 10 11 12
2013-01-04 13 14 15 16
2013-01-05 21 22 23 24
2013-01-06 5 6 7 8""")
executeAssertNone("""
df2 = pd.DataFrame({ 'A' : 1.,
'B' : pd.Timestamp('20130102'),
'C' : pd.Series(1,index=list(range(4)),dtype='float32'),
'D' : np.array([3] * 4,dtype='int32'),
'E' : pd.Categorical(["test","train","test","train"]),
'F' : 'foo' })""")
result = executeCell("return df2")
self.assertEqual(
result,
""" A B C D E F
0 1.0 2013-01-02 1.0 3 test foo
1 1.0 2013-01-02 1.0 3 train foo
2 1.0 2013-01-02 1.0 3 test foo
3 1.0 2013-01-02 1.0 3 train foo""")
result = executeCell("return df2.dtypes")
self.assertEqual(
result, """\
A float64
B datetime64[ns]
C float32
D int32
E category
F object
dtype: object"""
)
result = executeCell("return df.head()")
self.assertEqual(result, """\
A B C D
2013-01-01 1 2 3 4
2013-01-02 17 18 19 20
2013-01-03 9 10 11 12
2013-01-04 13 14 15 16
2013-01-05 21 22 23 24""")
result = executeCell("return df.tail(3)")
self.assertEqual(result, """\
A B C D
2013-01-04 13 14 15 16
2013-01-05 21 22 23 24
2013-01-06 5 6 7 8""")
result = executeCell("return df.index")
self.assertEqual(
result,
"""DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
'2013-01-05', '2013-01-06'],
dtype='datetime64[ns]', freq='D')"""
)
result = executeCell("return df.columns")
self.assertEqual(
result,
"""Index([u'A', u'B', u'C', u'D'], dtype='object')""")
result = executeCell("return df.values")
self.assertEqual(result, """\
[[ 1 2 3 4]
[17 18 19 20]
[ 9 10 11 12]
[13 14 15 16]
[21 22 23 24]
[ 5 6 7 8]]""")
result = executeCell("return df.describe()")
self.assertEqual(result, """\
A B C D
count 6.000000 6.000000 6.000000 6.000000
mean 11.000000 12.000000 13.000000 14.000000
std 7.483315 7.483315 7.483315 7.483315
min 1.000000 2.000000 3.000000 4.000000
25% 6.000000 7.000000 8.000000 9.000000
50% 11.000000 12.000000 13.000000 14.000000
75% 16.000000 17.000000 18.000000 19.000000
max 21.000000 22.000000 23.000000 24.000000""")
result = executeCell("return df.T")
self.assertEqual(result, """\
2013-01-01 2013-01-02 2013-01-03 2013-01-04 2013-01-05 2013-01-06
A 1 17 9 13 21 5
B 2 18 10 14 22 6
C 3 19 11 15 23 7
D 4 20 12 16 24 8""")
result = executeCell("return df.sort_index(axis=1, ascending=False)")
self.assertEqual(result, """\
D C B A
2013-01-01 4 3 2 1
2013-01-02 20 19 18 17
2013-01-03 12 11 10 9
2013-01-04 16 15 14 13
2013-01-05 24 23 22 21
2013-01-06 8 7 6 5""")
result = executeCell("return df.sort_values(by='B')")
self.assertEqual(result, """\
A B C D
2013-01-01 1 2 3 4
2013-01-06 5 6 7 8
2013-01-03 9 10 11 12
2013-01-04 13 14 15 16
2013-01-02 17 18 19 20
2013-01-05 21 22 23 24""")
result = executeCell("return df['A']")
self.assertEqual(result, """\
2013-01-01 1
2013-01-02 17
2013-01-03 9
2013-01-04 13
2013-01-05 21
2013-01-06 5
Freq: D, Name: A, dtype: int64""")
result = executeCell("return df[0:3]")
self.assertEqual(result, """\
A B C D
2013-01-01 1 2 3 4
2013-01-02 17 18 19 20
2013-01-03 9 10 11 12""")
result = executeCell("return df['20130102':'20130104']")
self.assertEqual(result, """\
A B C D
2013-01-02 17 18 19 20
2013-01-03 9 10 11 12
2013-01-04 13 14 15 16""")
result = executeCell("return df.loc['20130102':'20130104',['A','B']]")
self.assertEqual(result, """\
A B
2013-01-02 17 18
2013-01-03 9 10
2013-01-04 13 14""")
result = executeCell("return df[df.A > 6]")
self.assertEqual(result, """\
A B C D
2013-01-02 17 18 19 20
2013-01-03 9 10 11 12
2013-01-04 13 14 15 16
2013-01-05 21 22 23 24""")
result = executeCell("return df.apply(np.cumsum)")
self.assertEqual(result, """\
A B C D
2013-01-01 1 2 3 4
2013-01-02 18 20 22 24
2013-01-03 27 30 33 36
2013-01-04 40 44 48 52
2013-01-05 61 66 71 76
2013-01-06 66 72 78 84""")
result = executeCell("return df.groupby('A').sum()")
self.assertEqual(result, """\
B C D
A
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
17 18 19 20
21 22 23 24""")
result = executeCell("return pd.pivot_table(df, values='D', index=['A', 'B'], columns=['C'])")
self.assertEqual(result, """\
C 3 7 11 15 19 23
A B
1 2 4.0 NaN NaN NaN NaN NaN
5 6 NaN 8.0 NaN NaN NaN NaN
9 10 NaN NaN 12.0 NaN NaN NaN
13 14 NaN NaN NaN 16.0 NaN NaN
17 18 NaN NaN NaN NaN 20.0 NaN
21 22 NaN NaN NaN NaN NaN 24.0""")
\ No newline at end of file
bt5/erp5_data_notebook/TestTemplateItem/portal_components/test.erp5.testClusterDataNotebook.xml 0 → 100644
View file @ b88ef920
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>testClusterDataNotebook</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test.erp5.testClusterDataNotebook</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.patches.WorkflowTool"/>
</pickle>
<pickle>
<tuple>
<none/>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</tuple>
</pickle>
</record>
</ZopeData>
bt5/erp5_data_notebook/bt/template_action_path_list
View file @ b88ef920
Cluster Data Notebook Line | view
Cluster Data Notebook Module | view Cluster Data Notebook Module | view
Cluster Data Notebook | post_execution Cluster Data Notebook | post_execution
Cluster Data Notebook | view Cluster Data Notebook | view
Cluster Data Script | proxy_role_view
Cluster Data Script | repository_history_view
Cluster Data Script | run
Cluster Data Script | view
Data Notebook Line | new_notebook_line Data Notebook Line | new_notebook_line
Data Notebook Line | view Data Notebook Line | view
Data Notebook Module | view Data Notebook Module | view
......
bt5/erp5_data_notebook/bt/template_document_id_list 0 → 100644
View file @ b88ef920
document.erp5.ClusterDataScript
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_extension_id_list
View file @ b88ef920
extension.erp5.JupyterCompile extension.erp5.JupyterCompile
\ No newline at end of file extension.erp5.ClusterDataNotebookExtension
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_portal_type_allowed_content_type_list
View file @ b88ef920
Cluster Data Notebook Line | Cluster Data Script
Cluster Data Notebook Module | Cluster Data Notebook Cluster Data Notebook Module | Cluster Data Notebook
Cluster Data Notebook | Cluster Data Notebook Line
Data Notebook Module | Data Notebook Data Notebook Module | Data Notebook
Data Notebook | Data Notebook Line Data Notebook | Data Notebook Line
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_portal_type_hidden_content_type_list 0 → 100644
View file @ b88ef920
Cluster Data Notebook Line | Cluster Data Script
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_portal_type_id_list
View file @ b88ef920
Cluster Data Notebook Cluster Data Notebook
Cluster Data Notebook Line
Cluster Data Notebook Module Cluster Data Notebook Module
Cluster Data Script
Data Notebook Data Notebook
Data Notebook Line Data Notebook Line
Data Notebook Module Data Notebook Module
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_portal_type_property_sheet_list
View file @ b88ef920
Cluster Data Notebook Line | ClusterDataNotebookLine
Cluster Data Notebook | ClusterDataNotebook
Data Notebook Line | JupyterNotebookLine Data Notebook Line | JupyterNotebookLine
Data Notebook | JupyterNotebook Data Notebook | JupyterNotebook
Data Notebook | Reference Data Notebook | Reference
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_portal_type_workflow_chain_list
View file @ b88ef920
Cluster Data Notebook Line | edit_workflow
Cluster Data Notebook | edit_workflow Cluster Data Notebook | edit_workflow
Data Notebook | edit_workflow Data Notebook | edit_workflow
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_property_sheet_id_list
View file @ b88ef920
JupyterNotebook JupyterNotebook
JupyterNotebookLine JupyterNotebookLine
DataNotebookPreference DataNotebookPreference
\ No newline at end of file ClusterDataNotebookLine
ClusterDataNotebook
\ No newline at end of file
bt5/erp5_data_notebook/bt/template_test_id_list
View file @ b88ef920
test.erp5.testExecuteJupyter test.erp5.testExecuteJupyter
\ No newline at end of file test.erp5.testClusterDataNotebook
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7