Skip to content

erp5
erp5
Commit d2b29af1 authored by Jérome Perrin's avatar Jérome Perrin
Browse files
Options

core,monaco_editor: python language support 🚧 

some work in progress changes to improve developer experience

monaco_editor: increase debounce timeout for pylint checks XXX

on very large python files (>1000 lines) sometimes they queue up and we
have to wait for all requests that were queued by zope.

XXX maybe this does not happen when accessing through haproxy/apache, I
am observing this when hitting zope directly

jedi: generate stubs WIP

ERP5: "quick and dirty" type annotations XXX

IIRC the only thing needed is that ERP5TypeTestCase.getPortal is an
ERP5Site

monaco_editor: also enable jedi for codelens ( WIP: ZMI only )

core: pass "language support url" to text editors XXX

For now this is just the portal_url, but I'm thinking it could be a
proper tool.

monaco_editor: enable pylint in gadget version

because pylint is a bit slow on large components, debounce every 2
seconds. TODO: this is too slow.

monaco_editor: enable formatting provider for python

This makes "Format Document" / "Format Selection" work.

monaco_editor: enable completion provider for python

this makes completions works when using Ctrl+space

monaco_editor: pass portal_type to checkPythonSourceCode

python_support: new business template to act as a language server for python

checkPythonSourceCode: add a cache

When using checkPythonSourceCode integrated in the source code editor,
for a scenario where developer edit a component and save we can benefit
from caching the check message for the source code content, because the
same check that the one happening in the editor will happen when the
component is saved.

This cache varies on:
 - "component_packages" cache cookie which is reset every time some
component code is edited.
 - zope startup time to take into account editions of file system code.
This assumes that after reseting file system code zope will be
restarted.
 - portal_type, because the checks performed by this function also
depend on portal type.

jedi wip

administration: keep using pylint only for now

monaco_editor: jedi WIP

core: use mypy to check python code ( WIP experiment )

Revert "core: use mypy to check python code ( WIP experiment )"

This reverts commit cfa27232.

ERP5TypeTestCase: jedi workarounds

monaco_editor: WIP reference provider for python

monaco_editor: jedi wip (no longer use /tmp/)

yapf: adjust config following up Gabriel feedback
parent 3cb786cc
Expand all Hide whitespace changes
Inline Side-by-side
Showing with 4368 additions and 4 deletions
bt5/erp5_monaco_editor/ExtensionTemplateItem/portal_components/extension.erp5.Jedi.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Extension Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>extension.erp5.Jedi</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Extension Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/ExtensionTemplateItem/portal_components/extension.erp5.YAPF.py 0 → 100644
View file @ d2b29af1
from yapf.yapflib import yapf_api
import json
import tempfile
import textwrap
import logging
logger = logging.getLogger(__name__)
def ERP5Site_formatPythonSourceCode(self, data, REQUEST=None):
if isinstance(data, basestring):
data = json.loads(data)
try:
extra = {}
if data['range']:
extra['lines'] = (
(data['range']['startLineNumber'], data['range']['endLineNumber']),)
with tempfile.NamedTemporaryFile(mode='w', suffix='.style.yapf') as f:
f.write(
textwrap.dedent(
'''
[style]
based_on_style = pep8
indent_width = 2
continuation_indent_width = 2
split_before_expression_after_opening_paren = true
blank_line_before_nested_class_or_def = false
allow_split_before_dict_value = false
split_before_first_argument = true
split_before_logical_operator = true
split_before_dot = true
'''))
f.flush()
formatted_code, changed = yapf_api.FormatCode(
data['code'], style_config=f.name, **extra)
except SyntaxError as e:
logger.exception("Error in source code")
return json.dumps(dict(error=True, error_line=e.lineno))
if REQUEST is not None:
REQUEST.RESPONSE.setHeader('content-type', 'application/json')
return json.dumps(dict(formatted_code=formatted_code, changed=changed))
bt5/erp5_monaco_editor/ExtensionTemplateItem/portal_components/extension.erp5.YAPF.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Extension Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>YAPF</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>extension.erp5.YAPF</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Extension Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/ERP5Site_dumpModuleCode.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>ERP5Site_dumpModuleCode</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ERP5Site_dumpModuleCode</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/ERP5Site_formatPythonSourceCode.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>ERP5Site_formatPythonSourceCode</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>YAPF</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ERP5Site_formatPythonSourceCode</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/ERP5Site_getPortalStub.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>ERP5Site_getPortalStub</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ERP5Site_getPortalStub</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/ERP5Site_getPythonSourceCodeCompletionList.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>ERP5Site_getPythonSourceCodeCompletionList</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>ERP5Site_getPythonSourceCodeCompletionList</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/PropertySheetTool_getStub.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>PropertySheetTool_getStub</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>PropertySheetTool_getStub</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/PropertySheet_getStub.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>PropertySheet_getStub</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>PropertySheet_getStub</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/SkinsTool_getStub.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>SkinsTool_getStub</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>SkinsTool_getStub</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/TypeInformation_getStub.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>TypeInformation_getStub</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>TypeInformation_getStub</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/TypesTool_getStub.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="ExternalMethod" module="Products.ExternalMethod.ExternalMethod"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_function</string> </key>
<value> <string>TypesTool_getStub</string> </value>
</item>
<item>
<key> <string>_module</string> </key>
<value> <string>Jedi</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>TypesTool_getStub</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_monaco_editor/SkinTemplateItem/portal_skins/erp5_monaco_editor/monaco_editor_support.zpt
View file @ d2b29af1
......@@ -47,6 +47,7 @@
<script tal:content='python: "var textarea_selector=" + modules["json"].dumps(options.get("textarea_selector"))'>
</script>
<script tal:content='python: "var bound_names=" + modules["json"].dumps(options.get("bound_names"))'></script>
<script tal:content='python: "var script_name=" + modules["json"].dumps(options.get("script_name"))'></script>
<script
tal:content='python: "window.monacoEditorWebPackResourceBaseUrl = " + modules["json"].dumps(options["portal_url"]) + " + \"/monaco-editor/\""'>
......@@ -254,10 +255,205 @@ $script.onload = function() {
function makeTimeoutFunction(ac){
return () => checkPythonSourceCode(ac)
}
timeout = setTimeout(makeTimeoutFunction(controller), 300);
timeout = setTimeout(makeTimeoutFunction(controller), 3000);
}
});
yapfDocumentFormattingProvider = {
_provideFormattingEdits: function(model, range, options, token) {
const controller = new AbortController();
token.onCancellationRequested(() => {controller.abort()})
const data = new FormData();
data.append("data", JSON.stringify({code: model.getValue(), range:range}));
return fetch(portal_url + "/ERP5Site_formatPythonSourceCode", {
method: "POST",
body: data,
signal: controller.signal
})
.then(response => response.json())
.then(data => {
if (data.error){
editor.revealLine(data.error_line);
return;
}
if (data.changed) {
return [
{
range: model.getFullModelRange(),
text: data.formatted_code,
},
];
};
}, e => {
if (!e instanceof DOMException /* AbortError */ ) {
throw e;
}
/* ignore aborted requests */
});
},
provideDocumentRangeFormattingEdits: function(model, range, options, token){
return this._provideFormattingEdits(model, range, options, token);
},
provideDocumentFormattingEdits: function(model, options, token) {
return this._provideFormattingEdits(model, null, options, token);
}
}
monaco.languages.registerDocumentFormattingEditProvider(
'python',
yapfDocumentFormattingProvider)
monaco.languages.registerDocumentRangeFormattingEditProvider(
'python',
yapfDocumentFormattingProvider)
monaco.languages.registerDefinitionProvider('python', {
provideDefinition: async function(model, position, token) {
const controller = new AbortController();
token.onCancellationRequested(() => {controller.abort()})
const data = new FormData();
const complete_parameters = {
code: model.getValue(),
position: {line: position.lineNumber, column: position.column}
};
// ZMI python scripts pass extra parameters to linter
if (bound_names) {
complete_parameters["script_name"] = script_name;
complete_parameters["bound_names"] = JSON.parse(bound_names);
complete_parameters["params"] = document.querySelector(
'input[name="params"]'
).value;
}
complete_parameters['xxx_definition'] = true;
data.append("data", JSON.stringify(complete_parameters));
return fetch(portal_url + "/ERP5Site_getPythonSourceCodeCompletionList", {
method: "POST",
body: data,
signal: controller.signal
})
.then(response => response.json())
.then(data => {
var definitions = [];
for (let i = 0; i < data.length; i++) {
if (data[i].code) {
// TODO: these models are not refreshed, if the file they refefer is modified,
// they show outdated content.
let definition_uri = monaco.Uri.from({
scheme: 'file',
path: data[i].uri,
});
let definition_model = monaco.editor.getModel(
definition_uri
);
if (!definition_model) {
definition_model = monaco.editor.createModel(
data[i].code,
'python',
definition_uri
);
}
data[i].uri = definition_model.uri;
}
definitions.push({
range: data[i].range,
uri: data[i].uri ? data[i].uri : model.uri,
});
}
return definitions;
}, e => {
if (!(e instanceof DOMException) /* AbortError */ ) {
throw e;
}
/* ignore aborted requests */
});
}
});
monaco.languages.registerCompletionItemProvider('python', {
provideCompletionItems: async function(model, position, context, token) {
const controller = new AbortController();
token.onCancellationRequested(() => {controller.abort()})
const data = new FormData();
const complete_parameters = {
code: model.getValue(),
position: {line: position.lineNumber, column: position.column}
};
// ZMI python scripts pass extra parameters to linter
if (bound_names) {
complete_parameters["script_name"] = script_name;
complete_parameters["bound_names"] = JSON.parse(bound_names);
complete_parameters["params"] = document.querySelector(
'input[name="params"]'
).value;
}
data.append("data", JSON.stringify(complete_parameters));
return fetch(portal_url + "/ERP5Site_getPythonSourceCodeCompletionList", {
method: "POST",
body: data,
signal: controller.signal
})
.then(response => response.json())
.then(data => {
return {suggestions: data.map(c => {
c.kind = monaco.languages.CompletionItemKind[c._kind];
// this makes monaco render documentation as markdown.
c.documentation = {value: c.documentation};
return c
})};
}, e => {
if (!e instanceof DOMException /* AbortError */ ) {
throw e;
}
/* ignore aborted requests */
});
}
});
monaco.languages.registerHoverProvider('python', {
provideHover: function (model, position, token) {
const controller = new AbortController();
token.onCancellationRequested(() => {controller.abort()})
const data = new FormData();
const complete_parameters = {
code: model.getValue(),
position: {line: position.lineNumber, column: position.column}
};
// ZMI python scripts pass extra parameters to linter
if (bound_names) {
complete_parameters["script_name"] = script_name;
complete_parameters["bound_names"] = JSON.parse(bound_names);
complete_parameters["params"] = document.querySelector(
'input[name="params"]'
).value;
}
complete_parameters['xxx_hover'] = true;
data.append("data", JSON.stringify(complete_parameters));
return fetch(portal_url + "/ERP5Site_getPythonSourceCodeCompletionList", {
method: "POST",
body: data,
signal: controller.signal
})
.then(response => response.json())
.then(data => {
return {
range: new monaco.Range(position.lineNumber, position.column, position.lineNumber, 0),
contents: [
{ value: data }, // XXX
]
}
}, e => {
if (!e instanceof DOMException /* AbortError */ ) {
throw e;
}
/* ignore aborted requests */
});
}
});
if (mode === "python") {
// Perform a first check when loading document.
checkPythonSourceCode(new AbortController());
......
bt5/erp5_monaco_editor/bt/template_extension_id_list 0 → 100644
View file @ d2b29af1
extension.erp5.Jedi
extension.erp5.YAPF
\ No newline at end of file
bt5/erp5_python_support/PortalTypeTemplateItem/portal_types/Python%20Support%20Tool.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Base Type" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>content_icon</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>description</string> </key>
<value> <string>Provide intellisense for python.</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Python Support Tool</string> </value>
</item>
<item>
<key> <string>init_script</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>permission</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Base Type</string> </value>
</item>
<item>
<key> <string>type_class</string> </key>
<value> <string>PythonSupportTool</string> </value>
</item>
<item>
<key> <string>type_interface</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>type_mixin</string> </key>
<value>
<tuple/>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_python_support/TestTemplateItem/portal_components/test.erp5.testPythonSupport.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Test Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>testPythonSupport</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>test.erp5.testPythonSupport</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Test Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_python_support/ToolComponentTemplateItem/portal_components/tool.erp5.PythonSupportTool.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Tool Component" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_recorded_property_dict</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>default_reference</string> </key>
<value> <string>PythonSupportTool</string> </value>
</item>
<item>
<key> <string>description</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>tool.erp5.PythonSupportTool</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Tool Component</string> </value>
</item>
<item>
<key> <string>sid</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>text_content_error_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>text_content_warning_message</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>version</string> </key>
<value> <string>erp5</string> </value>
</item>
<item>
<key> <string>workflow_history</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAM=</string> </persistent>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary/>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="3" aka="AAAAAAAAAAM=">
<pickle>
<global name="PersistentMapping" module="Persistence.mapping"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>data</string> </key>
<value>
<dictionary>
<item>
<key> <string>component_validation_workflow</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAQ=</string> </persistent>
</value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</pickle>
</record>
<record id="4" aka="AAAAAAAAAAQ=">
<pickle>
<global name="WorkflowHistoryList" module="Products.ERP5Type.Workflow"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_log</string> </key>
<value>
<list>
<dictionary>
<item>
<key> <string>action</string> </key>
<value> <string>validate</string> </value>
</item>
<item>
<key> <string>validation_state</string> </key>
<value> <string>validated</string> </value>
</item>
</dictionary>
</list>
</value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_python_support/ToolTemplateItem/portal_python_support.xml 0 → 100644
View file @ d2b29af1
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Python Support Tool" module="erp5.portal_type"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>id</string> </key>
<value> <string>portal_python_support</string> </value>
</item>
<item>
<key> <string>portal_type</string> </key>
<value> <string>Python Support Tool</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_python_support/bt/template_portal_type_id_list 0 → 100644
View file @ d2b29af1
Python Support Tool
\ No newline at end of file
bt5/erp5_python_support/bt/template_test_id_list 0 → 100644
View file @ d2b29af1
test.erp5.testPythonSupport
\ No newline at end of file
bt5/erp5_python_support/bt/template_tool_component_id_list 0 → 100644
View file @ d2b29af1
tool.erp5.PythonSupportTool
\ No newline at end of file
bt5/erp5_python_support/bt/template_tool_id_list 0 → 100644
View file @ d2b29af1
portal_python_support
\ No newline at end of file
bt5/erp5_python_support/bt/title 0 → 100644
View file @ d2b29af1
erp5_python_support
\ No newline at end of file
product/ERP5/bootstrap/erp5_core/ExtensionTemplateItem/portal_components/extension.erp5.PythonCodeUtils.py
View file @ d2b29af1
from six import string_types as basestring
import time
import json
import logging
from Products.ERP5Type.Utils import checkPythonSourceCode
logger = logging.getLogger('extension.erp5.PythonCodeUtils')
def checkPythonSourceCodeAsJSON(self, data, REQUEST=None):
"""
Check Python source suitable for Source Code Editor and return a JSON object
"""
import json
# XXX data is encoded as json, because jQuery serialize lists as []
if isinstance(data, basestring):
......@@ -17,6 +22,7 @@ def checkPythonSourceCodeAsJSON(self, data, REQUEST=None):
return ''.join((" " + line) for line in text.splitlines(True))
# don't show 'undefined-variable' errors for {Python,Workflow} Script parameters
script_name = data.get('script_name') or 'unknown.py'
is_script = 'bound_names' in data
if is_script:
signature_parts = data['bound_names']
......@@ -31,7 +37,86 @@ def checkPythonSourceCodeAsJSON(self, data, REQUEST=None):
else:
body = data['code']
message_list = checkPythonSourceCode(body.encode('utf8'), data.get('portal_type'))
start = time.time()
code = body.encode('utf8')
import pyflakes.api
import pyflakes.reporter
import pyflakes.messages
class Reporter(pyflakes.reporter.Reporter):
def __init__(self): # pylint: disable=super-init-not-called
self.message_list = []
def addMessage(self, row, column, level, text):
self.message_list.append(
dict(row=row, column=column, type=level, text=text))
def flake(self, message):
# type: (pyflakes.messages.Message,) -> None
self.addMessage(
row=message.lineno,
column=message.col,
text=message.message % (message.message_args),
level='W')
def syntaxError(self, filename, msg, lineno, offset, text):
self.addMessage(
row=lineno,
column=offset,
text='SyntaxError: {}'.format(text),
level='E')
def unexpectedError(self, filename, msg):
# TODO: extend interface to have range and in this case whole range is wrong ?
# or use parse with python in that case ?
# repro: function(a="b", c)
self.addMessage(
row=0, column=0, text='Unexpected Error: {}'.format(msg), level='E')
start = time.time()
reporter = Reporter()
pyflakes.api.check(code, script_name, reporter)
logger.info(
'pyflake checked %d lines in %.2f',
len(code.splitlines()),
time.time() - start
)
message_list = reporter.message_list
import lib2to3.refactor
import lib2to3.pgen2.parse
refactoring_tool = lib2to3.refactor.RefactoringTool(fixer_names=('lib2to3.fixes.fix_except', ))
old_code = code.decode('utf-8')
try:
new_code = unicode(refactoring_tool.refactor_string(old_code, script_name))
except lib2to3.pgen2.parse.ParseError as e:
message, (row, column) = e.context
message_list.append(
dict(row=row, column=column, type='E', text=message))
else:
if new_code != old_code:
i = 0
for new_line, old_line in zip(new_code.splitlines(), old_code.splitlines()):
i += 1
#print ('new_line', new_line, 'old_line', old_line)
if new_line != old_line:
message_list.append(
dict(row=i, column=0, type='W', text=u'-{}\n+{}'.format(old_line, new_line)))
# import pdb; pdb.set_trace()
pylint_message_list = []
if 1:
start = time.time()
pylint_message_list = checkPythonSourceCode(code, data.get('portal_type'))
logger.info(
'pylint checked %d lines in %.2f',
len(code.splitlines()),
time.time() - start
)
message_list = pylint_message_list
for message_dict in message_list:
if is_script:
message_dict['row'] = message_dict['row'] - 2
......@@ -46,3 +131,4 @@ def checkPythonSourceCodeAsJSON(self, data, REQUEST=None):
if REQUEST is not None:
REQUEST.RESPONSE.setHeader('content-type', 'application/json')
return json.dumps(dict(annotations=message_list))
\ No newline at end of file
product/ERP5/bootstrap/erp5_core/SkinTemplateItem/portal_skins/erp5_core/ComponentMixin_view/my_text_content.xml
View file @ d2b29af1
......@@ -204,6 +204,16 @@
<key> <string>width</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>renderjs_extra</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</value>
</item>
......@@ -271,6 +281,12 @@
<key> <string>text_editor</string> </key>
<value> <string>text_area</string> </value>
</item>
<item>
<key> <string>renderjs_extra</string> </key>
<value>
<list/>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Source Code</string> </value>
......
product/ERP5/bootstrap/erp5_core/SkinTemplateItem/portal_skins/erp5_core/ComponentMixin_viewAsWeb/my_text_content.xml
View file @ d2b29af1
......@@ -10,6 +10,7 @@
<key> <string>delegated_list</string> </key>
<value>
<list>
<string>renderjs_extra</string>
<string>title</string>
</list>
</value>
......@@ -56,6 +57,12 @@
<key> <string>form_id</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>renderjs_extra</string> </key>
<value>
<persistent> <string encoding="base64">AAAAAAAAAAI=</string> </persistent>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
......@@ -75,6 +82,12 @@
<key> <string>form_id</string> </key>
<value> <string>Base_viewFieldLibrary</string> </value>
</item>
<item>
<key> <string>renderjs_extra</string> </key>
<value>
<list/>
</value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string>Source Code</string> </value>
......@@ -85,4 +98,17 @@
</dictionary>
</pickle>
</record>
<record id="2" aka="AAAAAAAAAAI=">
<pickle>
<global name="TALESMethod" module="Products.Formulator.TALESField"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_text</string> </key>
<value> <string>python: [(\'editor\', context.Base_getEditorFieldPreferredTextEditor()), (\'portal_type\', context.getPortalType()), (\'maximize\', \'listbox\' not in field.id), (\'content_type\', context.getProperty(\'content_type\')), (\'language_support_url\', context.getPortalObject().portal_url())]</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
product/ERP5/bootstrap/erp5_core/SkinTemplateItem/portal_skins/erp5_core/gadget_editor.js.js
View file @ d2b29af1
......@@ -15,6 +15,8 @@
* @property {string} language the user language, if the editor supports
* localisation it will be displayed in this language
* @property {string} password a password to decrypt the content
* @property {string} language_support_url URL of language support endpoint
* for completion, diagnostics etc in the editor
* @property {boolean} run a hack for jsmd editor
* @property {string} key Key for ERP5 form
*/
......@@ -90,6 +92,7 @@
run: options.run || false,
key: options.key,
password: options.password,
language_support_url: options.language_support_url || "",
// Force calling subfield render
// as user may have modified the input value
render_timestamp: new Date().getTime()
......
product/ERP5Type/Core/Folder.py
View file @ d2b29af1
......@@ -56,6 +56,11 @@ from Products.ERP5Type.Utils import sortValueList
from Products.ERP5Type import Permissions
from Products.ERP5Type.Globals import InitializeClass
from Products.ERP5Type.Accessor import Base as BaseAccessor
try:
from typing import List
except ImportError:
pass
try:
from Products.CMFCore.CMFBTreeFolder import CMFBTreeFolder
except ImportError:
......@@ -206,6 +211,7 @@ class FolderMixIn(ExtensionClass.Base):
security.declarePublic('newContent')
def newContent(self, id=None, portal_type=None, id_group=None,
default=None, method=None, container=None, temp_object=0, **kw):
# type: (...) -> Folder
"""Creates a new content.
This method is public, since TypeInformation.constructInstance will perform
the security check.
......@@ -421,6 +427,7 @@ class FolderMixIn(ExtensionClass.Base):
# Get the content
security.declareProtected(Permissions.AccessContentsInformation, 'searchFolder')
def searchFolder(self, **kw):
# type: (str) -> List[Folder]
"""
Search the content of a folder by calling
the portal_catalog.
......@@ -1521,6 +1528,7 @@ class Folder(FolderMixIn, CopyContainer, ObjectManager, Base, OFSFolder2, CMFBTr
def objectValues(self, spec=None, meta_type=None, portal_type=None,
sort_on=None, sort_order=None, checked_permission=None,
**kw):
# type: () -> List[Folder]
# Returns list of objects contained in this folder.
# (no docstring to prevent publishing)
if meta_type is not None:
......@@ -1552,6 +1560,7 @@ class Folder(FolderMixIn, CopyContainer, ObjectManager, Base, OFSFolder2, CMFBTr
security.declareProtected( Permissions.AccessContentsInformation,
'contentValues' )
def contentValues(self, *args, **kw):
# type: () -> List[Folder]
# Returns a list of documents contained in this folder.
# ( no docstring to prevent publishing )
portal_type_id_list = self._getTypesTool().listContentTypes()
......
product/ERP5Type/Utils.py
View file @ d2b29af1
......@@ -431,14 +431,35 @@ def fill_args_from_request(*optional_args):
_pylint_message_re = re.compile(
'^(?P<type>[CRWEF]):\s*(?P<row>\d+),\s*(?P<column>\d+):\s*(?P<message>.*)$')
zope_startup_time = time.time()
def checkPythonSourceCode(source_code_str, portal_type=None):
"""
Check source code with pylint or compile() builtin if not available.
`portal_type` argument can be passed to the checker, to adapt the checks
based on the portal type.
This function is cached, so checkin the same code several times is instant.
TODO-arnau: Get rid of NamedTemporaryFile (require a patch on pylint to
allow passing a string) and this should probably return a proper
ERP5 object rather than a dict...
"""
# late imports because we have a circular import dependency here.
from Products.ERP5Type.Cache import CachingMethod
from Products.ERP5.ERP5Site import getSite
checkPythonSourceCode = CachingMethod(
_checkPythonSourceCode,
'_checkPythonSourceCode.{}.{}.{}'.format(
zope_startup_time,
md5_new(source_code_str).hexdigest(),
getSite().getCacheCookie('component_packages')))
# normalize type of portal_type argument to maximize cache hits.
if isinstance(portal_type, unicode):
portal_type = portal_type.encode()
return checkPythonSourceCode(source_code_str, portal_type)
def _checkPythonSourceCode(source_code_str, portal_type):
if not source_code_str:
return []
......
product/ERP5Type/patches/SourceCodeEditorZMI.py
View file @ d2b29af1
......@@ -108,6 +108,7 @@ def manage_page_footer(self):
textarea_selector=textarea_selector,
portal_url=portal_url,
bound_names=bound_names,
script_name=document.getId(),
mode=mode).encode('utf-8'))
return default
......
product/ERP5Type/tests/ERP5TypeTestCase.py
View file @ d2b29af1
......@@ -19,6 +19,7 @@ import sys
import time
import traceback
import urllib
import unittest
import ConfigParser
from contextlib import contextmanager
from cStringIO import StringIO
......@@ -214,7 +215,14 @@ def _parse_args(self, *args, **kw):
_parse_args._original = DateTime._original_parse_args
DateTime._parse_args = _parse_args
class ERP5TypeTestCaseMixin(ProcessingNodeTestCase, PortalTestCase):
try:
from erp5.portal_type import ERP5Site as erp5_portal_type_ERP5Site
except ImportError:
pass
class ERP5TypeTestCaseMixin(ProcessingNodeTestCase, PortalTestCase, unittest.TestCase, object):
"""Mixin class for ERP5 based tests.
"""
def __init__(self, *args, **kw):
......@@ -990,6 +998,7 @@ class ERP5TypeCommandLineTestCase(ERP5TypeTestCaseMixin):
return portal_name + '_' + m.hexdigest()
def getPortal(self):
# type: () -> erp5_portal_type_ERP5Site
"""Returns the portal object, i.e. the "fixture root".
It also does some initialization, as if the portal was accessed for the
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7