Skip to content

erp5
erp5
Commit c39b8e0a authored by Kazuhiko Shiozaki's avatar Kazuhiko Shiozaki
1 Browse files
Options

defer fulltext index in a lower priority activity. 

also raise an exception in case of error, like cloudooo is not running.
parent 10c9f862
Hide whitespace changes
Inline Side-by-side
Showing with 398 additions and 8 deletions
bt5/erp5_full_text_mroonga_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/SQLCatalog_deferFullTextIndex.catalog_keys.xml 0 → 100644
View file @ c39b8e0a
<catalog_method>
<item key="sql_catalog_object_list" type="int">
<value>1</value>
</item>
</catalog_method>
bt5/erp5_full_text_mroonga_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/SQLCatalog_deferFullTextIndex.xml 0 → 100644
View file @ c39b8e0a
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string># This script is called to defer fulltext indexing in a lower priority.\n
context.activate(activity=\'SQLQueue\', priority=4).SQLCatalog_deferFullTextIndexActivity(path_list=list(getPath))\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>getPath</string> </value>
</item>
<item>
<key> <string>_proxy_roles</string> </key>
<value>
<tuple>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>SQLCatalog_deferFullTextIndex</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_full_text_mroonga_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/SQLCatalog_deferFullTextIndexActivity.xml 0 → 100644
View file @ c39b8e0a
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>from Products.ERP5Type.Utils import UpperCase\n
from ZODB.POSException import ConflictError\n
from zExceptions import Unauthorized\n
\n
method = context.z_catalog_fulltext_list\n
property_list = method.arguments_src.split()\n
parameter_dict = {}\n
failed_path_list = []\n
restrictedTraverse = context.getPortalObject().restrictedTraverse\n
for path in path_list:\n
if not path: # should happen in tricky testERP5Catalog tests only \n
continue\n
obj = restrictedTraverse(path, None)\n
if obj is None:\n
continue\n
try:\n
tmp_dict = {}\n
for property in property_list:\n
if property == \'SearchableText\':\n
value = obj.SearchableText()\n
else:\n
value = getattr(obj, \'get%s\' % UpperCase(property))()\n
tmp_dict[property] = value\n
except ConflictError:\n
raise\n
except Unauthorized: # should happen in tricky testERP5Catalog tests only \n
continue\n
except Exception, e:\n
exception = e\n
failed_path_list.append(path)\n
else:\n
for property, value in tmp_dict.items():\n
parameter_dict.setdefault(property, []).append(value)\n
\n
if len(failed_path_list):\n
if len(parameter_dict):\n
# reregister activity for failed objects only\n
context.activate(activity=\'SQLQueue\', priority=5).SQLCatalog_deferFullTextIndexActivity(path_list=failed_path_list)\n
else:\n
# if all objects are failed one, just raise an exception to avoid infinite loop.\n
raise AttributeError, \'exception %r raised in indexing %r\' % (exception, failed_path_list)\n
\n
if parameter_dict:\n
return method(**parameter_dict)\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>path_list</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>SQLCatalog_deferFullTextIndexActivity</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_full_text_mroonga_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/z_catalog_fulltext_list.catalog_keys.xml
View file @ c39b8e0a
<catalog_method>
<item key="sql_catalog_object_list" type="int">
<value>1</value>
</item>
</catalog_method>
bt5/erp5_full_text_mroonga_catalog/bt/change_log
View file @ c39b8e0a
2013-04-04 Kazuhiko
* defer fulltext index in a lower priority activity.
2012-01-30 Kazuhiko
* engine name is mroonga instead of groonga since mroonga-1.20.
......
bt5/erp5_full_text_mroonga_catalog/bt/revision
View file @ c39b8e0a
4
\ No newline at end of file
7
\ No newline at end of file
bt5/erp5_full_text_mroonga_catalog/bt/template_catalog_method_id_list
View file @ c39b8e0a
erp5_mysql_innodb/SQLCatalog_deferFullTextIndex
erp5_mysql_innodb/SQLCatalog_deferFullTextIndexActivity
erp5_mysql_innodb/SQLCatalog_makeFullTextQuery
erp5_mysql_innodb/z0_drop_content_translation
erp5_mysql_innodb/z0_drop_fulltext
......
bt5/erp5_full_text_myisam_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/SQLCatalog_deferFullTextIndex.catalog_keys.xml 0 → 100644
View file @ c39b8e0a
<catalog_method>
<item key="sql_catalog_object_list" type="int">
<value>1</value>
</item>
</catalog_method>
bt5/erp5_full_text_myisam_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/SQLCatalog_deferFullTextIndex.xml 0 → 100644
View file @ c39b8e0a
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string># This script is called to defer fulltext indexing in a lower priority.\n
context.activate(activity=\'SQLQueue\', priority=4).SQLCatalog_deferFullTextIndexActivity(path_list=list(getPath))\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>getPath</string> </value>
</item>
<item>
<key> <string>_proxy_roles</string> </key>
<value>
<tuple>
<string>Manager</string>
</tuple>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>SQLCatalog_deferFullTextIndex</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_full_text_myisam_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/SQLCatalog_deferFullTextIndexActivity.xml 0 → 100644
View file @ c39b8e0a
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>from Products.ERP5Type.Utils import UpperCase\n
from ZODB.POSException import ConflictError\n
from zExceptions import Unauthorized\n
\n
method = context.z_catalog_fulltext_list\n
property_list = method.arguments_src.split()\n
parameter_dict = {}\n
failed_path_list = []\n
restrictedTraverse = context.getPortalObject().restrictedTraverse\n
for path in path_list:\n
if not path: # should happen in tricky testERP5Catalog tests only \n
continue\n
obj = restrictedTraverse(path, None)\n
if obj is None:\n
continue\n
try:\n
tmp_dict = {}\n
for property in property_list:\n
if property == \'SearchableText\':\n
value = obj.SearchableText()\n
else:\n
value = getattr(obj, \'get%s\' % UpperCase(property))()\n
tmp_dict[property] = value\n
except ConflictError:\n
raise\n
except Unauthorized: # should happen in tricky testERP5Catalog tests only \n
continue\n
except Exception, e:\n
exception = e\n
failed_path_list.append(path)\n
else:\n
for property, value in tmp_dict.items():\n
parameter_dict.setdefault(property, []).append(value)\n
\n
if len(failed_path_list):\n
if len(parameter_dict):\n
# reregister activity for failed objects only\n
context.activate(activity=\'SQLQueue\', priority=5).SQLCatalog_deferFullTextIndexActivity(path_list=failed_path_list)\n
else:\n
# if all objects are failed one, just raise an exception to avoid infinite loop.\n
raise AttributeError, \'exception %r raised in indexing %r\' % (exception, failed_path_list)\n
\n
if parameter_dict:\n
return method(**parameter_dict)\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>path_list</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>SQLCatalog_deferFullTextIndexActivity</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_full_text_myisam_catalog/CatalogMethodTemplateItem/portal_catalog/erp5_mysql_innodb/z_catalog_fulltext_list.catalog_keys.xml
View file @ c39b8e0a
<catalog_method>
<item key="sql_catalog_object_list" type="int">
<value>1</value>
</item>
</catalog_method>
bt5/erp5_full_text_myisam_catalog/bt/change_log
View file @ c39b8e0a
2013-04-04 Kazuhiko
* defer fulltext index in a lower priority activity.
2012-01-05 Kazuhiko
* move create/drop table SQL for content_translation table from erp5_content_translation to erp5_full_text_*_catalog BT.
......
bt5/erp5_full_text_myisam_catalog/bt/revision
View file @ c39b8e0a
6
\ No newline at end of file
9
\ No newline at end of file
bt5/erp5_full_text_myisam_catalog/bt/template_catalog_method_id_list
View file @ c39b8e0a
erp5_mysql_innodb/SQLCatalog_deferFullTextIndex
erp5_mysql_innodb/SQLCatalog_deferFullTextIndexActivity
erp5_mysql_innodb/SQLCatalog_makeFullTextQuery
erp5_mysql_innodb/z0_drop_content_translation
erp5_mysql_innodb/z0_drop_fulltext
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7