Skip to content

erp5
erp5
Commit 3bd8c2ba authored by Ivan Tyagov's avatar Ivan Tyagov
Browse files
Options

Initial import. 

git-svn-id: https://svn.erp5.org/repos/public/erp5/trunk@43801 20353a03-c40f-0410-a6d1-a30d3c3de9de
parent 7f3ff26e
Hide whitespace changes
Inline Side-by-side
Showing with 540 additions and 0 deletions
bt5/erp5_km_sphinx_full_text_search/SkinTemplateItem/portal_skins/erp5_km_sphinx_full_text_search.xml 0 → 100644
View file @ 3bd8c2ba
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="Folder" module="OFS.Folder"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_objects</string> </key>
<value>
<tuple/>
</value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>erp5_km_sphinx_full_text_search</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_km_sphinx_full_text_search/SkinTemplateItem/portal_skins/erp5_km_sphinx_full_text_search/Base_redirectToPersonByReference.xml 0 → 100644
View file @ 3bd8c2ba
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>"""\n
This script will redirect (HTTP redirect) to respective ERP5 Person object by reference.\n
This script is used in "NO ZODB" approach mode although it can be used in other UI parts\n
as well.\n
"""\n
person = context.ERP5Site_getAuthenticatedMemberPersonValue(reference)\n
if person is not None:\n
person.Base_redirect(form_id=\'view\')\n
else:\n
# logged in user (or anonymous) can\'t access or no such user exists\n
context.Base_redirect(\n
form_id = \'view\',\n
keep_items = {\'portal_status_message\': \n
context.Base_translateString(\'You can not access person object.\')})\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>reference</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>Base_redirectToPersonByReference</string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_km_sphinx_full_text_search/SkinTemplateItem/portal_skins/erp5_km_sphinx_full_text_search/WebSite_getFullTextSearchResultList.xml 0 → 100644
View file @ 3bd8c2ba
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>"""\n
Return the result list of all documents found by specified agruments.\n
Include an optimisation for \'search\' mode which returns all documents info like \n
(reference, state, web sections ...) without getting a ZODB object (i.e. get everything from \n
MySQL - "No ZODB" approach).\n
"""\n
from Products.ERP5Type.Document import newTempBase\n
\n
request = context.REQUEST\n
portal = context.getPortalObject()\n
web_site = context.getWebSiteValue()\n
\n
selection = kw.get(\'selection\', {})\n
list_style = kw.get(\'list_style\', \\\n
selection.get(\'list_style\', request.get(\'list_style\', \'table\')))\n
\n
search_text = kw[\'search_text\']\n
parsed_dict = context.Base_parseSearchString(search_text)\n
search_string = parsed_dict[\'searchabletext\']\n
\n
if list_style != \'search\':\n
return context.portal_catalog(**kw)\n
else:\n
# search mode requires optimization, use catalog to get more data from it\n
result_list = []\n
result_set_dict_list = [] \n
repeating_uid_category_map = {}\n
portal_types = portal.portal_types\n
\n
# get Web Site predicate info\n
category_section_map, base_category_uid_list = web_site.WebSite_getWebSectionPredicateMapAndUidList()\n
\n
# XXX: using catalog API instead of script should be researched as a more maintainable alternative\n
found_result_list = web_site.WebSite_zGetAdvancedSearchResultList(\n
base_category_uid_list = base_category_uid_list,\n
search_string = search_string,\n
is_full_text_search_on = 1,\n
use_text_excerpts = 1,\n
kw = kw)\n
for line in found_result_list:\n
uid = line[\'uid\']\n
if uid not in repeating_uid_category_map.keys():\n
# first time \n
repeating_uid_category_map[uid] = []\n
category_relative_url = line[\'category_relative_url\']\n
if category_relative_url is not None:\n
# exactly matches, document("group/nexedi") belongs to section("group/nexedi")\n
sections = category_section_map.get(category_relative_url, []) \n
if not len(sections):\n
# try to find by similarity if no exact match so if document belongs to \'group/nexedi/hq\'\n
# and we have a section \'group/nexedi\' it will belong to this section\n
for key,value in category_section_map.items():\n
if category_relative_url.startswith(key):\n
sections.extend(value)\n
repeating_uid_category_map[uid].extend(sections)\n
# turn into a relative URL\n
path = line[\'path\'].replace(\'/%s/\' %portal.getId(), \'\')\n
result_set_dict_list.append({\'uid\': uid,\n
\'object_portal_type\': line[\'portal_type\'],\n
\'object_icon\': portal_types[line[\'portal_type\']].getIcon(),\n
\'path\': path,\n
\'title\': line[\'title\'],\n
\'text\': getattr(line, \'text\', \'\'),\n
\'modification_date\': line[\'modification_date\'],\n
\'reference\': line[\'reference\'],\n
\'category_relative_url\': line[\'category_relative_url\'],\n
\'owner\': line[\'owner\'],\n
\'web_site\': web_site.getRelativeUrl()})\n
\n
# one document can belong to n categories, we need show only one doc\n
# and all sections it belongs to\n
found_uids = []\n
for line in result_set_dict_list:\n
uid = line[\'uid\']\n
if uid not in found_uids:\n
found_uids.append(uid)\n
# show only unique sections\n
unique_sections = {}\n
sections = repeating_uid_category_map[uid]\n
for section in sections:\n
unique_sections[section[\'uid\']] = section[\'relative_url\']\n
line[\'section_list\'] = unique_sections.values()\n
result_list.append(line)\n
return [newTempBase(portal, x[\'title\'], **x) for x in result_list]\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string>**kw</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>WebSite_getFullTextSearchResultList</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_km_sphinx_full_text_search/SkinTemplateItem/portal_skins/erp5_km_sphinx_full_text_search/WebSite_getWebSectionPredicateMapAndUidList.xml 0 → 100644
View file @ 3bd8c2ba
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="PythonScript" module="Products.PythonScripts.PythonScript"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>Script_magic</string> </key>
<value> <int>3</int> </value>
</item>
<item>
<key> <string>_bind_names</string> </key>
<value>
<object>
<klass>
<global name="NameAssignments" module="Shared.DC.Scripts.Bindings"/>
</klass>
<tuple/>
<state>
<dictionary>
<item>
<key> <string>_asgns</string> </key>
<value>
<dictionary>
<item>
<key> <string>name_container</string> </key>
<value> <string>container</string> </value>
</item>
<item>
<key> <string>name_context</string> </key>
<value> <string>context</string> </value>
</item>
<item>
<key> <string>name_m_self</string> </key>
<value> <string>script</string> </value>
</item>
<item>
<key> <string>name_subpath</string> </key>
<value> <string>traverse_subpath</string> </value>
</item>
</dictionary>
</value>
</item>
</dictionary>
</state>
</object>
</value>
</item>
<item>
<key> <string>_body</string> </key>
<value> <string>"""\n
Examine Web Site\'s Web Sections and return mapping between sections\' uid and respective\n
category used in sections\' predicate.\n
This script is used in "No ZODB" approach to get fast search results (including list of \n
sections a object belongs to).\n
"""\n
from Products.ERP5Type.Cache import CachingMethod\n
\n
website = context.getWebSiteValue()\n
\n
def getWebSectionList(section):\n
result = [{\'uid\': section.getUid(),\n
\'relative_url\': section.getRelativeUrl(),\n
\'membership_base_category_list\': section.getMembershipCriterionBaseCategoryList(),\n
\'multi_membership_base_category_list\': section.getMultimembershipCriterionBaseCategoryList(),\n
\'membership_category_list\': section.getMembershipCriterionCategoryList()}]\n
for section in section.contentValues(portal_type=\'Web Section\'):\n
result.extend(getWebSectionList(section))\n
return result\n
\n
def getWebSectionPredicateValueList():\n
category_map = {}\n
base_category_uid_list = []\n
portal_categories = context.portal_categories\n
for section in getWebSectionList(website):\n
# calc category_path : section map \n
for category in section[\'membership_category_list\']:\n
# remove leading \'follow_up\' from category\n
if category.startswith(\'follow_up/\'):\n
category = category.replace(\'follow_up/\', \'\', 1)\n
if not category_map.has_key(category):\n
category_map[category] = []\n
category_map[category].append({\'uid\': section[\'uid\'], \'relative_url\':section[\'relative_url\']})\n
# get base_categories we care for\n
section_category_list = section[\'membership_base_category_list\']+section[\'multi_membership_base_category_list\']\n
for category_id in section_category_list:\n
category = getattr(portal_categories, category_id, None)\n
if category is not None and category.getUid() not in base_category_uid_list:\n
base_category_uid_list.append(category.getUid())\n
return category_map, base_category_uid_list\n
\n
getWebSectionPredicateValueList = CachingMethod(getWebSectionPredicateValueList,\n
id = \'WebSite_getWebSectionPredicateMapAndUidList\',\n
cache_factory = \'erp5_content_medium\')\n
return getWebSectionPredicateValueList()\n
</string> </value>
</item>
<item>
<key> <string>_params</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>WebSite_getWebSectionPredicateMapAndUidList</string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_km_sphinx_full_text_search/SkinTemplateItem/portal_skins/erp5_km_sphinx_full_text_search/WebSite_zGetAdvancedSearchResultList.xml 0 → 100644
View file @ 3bd8c2ba
<?xml version="1.0"?>
<ZopeData>
<record id="1" aka="AAAAAAAAAAE=">
<pickle>
<global name="SQL" module="Products.ZSQLMethods.SQL"/>
</pickle>
<pickle>
<dictionary>
<item>
<key> <string>_Use_Database_Methods_Permission</string> </key>
<value>
<list>
<string>Anonymous</string>
<string>Assignee</string>
<string>Assignor</string>
<string>Associate</string>
<string>Auditor</string>
<string>Author</string>
<string>Manager</string>
</list>
</value>
</item>
<item>
<key> <string>allow_simple_one_argument_traversal</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>arguments_src</string> </key>
<value> <string>base_category_uid_list\r\n
kw\r\n
search_string\r\n
is_full_text_search_on\r\n
use_text_excerpts\r\n
</string> </value>
</item>
<item>
<key> <string>cache_time_</string> </key>
<value> <int>0</int> </value>
</item>
<item>
<key> <string>class_file_</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>class_name_</string> </key>
<value> <string></string> </value>
</item>
<item>
<key> <string>connection_hook</string> </key>
<value>
<none/>
</value>
</item>
<item>
<key> <string>connection_id</string> </key>
<value> <string>erp5_sql_connection</string> </value>
</item>
<item>
<key> <string>id</string> </key>
<value> <string>WebSite_zGetAdvancedSearchResultList</string> </value>
</item>
<item>
<key> <string>max_cache_</string> </key>
<value> <int>100</int> </value>
</item>
<item>
<key> <string>max_rows_</string> </key>
<value> <int>1000</int> </value>
</item>
<item>
<key> <string>src</string> </key>
<value> <string encoding="cdata"><![CDATA[
<dtml-comment>\n
Use SQL method rather that catalog to speed up searching\n
</dtml-comment>\n
\n
<dtml-let query="buildSQLQuery(query=portal_catalog.getSecurityQuery(**kw), **kw)" \n
fix="query[\'from_table_list\'].append((\'full_text\',\'full_text\'))">\n
\n
SELECT catalog.uid, \n
catalog.path, \n
catalog.portal_type, \n
catalog.title,\n
catalog.reference, \n
catalog.modification_date,\n
catalog.owner,\n
<dtml-if is_full_text_search_on> text, </dtml-if>\n
category_uid,\n
base_category_uid,\n
category_relative_url\n
\n
FROM catalog,\n
(SELECT catalog.uid\n
<dtml-if is_full_text_search_on>\n
<dtml-if use_text_excerpts>\n
/* MySQL server can produc text excerpts */\n
, sphinx_snippets(full_text.SearchableText, \'erp5\', \'<dtml-var "search_string">\') as text\n
<dtml-else>\n
/* Return all searchable text to server which will extract found text excerpts */\n
, full_text.SearchableText as text\n
</dtml-if>\n
</dtml-if>\n
<dtml-if "query[\'select_expression\']">\n
,<dtml-var "query[\'select_expression\']">\n
</dtml-if>\n
FROM\n
<dtml-in prefix="table" expr="query[\'from_table_list\']">\n
<dtml-if sequence-end>\n
<dtml-var table_item> AS <dtml-var table_key>\n
<dtml-else>\n
<dtml-var table_item> AS <dtml-var table_key>,\n
</dtml-if>\n
</dtml-in>\n
WHERE <dtml-var "query[\'where_expression\']"> AND `catalog`.`uid` = `full_text`.`uid`\n
\n
<dtml-if "query[\'order_by_expression\']"> ORDER BY <dtml-var "query[\'order_by_expression\']"> </dtml-if>\n
\n
<dtml-if "query[\'limit_expression\']"> LIMIT <dtml-var "query[\'limit_expression\']"> \n
<dtml-else> LIMIT 1000 </dtml-if>) \n
\n
AS search_results LEFT JOIN \n
(SELECT category.uid as join_category_uid, \n
category.base_category_uid AS base_category_uid,\n
category.category_uid AS category_uid,\n
catalog.relative_url as category_relative_url\n
FROM category, catalog\n
WHERE category.category_strict_membership = 1\n
and category.base_category_uid in \n
(<dtml-in prefix="loop" expr="base_category_uid_list">\n
<dtml-if sequence-end>\n
<dtml-sqlvar expr="loop_item" type="int">\n
<dtml-else>\n
<dtml-sqlvar expr="loop_item" type="int">,\n
</dtml-if>\n
</dtml-in>)\n
and category.category_uid = catalog.uid\n
\n
) AS join_category\n
ON search_results.uid = join_category.join_category_uid\n
\n
WHERE search_results.uid = catalog.uid\n
\n
</dtml-let>
]]></string> </value>
</item>
<item>
<key> <string>title</string> </key>
<value> <string></string> </value>
</item>
</dictionary>
</pickle>
</record>
</ZopeData>
bt5/erp5_km_sphinx_full_text_search/bt/change_log 0 → 100644
View file @ 3bd8c2ba
2011-02-28 Ivan
* Initial import
\ No newline at end of file
bt5/erp5_km_sphinx_full_text_search/bt/dependency_list 0 → 100644
View file @ 3bd8c2ba
erp5_km
erp5_full_text_sphinxse_catalog
\ No newline at end of file
bt5/erp5_km_sphinx_full_text_search/bt/description 0 → 100644
View file @ 3bd8c2ba
KM full text search using sphinx.
Use "NO ZODB" approach -i.e. search results are delivered entirely by MySQL backend together with text excerpts from Sphinx search engine.
See http://www.erp5.org/HowToUseSphinxSE
\ No newline at end of file
bt5/erp5_km_sphinx_full_text_search/bt/template_skin_id_list 0 → 100644
View file @ 3bd8c2ba
erp5_km_sphinx_full_text_search
\ No newline at end of file
bt5/erp5_km_sphinx_full_text_search/bt/title 0 → 100644
View file @ 3bd8c2ba
erp5_km_sphinx_full_text_search
\ No newline at end of file
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7