stack/erp5: Add support for {activity,publisher}-timeout zope settings.

Call them deadlines, as this is what they really are, but the "wrong" name
stuck on ERP5 side (for now).
Also, introduce a per-family-override mechanism, to avoid having per-
option equivalents for such mechanism.

software/erp5/instance-erp5-input-schema.json

@@ -81,6 +81,37 @@
       "uniqueItems": true,
       "type": "array"
     },
+    "activity-deadline": {
+      "description": "How long a CMFActivity-initiated transaction may last, in seconds",
+      "default": null,
+      "type": ["number", "null"]
+    },
+    "publication-deadline": {
+      "description": "How long a publisher-initiated transaction may last, in seconds",
+      "default": null,
+      "type": ["number", "null"]
+    },
+    "family-override": {
+      "description": "Family-wide options, possibly overriding global options",
+      "default": {},
+      "patternProperties": {
+        ".*": {
+          "default": {},
+          "properties": {
+            "activity-deadline": {
+              "description": "Override global activity deadline",
+              "type": ["number", "null"]
+            },
+            "publication-deadline": {
+              "description": "Override global publication deadline",
+              "type": ["number", "null"]
+            }
+          },
+          "type": "object"
+        }
+      },
+      "type": "object"
+    },
     "hostalias-dict": {
       "description": "Hostname-to-domain-name mapping",
       "default": {},

stack/erp5/buildout.hash.cfg

@@ -34,7 +34,7 @@ md5sum = e91c0fbd0df441884f7422fa7976053c
 
 [template-zope-conf]
 filename = zope.conf.in
-md5sum = c43da8f7b4db22e40a4864e6cfcaef44
+md5sum = 2631a576ffe74c9e6a08cc0a16883765
 
 [site-zcml]
 filename = site.zcml
@@ -74,7 +74,7 @@ md5sum = b5ac16fdeed8863e465e955ba6d1e12a
 
 [template-erp5]
 filename = instance-erp5.cfg.in
-md5sum = 82b5153807b1a061a86081d490129582
+md5sum = 1d380028856c488f5ed87ca2b78ba79a
 
 [template-zeo]
 filename = instance-zeo.cfg.in
@@ -82,7 +82,7 @@ md5sum = 0648e38bd5d3a15bb9f93264932740b9
 
 [template-zope]
 filename = instance-zope.cfg.in
-md5sum = c03f93f95333e6a61b857dcfab7f9c0e
+md5sum = e31bbec1580d67ba84563b1d59c386b8
 
 [template-balancer]
 filename = instance-balancer.cfg.in

stack/erp5/instance-erp5.cfg.in

@@ -24,6 +24,7 @@
 {% set use_ipv6 = slapparameter_dict.get('use-ipv6', False) -%}
 {% set partition_thread_count_list = [] -%}
 {% set zope_partition_dict = slapparameter_dict.get('zope-partition-dict', {'1': {}}) -%}
+{% set zope_family_override_dict = slapparameter_dict.get('family-override', {}) -%}
 {% for zope_parameter_dict in zope_partition_dict.values() -%}
 {#   Apply some zope_parameter_dict default values, to avoid duplication. -#}
 {%   do zope_parameter_dict.setdefault('thread-amount', 4) -%}
@@ -225,6 +226,8 @@ config-tidstorage-port = ${request-zodb:connection-tidstorage-port}
 {% endfor -%}
 software-type = zope
 
+{% set global_publication_deadline = slapparameter_dict.get('publication-deadline') -%}
+{% set global_activity_deadline = slapparameter_dict.get('activity-deadline') -%}
 {% set zope_family_dict = {} -%}
 {% set zope_family_name_list = [] -%}
 {% set zope_backend_path_dict = {} -%}
@@ -245,6 +248,7 @@ software-type = zope
 {%   do zope_family_dict.setdefault(zope_family, []).append(section_name) -%}
 {%   do zope_backend_path_dict.__setitem__(zope_family, backend_path) -%}
 {%   do ssl_authentication_dict.__setitem__(zope_family, zope_parameter_dict.get('ssl-authentication', False)) -%}
+{%   set current_zope_family_override_dict = zope_family_override_dict.get(zope_family, {}) -%}
 [{{ section_name }}]
 <= request-zope-base
 name = {{ partition_name }}
@@ -260,6 +264,8 @@ config-longrequest-logger-timeout = {{ dumps(zope_parameter_dict.get('longreques
 config-large-file-threshold = {{ dumps(zope_parameter_dict.get('large-file-threshold', "10MB")) }}
 config-port-base = {{ dumps(zope_parameter_dict.get('port-base', 2200)) }}
 config-webdav = {{ dumps(zope_parameter_dict.get('webdav', False)) }}
+config-publication-deadline = {{ dumps(current_zope_family_override_dict.get('publication-deadline', global_publication_deadline)) }}
+config-activity-deadline = {{ dumps(current_zope_family_override_dict.get('activity-deadline', global_activity_deadline)) }}
 {%   if test_runner_enabled -%}
 config-test-runner-apache-url-list = ${publish-early:{{ zope_family }}-test-runner-url-list}
 

stack/erp5/instance-zope.cfg.in

@@ -274,6 +274,8 @@ cloudooo-url = {{ (cloudooo if cloudooo.port == None else
 {%   endif -%}
 {% endfor -%}
 developer-list = {{ dumps(slapparameter_dict['developer-list']) }}
+publication-deadline = {{ dumps(slapparameter_dict['publication-deadline']) }}
+activity-deadline = {{ dumps(slapparameter_dict['activity-deadline']) }}
 instance = ${directory:instance}
 instance-products = ${directory:instance-products}
 deadlock-path = /manage_debug_threads

stack/erp5/zope.conf.in

@@ -140,10 +140,14 @@ large-file-threshold {{ parameter_dict['large-file-threshold'] }}
 </zodb_db>
 
 {% set developer_list = parameter_dict['developer-list'] -%}
-{% if developer_list -%}
+{% set publication_deadline = parameter_dict['publication-deadline'] -%}
+{% set activity_deadline = parameter_dict['activity-deadline'] -%}
+{% if developer_list or publication_deadline or activity_deadline -%}
 %import Products.ERP5Type
 <ERP5Type erp5>
-  developers {{ developer_list | join(' ') }}
+  {% if developer_list %}developers {{ developer_list | join(' ') }}{% endif %}
+  {% if publication_deadline %}publisher-timeout {{ publication_deadline }}{% endif %}
+  {% if activity_deadline %}activity-timeout {{ activity_deadline }}{% endif %}
 </ERP5Type>
 
 {% endif -%}

[Jérome Perrin]

Did we consider enabling publisher-timeout by default ?

I'm thinking that it may be a better default value to have it enabled. There's already a timeout that users will see in haproxy, we could use the same value.

Until today, I was assuming that it was enabled by default.

[Vincent Pelletier]

I do not think enabling it by default was considered.

I think there is a problem with how these settings are layered, though (see family-override below): I would like to be able to specify different timeouts per zope families (ex: admin family would get a different setting), and the proxy timeout for a family would translate into a deadline for zopes in that family.

What about reversing the relation between haproxy timeout and publication deadline compared to what you suggest ? I believe there is currently no way of specifying the timeout, so why not set publication-deadline default value to the current hard-coded global proxy timeout, and using the deadlines (with per-family granularity) to set the proxy timeout for each zope family ? Is this possible ?

[Jérome Perrin]

This makes sense and it sounds possible, I'll try this, thanks !

[Jérome Perrin]

I have prepared some changes in 369fa364 and d25ab7bd ( !1192 (merged) )