Unhardcode ports.

Also, replace cloudooo-json by exposing its internals directly.

software/erp5/README.txt

@@ -14,6 +14,48 @@ Parameters are expected to be passed as of *.serialised recipes expect them::
 
 where `...` is a json expression (typically a dict).
 
+TCPv4 ports allocation
+----------------------
+Service listening ports are allocated in the following pattern.
+Base port of each software can be overridden, those are the default values.
+- kumofs (persistent)
+  2000: manager
+  2001: server port (?)
+  2002: server listen port (?)
+  2003: gateway port (?)
+- kumofs (volatile)
+  2010: manager
+  2011: server port (?)
+  2012: server listen port (?)
+  2013: gateway port (?)
+- cloudooo
+  2020: cloudooo
+  2021: openoffice
+- mariadb
+  2099: mariadb
+- zeo & tidstorage
+  2100: tidstorage
+  2101: first zeo
+  2102: second zeo
+  (etc)
+- haproxy
+  2150: first haproxy
+  2151: first apache
+  2152: second haproxy
+  2153: second apache
+  (etc)
+- cluster-zope
+  2200: first zope
+  2201: second zope
+  (etc)
+Note: these are not applicable when (yet unsupported) ipv6 mode is enabled, as
+stunnel (used to tunnel ipv4-only services over ipv6) needs its own listening
+ports.
+This pattern was chosen to make it possible to deploy this software release in
+a setup where all partitions would share the same IPv4 without having to
+provide many parameters.
+Some ports are unused in the overall range to allow software types to grow.
+
 common
 ------
 Parameters which are available to all software types.
@@ -24,6 +66,9 @@ MariaDB (used for catalog, activity tables and id generator) instance
 parameters.
 Defaults to {}.
 Possible keys and associated value types:
+'tcpv4-port' (int, optional)
+  TCPv4 port to listen on.
+  Defaults to 2099.
 'database-list' (list, optional)
   Define the list of databases mariadb must provide, and the user having entire
   access to each database. Each entry in the list is a dict, with these
@@ -122,6 +167,12 @@ Storage mechanism to use. To know the list of supported values, see all keys in
 instance.cfg.in's section [switch-softwaretype] which start with "zodb-".
 Defaults to 'zeo'.
 
+'zodb-tcpv4-port' (int, optional)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Base TCPv4 port for ZODB provider, if applicable (depends on chosen
+software-type).
+Defaults to 2100.
+
 'zodb-dict' (dict, optional)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Describes ZODBs to create and use in the instance. At least one entry is
@@ -216,7 +267,7 @@ value (dict)
     Start allocating ports at this value. Useful if one needs to make several
     partitions share the same port range (ie, several partitions bound to a
     single address).
-    Defaults to 2000.
+    Defaults to 2200.
 
 'haproxy-maxconn' (int, optional)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -224,6 +275,11 @@ The number of connections haproxy accepts for a given backend.
 See haproxy's "server maxconn" setting.
 Defaults to 1 (correct for single-worker-threaded zopes).
 
+'haproxy-tcpv4-port' (int, optional)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Base TCPv4 port for load-balancer (haproxy + backend apache).
+Defaults to 2150.
+
 'haproxy-server-check-path' (str, optional)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 The path haproxy accesses on a each backend to test their responsiveness.
@@ -266,26 +322,42 @@ Possible keys and associated value types:
 'company' (str, optional)
   Company name. Defaults to 'Dummy Company'.
 
-'use-ipv6' (boolean, optional)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Use IPv6 to communicate between partitions.
-Defaults to False.
-
 'mariadb-computer-guid' (str, optional)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Computer GUID identifying the partition mariadb is to be requested on.
 Defaults to "cluster" software type's partition's effective computer GUID.
 
+'cloudooo-tcpv4-port' (int, optional)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Base TCPv4 port for cloudooo.
+Defaults to 2020.
+
 'cloudooo-computer-guid' (str, optional)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Computer GUID identifying the partition cloudooo is to be requested on.
 Defaults to "cluster" software type's partition's effective computer GUID.
 
+'memcached-size' (int, optional)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Megabytes of ram to allocate for volatile memcached use.
+Defaults to 64.
+Negative/zero values cause undefined behaviour which may change in the future.
+
+'memcached-tcpv4-port'
+~~~~~~~~~~~~~~~~~~~~~~
+Base TCPv4 port for volatile memcached.
+Defaults to 2010.
+
 'memcached-computer-guid' (str, optional)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Computer GUID identifying the partition memcached is to be requested on.
 Defaults to "cluster" software type's partition's effective computer GUID.
 
+'kumofs-tcpv4-port'
+~~~~~~~~~~~~~~~~~~~
+Base TCPv4 port for persistent memcached.
+Defaults to 2000.
+
 'kumofs-computer-guid' (str, optional)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Computer GUID identifying the partition kumofs is to be requested on.
@@ -302,10 +374,10 @@ Computer GUID identifying the partition balander (haproxy, apache, some HTTP
 cache) is to be requested on.
 Defaults to "cluster" software type's partition's effective computer GUID.
 
-'cloudooo-json' (str, optional)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-XXX: what is this ?
-XXX: should not require serialising json by hand as parameter value
+'font-url-list' (list of strings, optional)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+List of extra fonts URLs to be used by cloudooo.
+Defaults to [].
 
 'bt5' (str, optional)
 ~~~~~~~~~~~~~~~~~~~~~

stack/erp5/buildout.cfg

@@ -188,7 +188,7 @@ context =
 [template-mariadb]
 < = download-base
 filename = instance-mariadb.cfg.in
-md5sum = 7c494ffbf6f0ad584a8d2604d8bcf4e3
+md5sum = 60f5e662d7301e3d56c51f934ea4a303
 
 [template-zope]
 < = download-base
@@ -198,12 +198,12 @@ md5sum = 2b68521ebde309d6453336f1a2e46381
 [template-kumofs]
 < = download-base
 filename = instance-kumofs.cfg.in
-md5sum = 8e0a34a2b6cdd3d9092ee12211d706a4
+md5sum = 2c403be4e1bc03fe98ee301877347584
 
 [template-cloudooo]
 < = download-base
 filename = instance-cloudoo.cfg.in
-md5sum = 3bb277c9181210753fa3a09901cffc61
+md5sum = 6660382993d14e1d538e737e881219ba
 
 [template-zope-conf]
 < = download-base
@@ -225,7 +225,7 @@ md5sum = 564006953b7d7a12d40a14b6648b32f0
 # XXX: "template.cfg" is hardcoded in instanciation recipe
 filename = template.cfg
 template = ${:_profile_base_location_}/instance.cfg.in
-md5sum = 80f0ded87cad5bf50c25165f6a9d326f
+md5sum = 55a4a8957ac0cb9462fadafe9a2fe378
 extra-context =
     key apache_location apache:location
     key aspell_location aspell:location
@@ -298,12 +298,12 @@ md5sum = 77ec4f095a22b5bd7dfea9dff63bade5
 [template-erp5-cluster]
 < = download-base
 filename = instance-erp5-cluster.cfg.in
-md5sum = e789d0b91dd08c3077a5b33df4878e56
+md5sum = 41a9ae2790acb79adb33985aec1abb8e
 
 [template-zeo]
 < = download-base
 filename = instance-zeo.cfg.in
-md5sum = c90fbf25252638d6d764831a7999fb83
+md5sum = 6d0bdee21ac4837f07852b98b6fcea36
 
 [template-cluster-zope]
 < = download-base
@@ -313,7 +313,7 @@ md5sum = 019f306860857e8536f26aba32101b30
 [template-balancer]
 < = download-base
 filename = instance-balancer.cfg.in
-md5sum = 896cfa69cf2529a3b64b7d783f73a447
+md5sum = 9c9741c4db7e46f3cd13c0b2f392fd5d
 
 [template-logrotate-base]
 < = template-jinja2-base

stack/erp5/instance-balancer.cfg.in

@@ -23,7 +23,7 @@ ipv4 = {{ ipv4 }}
 {% endif -%}
 {% set haproxy_dict = {} -%}
 {% set apache_dict = {} -%}
-{% set next_port = 9000 -%}
+{% set next_port = slapparameter_dict['tcpv4-port'] -%}
 {% for family_name, parameter_id_list in slapparameter_dict['zope-family-dict'].items() -%}
 {%   set zope_family_address_list = [] -%}
 {%   for parameter_id in parameter_id_list -%}

stack/erp5/instance-cloudoo.cfg.in

 {% if software_type == slap_software_type -%}
-{% set json = json_module.loads(parameter_dict.get('cloudooo-json', '{}')) -%}
 {% set bin_directory = parameter_dict['buildout-bin-directory'] -%}
 {% set use_ipv6 = slapparameter_dict.get('use-ipv6', False) -%}
 [buildout]
@@ -26,8 +25,9 @@ recipe = slapos.cookbook:generic.cloudooo
 
 # Network options
 ip = ${slap-network-information:local-ipv4}
-port = 23000
-openoffice-port = 23060
+{% set tcpv4_port = slapparameter_dict['tcpv4-port'] -%}
+port = {{ tcpv4_port }}
+openoffice-port = {{ tcpv4_port + 1 }}
 
 # Paths
 configuration-file = ${directory:etc}/cloudooo.cfg
@@ -51,7 +51,7 @@ recipe = slapos.cookbook:fontconfig
 conf-path = ${directory:etc}/font.conf
 font-system-folder = {{ parameter_dict['fonts'] }}
 font-folder = ${directory:font}
-url-list = {{ json.get('font_url_list', []) | join(' ') }}
+url-list = {{ slapparameter_dict.get('font-url-list', []) | join(' ') }}
 service-folder = ${directory:service}
 onetimedownload_path = {{ bin_directory }}/onetimedownload
 

stack/erp5/instance-erp5-cluster.cfg.in

 {% if slap_software_type == software_type -%}
 {% set has_frontend = slapparameter_dict.get('frontend-software-url', '') != '' -%}
-{% set port_base = 2000 -%}
 {% set site_id = slapparameter_dict.get('site-id', 'erp5') -%}
 [request-common]
 recipe = slapos.cookbook:request.serialised
@@ -32,8 +31,11 @@ extra-config = {{ ' '.join(mariadb_dict) }}
 [request-cloudooo]
 <=request-common
 name = Cloudooo
-extra-config = cloudooo-json
-config-cloudooo-json = {{ slapparameter_dict.get('cloudooo-json', '') }}
+extra-config =
+  font-url-list
+  tcpv4-port
+config-font-url-list = {{ dumps(slapparameter_dict.get('font-url-list', [])) }}
+tcpv4-port = {{ dumps(slapparameter_dict.get(cloudooo-tcpv4-port, 2020)) }}
 software-type = cloudooo
 sla-computer_guid = {{ slapparameter_dict.get('cloudooo-computer-guid', computer_id) }}
 
@@ -44,13 +46,18 @@ software-type = kumofs
 sla-computer_guid = {{ slapparameter_dict.get('memcached-computer-guid', computer_id) }}
 extra-config =
   ram-storage-size
-config-ram-storage-size = 64m
+  tcpv4-port
+config-ram-storage-size = {{ slapparameter_dict.get('memcached-size', 64) ~ 'm' }}
+config-tcpv4-port = {{ dumps(slapparameter_dict.get('memcached-tcpv4-port', 2010)) }}
 
 [request-kumofs]
 <=request-common
 name = KumoFS
 software-type = kumofs
 sla-computer_guid = {{ slapparameter_dict.get('kumofs-computer-guid', computer_id) }}
+extra-config =
+  tcpv4-port
+config-tcpv4-port = {{ dumps(slapparameter_dict.get('kumofs-tcpv4-port', 2000)) }}
 
 [request-zope-base]
 <=request-common
@@ -102,6 +109,8 @@ return =
 extra-config =
   zodb-dict
   tidstorage-dict
+  tcpv4-port
+config-tcpv4-port = {{ dumps(slapparameter_dict.get('zodb-tcpv4-port', 2100)) }}
 config-zodb-dict = {{ dumps(slapparameter_dict.get('zodb-dict', {'root': {}})) }}
 config-tidstorage-dict = {{ dumps(slapparameter_dict.get('tidstorage-dict')) }}
 software-type = zodb-{{ slapparameter_dict.get('zodb-software-type', 'zeo') }}
@@ -120,7 +129,7 @@ config-thread-amount = {{ dumps(zope_parameter_dict.get('thread-amount', 1)) }}
 config-timerserver-interval = {{ dumps(zope_parameter_dict.get('timerserver-interval', 5)) }}
 config-longrequest-logger-interval = {{ dumps(zope_parameter_dict.get('longrequest-logger-interval', -1)) }}
 config-longrequest-logger-timeout = {{ dumps(zope_parameter_dict.get('longrequest-logger-timeout', 1)) }}
-config-port-base = {{ dumps(zope_parameter_dict.get('port-base', port_base)) }}
+config-port-base = {{ dumps(zope_parameter_dict.get('port-base', 2200)) }}
 sla-computer_guid = {{ zope_parameter_dict.get('computer-guid', computer_id) }}
 {% endfor -%}
 
@@ -153,6 +162,7 @@ name = balancer
 software-type = balancer
 sla-computer_guid = {{ slapparameter_dict.get('balancer-computer-guid', computer_id) }}
 extra-config =
+  tcpv4-port
   haproxy-maxconn
   haproxy-server-check-path
   apache-access-control-string
@@ -164,6 +174,7 @@ extra-config =
 return =
   {{ zope_family_dict.keys() | join(' ') }}
 config-zope-family-dict = {{ dumps(zope_family_parameter_dict) }}
+config-tcpv4-port = {{ dumps(slapparameter_dict.get('', 2150)) }}
 {% for zope_section_id, name in zope_address_list_id_dict.items() -%}
 config-{{ name }} = {{ ' ${' ~ zope_section_id ~ ':connection-zope-address-list}' }}
 {% endfor -%}

stack/erp5/instance-kumofs.cfg.in

@@ -36,10 +36,11 @@ address-family = inet6
 ip = ${slap-network-information:local-ipv4}
 address-family = inet4
 {% endif -%}
-manager-port = 13101
-server-port = 13201
-server-listen-port = 13202
-gateway-port = 13301
+{% set tcpv4_port = slapparameter_dict['tcpv4-port'] -%}
+manager-port = {{ tcpv4_port }}
+server-port = {{ tcpv4_port + 1 }}
+server-listen-port = {{ tcpv4_port + 2 }}
+gateway-port = {{ tcpv4_port + 3 }}
 # Paths: Data
 {% set ram_storage_size = slapparameter_dict.get('ram-storage-size') -%}
 {% if ram_storage_size -%}

stack/erp5/instance-mariadb.cfg.in

@@ -54,7 +54,7 @@ ip = {{ (ipv6_set | list)[0] }}
 {% else -%}
 ip = {{ (ipv4_set | list)[0] }}
 {% endif -%}
-port = 45678
+port = {{ slapparameter_dict['tcpv4-port'] }}
 socket = ${directory:run}/mariadb.sock
 data-directory = ${directory:mariadb-data}
 pid-file = ${directory:run}/mariadb.pid

stack/erp5/instance-zeo.cfg.in

 {% if software_type == slap_software_type -%}
 {% set use_ipv6 = slapparameter_dict.get('use-ipv6', False) -%}
-{% set next_port = 2000 -%}
+{% set next_port = slapparameter_dict['tcpv4-port'] -%}
 {% set part_list = [] -%}
 {% set storage_dict = {} -%}
 {% set zodb_dict = {} -%}

stack/erp5/instance.cfg.in

@@ -54,7 +54,6 @@ template = {{ template_cloudooo }}
 filename = instance-cloudoo.cfg
 extra-context =
     section parameter_dict dynamic-template-cloudooo-parameters
-    import json_module json
 # Must match the key id in [switch-softwaretype] which uses this section.
     raw software_type cloudooo