Skip to content

slapos
slapos
Commit be08f156 authored by Alain Takoudjou's avatar Alain Takoudjou
Browse files
Options

Update Release Candidate

parents 121aff64 06d4db50
Hide whitespace changes
Inline Side-by-side
Showing with 1119 additions and 194 deletions
software/caddy-frontend/TODO.rst
View file @ be08f156
Generally things to be done with ``caddy-frontend``:
* tests: add assertion with results of promises in etc/promise for each partition
* generated files: ``| trim`` values (like ``slave_password[slave]`` in ``templates/template-log-access.conf.in``) in generated configuration files to have them renfered correctly
* check the whole frontend slave snippet with ``caddy -validate`` during buildout run, and reject if does not pass validation
* check that all options from ``instance-slave-caddy-input-schema.json`` are safe to be used
* ``apache-ca-certificate`` shall be merged with ``apache-certificate``
* ``apache-ca-certificate`` shall be appended to ``apache-certificate`` if not already there
......@@ -26,7 +26,6 @@ Generally things to be done with ``caddy-frontend``:
* ``apache-ca-certificate``
* ``apache-certificate`` and ``apache-key``
* change ``switch-softwaretype`` to way how ``software/erp5`` does, which will help with dropping jinja2 template for ``caddy-wrapper``, which is workaround for current situation, cf `note_62678 <https://lab.nexedi.com/nexedi/slapos/merge_requests/312#note_62678>`_
* use `slapos!326 <https://lab.nexedi.com/nexedi/slapos/merge_requests/326>`_, and especially `note about complex restart scenarios <https://lab.nexedi.com/nexedi/slapos/merge_requests/326#note_60198>`_, instead of self-developed graceful restart scripts
* move out ``test/utils.py`` and use it from shared python distribution
* provide various tricks for older browsers::
......@@ -61,7 +60,7 @@ Generally things to be done with ``caddy-frontend``:
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
* reduce the time of configuration validation (in ``instance-apache-frontend.cfg`` sections ``[configtest]``, ``[caddy-configuration]``, ``[nginx-configuration]``), as it is not scalable on frontend with 2000+ slaves (takes few minutes instead of few, < 5, seconds), issue posted `upstream <https://github.com/mholt/caddy/issues/2220>`_
* reduce the time of configuration validation (in ``instance-apache-frontend.cfg.in`` sections ``[configtest]``, ``[caddy-configuration]``, ``[nginx-configuration]``), as it is not scalable on frontend with 2000+ slaves (takes few minutes instead of few, < 5, seconds), issue posted `upstream <https://github.com/mholt/caddy/issues/2220>`_
* drop ``6tunnel`` and use ``bind`` in Caddy configuration, as soon as multiple binds will be possible, tracked in upstream `bind: support multiple values <https://github.com/mholt/caddy/pull/2128>`_ and `ipv6: does not bind on ipv4 and ipv6 for sites that resolve to both <https://github.com/mholt/caddy/issues/864>`_
* use caddy-frontend in `standalone style playbooks <https://lab.nexedi.com/nexedi/slapos.package/tree/master/playbook/roles/standalone-shared>`_
* ensure `QUIC <https://en.wikipedia.org/wiki/QUIC>`_ is used by caddy
......
software/caddy-frontend/buildout.hash.cfg
View file @ be08f156
......@@ -13,20 +13,24 @@
# section inheritance (< = ...) are NOT supported (but you should really
# not need these here).
[template]
filename = instance.cfg
md5sum = b73505ae80d6325a244f5094f8edc0ae
filename = instance.cfg.in
md5sum = d649e128d36cf76f870c189c53985569
[template-common]
filename = instance-common.cfg.in
md5sum = c801b7f9f11f0965677c22e6bbe9281b
[template-apache-frontend]
filename = instance-apache-frontend.cfg
md5sum = b170d0987563b481eb71cf705c3658ab
filename = instance-apache-frontend.cfg.in
md5sum = 64fb8005a62f0a3a9987de2e336b68e1
[template-apache-replicate]
filename = instance-apache-replicate.cfg.in
md5sum = 27e98547061bd81e5f84cb7dd21b683b
md5sum = 8d34141a9cd1e51462aba845c7bea85b
[template-slave-list]
filename = templates/apache-custom-slave-list.cfg.in
md5sum = fb6c93f42f232e381174a5951c3fc222
md5sum = 8f29aaf247a6b8354292c78abe7a5ad6
[template-slave-configuration]
filename = templates/custom-virtualhost.conf.in
......@@ -34,15 +38,15 @@ md5sum = 54ae95597a126ae552c3a913ddf29e5e
[template-replicate-publish-slave-information]
filename = templates/replicate-publish-slave-information.cfg.in
md5sum = 8d318af17da5631d4242c0d6d1531066
md5sum = 6a308c29b54d53cfd82ae23ba77a35dd
[template-caddy-frontend-configuration]
filename = templates/Caddyfile.in
md5sum = 6689d96fc18d9aad78d77fe87770d4da
md5sum = 7c987ad75fcce6f5b925c7696ff41971
[template-custom-slave-list]
filename = templates/apache-custom-slave-list.cfg.in
md5sum = fb6c93f42f232e381174a5951c3fc222
md5sum = 8f29aaf247a6b8354292c78abe7a5ad6
[caddy-backend-url-validator]
filename = templates/caddy-backend-url-validator.in
......@@ -54,15 +58,15 @@ md5sum = f20d6c3d2d94fb685f8d26dfca1e822b
[template-default-slave-virtualhost]
filename = templates/default-virtualhost.conf.in
md5sum = 6da56d875f5cf396f8fd0685cf1a9a7a
md5sum = 8ed87061b9e20e2ad74aae9f80d1b53d
[template-cached-slave-virtualhost]
filename = templates/cached-virtualhost.conf.in
md5sum = 0e7d8df879ec363f771740d017cb7512
md5sum = f149ab15334d7d15d8c525f02fc4d968
[template-log-access]
filename = templates/template-log-access.conf.in
md5sum = cbf492b2fd8a955b0f92eb512fe9163f
md5sum = f2a74f88c7248f199011fa9ec6182f73
[template-empty]
filename = templates/empty.in
......@@ -72,10 +76,6 @@ md5sum = c2314c3a9c3412a38d14b312d3df83c1
filename = templates/wrapper.in
md5sum = 8cde04bfd0c0e9bd56744b988275cfd8
[template-caddy-wrapper]
filename = templates/caddy-wrapper.in
md5sum = c5816275757124613920078b6bec1caf
[template-trafficserver-records-config]
filename = templates/trafficserver/records.config.jinja2
md5sum = 84baef0a49c9a65e8f2d2ffdb8c1d39c
......@@ -90,11 +90,11 @@ md5sum = fadb2fcaf0f2b4fe735617fac222f7ed
[template-nginx-eventsource-slave-virtualhost]
filename = templates/nginx-eventsource-slave.conf.in
md5sum = 69d65e461cd7cd5ef5b1ccd0098b50c8
md5sum = 176cbca2070734a185a7ae5a4d1181c5
[template-nginx-notebook-slave-virtualhost]
filename = templates/nginx-notebook-slave.conf.in
md5sum = 21a102ac2ee98f9a7f168fa0a1390068
md5sum = e018935e2cec2368991f743cab725741
[template-apache-lazy-script-call]
filename = templates/apache-lazy-script-call.sh.in
......
software/caddy-frontend/common.cfg
View file @ be08f156
......@@ -34,16 +34,71 @@ eggs +=
websockify
erp5.util
[template-common]
recipe = slapos.recipe.template:jinja2
template = ${:_profile_base_location_}/instance-common.cfg.in
rendered = ${buildout:directory}/instance-common.cfg
mode = 0644
context =
key develop_eggs_directory buildout:develop-eggs-directory
key eggs_directory buildout:eggs-directory
[template-frontend-parameter-section]
common_profile = ${template-common:rendered}
bin_directory = ${buildout:bin-directory}
sixtunnel = ${6tunnel:location}
caddy = ${caddy:output}
caddy_location = ${caddy:location}
curl = ${curl:location}
dash = ${dash:location}
dcron = ${dcron:location}
gzip = ${gzip:location}
logrotate = ${logrotate:location}
openssl = ${openssl:location}
trafficserver = ${trafficserver:location}
monitor_template = ${monitor-template:output}
template_cached_slave_virtualhost = ${template-cached-slave-virtualhost:target}
template_caddy_frontend_configuration = ${template-caddy-frontend-configuration:target}
template_caddy_graceful_script = ${template-caddy-graceful-script:target}
template_caddy_lazy_script_call = ${template-caddy-lazy-script-call:target}
template_default_slave_virtualhost = ${template-default-slave-virtualhost:target}
template_empty = ${template-empty:target}
template_log_access = ${template-log-access:target}
template_nging_configuration = ${template-nginx-configuration:output}
template_nginx_eventsource_slave_virtualhost = ${template-nginx-eventsource-slave-virtualhost:target}
template_nginx_notebook_slave_virtualhost = ${template-nginx-notebook-slave-virtualhost:target}
template_not_found_html = ${template-not-found-html:target}
template_slave_configuration = ${template-slave-configuration:target}
template_slave_list = ${template-slave-list:target}
template_trafficserver_records_config = ${template-trafficserver-records-config:location}
template_trafficserver_records_config_filename = ${template-trafficserver-records-config:filename}
template_trafficserver_records_config_location = ${template-trafficserver-records-config:location}
template_trafficserver_storage_config_filename = ${template-trafficserver-storage-config:filename}
template_trafficserver_storage_config_location = ${template-trafficserver-storage-config:location}
template_wrapper = ${template-wrapper:output}
[template]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance.cfg
output = ${buildout:directory}/template.cfg
recipe = slapos.recipe.template:jinja2
template = ${:_profile_base_location_}/instance.cfg.in
rendered = ${buildout:directory}/template.cfg
mode = 0644
context =
key common_profile template-common:rendered
key monitor2_template monitor2-template:rendered
key template_caddy_frontend template-caddy-frontend:target
key template_caddy_replicate template-caddy-replicate:target
key template_replicate_publish_slave_information template-replicate-publish-slave-information:target
key caddy_backend_url_validator caddy-backend-url-validator:output
section template_frontend_parameter_dict template-frontend-parameter-section
[template-caddy-frontend]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/instance-apache-frontend.cfg
output = ${buildout:directory}/template-caddy-frontend.cfg
recipe = slapos.recipe.build:download
url = ${:_profile_base_location_}/instance-apache-frontend.cfg.in
mode = 0644
[caddy-backend-url-validator]
......@@ -103,12 +158,6 @@ filename = template-log-access.conf.in
<=download-template
filename = empty.in
[template-caddy-wrapper]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/templates/caddy-wrapper.in
output = ${buildout:directory}/template-caddy-wrapper.cfg
mode = 0644
[template-wrapper]
recipe = slapos.recipe.template
url = ${:_profile_base_location_}/templates/wrapper.in
......@@ -116,22 +165,14 @@ output = ${buildout:directory}/template-wrapper.cfg
mode = 0644
[template-trafficserver-records-config]
recipe = hexagonit.recipe.download
ignore-existing = true
<=download-template
url = ${:_profile_base_location_}/templates/trafficserver/${:filename}
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = records.config.jinja2
download-only = true
mode = 0644
[template-trafficserver-storage-config]
recipe = hexagonit.recipe.download
ignore-existing = true
<=download-template
url = ${:_profile_base_location_}/templates/trafficserver/${:filename}
location = ${buildout:parts-directory}/${:_buildout_section_name_}
filename = storage.config.jinja2
download-only = true
mode = 0644
# NGINX Configuration
[template-nginx-configuration]
......
software/caddy-frontend/instance-apache-frontend.cfg → software/caddy-frontend/instance-apache-frontend.cfg.in
View file @ be08f156
[buildout]
extends =
{{ parameter_dict['common_profile'] }}
{{ parameter_dict['monitor_template'] }}
parts =
directory
configtest
......@@ -51,134 +55,97 @@ parts =
monitor-caddy-server-status-wrapper
monitor-verify-re6st-connectivity
extends = ${monitor-template:output}
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
# Create all needed directories
[directory]
recipe = slapos.cookbook:mkdirectory
bin = $${buildout:directory}/bin/
etc = $${buildout:directory}/etc/
srv = $${buildout:directory}/srv/
var = $${buildout:directory}/var/
template = $${buildout:directory}/template/
bin = ${buildout:directory}/bin/
etc = ${buildout:directory}/etc/
srv = ${buildout:directory}/srv/
var = ${buildout:directory}/var/
template = ${buildout:directory}/template/
backup = $${:srv}/backup
log = $${:var}/log
run = $${:var}/run
service = $${:etc}/service
etc-run = $${:etc}/run
promise = $${:etc}/promise
backup = ${:srv}/backup
log = ${:var}/log
run = ${:var}/run
service = ${:etc}/service
etc-run = ${:etc}/run
promise = ${:etc}/promise
logrotate-backup = $${:backup}/logrotate
logrotate-entries = $${:etc}/logrotate.d
logrotate-backup = ${:backup}/logrotate
logrotate-entries = ${:etc}/logrotate.d
cron-entries = $${:etc}/cron.d
crontabs = $${:etc}/crontabs
cronstamps = $${:etc}/cronstamps
ca-dir = $${:srv}/ssl
cron-entries = ${:etc}/cron.d
crontabs = ${:etc}/crontabs
cronstamps = ${:etc}/cronstamps
ca-dir = ${:srv}/ssl
varnginx = $${:var}/nginx
varnginx = ${:var}/nginx
[switch-caddy-softwaretype]
recipe = slapos.cookbook:softwaretype
single-default = $${dynamic-custom-personal-template-slave-list:rendered}
single-custom-personal = $${dynamic-custom-personal-template-slave-list:rendered}
[instance-parameter]
# Fetches parameters defined in SlapOS Master for this instance.
# Always the same.
recipe = slapos.cookbook:slapconfiguration.serialised
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
# Define default parameter(s) that will be used later, in case user didn't
# specify it
# All parameters are available through the configuration.XX syntax.
# All possible parameters should have a default.
configuration.domain = example.org
configuration.public-ipv4 =
configuration.port = 4443
configuration.plain_http_port = 8080
configuration.plain_nginx_port = 8081
configuration.nginx_port = 9443
configuration.server-admin = admin@example.com
# BBB: apache_custom_https and apache_custom_http
configuration.apache_custom_https = ""
configuration.apache_custom_http = ""
configuration.caddy_custom_https = ""
configuration.caddy_custom_http = ""
configuration.apache-key =
configuration.apache-certificate =
configuration.apache-ca-certificate =
configuration.open-port = 80 443
configuration.extra_slave_instance_list =
configuration.disk-cache-size = 8G
configuration.ram-cache-size = 1G
configuration.trafficserver-autoconf-port = 8083
configuration.trafficserver-mgmt-port = 8084
configuration.re6st-verification-url = http://[2001:67c:1254:4::1]/index.html
configuration.enable-http2-by-default = true
configuration.mpm-graceful-shutdown-timeout = 5
configuration.monitor-httpd-port = 8072
single-default = ${dynamic-custom-personal-template-slave-list:rendered}
single-custom-personal = ${dynamic-custom-personal-template-slave-list:rendered}
[frontend-configuration]
template-log-access = ${template-log-access:target}
log-access-configuration = $${directory:etc}/log-access.conf
caddy-directory = ${caddy:location}
caddy-ipv6 = $${instance-parameter:ipv6-random}
caddy-https-port = $${instance-parameter:configuration.port}
template-log-access = {{ parameter_dict['template_log_access'] }}
log-access-configuration = ${directory:etc}/log-access.conf
caddy-directory = {{ parameter_dict['caddy_location'] }}
caddy-ipv6 = {{ instance_parameter['ipv6-random'] }}
caddy-https-port = ${configuration:port}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
rendered = ${buildout:directory}/${:filename}
extra-context =
slapparameter_dict = {{ dumps(instance_parameter['configuration']) }}
slap_software_type = {{ dumps(instance_parameter['slap-software-type']) }}
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type instance-parameter:slap-software-type
key slapparameter_dict instance-parameter:configuration
raw common_profile {{ parameter_dict['common_profile'] }}
key slap_software_type :slap_software_type
key slapparameter_dict :slapparameter_dict
section directory directory
$${:extra-context}
${:extra-context}
[software-release-path]
template-empty = ${template-empty:target}
template-slave-configuration = ${template-slave-configuration:target}
template-default-slave-virtualhost = ${template-default-slave-virtualhost:target}
template-cached-slave-virtualhost = ${template-cached-slave-virtualhost:target}
caddy-location = ${caddy:location}
template-nginx-eventsource-slave-virtualhost = ${template-nginx-eventsource-slave-virtualhost:target}
template-nginx-notebook-slave-virtualhost = ${template-nginx-notebook-slave-virtualhost:target}
template-empty = {{ parameter_dict['template_empty'] }}
template-slave-configuration = {{ parameter_dict['template_slave_configuration'] }}
template-default-slave-virtualhost = {{ parameter_dict['template_default_slave_virtualhost'] }}
template-cached-slave-virtualhost = {{ parameter_dict['template_cached_slave_virtualhost'] }}
caddy-location = {{ parameter_dict['caddy_location'] }}
template-nginx-eventsource-slave-virtualhost = {{ parameter_dict['template_nginx_eventsource_slave_virtualhost'] }}
template-nginx-notebook-slave-virtualhost = {{ parameter_dict['template_nginx_notebook_slave_virtualhost'] }}
[dynamic-custom-personal-template-slave-list]
< = jinja2-template-base
template = ${template-slave-list:target}
template = {{ parameter_dict['template_slave_list'] }}
filename = custom-personal-instance-slave-list.cfg
extensions = jinja2.ext.do
slave_instance_list = {{ dumps(instance_parameter['slave-instance-list']) }}
extra_slave_instance_list = ${configuration:extra_slave_instance_list}
local_ipv4 = {{ dumps(instance_parameter['ipv4-random']) }}
local_ipv6 = {{ dumps(instance_parameter['ipv6-random']) }}
software_type = single-custom-personal
bin_directory = {{ parameter_dict['bin_directory'] }}
sixtunnel_executable = {{ parameter_dict['sixtunnel'] }}/bin/6tunnel
service_directory = ${directory:service}
extra-context =
key caddy_configuration_directory caddy-directory:slave-configuration
key nginx_configuration_directory caddy-directory:nginx-slave-configuration
key caddy_cached_configuration_directory caddy-directory:slave-with-cache-configuration
key slave_with_cache_configuration_directory caddy-directory:slave-with-cache-configuration
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key nginx_http_port instance-parameter:configuration.plain_nginx_port
key nginx_https_port instance-parameter:configuration.nginx_port
key public_ipv4 instance-parameter:configuration.public-ipv4
key slave_instance_list instance-parameter:slave-instance-list
key extra_slave_instance_list instance-parameter:configuration.extra_slave_instance_list
key http_port configuration:plain_http_port
key https_port configuration:port
key nginx_http_port configuration:plain_nginx_port
key nginx_https_port configuration:nginx_port
key public_ipv4 configuration:public-ipv4
key slave_instance_list :slave_instance_list
key extra_slave_instance_list :extra_slave_instance_list
key custom_ssl_directory caddy-directory:vh-ssl
key caddy_log_directory caddy-directory:slave-log
key local_ipv4 instance-parameter:ipv4-random
key local_ipv6 instance-parameter:ipv6-random
key local_ipv4 :local_ipv4
key local_ipv6 :local_ipv6
key global_ipv6 slap-network-information:global-ipv6
key varnginx directory:varnginx
key empty_template software-release-path:template-empty
......@@ -187,7 +154,7 @@ extra-context =
key template_cached_slave_configuration software-release-path:template-cached-slave-virtualhost
key template_eventsource_slave_configuration software-release-path:template-nginx-eventsource-slave-virtualhost
key template_notebook_slave_configuration software-release-path:template-nginx-notebook-slave-virtualhost
raw software_type single-custom-personal
key software_type :software_type
key frontend_lazy_graceful_reload frontend-caddy-lazy-graceful:rendered
section logrotate_dict logrotate
section frontend_configuration frontend-configuration
......@@ -196,41 +163,41 @@ extra-context =
key monitor_base_url monitor-instance-parameter:monitor-base-url
key promise_directory monitor-directory:promises
key report_directory monitor-directory:reports
raw bin_directory ${buildout:bin-directory}
key bin_directory :bin_directory
key login_certificate ca-frontend:cert-file
key login_key ca-frontend:key-file
key login_ca_crt ca-custom-frontend:rendered
key enable_http2_by_default instance-parameter:configuration.enable-http2-by-default
key enable_http2_by_default configuration:enable-http2-by-default
key access_log caddy-configuration:access-log
key error_log caddy-configuration:error-log
raw sixtunnel_executable ${6tunnel:location}/bin/6tunnel
raw service_directory $${directory:service}
key sixtunnel_executable :sixtunnel_executable
key service_directory directory:service
key not_found_file caddy-configuration:not-found-file
[dynamic-virtualhost-template-slave]
<= jinja2-template-base
template = ${template-slave-configuration:target}
rendered = $${directory:template}/slave-virtualhost.conf.in
template = {{ parameter_dict['template_slave_configuration'] }}
rendered = ${directory:template}/slave-virtualhost.conf.in
extensions = jinja2.ext.do
# BBB: apache_custom_https and apache_custom_http
extra-context =
key https_port instance-parameter:configuration.port
key http_port instance-parameter:configuration.plain_http_port
key apache_custom_https instance-parameter:configuration.apache_custom_https
key apache_custom_http instance-parameter:configuration.apache_custom_http
key caddy_custom_https instance-parameter:configuration.caddy_custom_https
key caddy_custom_http instance-parameter:configuration.caddy_custom_http
key https_port configuration:port
key http_port configuration:plain_http_port
key apache_custom_https configuration:apache_custom_https
key apache_custom_http configuration:apache_custom_http
key caddy_custom_https configuration:caddy_custom_https
key caddy_custom_http configuration:caddy_custom_http
# Deploy Caddy Frontend with Jinja power
[dynamic-caddy-frontend-template]
< = jinja2-template-base
template = ${template-caddy-frontend-configuration:target}
rendered = $${caddy-configuration:frontend-configuration}
template = {{ parameter_dict['template_caddy_frontend_configuration'] }}
rendered = ${caddy-configuration:frontend-configuration}
local_ipv4 = {{ dumps(instance_parameter['ipv4-random']) }}
extra-context =
key httpd_home software-release-path:caddy-location
key httpd_mod_ssl_cache_directory caddy-directory:mod-ssl
key instance_home buildout:directory
key server_admin instance-parameter:configuration.server-admin
key login_certificate ca-frontend:cert-file
key login_key ca-frontend:key-file
key login_ca_crt ca-custom-frontend:rendered
......@@ -242,150 +209,151 @@ extra-context =
key ssl_cached_port caddy-configuration:ssl-cache-through-port
key slave_with_cache_configuration_directory caddy-directory:slave-with-cache-configuration
section frontend_configuration frontend-configuration
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key local_ipv4 instance-parameter:ipv4-random
key http_port configuration:plain_http_port
key https_port configuration:port
key local_ipv4 :local_ipv4
key global_ipv6 slap-network-information:global-ipv6
key error_log caddy-configuration:error-log
key not_found_file caddy-configuration:not-found-file
key username slap-parameter:monitor-username
key password slap-parameter:monitor-password
key username monitor-instance-parameter:username
key password monitor-htpasswd:passwd
[caddy-wrapper]
< = jinja2-template-base
template = ${template-caddy-wrapper:output}
rendered = $${directory:bin}/caddy-wrapper
mode = 0700
extra-context =
raw caddy ${caddy:output}
key conf dynamic-caddy-frontend-template:rendered
key log caddy-configuration:error-log
key grace instance-parameter:configuration.mpm-graceful-shutdown-timeout
recipe = slapos.cookbook:wrapper
command-line = {{ parameter_dict['caddy'] }}
-conf ${dynamic-caddy-frontend-template:rendered}
-log ${caddy-configuration:error-log}
-http2=true
-grace {{ instance_parameter['configuration.mpm-graceful-shutdown-timeout'] }}s
-disable-http-challenge
-disable-tls-sni-challenge
wrapper-path = ${directory:bin}/caddy-wrapper
[caddy-frontend]
recipe = slapos.cookbook:wrapper
command-line = $${caddy-wrapper:rendered} -pidfile $${caddy-configuration:pid-file}
wrapper-path = $${directory:service}/frontend_caddy
command-line = ${caddy-wrapper:wrapper-path} -pidfile ${caddy-configuration:pid-file}
wrapper-path = ${directory:service}/frontend_caddy
wait-for-files =
$${ca-frontend:cert-file}
$${ca-frontend:key-file}
${ca-frontend:cert-file}
${ca-frontend:key-file}
[not-found-html]
recipe = slapos.cookbook:symbolic.link
target-directory = $${caddy-directory:document-root}
target-directory = ${caddy-directory:document-root}
link-binary =
${template-not-found-html:target}
{{ parameter_dict['template_not_found_html'] }}
[caddy-directory]
recipe = slapos.cookbook:mkdirectory
document-root = $${directory:srv}/htdocs
slave-configuration = $${directory:etc}/caddy-slave-conf.d/
slave-with-cache-configuration = $${directory:etc}/caddy-slave-with-cache-conf.d/
cache = $${directory:var}/cache
mod-ssl = $${:cache}/httpd_mod_ssl
vh-ssl = $${:slave-configuration}/ssl
slave-log = $${directory:log}/httpd
nginx-slave-configuration = $${directory:etc}/nginx-slave-conf.d/
document-root = ${directory:srv}/htdocs
slave-configuration = ${directory:etc}/caddy-slave-conf.d/
slave-with-cache-configuration = ${directory:etc}/caddy-slave-with-cache-conf.d/
cache = ${directory:var}/cache
mod-ssl = ${:cache}/httpd_mod_ssl
vh-ssl = ${:slave-configuration}/ssl
slave-log = ${directory:log}/httpd
nginx-slave-configuration = ${directory:etc}/nginx-slave-conf.d/
[caddy-configuration]
frontend-configuration = $${directory:etc}/Caddyfile
access-log = $${directory:log}/frontend-access.log
error-log = $${directory:log}/frontend-error.log
pid-file = $${directory:run}/httpd.pid
frontend-configuration-verification = $${caddy-wrapper:rendered} -validate > /dev/null
frontend-graceful-command = $${:frontend-configuration-verification}; if [ $? -eq 0 ]; then kill -USR1 $(cat $${:pid-file}); fi
not-found-file = $${caddy-directory:document-root}/notfound.html
frontend-configuration = ${directory:etc}/Caddyfile
access-log = ${directory:log}/frontend-access.log
error-log = ${directory:log}/frontend-error.log
pid-file = ${directory:run}/httpd.pid
frontend-configuration-verification = ${caddy-wrapper:wrapper-path} -validate > /dev/null
frontend-graceful-command = ${:frontend-configuration-verification}; if [ $? -eq 0 ]; then kill -USR1 $(cat ${:pid-file}); fi
not-found-file = ${caddy-directory:document-root}/notfound.html
# Communication with ATS
cache-port = $${trafficserver-variable:input-port}
cache-port = ${trafficserver-variable:input-port}
cache-through-port = 26011
ssl-cache-through-port = 26012
[configtest]
recipe = slapos.cookbook:wrapper
command-line = $${caddy-wrapper:rendered} -validate
wrapper-path = $${directory:bin}/caddy-configtest
command-line = ${caddy-wrapper:wrapper-path} -validate
wrapper-path = ${directory:bin}/caddy-configtest
[certificate-authority]
recipe = slapos.cookbook:certificate_authority
openssl-binary = ${openssl:location}/bin/openssl
ca-dir = $${directory:ca-dir}
requests-directory = $${cadirectory:requests}
wrapper = $${directory:service}/certificate_authority
ca-private = $${cadirectory:private}
ca-certs = $${cadirectory:certs}
ca-newcerts = $${cadirectory:newcerts}
ca-crl = $${cadirectory:crl}
openssl-binary = {{ parameter_dict['openssl'] }}/bin/openssl
ca-dir = ${directory:ca-dir}
requests-directory = ${cadirectory:requests}
wrapper = ${directory:service}/certificate_authority
ca-private = ${cadirectory:private}
ca-certs = ${cadirectory:certs}
ca-newcerts = ${cadirectory:newcerts}
ca-crl = ${cadirectory:crl}
[cadirectory]
recipe = slapos.cookbook:mkdirectory
requests = $${directory:ca-dir}/requests/
private = $${directory:ca-dir}/private/
certs = $${directory:ca-dir}/certs/
newcerts = $${directory:ca-dir}/newcerts/
crl = $${directory:ca-dir}/crl/
requests = ${directory:ca-dir}/requests/
private = ${directory:ca-dir}/private/
certs = ${directory:ca-dir}/certs/
newcerts = ${directory:ca-dir}/newcerts/
crl = ${directory:ca-dir}/crl/
[ca-frontend]
<= certificate-authority
recipe = slapos.cookbook:certificate_authority.request
key-file = $${cadirectory:certs}/frontend.key
cert-file = $${cadirectory:certs}/frontend.crt
executable = $${directory:service}/frontend_caddy
wrapper = $${directory:service}/frontend_caddy
key-content = $${instance-parameter:configuration.apache-key}
cert-content = $${instance-parameter:configuration.apache-certificate}
key-file = ${cadirectory:certs}/frontend.key
cert-file = ${cadirectory:certs}/frontend.crt
executable = ${directory:service}/frontend_caddy
wrapper = ${directory:service}/frontend_caddy
key-content = ${configuration:apache-key}
cert-content = ${configuration:apache-certificate}
# Put domain name
name = $${instance-parameter:configuration.domain}
name = ${configuration:domain}
[ca-custom-frontend]
< = jinja2-template-base
template = ${template-empty:target}
rendered = $${cadirectory:certs}/frontend.ca.crt
template = {{ parameter_dict['template_empty'] }}
rendered = ${cadirectory:certs}/frontend.ca.crt
apache-ca-certificate = ${configuration:apache-ca-certificate}
extra-context =
key content instance-parameter:configuration.apache-ca-certificate
key content :apache-ca-certificate
[cron]
recipe = slapos.cookbook:cron
dcrond-binary = ${dcron:location}/sbin/crond
cron-entries = $${directory:cron-entries}
crontabs = $${directory:crontabs}
cronstamps = $${directory:cronstamps}
catcher = $${cron-simplelogger:wrapper}
binary = $${directory:service}/crond
dcrond-binary = {{ parameter_dict['dcron'] }}/sbin/crond
cron-entries = ${directory:cron-entries}
crontabs = ${directory:crontabs}
cronstamps = ${directory:cronstamps}
catcher = ${cron-simplelogger:wrapper}
binary = ${directory:service}/crond
[cron-simplelogger]
recipe = slapos.cookbook:simplelogger
wrapper = $${directory:bin}/cron_simplelogger
log = $${directory:log}/cron.log
wrapper = ${directory:bin}/cron_simplelogger
log = ${directory:log}/cron.log
[cron-entry-logrotate]
<= cron
recipe = slapos.cookbook:cron.d
name = logrotate
frequency = 0 0 * * *
command = $${logrotate:wrapper}
command = ${logrotate:wrapper}
# Deploy Logrotate
[logrotate]
recipe = slapos.cookbook:logrotate
# Binaries
logrotate-binary = ${logrotate:location}/sbin/logrotate
gzip-binary = ${gzip:location}/bin/gzip
gunzip-binary = ${gzip:location}/bin/gunzip
logrotate-binary = {{ parameter_dict['logrotate'] }}/sbin/logrotate
gzip-binary = {{ parameter_dict['gzip'] }}/bin/gzip
gunzip-binary = {{ parameter_dict['gzip'] }}/bin/gunzip
# Directories
wrapper = $${directory:bin}/logrotate
conf = $${directory:etc}/logrotate.conf
logrotate-entries = $${directory:logrotate-entries}
backup = $${directory:logrotate-backup}
state-file = $${directory:srv}/logrotate.status
wrapper = ${directory:bin}/logrotate
conf = ${directory:etc}/logrotate.conf
logrotate-entries = ${directory:logrotate-entries}
backup = ${directory:logrotate-backup}
state-file = ${directory:srv}/logrotate.status
[logrotate-entry-caddy]
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = caddy
log = $${caddy-configuration:error-log} $${caddy-configuration:access-log}
log = ${caddy-configuration:error-log} ${caddy-configuration:access-log}
frequency = daily
rotatep-num = 30
post = $${frontend-caddy-lazy-graceful:rendered} &
post = ${frontend-caddy-lazy-graceful:rendered} &
sharedscripts = true
notifempty = true
create = true
......@@ -394,10 +362,10 @@ create = true
<= logrotate
recipe = slapos.cookbook:logrotate.d
name = caddy-nginx
log = $${nginx-configuration:error_log} $${nginx-configuration:access_log}
log = ${nginx-configuration:error_log} ${nginx-configuration:access_log}
frequency = daily
rotatep-num = 30
post = $${nginx-configuration:nginx-graceful-command}
post = ${nginx-configuration:nginx-graceful-command}
sharedscripts = true
notifempty = true
create = true
......@@ -407,93 +375,93 @@ create = true
#################
[trafficserver-directory]
recipe = slapos.cookbook:mkdirectory
configuration = $${directory:etc}/trafficserver
local-state = $${directory:var}/trafficserver
bin_path = ${trafficserver:location}/bin
log = $${directory:log}/trafficserver
cache-path = $${directory:srv}/ats_cache
configuration = ${directory:etc}/trafficserver
local-state = ${directory:var}/trafficserver
bin_path = {{ parameter_dict['trafficserver'] }}/bin
log = ${directory:log}/trafficserver
cache-path = ${directory:srv}/ats_cache
[trafficserver-variable]
wrapper-path = $${directory:service}/trafficserver
reload-path = $${directory:etc-run}/trafficserver-reload
local-ip = $${instance-parameter:ipv4-random}
wrapper-path = ${directory:service}/trafficserver
reload-path = ${directory:etc-run}/trafficserver-reload
local-ip = {{ instance_parameter['ipv4-random'] }}
input-port = 23432
hostname = $${instance-parameter:configuration.frontend-name}
remap = map /HTTPS/ http://$${instance-parameter:ipv4-random}:$${caddy-configuration:ssl-cache-through-port}
map / http://$${instance-parameter:ipv4-random}:$${caddy-configuration:cache-through-port}
hostname = ${configuration:frontend-name}
remap = map /HTTPS/ http://{{ instance_parameter['ipv4-random'] }}:${caddy-configuration:ssl-cache-through-port}
map / http://{{ instance_parameter['ipv4-random'] }}:${caddy-configuration:cache-through-port}
plugin-config = ${trafficserver:location}/libexec/trafficserver/rfc5861.so
cache-path = $${trafficserver-directory:cache-path}
disk-cache-size = $${instance-parameter:configuration.disk-cache-size}
autoconf-port = $${instance-parameter:configuration.trafficserver-autoconf-port}
mgmt-port = $${instance-parameter:configuration.trafficserver-mgmt-port}
ram-cache-size = $${instance-parameter:configuration.ram-cache-size}
plugin-config = {{ parameter_dict['trafficserver'] }}/libexec/trafficserver/rfc5861.so
cache-path = ${trafficserver-directory:cache-path}
disk-cache-size = ${configuration:disk-cache-size}
autoconf-port = ${configuration:trafficserver-autoconf-port}
mgmt-port = ${configuration:trafficserver-mgmt-port}
ram-cache-size = ${configuration:ram-cache-size}
[trafficserver-configuration-directory]
recipe = plone.recipe.command
command = cp -rn ${trafficserver:location}/etc/trafficserver/* $${:target}
target = $${trafficserver-directory:configuration}
command = cp -rn {{ parameter_dict['trafficserver'] }}/etc/trafficserver/* ${:target}
target = ${trafficserver-directory:configuration}
[trafficserver-launcher]
recipe = slapos.cookbook:wrapper
command-line = ${trafficserver:location}/bin/traffic_cop
wrapper-path = $${trafficserver-variable:wrapper-path}
environment = TS_ROOT=$${buildout:directory}
command-line = {{ parameter_dict['trafficserver'] }}/bin/traffic_cop
wrapper-path = ${trafficserver-variable:wrapper-path}
environment = TS_ROOT=${buildout:directory}
[trafficserver-reload]
recipe = slapos.cookbook:wrapper
command-line = ${trafficserver:location}/bin/traffic_line -x
wrapper-path = $${trafficserver-variable:reload-path}
environment = TS_ROOT=$${buildout:directory}
command-line = {{ parameter_dict['trafficserver'] }}/bin/traffic_line -x
wrapper-path = ${trafficserver-variable:reload-path}
environment = TS_ROOT=${buildout:directory}
# XXX Dedicated Jinja Section without slapparameter
[trafficserver-jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${trafficserver-directory:configuration}/$${:filename}
rendered = ${trafficserver-directory:configuration}/${:filename}
extra-context =
mode = 600
context =
section ats_directory trafficserver-directory
section ats_configuration trafficserver-variable
$${:extra-context}
${:extra-context}
[trafficserver-records-config]
< = trafficserver-jinja2-template-base
template = ${template-trafficserver-records-config:location}/${template-trafficserver-records-config:filename}
template = {{ parameter_dict['template_trafficserver_records_config_location'] }}/{{ parameter_dict['template_trafficserver_records_config_filename'] }}
filename = records.config
extra-context =
import os_module os
[trafficserver-storage-config]
< = trafficserver-jinja2-template-base
template = ${template-trafficserver-storage-config:location}/${template-trafficserver-storage-config:filename}
template = {{ parameter_dict['template_trafficserver_storage_config_location'] }}/{{ parameter_dict['template_trafficserver_storage_config_filename'] }}
filename = storage.config
[trafficserver-remap-config]
< = trafficserver-jinja2-template-base
template = ${template-empty:target}
template = {{ parameter_dict['template_empty'] }}
filename = remap.config
context =
key content trafficserver-variable:remap
[trafficserver-plugin-config]
< = trafficserver-jinja2-template-base
template = ${template-empty:target}
template = {{ parameter_dict['template_empty'] }}
filename = plugin.config
context =
key content trafficserver-variable:plugin-config
[trafficserver-promise-listen-port]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/trafficserver-port-listening
hostname = $${trafficserver-variable:local-ip}
port = $${trafficserver-variable:input-port}
path = ${directory:promise}/trafficserver-port-listening
hostname = ${trafficserver-variable:local-ip}
port = ${trafficserver-variable:input-port}
[trafficserver-line]
recipe = slapos.cookbook:wrapper
command-line = ${trafficserver:location}/bin/traffic_line
wrapper-path = $${directory:bin}/traffic_line
environment = TS_ROOT=$${buildout:directory}
command-line = {{ parameter_dict['trafficserver'] }}/bin/traffic_line
wrapper-path = ${directory:bin}/traffic_line
environment = TS_ROOT=${buildout:directory}
[trafficserver-promise-cache-availability]
recipe = collective.recipe.template
......@@ -501,12 +469,12 @@ input =
inline:#!${buildout:executable}
import subprocess
import sys
traffic_line = "$${trafficserver-line:wrapper-path}"
traffic_line = "${trafficserver-line:wrapper-path}"
result = float(subprocess.check_output([traffic_line, '-r', 'proxy.node.cache.percent_free' ]))
if result != 0: sys.exit(0)
sys.stderr.write("Cache not available, availability: %s" % result)
sys.exit(127)
output = $${directory:promise}/trafficserver-cache-availability
output = ${directory:promise}/trafficserver-cache-availability
mode = 700
### End of ATS sections
......@@ -514,16 +482,16 @@ mode = 700
### Caddy Graceful and promises
[frontend-caddy-graceful-bin]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:bin}/frontend-caddy-safe-graceful
template = {{ parameter_dict['template_wrapper'] }}
rendered = ${directory:bin}/frontend-caddy-safe-graceful
mode = 0700
extra-context =
key content caddy-configuration:frontend-graceful-command
[frontend-caddy-graceful]
< = jinja2-template-base
template = ${template-caddy-graceful-script:target}
rendered = $${directory:etc-run}/frontend-caddy-safe-graceful
template = {{ parameter_dict['template_caddy_graceful_script'] }}
rendered = ${directory:etc-run}/frontend-caddy-safe-graceful
mode = 0700
extra-context =
key directory_run directory:run
......@@ -533,179 +501,150 @@ extra-context =
[frontend-caddy-lazy-graceful]
< = jinja2-template-base
template = ${template-caddy-lazy-script-call:target}
rendered = $${directory:bin}/frontend-caddy-lazy-graceful
template = {{ parameter_dict['template_caddy_lazy_script_call'] }}
rendered = ${directory:bin}/frontend-caddy-lazy-graceful
mode = 0700
pid-file = $${directory:run}/lazy-graceful.pid
pid-file = ${directory:run}/lazy-graceful.pid
wait_time = 60
extra-context =
key pid_file :pid-file
raw wait_time 60
key wait_time :wait_time
key lazy_command caddy-configuration:frontend-graceful-command
# Promises checking configuration:
[promise-frontend-caddy-configuration]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:promise}/frontend-caddy-configuration-promise
template = {{ parameter_dict['template_wrapper'] }}
rendered = ${directory:promise}/frontend-caddy-configuration-promise
mode = 0700
extra-context =
key content caddy-configuration:frontend-configuration-verification
[promise-caddy-frontend-v4-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/caddy_frontend_ipv4_https
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.port}
path = ${directory:promise}/caddy_frontend_ipv4_https
hostname = {{ instance_parameter['ipv4-random'] }}
port = ${configuration:port}
[promise-caddy-frontend-v4-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/caddy_frontend_ipv4_http
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.plain_http_port}
path = ${directory:promise}/caddy_frontend_ipv4_http
hostname = {{ instance_parameter['ipv4-random'] }}
port = ${configuration:plain_http_port}
[promise-caddy-frontend-v6-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/caddy_frontend_ipv6_https
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.port}
path = ${directory:promise}/caddy_frontend_ipv6_https
hostname = {{ instance_parameter['ipv6-random'] }}
port = ${configuration:port}
[promise-caddy-frontend-v6-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/caddy_frontend_ipv6_http
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.plain_http_port}
path = ${directory:promise}/caddy_frontend_ipv6_http
hostname = {{ instance_parameter['ipv6-random'] }}
port = ${configuration:plain_http_port}
[promise-caddy-frontend-cached]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/caddy_cached
hostname = $${instance-parameter:ipv4-random}
port = $${caddy-configuration:cache-through-port}
path = ${directory:promise}/caddy_cached
hostname = {{ instance_parameter['ipv4-random'] }}
port = ${caddy-configuration:cache-through-port}
[promise-caddy-frontend-ssl-cached]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/caddy_ssl_cached
hostname = $${instance-parameter:ipv4-random}
port = $${caddy-configuration:ssl-cache-through-port}
path = ${directory:promise}/caddy_ssl_cached
hostname = {{ instance_parameter['ipv4-random'] }}
port = ${caddy-configuration:ssl-cache-through-port}
[promise-caddy-is-process-older-than-dependency-set]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/is-process-older-than-dependency-set $${caddy-configuration:pid-file}
wrapper-path = $${directory:promise}/caddy-frontend-is-running-actual-software-release
[slap_connection]
# Kept for backward compatibility
computer_id = $${slap-connection:computer-id}
partition_id = $${slap-connection:partition-id}
server_url = $${slap-connection:server-url}
software_release_url = $${slap-connection:software-release-url}
key_file = $${slap-connection:key-file}
cert_file = $${slap-connection:cert-file}
[slap-parameter]
# Define default parameter(s) that will be used later, in case user didn't
# specify it
# All parameters are available through the configuration.XX syntax.
# All possible parameters should have a default.
domain = example.org
public-ipv4 =
port = 4443
plain_http_port = 8080
server-admin = admin@example.com
# BBB: apache_custom_https and apache_custom_http
apache_custom_https = ""
apache_custom_http = ""
caddy_custom_https = ""
caddy_custom_http = ""
apache-key =
apache-certificate =
open-port = 80 443
extra_slave_instance_list =
frontend-name =
monitor-cors-domains =
monitor-username = $${monitor-instance-parameter:username}
monitor-password = $${monitor-htpasswd:passwd}
command-line = {{ parameter_dict['bin_directory'] }}/is-process-older-than-dependency-set ${caddy-configuration:pid-file}
wrapper-path = ${directory:promise}/caddy-frontend-is-running-actual-software-release
#######
# Monitoring sections
#
[monitor-instance-parameter]
monitor-httpd-port = $${instance-parameter:configuration.monitor-httpd-port}
cors-domains = $${slap-parameter:monitor-cors-domains}
username = $${slap-parameter:monitor-username}
password = $${slap-parameter:monitor-password}
# Note: Workaround for monitor stack, which uses monitor-httpd-port parameter
# directly, and in our case it can come from the network, thus resulting
# with need to strip !py!'u'
{% set monitor_httpd_port = instance_parameter.get('configuration.monitor-httpd-port') %}
{% if monitor_httpd_port %}
monitor-httpd-port = {{ monitor_httpd_port | int }}
{% endif -%}
[monitor-conf-parameters]
private-path-list +=
$${directory:logrotate-backup}
${directory:logrotate-backup}
[monitor-traffic-summary-last-stats-wrapper]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${monitor-directory:reports}/traffic-summary-last-stats_every_1_hour
template = {{ parameter_dict['template_wrapper'] }}
rendered = ${monitor-directory:reports}/traffic-summary-last-stats_every_1_hour
mode = 0700
command = export TS_ROOT=$${buildout:directory} && echo "<pre>$(${trafficserver:location}/bin/traffic_logstats -f $${trafficserver-directory:log}/squid.blog)</pre>"
command = export TS_ROOT=${buildout:directory} && echo "<pre>$({{ parameter_dict['trafficserver'] }}/bin/traffic_logstats -f ${trafficserver-directory:log}/squid.blog)</pre>"
extra-context =
key content monitor-traffic-summary-last-stats-wrapper:command
# Produce ATS Cache stats
[monitor-ats-cache-stats-wrapper]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${monitor-directory:reports}/ats-cache-stats_every_1_hour
template = {{ parameter_dict['template_wrapper'] }}
rendered = ${monitor-directory:reports}/ats-cache-stats_every_1_hour
mode = 0700
command = export TS_ROOT=$${buildout:directory} && echo "<pre>$(${trafficserver:location}/bin/traffic_shell $${monitor-ats-cache-stats-config:rendered})</pre>"
command = export TS_ROOT=${buildout:directory} && echo "<pre>$({{ parameter_dict['trafficserver'] }}/bin/traffic_shell ${monitor-ats-cache-stats-config:rendered})</pre>"
extra-context =
key content monitor-ats-cache-stats-wrapper:command
[monitor-caddy-server-status-wrapper]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${monitor-directory:reports}/monitor-caddy-server-status-wrapper
template = {{ parameter_dict['template_wrapper'] }}
rendered = ${monitor-directory:reports}/monitor-caddy-server-status-wrapper
mode = 0700
command = ${curl:location}/bin/curl -s http://$${instance-parameter:ipv4-random}:$${instance-parameter:configuration.plain_http_port}/server-status -u $${monitor-instance-parameter:username}:$${monitor-htpasswd:passwd} 2>&1
command = {{ parameter_dict['curl'] }}/bin/curl -s http://{{ instance_parameter['ipv4-random'] }}:${configuration:plain_http_port}/server-status -u ${monitor-instance-parameter:username}:${monitor-htpasswd:passwd} 2>&1
extra-context =
key content monitor-caddy-server-status-wrapper:command
[monitor-ats-cache-stats-config]
< = jinja2-template-base
template = ${template-empty:target}
rendered = $${trafficserver-configuration-directory:target}/cache-config.stats
template = {{ parameter_dict['template_empty'] }}
rendered = ${trafficserver-configuration-directory:target}/cache-config.stats
mode = 644
context =
raw content show:cache-stats
[monitor-verify-re6st-connectivity]
recipe = slapos.cookbook:check_url_available
path = $${directory:promise}/re6st-connectivity
url = $${instance-parameter:configuration.re6st-verification-url}
dash_path = ${dash:location}/bin/dash
curl_path = ${curl:location}/bin/curl
path = ${directory:promise}/re6st-connectivity
url = ${configuration:re6st-verification-url}
dash_path = {{ parameter_dict['dash'] }}/bin/dash
curl_path = {{ parameter_dict['curl'] }}/bin/curl
#######################
# Nginx
#
[nginx-wrapper]
< = jinja2-template-base
template = ${template-caddy-wrapper:output}
rendered = $${directory:bin}/nginx-wrapper
mode = 0700
extra-context =
raw caddy ${caddy:output}
key conf dynamic-nginx-frontend-template:rendered
key log nginx-configuration:error_log
key grace instance-parameter:configuration.mpm-graceful-shutdown-timeout
recipe = slapos.cookbook:wrapper
command-line = {{ parameter_dict['caddy'] }}
-conf ${dynamic-nginx-frontend-template:rendered}
-log ${nginx-configuration:error_log}
-http2=true
-grace {{ instance_parameter['configuration.mpm-graceful-shutdown-timeout'] }}s
-disable-http-challenge
-disable-tls-sni-challenge
wrapper-path = ${directory:bin}/nginx-wrapper
[nginx-frontend]
recipe = slapos.cookbook:wrapper
command-line = $${nginx-wrapper:rendered} -pidfile $${nginx-configuration:pid-file}
wrapper-path = $${directory:service}/frontend_nginx
command-line = ${nginx-wrapper:wrapper-path} -pidfile ${nginx-configuration:pid-file}
wrapper-path = ${directory:service}/frontend_nginx
[dynamic-nginx-frontend-template]
< = jinja2-template-base
template = ${template-nginx-configuration:output}
rendered = $${directory:etc}/nginx.cfg
template = {{ parameter_dict['template_nging_configuration'] }}
rendered = ${directory:etc}/nginx.cfg
mode = 0600
extra-context =
key port nginx-configuration:port
......@@ -719,72 +658,79 @@ extra-context =
key not_found_file caddy-configuration:not-found-file
[nginx-configuration]
access_log = $${directory:log}/nginx-access.log
error_log = $${directory:log}/nginx-error.log
ip = $${slap-network-information:global-ipv6}
local_ip = $${slap-network-information:local-ipv4}
port = $${instance-parameter:configuration.nginx_port}
plain_port = $${instance-parameter:configuration.plain_nginx_port}
access_log = ${directory:log}/nginx-access.log
error_log = ${directory:log}/nginx-error.log
ip = ${slap-network-information:global-ipv6}
local_ip = ${slap-network-information:local-ipv4}
port = ${configuration:nginx_port}
plain_port = ${configuration:plain_nginx_port}
worker_processes = 4
worker_connections = 1024
slave-configuration-directory = $${caddy-directory:nginx-slave-configuration}
pid-file = $${directory:run}/nginx.pid
nginx-graceful-command = $${:nginx-configuration-verification}; if [ $? -eq 0 ]; then kill -HUP $(cat $${:pid-file}); fi
nginx-configuration-verification = $${nginx-wrapper:rendered} -validate
ssl_certificate = $${ca-frontend:cert-file}
ssl_key = $${ca-frontend:key-file}
slave-configuration-directory = ${caddy-directory:nginx-slave-configuration}
pid-file = ${directory:run}/nginx.pid
nginx-graceful-command = ${:nginx-configuration-verification}; if [ $? -eq 0 ]; then kill -HUP $(cat ${:pid-file}); fi
nginx-configuration-verification = ${nginx-wrapper:wrapper-path} -validate
ssl_certificate = ${ca-frontend:cert-file}
ssl_key = ${ca-frontend:key-file}
[frontend-nginx-graceful]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:etc-run}/frontend-nginx-safe-graceful
template = {{ parameter_dict['template_wrapper'] }}
rendered = ${directory:etc-run}/frontend-nginx-safe-graceful
mode = 0700
extra-context =
key content nginx-configuration:nginx-graceful-command
[promise-nginx-configuration]
< = jinja2-template-base
template = ${template-wrapper:output}
rendered = $${directory:promise}/nginx-configuration-promise
template = {{ parameter_dict['template_wrapper'] }}
rendered = ${directory:promise}/nginx-configuration-promise
mode = 0700
extra-context =
key content nginx-configuration:nginx-configuration-verification
[promise-nginx-frontend-v4-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv4_https
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.nginx_port}
path = ${directory:promise}/nginx_frontend_ipv4_https
hostname = {{ instance_parameter['ipv4-random'] }}
port = ${configuration:nginx_port}
[promise-nginx-frontend-v4-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv4_http
hostname = $${instance-parameter:ipv4-random}
port = $${instance-parameter:configuration.plain_nginx_port}
path = ${directory:promise}/nginx_frontend_ipv4_http
hostname = {{ instance_parameter['ipv4-random'] }}
port = ${configuration:plain_nginx_port}
[promise-nginx-frontend-v6-https]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv6_https
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.nginx_port}
path = ${directory:promise}/nginx_frontend_ipv6_https
hostname = {{ instance_parameter['ipv6-random'] }}
port = ${configuration:nginx_port}
[promise-nginx-frontend-v6-http]
recipe = slapos.cookbook:check_port_listening
path = $${directory:promise}/nginx_frontend_ipv6_http
hostname = $${instance-parameter:ipv6-random}
port = $${instance-parameter:configuration.plain_nginx_port}
path = ${directory:promise}/nginx_frontend_ipv6_http
hostname = {{ instance_parameter['ipv6-random'] }}
port = ${configuration:plain_nginx_port}
[promise-nginx-is-process-older-than-dependency-set]
recipe = slapos.cookbook:wrapper
command-line = ${buildout:bin-directory}/is-process-older-than-dependency-set $${nginx-configuration:pid-file}
wrapper-path = $${directory:promise}/promise-nginx-is-process-older-than-dependency-set
command-line = {{ parameter_dict['bin_directory'] }}/is-process-older-than-dependency-set ${nginx-configuration:pid-file}
wrapper-path = ${directory:promise}/promise-nginx-is-process-older-than-dependency-set
[port-redirection]
<= jinja2-template-base
template = inline:
[{"srcPort": 80, "destPort": {{ http_port }}}, {"srcPort": 443, "destPort": {{ https_port }}}]
rendered = $${buildout:directory}/.slapos-port-redirect
[{"srcPort": 80, "destPort": {{ '{{' }} http_port {{ '}}' }}}, {"srcPort": 443, "destPort": {{ '{{' }} https_port {{ '}}' }}}]
rendered = ${buildout:directory}/.slapos-port-redirect
mode = 0644
extra-context =
key http_port instance-parameter:configuration.plain_http_port
key https_port instance-parameter:configuration.port
key http_port configuration:plain_http_port
key https_port configuration:port
[configuration]
{%- for key, value in instance_parameter.iteritems() -%}
{%- if key.startswith('configuration.') %}
{{ key.replace('configuration.', '') }} = {{ dumps(value) }}
{%- endif -%}
{%- endfor -%}
software/caddy-frontend/instance-apache-replicate.cfg.in
View file @ be08f156
......@@ -6,10 +6,7 @@ rendered = ${buildout:directory}/${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-parameter:slap_software_type
key slave_instance_list slap-parameter:slave_instance_list
raw common_profile {{ common_profile }}
${:extra-context}
{% set part_list = [] %}
......@@ -173,7 +170,9 @@ monitor-url-list +=
{% endfor %}
[buildout]
extends = {{ template_monitor }}
extends =
{{ common_profile }}
{{ template_monitor }}
parts =
monitor-base
publish-slave-information
......@@ -182,23 +181,4 @@ parts =
{{ ' %s' % part }}
{% endfor %}
# publish-information
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[slap_connection]
# Kept for backward compatibility
computer_id = ${slap-connection:computer-id}
partition_id = ${slap-connection:partition-id}
server_url = ${slap-connection:server-url}
software_release_url = ${slap-connection:software-release-url}
key_file = ${slap-connection:key-file}
cert_file = ${slap-connection:cert-file}
[slap-parameter]
slave_instance_list =
-frontend-quantity = 1
-frontend-type = single-default
{% endif %}
software/caddy-frontend/instance-common.cfg.in 0 → 100644
View file @ be08f156
[buildout]
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
[slap_connection]
# Kept for backward compatibility
computer_id = ${slap-connection:computer-id}
partition_id = ${slap-connection:partition-id}
server_url = ${slap-connection:server-url}
software_release_url = ${slap-connection:software-release-url}
key_file = ${slap-connection:key-file}
cert_file = ${slap-connection:cert-file}
\ No newline at end of file
software/caddy-frontend/instance.cfg deleted 100644 → 0
View file @ 121aff64
[buildout]
parts =
dynamic-template-caddy-replicate
switch-softwaretype
eggs-directory = ${buildout:eggs-directory}
develop-eggs-directory = ${buildout:develop-eggs-directory}
offline = true
[slap-parameters]
recipe = slapos.cookbook:slapconfiguration
computer = $${slap-connection:computer-id}
partition = $${slap-connection:partition-id}
url = $${slap-connection:server-url}
key = $${slap-connection:key-file}
cert = $${slap-connection:cert-file}
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = $${buildout:directory}/$${:filename}
extra-context =
context =
import json_module json
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
key slap_software_type slap-parameters:slap-software-type
key slapparameter_dict slap-parameters:configuration
key slave_instance_list slap-parameters:slave-instance-list
$${:extra-context}
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = $${dynamic-template-caddy-replicate:rendered}
RootSoftwareInstance = $${dynamic-template-caddy-replicate:rendered}
custom-personal = $${dynamic-template-caddy-replicate:rendered}
single-default = ${template-caddy-frontend:output}
single-custom-personal = ${template-caddy-frontend:output}
replicate = $${dynamic-template-caddy-replicate:rendered}
[dynamic-template-caddy-replicate]
< = jinja2-template-base
template = ${template-caddy-replicate:target}
filename = instance-caddy-replicate.cfg
extensions = jinja2.ext.do
extra-context =
import subprocess_module subprocess
raw caddy_backend_url_validator ${caddy-backend-url-validator:output}
raw template_publish_slave_information ${template-replicate-publish-slave-information:target}
# Must match the key id in [switch-softwaretype] which uses this section.
raw software_type RootSoftwareInstance-default-custom-personal-replicate
raw template_monitor ${monitor2-template:rendered}
software/caddy-frontend/instance.cfg.in 0 → 100644
View file @ be08f156
[buildout]
extends = {{ common_profile }}
parts =
dynamic-template-caddy-replicate
switch-softwaretype
[jinja2-template-base]
recipe = slapos.recipe.template:jinja2
rendered = ${buildout:directory}/${:filename}
extra-context =
context =
import json_module json
key slap_software_type instance-parameter:slap-software-type
key slapparameter_dict instance-parameter:configuration
key slave_instance_list instance-parameter:slave-instance-list
section instance_parameter instance-parameter
${:extra-context}
[switch-softwaretype]
recipe = slapos.cookbook:softwaretype
default = ${dynamic-template-caddy-replicate:rendered}
RootSoftwareInstance = ${dynamic-template-caddy-replicate:rendered}
custom-personal = ${dynamic-template-caddy-replicate:rendered}
single-default = ${dynamic-template-caddy-frontend:rendered}
single-custom-personal = ${dynamic-template-caddy-frontend:rendered}
replicate = ${dynamic-template-caddy-replicate:rendered}
[dynamic-template-caddy-frontend-parameters]
{% for key,value in template_frontend_parameter_dict.iteritems() %}
{{ key }} = {{ dumps(value) }}
{% endfor -%}
[dynamic-template-caddy-frontend]
< = jinja2-template-base
template = {{ template_caddy_frontend }}
filename = instance-caddy-frontend.cfg
extensions = jinja2.ext.do
extra-context =
section parameter_dict dynamic-template-caddy-frontend-parameters
[dynamic-template-caddy-replicate]
< = jinja2-template-base
template = {{ template_caddy_replicate }}
filename = instance-caddy-replicate.cfg
extensions = jinja2.ext.do
extra-context =
import subprocess_module subprocess
raw caddy_backend_url_validator {{ caddy_backend_url_validator }}
raw template_publish_slave_information {{ template_replicate_publish_slave_information }}
# Must match the key id in [switch-softwaretype] which uses this section.
raw software_type RootSoftwareInstance-default-custom-personal-replicate
raw template_monitor {{ monitor2_template }}
raw common_profile {{ common_profile }}
[instance-parameter]
# Fetches parameters defined in SlapOS Master for this instance.
# Always the same.
recipe = slapos.cookbook:slapconfiguration.serialised
computer = ${slap-connection:computer-id}
partition = ${slap-connection:partition-id}
url = ${slap-connection:server-url}
key = ${slap-connection:key-file}
cert = ${slap-connection:cert-file}
# Define default parameter(s) that will be used later, in case user didn't
# specify it
# All parameters are available through the configuration.XX syntax.
# All possible parameters should have a default.
configuration.domain = example.org
configuration.public-ipv4 =
configuration.port = 4443
configuration.plain_http_port = 8080
configuration.plain_nginx_port = 8081
configuration.nginx_port = 9443
# BBB: apache_custom_https and apache_custom_http
configuration.apache_custom_https = ""
configuration.apache_custom_http = ""
configuration.caddy_custom_https = ""
configuration.caddy_custom_http = ""
configuration.apache-key =
configuration.apache-certificate =
configuration.apache-ca-certificate =
configuration.open-port = 80 443
configuration.extra_slave_instance_list =
configuration.disk-cache-size = 8G
configuration.ram-cache-size = 1G
configuration.trafficserver-autoconf-port = 8083
configuration.trafficserver-mgmt-port = 8084
configuration.re6st-verification-url = http://[2001:67c:1254:4::1]/index.html
configuration.enable-http2-by-default = true
configuration.mpm-graceful-shutdown-timeout = 5
configuration.monitor-httpd-port = 8072
configuration.frontend-name =
\ No newline at end of file
software/caddy-frontend/templates/Caddyfile.in
View file @ be08f156
......@@ -34,7 +34,7 @@ https://[{{ global_ipv6 }}]:{{ https_port }}/server-status, https://{{ local_ipv
# Compress the output
gzip
bind {{ local_ipv4 }}
basicauth "{{ username }}" {{ password }} {
basicauth "{{ username }}" {{ password | trim }} {
"Server Status"
/
}
......
software/caddy-frontend/templates/apache-custom-slave-list.cfg.in
View file @ be08f156
......@@ -20,8 +20,7 @@ recipe = slapos.recipe.template:jinja2
extensions = jinja2.ext.do
extra-context =
context =
key eggs_directory buildout:eggs-directory
key develop_eggs_directory buildout:develop-eggs-directory
raw common_profile {{ common_profile }}
${:extra-context}
{% do logrotate_dict.pop('recipe') %}
......@@ -194,6 +193,13 @@ cert-content = {{ dumps(slave_instance.get('ssl_crt')) }}
{# ########################################## #}
{# Set Slave Configuration #}
[{{ slave_configuration_section_name }}]
https_port = {{ dumps(https_port) }}
http_port = {{ dumps(http_port) }}
local_ipv4 = {{ dumps(local_ipv4) }}
nginx_http_port = {{ dumps(nginx_http_port) }}
nginx_https_port = {{ dumps(nginx_https_port) }}
cached_port = {{ dumps(cached_port) }}
ssl_cached_port = {{ (ssl_cached_port) }}
{# BBB: apache_custom_https and apache_custom_http #}
{% set caddy_custom_http = ((slave_instance.pop('caddy_custom_http', slave_instance.pop('apache_custom_http', ''))) % slave_parameter_dict) %}
{% set caddy_custom_https = ((slave_instance.pop('caddy_custom_https', slave_instance.pop('apache_custom_https', ''))) % slave_parameter_dict) %}
......@@ -225,11 +231,6 @@ template = {{ template_default_slave_configuration }}
filename = {{ '%s.conf' % slave_reference }}
extra-context =
raw https_port {{ https_port }}
raw http_port {{ http_port }}
raw local_ipv4 {{ local_ipv4 }}
raw nginx_http_port {{ nginx_http_port }}
raw nginx_https_port {{ nginx_https_port }}
section slave_parameter {{ slave_configuration_section_name }}
{{ '\n' }}
......@@ -316,9 +317,6 @@ rendered = {{ caddy_cached_configuration_directory }}/${:filename}
extensions = jinja2.ext.do
extra-context =
section slave_parameter {{ slave_configuration_section_name }}
raw cached_port {{ cached_port }}
raw ssl_cached_port {{ ssl_cached_port }}
raw local_ipv4 {{ local_ipv4 }}
{{ '\n' }}
{% endfor %}
......@@ -365,6 +363,19 @@ ipv4-port = {{ nginx_https_port }}
ipv6-port = {{ nginx_https_port }}
{# Define log access #}
[caddy-log-access-parameters]
caddy_log_directory = {{ dumps(caddy_log_directory) }}
caddy_configuration_directory = {{ dumps(caddy_configuration_directory) }}
local_ipv4 = {{ dumps(local_ipv4) }}
global_ipv6 = {{ dumps(global_ipv6) }}
https_port = {{ dumps(https_port) }}
http_port = {{ dumps(http_port) }}
login_certificate = {{ dumps(login_certificate) }}
login_key = {{ dumps(login_key) }}
access_log = {{ dumps(access_log) }}
error_log = {{ dumps(error_log) }}
not_found_file = {{ dumps(not_found_file) }}
[caddy-log-access]
< = jinja2-template-base
template = {{frontend_configuration.get('template-log-access')}}
......@@ -372,17 +383,7 @@ rendered = {{frontend_configuration.get('log-access-configuration')}}
extra-context =
section slave_log_directory slave-log-directory-dict
section slave_password slave-password
raw caddy_log_directory {{caddy_log_directory}}
raw caddy_configuration_directory {{caddy_configuration_directory}}
raw local_ipv4 {{ local_ipv4 }}
raw global_ipv6 {{ global_ipv6 }}
raw https_port {{ https_port }}
raw http_port {{ http_port }}
raw login_certificate {{ login_certificate }}
raw login_key {{ login_key }}
raw access_log {{ access_log }}
raw error_log {{ error_log }}
raw not_found_file {{ not_found_file }}
section parameter_dict caddy-log-access-parameters
{# Publish information for the instance #}
[publish-caddy-information]
......@@ -395,6 +396,7 @@ slave-instance-information-list = {{ json_module.dumps(slave_instance_informatio
monitor-base-url = {{ monitor_base_url }}
[buildout]
extends = {{ common_profile }}
parts +=
slave-log-directories
{% for part in part_list %}
......@@ -409,9 +411,6 @@ parts +=
tunnel-6to4-base-nginx_http_port
tunnel-6to4-base-nginx_https_port
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
cache-access = {{ cache_access }}
{% endif %}
software/caddy-frontend/templates/cached-virtualhost.conf.in
View file @ be08f156
......@@ -5,13 +5,13 @@
{%- set http_backend_host_list = [] %}
{%- set https_backend_host_list = [] %}
{%- for host in host_list %}
{%- do http_backend_host_list.append('http://%s:%s' % (host, cached_port)) %}
{%- do https_backend_host_list.append('http://%s:%s' % (host, ssl_cached_port)) %}
{%- do http_backend_host_list.append('http://%s:%s' % (host, slave_parameter['cached_port'])) %}
{%- do https_backend_host_list.append('http://%s:%s' % (host, slave_parameter['ssl_cached_port'])) %}
{%- endfor %}
# SSL-disabled backends
{{ http_backend_host_list|join(', ') }} {
bind {{ local_ipv4 }}
bind {{ slave_parameter['local_ipv4'] }}
# Compress the output
gzip
{%- if ssl_proxy_verify and 'ssl_proxy_ca_crt' in slave_parameter %}
......@@ -35,7 +35,7 @@
# SSL-enabled backends
{{ https_backend_host_list|join(', ') }} {
bind {{ local_ipv4 }}
bind {{ slave_parameter['local_ipv4'] }}
# Compress the output
gzip
{%- if ssl_proxy_verify and 'ssl_proxy_ca_crt' in slave_parameter %}
......
software/caddy-frontend/templates/caddy-wrapper.in deleted 100644 → 0
View file @ 121aff64
#!${dash-output:dash}
exec {{ caddy }} \
-conf {{ conf }} \
-log {{ log }} \
-http2=true \
-grace {{ grace }}s \
-disable-http-challenge \
-disable-tls-sni-challenge \
"$@"
software/caddy-frontend/templates/default-virtualhost.conf.in
View file @ be08f156
......@@ -17,13 +17,13 @@
{%- set http_host_list = [] %}
{%- set https_host_list = [] %}
{%- for host in host_list %}
{%- do http_host_list.append('http://%s:%s' % (host, http_port)) %}
{%- do https_host_list.append('https://%s:%s' % (host, https_port)) %}
{%- do http_host_list.append('http://%s:%s' % (host, slave_parameter['http_port'] )) %}
{%- do https_host_list.append('https://%s:%s' % (host, slave_parameter['https_port'] )) %}
{%- endfor %} {#- for host in host_list #}
# SSL enabled hosts
{{ https_host_list|join(', ') }} {
bind {{ local_ipv4 }}
bind {{ slave_parameter['local_ipv4'] }}
# Compress the output
gzip
{%- if ssl_proxy_verify and 'ssl_proxy_ca_crt' in slave_parameter %}
......@@ -144,7 +144,7 @@
# SSL-disabled hosts
{{ http_host_list|join(', ') }} {
bind {{ local_ipv4 }}
bind {{ slave_parameter['local_ipv4'] }}
# Compress the output
gzip
{%- if ssl_proxy_verify and 'ssl_proxy_ca_crt' in slave_parameter %}
......
software/caddy-frontend/templates/nginx-eventsource-slave.conf.in
View file @ be08f156
......@@ -21,7 +21,7 @@
# TODO-Caddy server {
# TODO-Caddy listen {{ local_ipv4 }}:{{ nginx_http_port }};
# TODO-Caddy listen {{ slave_parameter['local_ipv4'] }}:{{ slave_parameter['nginx_http_port'] }};
# TODO-Caddy
# TODO-Caddy server_name {{ slave_parameter.get('custom_domain') }};
# TODO-Caddy
......@@ -60,7 +60,7 @@
# TODO-Caddy
# TODO-Caddy server {
# TODO-Caddy listen {{ local_ipv4 }}:{{ nginx_https_port }} ssl;
# TODO-Caddy listen {{ slave_parameter['local_ipv4'] }}:{{ slave_parameter['nginx_https_port'] }} ssl;
# TODO-Caddy
# TODO-Caddy server_name {{ slave_parameter.get('custom_domain') }};
# TODO-Caddy
......
software/caddy-frontend/templates/nginx-notebook-slave.conf.in
View file @ be08f156
......@@ -5,8 +5,8 @@
{%- set https_upstream = https_url.split("/")[2] %}
# SSL-enabled
https://{{ slave_parameter.get('custom_domain') }}:{{ nginx_https_port }} {
bind {{ local_ipv4 }}
https://{{ slave_parameter.get('custom_domain') }}:{{ slave_parameter['nginx_https_port'] }} {
bind {{ slave_parameter['local_ipv4'] }}
# Compress the output
gzip
log / {{ slave_parameter.get('access_log') }} "{remote} {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}"
......@@ -37,8 +37,8 @@ https://{{ slave_parameter.get('custom_domain') }}:{{ nginx_https_port }} {
}
# SSL-disabled
http://{{ slave_parameter.get('custom_domain') }}:{{ nginx_http_port }} {
bind {{ local_ipv4 }}
http://{{ slave_parameter.get('custom_domain') }}:{{ slave_parameter['nginx_http_port'] }} {
bind {{ slave_parameter['local_ipv4'] }}
# Compress the output
gzip
log / {{ slave_parameter.get('access_log') }} "{remote} {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}"
......
software/caddy-frontend/templates/replicate-publish-slave-information.cfg.in
View file @ be08f156
......@@ -41,11 +41,8 @@ log-access-url = {{ json_module.dumps(slave_information.pop('log-access-urls', 1
{% endfor %}
[buildout]
extends = {{ common_profile }}
parts =
{% for part in part_list %}
{{ ' %s' % part }}
{% endfor %}
eggs-directory = {{ eggs_directory }}
develop-eggs-directory = {{ develop_eggs_directory }}
offline = true
\ No newline at end of file
{% endfor %}
\ No newline at end of file
software/caddy-frontend/templates/template-log-access.conf.in
View file @ be08f156
{% for slave, directory in slave_log_directory.iteritems() %}
https://[{{ global_ipv6 }}]:{{ https_port }}/{{ slave }}, https://{{ local_ipv4 }}:{{ https_port }}/{{ slave }} {
bind {{ local_ipv4 }}
root {{directory}}/
https://[{{ parameter_dict['global_ipv6'] }}]:{{ parameter_dict['https_port'] }}/{{ slave }}, https://{{ parameter_dict['local_ipv4'] }}:{{ parameter_dict['https_port'] }}/{{ slave }} {
bind {{ parameter_dict['local_ipv4'] }}
root {{ directory }}/
browse
tls {{ login_certificate }} {{ login_key }}
basicauth "{{ slave }}" {{ slave_password[slave] }} {
tls {{ parameter_dict['login_certificate'] }} {{ parameter_dict['login_key'] }}
basicauth "{{ slave }}" {{ slave_password[slave] | trim }} {
"Log Access {{ slave }}"
/
}
log / {{ access_log }} "{remote} {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}"
errors {{ error_log }} {
* {{ not_found_file }}
log / {{ parameter_dict['access_log'] }} "{remote} {>REMOTE_USER} [{when}] \"{method} {uri} {proto}\" {status} {size} \"{>Referer}\" \"{>User-Agent}\" {latency_ms}"
errors {{ parameter_dict['error_log'] }} {
* {{ parameter_dict['not_found_file'] }}
}
}
{% endfor %}
software/caddy-frontend/test/test.py
View file @ be08f156
......@@ -758,6 +758,17 @@ class TestSlave(SlaveHttpFrontendTestCase, TestDataMixin):
os.path.join(
partition_path, 'etc', 'httpd-cors.cfg'), 'r').read().strip())
def test_promise_monitor_httpd_listening_on_tcp(self):
result = set([
subprocess.call(q) for q in glob.glob(
os.path.join(
self.instance_path, '*', 'etc', 'promise',
'monitor-httpd-listening-on-tcp'))])
self.assertEqual(
result,
set([0])
)
@skipIf(not IS_CADDY, 'Will NOT be covered on apache-frontend')
def test_slave_partition_state(self):
partition_path = self.getSlavePartitionPath()
......@@ -2780,3 +2791,42 @@ class TestMalformedBackenUrlSlave(SlaveHttpFrontendTestCase,
self.assertEqual(
parameter_dict, {}
)
class TestDefaultMonitorHttpdPort(SlaveHttpFrontendTestCase, TestDataMixin):
@classmethod
def getInstanceParameterDict(cls):
return {
'-frontend-1-state': 'stopped',
}
@classmethod
def getSlaveParameterDictDict(cls):
return {
'test': {
'url': cls.backend_url,
},
}
def test(self):
parameter_dict = self.slave_connection_parameter_dict_dict[
'test']
self.assertKeyWithPop('log-access-url', parameter_dict)
self.assertEqual(
parameter_dict,
{
'domain': 'test.None', 'replication_number': '1',
'url': 'http://test.None', 'site_url': 'http://test.None',
'secure_access': 'https://test.None', 'public-ipv4': None}
)
master_monitor_conf = open(os.path.join(
self.instance_path, 'TestDefaultMonitorHttpdPort-0', 'etc',
'monitor-httpd.conf')).read()
slave_monitor_conf = open(os.path.join(
self.instance_path, 'TestDefaultMonitorHttpdPort-1', 'etc',
'monitor-httpd.conf')).read()
self.assertTrue(
'Listen [%s]:8196' % (utils.GLOBAL_IPV6,) in master_monitor_conf)
self.assertTrue(
'Listen [%s]:8072' % (utils.GLOBAL_IPV6,) in slave_monitor_conf)
software/caddy-frontend/test/test_data/test.TestDefaultMonitorHttpdPort.test_file_list_run-CADDY.txt 0 → 100644
View file @ be08f156
TestDefaultMonitorHttpdPort-0/var/run/monitor/monitor-bootstrap.pid
\ No newline at end of file
software/caddy-frontend/test/test_data/test.TestDefaultMonitorHttpdPort.test_monitor_promise_list-CADDY.txt 0 → 100644
View file @ be08f156
TestDefaultMonitorHttpdPort-1/etc/monitor-promise/check-_test-error-log-last-day
TestDefaultMonitorHttpdPort-1/etc/monitor-promise/check-_test-error-log-last-hour
\ No newline at end of file
software/caddy-frontend/test/test_data/test.TestDefaultMonitorHttpdPort.test_promise_list-CADDY.txt 0 → 100644
View file @ be08f156
TestDefaultMonitorHttpdPort-0/etc/promise/check-free-disk-space
TestDefaultMonitorHttpdPort-0/etc/promise/monitor-http-frontend
TestDefaultMonitorHttpdPort-0/etc/promise/monitor-httpd-listening-on-tcp
TestDefaultMonitorHttpdPort-0/etc/promise/promise-monitor-httpd-is-process-older-than-dependency-set
TestDefaultMonitorHttpdPort-1/etc/promise/caddy-frontend-is-running-actual-software-release
TestDefaultMonitorHttpdPort-1/etc/promise/caddy_cached
TestDefaultMonitorHttpdPort-1/etc/promise/caddy_frontend_ipv4_http
TestDefaultMonitorHttpdPort-1/etc/promise/caddy_frontend_ipv4_https
TestDefaultMonitorHttpdPort-1/etc/promise/caddy_frontend_ipv6_http
TestDefaultMonitorHttpdPort-1/etc/promise/caddy_frontend_ipv6_https
TestDefaultMonitorHttpdPort-1/etc/promise/caddy_ssl_cached
TestDefaultMonitorHttpdPort-1/etc/promise/check-free-disk-space
TestDefaultMonitorHttpdPort-1/etc/promise/frontend-caddy-configuration-promise
TestDefaultMonitorHttpdPort-1/etc/promise/monitor-http-frontend
TestDefaultMonitorHttpdPort-1/etc/promise/monitor-httpd-listening-on-tcp
TestDefaultMonitorHttpdPort-1/etc/promise/nginx-configuration-promise
TestDefaultMonitorHttpdPort-1/etc/promise/nginx_frontend_ipv4_http
TestDefaultMonitorHttpdPort-1/etc/promise/nginx_frontend_ipv4_https
TestDefaultMonitorHttpdPort-1/etc/promise/nginx_frontend_ipv6_http
TestDefaultMonitorHttpdPort-1/etc/promise/nginx_frontend_ipv6_https
TestDefaultMonitorHttpdPort-1/etc/promise/promise-monitor-httpd-is-process-older-than-dependency-set
TestDefaultMonitorHttpdPort-1/etc/promise/promise-nginx-is-process-older-than-dependency-set
TestDefaultMonitorHttpdPort-1/etc/promise/re6st-connectivity
TestDefaultMonitorHttpdPort-1/etc/promise/trafficserver-cache-availability
TestDefaultMonitorHttpdPort-1/etc/promise/trafficserver-port-listening
\ No newline at end of file
software/caddy-frontend/test/test_data/test.TestDefaultMonitorHttpdPort.test_supervisor_state-CADDY.txt 0 → 100644
View file @ be08f156
TestDefaultMonitorHttpdPort-0:bootstrap-monitor EXITED
TestDefaultMonitorHttpdPort-0:certificate_authority-on-watch RUNNING
TestDefaultMonitorHttpdPort-0:crond RUNNING
TestDefaultMonitorHttpdPort-0:monitor-httpd-graceful EXITED
TestDefaultMonitorHttpdPort-0:monitor-httpd-on-watch EXITED
TestDefaultMonitorHttpdPort-1:6tunnel-26011-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:6tunnel-26012-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:6tunnel-4443-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:6tunnel-8080-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:6tunnel-8081-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:6tunnel-9443-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:bootstrap-monitor STOPPED
TestDefaultMonitorHttpdPort-1:certificate_authority-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:crond-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:frontend-caddy-safe-graceful STOPPED
TestDefaultMonitorHttpdPort-1:frontend-nginx-safe-graceful STOPPED
TestDefaultMonitorHttpdPort-1:frontend_caddy-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:frontend_nginx-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:monitor-httpd-graceful STOPPED
TestDefaultMonitorHttpdPort-1:monitor-httpd-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:trafficserver-on-watch STOPPED
TestDefaultMonitorHttpdPort-1:trafficserver-reload STOPPED
watchdog:watchdog RUNNING
\ No newline at end of file
software/kvm/instance-kvm-cluster.cfg.jinja2.in
View file @ be08f156
......@@ -91,7 +91,7 @@ config-httpd-port = {{ dumps(kvm_parameter_dict.get('httpd-port', 8081)) }}
config-disable-ansible-promise = {{ dumps(kvm_parameter_dict.get('disable-ansible-promise', False)) }}
config-monitor-cors-domains = {{ slapparameter_dict.get('monitor-cors-domains', 'monitor.app.officejs.com') }}
config-monitor-username = ${monitor-instance-parameter:username}
config-monitor-password = ${monitor-htpasswd:passwd}
config-monitor-password = ${publish-early:monitor-password}
# Enable disk wipe options
{% if kvm_parameter_dict.get('wipe-disk-ondestroy', False) -%}
config-wipe-disk-ondestroy = True
......@@ -238,6 +238,11 @@ mode = {{ mode }}
{{ writefile('cluster-data-content', '${directory:webroot}/${hash-code:passwd}/data', slapparameter_dict.get('cluster-data', ''), '700') }}
{% endif -%}
[publish-early]
recipe = slapos.cookbook:publish-early
-init =
monitor-password monitor-htpasswd:passwd
[monitor-instance-parameter]
monitor-httpd-port = 8060
cors-domains = {{ slapparameter_dict.get('monitor-cors-domains', 'monitor.app.officejs.com') }}
......@@ -260,6 +265,8 @@ recipe = slapos.cookbook:publish
{% endfor %}
{% set monitor_interface_url = slapparameter_dict.get('monitor-interface-url', 'https://monitor.app.officejs.com') -%}
monitor-setup-url = {{ monitor_interface_url }}/#page=settings_configurator&url=${monitor-publish-parameters:monitor-url}&username=${monitor-publish-parameters:monitor-user}&password=${monitor-publish-parameters:monitor-password}
monitor-password = ${publish-early:monitor-password}
monitor-user = ${monitor-publish-parameters:monitor-user}
{% do part_list.append('monitor-base') -%}
[buildout]
......@@ -273,7 +280,6 @@ parts =
httpd-promise
publish-connection-information
directory-doc
monitor-htpasswd
# Complete parts with sections
{{ part_list | join('\n ') }}
......
software/kvm/instance-kvm-resilient.cfg.jinja2
View file @ be08f156
......@@ -21,7 +21,7 @@ offline = true
# += because we need to take up parts (like instance-custom, slapmonitor etc) from the profile we extended
parts +=
monitor-htpasswd
publish-early
{{ parts.replicate("kvm", backup_amount) }}
publish-connection-information
kvm-frontend-url-promise
......@@ -37,13 +37,18 @@ storage-path = ${directory:etc}/.monitor_user
bytes = 8
username = admin
[publish-early]
recipe = slapos.cookbook:publish-early
-init =
monitor-password monitor-htpasswd:passwd
# XXX Monitoring Main Instane
[monitor-instance-parameter]
monitor-httpd-port = 8160
cors-domains = {{ monitor_parameter.get('monitor-cors-domains', '') }}
{% do monitor_parameter.__setitem__('monitor-username', slapparameter_dict.get('monitor-username', 'admin'))%}
{% do monitor_parameter.__setitem__('monitor-password', slapparameter_dict.get('monitor-password', '${monitor-htpasswd:passwd}'))%}
{% do monitor_parameter.__setitem__('monitor-password', slapparameter_dict.get('monitor-password', '${publish-early:monitor-password}'))%}
{% endif -%}
{{ replicated.replicate("kvm", backup_amount, "kvm-export", "kvm-import", slapparameter_dict=slapparameter_dict, monitor_parameter_dict=monitor_dict) }}
......@@ -68,9 +73,11 @@ recipe = slapos.cookbook:publish
backend-url = ${request-kvm:connection-backend-url}
url = ${request-kvm:connection-url}
ipv6 = ${request-kvm:connection-ip}
monitor-password = ${publish-early:monitor-password}
monitor-user = ${monitor-publish-parameters:monitor-user}
{% if monitor_dict -%}
monitor-base-url = ${monitor-publish-parameters:monitor-base-url}
monitor-setup-url = {{ monitor_interface_url }}/#page=settings_configurator&url=${monitor-publish-parameters:monitor-url}&username=${monitor-publish-parameters:monitor-user}&password=${monitor-publish-parameters:monitor-password}
monitor-setup-url = {{ monitor_interface_url }}/#page=settings_configurator&url=${monitor-publish-parameters:monitor-url}&username=${monitor-publish-parameters:monitor-user}&password=${publish-early:monitor-password}
{% endif -%}
[kvm-frontend-url-promise]
......
software/kvm/software.cfg
View file @ be08f156
......@@ -108,7 +108,7 @@ recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/instance-kvm-cluster.cfg.jinja2.in
mode = 644
md5sum = d9fe920d31f1ef0e377aa768ccd24f4c
md5sum = 6d165aec7d236ea3944765236d11940f
download-only = true
on-update = true
......@@ -117,7 +117,7 @@ recipe = hexagonit.recipe.download
ignore-existing = true
url = ${:_profile_base_location_}/instance-kvm-resilient.cfg.jinja2
mode = 644
md5sum = 1095968487282784a735735aa1b37d35
md5sum = a5fd0cbe6be757d57c8b6903bb7a1d8b
download-only = true
on-update = true
......
software/re6stnet/instance-re6stnet-input-schema.json
View file @ be08f156
......@@ -72,6 +72,12 @@
"description": "Specify that tunnels should be encrypted.",
"type": "boolean",
"default": false
},
"same-country": {
"title": "Same Country",
"description": "Prevent tunnelling accross borders of listed countries",
"type": "string",
"default": ""
}
}
}
\ No newline at end of file
software/re6stnet/instance-re6stnet.cfg.in
View file @ be08f156
......@@ -125,6 +125,7 @@ max-clients = {{ slapparameter_dict.get('max-clients', 0) }}
hello = {{ slapparameter_dict.get('hello', 15) }}
min-protocol = {{ slapparameter_dict.get('min-protocol', -1) }}
encrypt = {{ slapparameter_dict.get('encrypt', 'False') }}
same-country = {{ slapparameter_dict.get('same-country', '') }}
[re6st-registry-conf]
recipe = slapos.recipe.template:jinja2
......
software/re6stnet/re6st-registry.conf.in
View file @ be08f156
......@@ -25,4 +25,7 @@ encrypt
{% endif -%}
{% if parameter_dict.get('max-clients') != '0' -%}
max-clients {{ parameter_dict['max-clients'] }}
{% endif -%}
{% if parameter_dict.get('same-country') -%}
same-country {{ parameter_dict['same-country'] }}
{% endif -%}
\ No newline at end of file
software/re6stnet/software.cfg
View file @ be08f156
......@@ -87,7 +87,7 @@ extra-context =
[template-re6stnet]
< = download-base
filename = instance-re6stnet.cfg.in
md5sum = 6f28b611a0e2415768238f5e8d29d36e
md5sum = 8c167f2adb2ed36aeaff773f59214981
[template-apache-conf]
< = download-base
......@@ -97,7 +97,7 @@ md5sum = d64cafda1139b740a49a9f5e30a1b57b
[template-re6st-registry-conf]
< = download-base
filename = re6st-registry.conf.in
md5sum = 5dc218f887faeffc466e41c7d6191e49
md5sum = b85375cd45c5f2fb0d68e449ae70e2a1
[template-wrapper]
< = download-base
......
software/slaprunner/buildout.hash.cfg
View file @ be08f156
......@@ -38,7 +38,7 @@ md5sum = 1a812a06cc02bb11636009f4ec043d54
[template-resilient]
filename = instance-resilient.cfg.jinja2
md5sum = 8ed180de711d207a540d0acb539b2536
md5sum = bed1c457aa9e54a59b64d167bdafe970
[template_nginx_conf]
filename = nginx_conf.in
......
software/slaprunner/instance-resilient.cfg.jinja2
View file @ be08f156
......@@ -28,7 +28,7 @@ offline = true
# += because we need to take up parts (like instance-custom, slapmonitor etc) from the profile we extended
parts +=
monitor-htpasswd
publish-early
{{ parts.replicate("runner", number_of_instances + 1) }}
publish-connection-information
......@@ -38,8 +38,13 @@ storage-path = ${directory:etc}/.monitor_user
bytes = 8
username = admin
[publish-early]
recipe = slapos.cookbook:publish-early
-init =
init-password monitor-htpasswd:passwd
{% do monitor_parameter.__setitem__('monitor-username', slapparameter_dict.get('monitor-username', 'admin'))%}
{% do monitor_parameter.__setitem__('monitor-password', slapparameter_dict.get('monitor-password', '${monitor-htpasswd:passwd}'))%}
{% do monitor_parameter.__setitem__('monitor-password', slapparameter_dict.get('monitor-password', '${publish-early:init-password}'))%}
{{ replicated.replicate("runner", number_of_instances + 1, "runner-export", "runner-import", slapparameter_dict=slapparameter_dict, monitor_parameter_dict=monitor_dict) }}
......@@ -60,7 +65,7 @@ recipe = slapos.cookbook:publish
backend-url = ${request-runner:connection-backend-url}
url = ${request-runner:connection-url}
init-user = ${request-runner:connection-init-user}
init-password = ${request-runner:connection-init-password}
init-password = ${publish-early:init-password}
ssh-command = ${request-runner:connection-ssh-command}
webdav-url = ${request-runner:connection-webdav-url}
public-url = ${request-runner:connection-public-url}
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7