stack/erp5: remove httpd and use haproxy instead
Two main differences of haproxy are file format for certificates and logs.
HAProxy also uses certificates in PEM format, but it expect its own server
certificate and the key to be in the same file (although recent version seems
to accept separate files, we don't use this now) and the CRL and CA certificates
also all together in the same file.
We change to use the same file for certificate and key and for CA and CRL, in
the updater script we we build PEM files by containing all CA certificates and
all CRL together.
Also, since haproxy needs to be reloaded when certificate change, we run it in
master-worker mode, with a pid file so that we can signal it to reload.
For the logs, since haproxy does not log to file, we introduce a rsyslogd to
log to a file. The log format is same as with httpd, except that timing are not
in microseconds but in milliseconds - this did not seem to be configurable.
This is a problem for apachedex reports on log, for that we plan to use an
updated version of apachedex with support for `%{ms}T` for durations.
HAProxy is configured with same timeouts, except:
- "connect" timeout has been increased a bit (from 5 to 10s), because the
comment "The connection should be immediate on LAN" was no longer true, now
that haproxy is accessed from frontend.
- the server entries for testrunner are a very long timeout (8h) because some
ERP5 functional tests exceeed the 305s timeout.
The SSL configuration is with current "modern" config from https://ssl-config.mozilla.org/
Tests have been modified a bit, because haproxy uses HTTP/2.0 and not 1.1
like httpd was doing several haproxy features (keep alive and gzip
compression) are only available when backend uses HTTP/1.1, so we adjusted
tests to use a 1.1 backend.
There was also differences with logs, because of the time being in milliseconds.
TestPublishedURLIsReachableMixin._checkERP5IsReachable was also updated, it
was working by chance because when accessed behind httpd->haproxy->zope, zope
was producing a redirect URL that was the URL of haproxy, which could be
resolved by chance. This test was updated to access zope with a path that
contains VirtualHostMonster magic, as the shared frontend ( with "zope" software
type) is supposed to set.
This should hopefuly solve the "502 Proxy Error" that we are observing with httpd.
Showing
| {# This file configures haproxy to redirect requests from ports to specific urls. | |||
| # It provides TLS support for server and optionnaly for client. | |||
| # | |||
| # All parameters are given through the `parameter_dict` variable, see the | |||
| # list entries : | |||
| # | |||
| # parameter_dict = { | |||
| # # Path of the PID file. HAProxy will write its own PID to this file | |||
| # # Sending USR2 signal to this pid will cause haproxy to reload | |||
| # # its configuration. | |||
| # "pidfile": "<file_path>", | |||
| # | |||
| # # AF_UNIX socket for logs. Syslog must be listening on this socket. | |||
| # "log-socket": "<file_path>", | |||
| # | |||
| # # AF_UNIX socket for statistics and control. | |||
| # # Haproxy will listen on this socket. | |||
| # "stats-socket": "<file_path>", | |||
| # | |||
| # # IPv4 to listen on | |||
| # # All backends from `backend-dict` will listen on this IP. | |||
| # "ipv4": "0.0.0.0", | |||
| # | |||
| # # IPv6 to listen on | |||
| # # All backends from `backend-dict` will listen on this IP. | |||
| # "ipv6": "::1", | |||
| # | |||
| # # Certificate and key in PEM format. All ports will serve TLS using | |||
| # # this certificate. | |||
| # "cert": "<file_path>", | |||
| # | |||
| # # CA to verify client certificates in PEM format. | |||
| # # If set, client certificates will be verified with these CAs. | |||
| # # If not set, client certificates are not verified. | |||
| # "ca-cert": "<file_path>", | |||
| # | |||
| # # An optional CRL in PEM format (the file can contain multiple CRL) | |||
| # # This is required if ca-cert is passed. | |||
| # "crl": "<file_path>", | |||
| # | |||
| # # Path to use for HTTP health check on backends from `backend-dict`. | |||
| # "server-check-path": "/", | |||
| # | |||
| # # The mapping of backends, keyed by family name | |||
| # "backend-dict": { | |||
| # "family-secure": { | |||
| # ( 8000, # port int | |||
| # 'https', # proto str | |||
| # True, # ssl_required bool | |||
| # [ # backends | |||
| # '10.0.0.10:8001', # netloc str | |||
| # 1, # max_connection_count int | |||
| # False, # is_web_dav bool | |||
| # ], | |||
| # ), | |||
| # }, | |||
| # "family-default": { | |||
| # ( 8002, # port int | |||
| # 'https', # proto str | |||
| # False, # ssl_required bool | |||
| # [ # backends | |||
| # '10.0.0.10:8003', # netloc str | |||
| # 1, # max_connection_count int | |||
| # False, # is_web_dav bool | |||
| # ], | |||
| # ), | |||
| # }, | |||
| # | |||
| # # The mapping of zope paths. | |||
| # # This is a Zope specific feature. | |||
| # # `enable_authentication` has same meaning as for `backend-list`. | |||
| # "zope-virtualhost-monster-backend-dict": { | |||
| # # {(ip, port): ( enable_authentication, {frontend_path: ( internal_url ) }, ) } | |||
| # ('[::1]', 8004): ( | |||
| # True, { | |||
| # 'zope-1': 'http://10.0.0.10:8001', | |||
| # 'zope-2': 'http://10.0.0.10:8002', | |||
| # }, | |||
| # ), | |||
| # }, | |||
| # } | |||
| # | |||
| # This sample of `parameter_dict` will make haproxy listening to : | |||
| # From to `backend-list`: | |||
| # For "family-secure": | |||
| # - 0.0.0.0:8000 redirecting internaly to http://10.0.0.10:8001 and | |||
| # - [::1]:8000 redirecting internaly to http://10.0.0.10:8001 | |||
| # only accepting requests from clients providing a verified TLS certificate | |||
| # emitted by a CA from `ca-cert` and not revoked in `crl`. | |||
| # For "family-default": | |||
| # - 0.0.0.0:8002 redirecting internaly to http://10.0.0.10:8003 | |||
| # - [::1]:8002 redirecting internaly to http://10.0.0.10:8003 | |||
| # accepting requests from any client. | |||
| # | |||
| # For both families, X-Forwarded-For header will be stripped unless | |||
| # client presents a certificate that can be verified with `ca-cert` and `crl`. | |||
| # | |||
| # From zope-virtualhost-monster-backend-dict`: | |||
| # - [::1]:8004 with some path based rewrite-rules redirecting to: | |||
| # * http://10.0.0.10/8001 when path matches /zope-1(.*) | |||
| # * http://10.0.0.10/8002 when path matches /zope-2(.*) | |||
| # with some VirtualHostMonster rewrite rules so zope writes URLs with | |||
| # [::1]:8004 as server name. | |||
| # For more details, refer to | |||
| # https://docs.zope.org/zope2/zope2book/VirtualHosting.html#using-virtualhostroot-and-virtualhostbase-together | |||
| -#} | |||
| {% set server_check_path = parameter_dict['server-check-path'] -%} | {% set server_check_path = parameter_dict['server-check-path'] -%} | ||
| global | global | ||
| maxconn 4096 | maxconn 4096 | ||
| stats socket {{ parameter_dict['socket-path'] }} level admin | master-worker | ||
| pidfile {{ parameter_dict['pidfile'] }} | |||
| # SSL configuration was generated with mozilla SSL Configuration Generator | |||
| # generated 2020-10-28, Mozilla Guideline v5.6, HAProxy 2.1, OpenSSL 1.1.1g, modern configuration | |||
| # https://ssl-config.mozilla.org/#server=haproxy&version=2.1&config=modern&openssl=1.1.1g&guideline=5.6 | |||
| ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 | |||
| ssl-default-bind-options prefer-client-ciphers no-sslv3 no-tlsv10 no-tlsv11 no-tlsv12 no-tls-tickets | |||
| ssl-default-server-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256 | |||
| ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tlsv12 no-tls-tickets | |||
| stats socket {{ parameter_dict['stats-socket'] }} level admin | |||
| defaults | defaults | ||
| mode http | mode http | ||
| retries 1 | retries 1 | ||
| option redispatch | option redispatch | ||
| maxconn 2000 | maxconn 2000 | ||
| cookie SERVERID rewrite | |||
| balance roundrobin | balance roundrobin | ||
| stats uri /haproxy | stats uri /haproxy | ||
| stats realm Global\ statistics | stats realm Global\ statistics | ||
| # it is useless to have timeout much bigger than the one of apache. | |||
| # By default apache use 300s, so we set slightly more in order to | timeout connect 10s | ||
| # make sure that apache will first stop the connection. | |||
| timeout server 305s | |||
| # Stop waiting in queue for a zope to become available. | |||
| # If no zope can be reached after one minute, consider the request will | |||
| # never succeed. | |||
| timeout queue 60s | timeout queue 60s | ||
| # The connection should be immediate on LAN, | timeout server 305s | ||
| # so we should not set more than 5 seconds, and it could be already too much | |||
| timeout connect 5s | |||
| # As requested in haproxy doc, make this "at least equal to timeout server". | |||
| timeout client 305s | timeout client 305s | ||
| # Use "option httpclose" to not preserve client & server persistent connections | |||
| # while handling every incoming request individually, dispatching them one after | option http-server-close | ||
| # another to servers, in HTTP close mode. This is really needed when haproxy | |||
| # is configured with maxconn to 1, without this option browsers are unable | # compress some content types | ||
| # to render a page | compression algo gzip | ||
| option httpclose | compression type application/font-woff application/font-woff2 application/hal+json application/javascript application/json application/rss+xml application/wasm application/x-font-opentype application/x-font-ttf application/x-javascript application/xml image/svg+xml text/cache-manifest text/css text/html text/javascript text/plain text/xml | ||
| {% for name, (port, backend_list) in sorted(parameter_dict['backend-dict'].iteritems()) -%} | log {{ parameter_dict['log-socket'] }} local0 info | ||
| listen {{ name }} | |||
| bind {{ parameter_dict['ip'] }}:{{ port }} | {% set bind_ssl_crt = 'ssl crt ' ~ parameter_dict['cert'] ~ ' alpn h2,http/1.1' %} | ||
| {% for name, (port, _, certificate_authentication, backend_list) in sorted(parameter_dict['backend-dict'].iteritems()) -%} | |||
| listen family_{{ name }} | |||
| {%- if parameter_dict.get('ca-cert') -%} | |||
| {%- set ssl_auth = ' ca-file ' ~ parameter_dict['ca-cert'] ~ ' verify' ~ ( ' required' if certificate_authentication else ' optional' ) ~ ' crl-file ' ~ parameter_dict['crl'] %} | |||
| {%- else %} | |||
| {%- set ssl_auth = '' %} | |||
| {%- endif %} | |||
| bind {{ parameter_dict['ipv4'] }}:{{ port }} {{ bind_ssl_crt }} {{ ssl_auth }} | |||
| bind {{ parameter_dict['ipv6'] }}:{{ port }} {{ bind_ssl_crt }} {{ ssl_auth }} | |||
| cookie SERVERID rewrite | |||
| http-request set-header X-Balancer-Current-Cookie SERVERID | http-request set-header X-Balancer-Current-Cookie SERVERID | ||
| # remove X-Forwarded-For unless client presented a verified certificate | |||
| acl client_cert_verified ssl_c_used ssl_c_verify 0 | |||
| http-request del-header X-Forwarded-For unless client_cert_verified | |||
| # set Remote-User if client presented a verified certificate | |||
| http-request del-header Remote-User | |||
| http-request set-header Remote-User %{+Q}[ssl_c_s_dn(cn)] if client_cert_verified | |||
| # logs | |||
| capture request header Referer len 512 | |||
| capture request header User-Agent len 512 | |||
| log-format "%{+Q}o %{-Q}ci - - [%trg] %r %ST %B %{+Q}[capture.req.hdr(0)] %{+Q}[capture.req.hdr(1)] %Tt" | |||
|
|||
| {% set has_webdav = [] -%} | {% set has_webdav = [] -%} | ||
| {% for address, connection_count, webdav in backend_list -%} | {% for address, connection_count, webdav in backend_list -%} | ||
| {% if webdav %}{% do has_webdav.append(None) %}{% endif -%} | {% if webdav %}{% do has_webdav.append(None) %}{% endif -%} | ||
| {% set server_name = name ~ '-' ~ loop.index0 -%} | {% set server_name = name ~ '-' ~ loop.index0 %} | ||
| server {{ server_name }} {{ address }} cookie {{ server_name }} check inter 3s rise 1 fall 2 maxqueue 5 maxconn {{ connection_count }} | server {{ server_name }} {{ address }} cookie {{ server_name }} check inter 3s rise 1 fall 2 maxqueue 5 maxconn {{ connection_count }} | ||
| {% endfor -%} | {%- endfor -%} | ||
| {%- if not has_webdav and server_check_path %} | {%- if not has_webdav and server_check_path %} | ||
| option httpchk GET {{ server_check_path }} | option httpchk GET {{ server_check_path }} | ||
| {% endif -%} | {%- endif %} | ||
| {% endfor %} | |||
| {% for (ip, port), (_, backend_dict) in sorted(parameter_dict['zope-virtualhost-monster-backend-dict'].iteritems()) -%} | |||
| {% set group_name = 'testrunner_' ~ loop.index0 %} | |||
| frontend frontend_{{ group_name }} | |||
| bind {{ ip }}:{{ port }} {{ bind_ssl_crt }} | |||
| timeout client 8h | |||
| # logs | |||
| capture request header Referer len 512 | |||
| capture request header User-Agent len 512 | |||
| log-format "%{+Q}o %{-Q}ci - - [%trg] %r %ST %B %{+Q}[capture.req.hdr(0)] %{+Q}[capture.req.hdr(1)] %Tt" | |||
| {% for name in sorted(backend_dict.keys()) %} | |||
| use_backend backend_{{ group_name }}_{{ name }} if { path -m beg /{{ name }} } | |||
| {%- endfor %} | |||
| {% for name, url in sorted(backend_dict.items()) %} | |||
| backend backend_{{ group_name }}_{{ name }} | |||
| http-request replace-path ^/{{ name }}(.*) /VirtualHostBase/https/{{ ip }}:{{ port }}/VirtualHostRoot/_vh_{{ name }}\1 | |||
| timeout server 8h | |||
| server {{ name }} {{ urlparse.urlparse(url).netloc }} | |||
| {%- endfor %} | |||
| {% endfor %} | {% endfor %} | ||
This diff is collapsed.
stack/erp5/rsyslogd.cfg.in
0 → 100644