Fix auto-renew of LetsEncrypt domains for Pages

As described in
https://community.letsencrypt.org/t/production-chain-changes/150739, the
LetsEncrypt DST Root CA X3 expired on September 30, 2021. Domains that
needed to be renewed via PagesDomainSslWorker would fail with the error,
"Certificate misses intermediates".

Newly-issued certificates would come with this chain of trust:

End-entity certificate (A) ← R3 (B) ← ISRG Root X1 (C) ← DST Root CA X3

Previously, this is what was happening:

1. LetsEncrypt returned a bundle containing A, B, and C.
2. `PagesDomain#has_intermediates?` took B and C and added them to the
OpenSSL certificate store.
3. `OpenSSL::X509::Store#verify` returned `false` because C was a
trusted certificate, but DST Root CA X3 had expired.

The crux of the problem is that we aren't using `verify` properly: we
should be passing in an untrusted chain and allow OpenSSL to verify that
C is indeed trusted from the system store. This emulates the behavior of
the `-untrusted` parameter in the `openssl` command-line:

```
All certificates (typically of intermediate CAs) are considered
untrusted and may be used to construct a certificate chain from the
target certificate to a trust anchor.
```

Relates to https://gitlab.com/gitlab-org/gitlab/-/issues/342326

Changelog: fixed

app/models/pages_domain.rb

@@ -129,18 +129,15 @@ class PagesDomain < ApplicationRecord
     store = OpenSSL::X509::Store.new
     store.set_default_paths
 
-    # This forces to load all intermediate certificates stored in `certificate`
-    Tempfile.open('certificate_chain') do |f|
-      f.write(certificate)
-      f.flush
-      store.add_file(f.path)
-    end
-
-    store.verify(x509)
+    store.verify(x509, untrusted_ca_certs_bundle)
   rescue OpenSSL::X509::StoreError
     false
   end
 
+  def untrusted_ca_certs_bundle
+    ::Gitlab::Email::Smime::Certificate.load_ca_certs_bundle(certificate)
+  end
+
   def expired?
     return false unless x509
 

spec/factories/pages_domains.rb

@@ -258,6 +258,18 @@ ZDXgrA==
       certificate_source { :gitlab_provided }
     end
 
+    # This contains:
+    # webdioxide.com
+    # Let's Encrypt R3
+    # ISRG Root X1 (issued by DST Root CA X3)
+    #
+    # DST Root CA X3 expired on 2021-09-30, but ISRG Root X1 should be trusted on most systems.
+    trait :letsencrypt_expired_x3_root do
+      certificate do
+        File.read(Rails.root.join('spec/fixtures/ssl', 'letsencrypt_expired_x3.pem'))
+      end
+    end
+
     trait :explicit_ecdsa do
       certificate do
         '-----BEGIN CERTIFICATE-----

spec/factories_spec.rb

@@ -29,6 +29,7 @@ RSpec.describe 'factories' do
       [:pages_domain, :with_trusted_chain],
       [:pages_domain, :with_trusted_expired_chain],
       [:pages_domain, :explicit_ecdsa],
+      [:pages_domain, :letsencrypt_expired_x3_root],
       [:project_member, :blocked],
       [:remote_mirror, :ssh],
       [:user_preference, :only_comments],

spec/fixtures/ssl/letsencrypt_expired_x3.pem

+-----BEGIN CERTIFICATE-----
+MIIGJDCCBQygAwIBAgISBOSAE/WwQGsTbDJI1vDL9+eKMA0GCSqGSIb3DQEBCwUA
+MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
+EwJSMzAeFw0yMTEwMDEyMjIxMTlaFw0yMTEyMzAyMjIxMThaMBkxFzAVBgNVBAMT
+DndlYmRpb3hpZGUuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA
+wf/TpE5AjzoLXMFQ+WHle7Dn5rlEe0bPee2JU386cZmMYnGFS5DR251FerSX28U4
+pqk2yS8oefHGi2PS6h8/MWxr+Zy/6hk3WkgwdIK3uPiUcfCdPV/btXDd4YqikEDm
+BoOE4fQlqKQwtLOnhEZu9y8FQoxxoQ+7DndHrDixDoMbpUloxpqUZwziQnH4QHXE
+32rQhq25+NUK/lVFGKOFnmZ2s/yUildKafqulHrLHOhumKMOEivzlFDZbtqP+RKt
+nsrJ3i9O+nSQz6j5dv3Du6eaResrtK7tT1MFDNhcg2cgjNW64VLXQdFXYXE1OYsw
+yAuXUnHNzWFhinyf80qeh2046YR21dlG8voIDQH4fGG5GmWLyu7glsWYVwQQ36VA
+TTxPmAoaqUTl8A7cnlJpAo+BJ00mS/9DwJ7pkgGC7dYOhJzWlI7lPqzEfmJ+o8pj
+CJlLIuqsn0vcCZQlmqCFMxK4asn+puLLnMjRLHIYEJKDNyPGHQEr2e5t4GUYZKaN
+MEpXMwJd97tUamUKWeBPNIND/kOuqexe+okbOTRp34VAsK5oCpawEJckoNkK+sv0
+OrSWFOdfLBHv66p9qsrz8LQXxmN5JUBUe51SBSUo1Ul4/vGYdhuKd/8KcLw9/Al+
+HJN2hAeo3v+2fVey4hgGna7XNe8e3+E+OEQb4zpQDLkCAwEAAaOCAkswggJHMA4G
+A1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDAYD
+VR0TAQH/BAIwADAdBgNVHQ4EFgQU4PbvqCKatjx6GZMXy7v9GwykZq4wHwYDVR0j
+BBgwFoAUFC6zF7dYVsuuUAlA5h+vnYsUwsYwVQYIKwYBBQUHAQEESTBHMCEGCCsG
+AQUFBzABhhVodHRwOi8vcjMuby5sZW5jci5vcmcwIgYIKwYBBQUHMAKGFmh0dHA6
+Ly9yMy5pLmxlbmNyLm9yZy8wGQYDVR0RBBIwEIIOd2ViZGlveGlkZS5jb20wTAYD
+VR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC3xMBAQEwKDAmBggrBgEFBQcCARYa
+aHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcwggEGBgorBgEEAdZ5AgQCBIH3BIH0
+APIAdwBc3EOS/uarRUSxXprUVuYQN/vV+kfcoXOUsl7m9scOygAAAXw+KYGHAAAE
+AwBIMEYCIQCqD6jMtHrGlE02Qh1FzFd4+qYzJTrChHmHBFIncPGQKAIhALeYk0Vf
+/Lw2tX2beVlKN4/h1o8srNJv+06xkr1N6XmiAHcAfT7y+I//iFVoJMLAyp5SiXkr
+xQ54CX8uapdomX4i8NcAAAF8PimBogAABAMASDBGAiEA0h883FFj1dSYKGym9+Wa
+XgJRj526X7YlkhkZ5J1TjioCIQDyjMPrbo5liVi/e5b8gfDw5Fd9WNiTu1W1LKKu
+UpE/qTANBgkqhkiG9w0BAQsFAAOCAQEAcx10nqp1kh2awwoqwf7Jo8Gycqx2bA2O
+E2rveQ/BK9UhwvrNeEpE9SG6liMsYJKxGar0vbbBHvxzuMU00bhGjXFtUT5XuQ8q
+FcU0OdycyZj8fjZmUNsJr82l8HvfJ50jfxFORTgj8Ln5MWVUFlbl0nD+06l28sDc
+V+r/B4394fkoMsKXtiTA4/ZeOD1tHNsdxQ7sNQtEfqCG0wFCYHK3rs7XTZ1K0F3c
+M051JShko1UKP/k5blrendOwVRwLtq+9pavGnJBeqNIVgugTER/IHlp4427WyhdY
+KYjKoytW+XQyWqxU/Mh/O4rxkD8cZaE+FdZpP67VZ185AuZMbn+LcQ==
+-----END CERTIFICATE-----
+
+-----BEGIN CERTIFICATE-----
+MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
+TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
+WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
+RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
+AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
+R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
+sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
+NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
+Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
+/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
+AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
+Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
+FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
+AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
+Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
+gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
+PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
+ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
+CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
+lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
+avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
+yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
+yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
+hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
+HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
+MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
+nLRbwHOoq7hHwg==
+-----END CERTIFICATE-----
+
+-----BEGIN CERTIFICATE-----
+MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/
+MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
+DkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1ow
+TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEB
+AQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XC
+ov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpL
+wYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+D
+LtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK
+4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5
+bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5y
+sR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZ
+Xmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4
+FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBc
+SLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2ql
+PRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TND
+TwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYw
+SwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1
+c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx
++tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEB
+ATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQu
+b3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9E
+U1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26Ztu
+MA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC
+5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW
+9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuG
+WCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9O
+he8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFC
+Dfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5
+-----END CERTIFICATE-----

spec/models/pages_domain_spec.rb

@@ -287,6 +287,19 @@ RSpec.describe PagesDomain do
 
       it { is_expected.to be_truthy }
     end
+
+    # The LetsEncrypt DST Root CA X3 expired on 2021-09-30, but the
+    # cross-sign in ISRG Root X1 enables it to function provided a chain
+    # of trust can be established with the system store. See:
+    #
+    # 1. https://community.letsencrypt.org/t/production-chain-changes/150739
+    # 2. https://letsencrypt.org/2020/12/21/extending-android-compatibility.html
+    # 3. https://www.openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire/
+    context 'with a LetsEncrypt bundle with an expired DST Root CA X3' do
+      let(:domain) { build(:pages_domain, :letsencrypt_expired_x3_root) }
+
+      it { is_expected.to be_truthy }
+    end
   end
 
   describe '#expired?' do