gitlab: simplify extends and parts

This is a workaround for what seem to be a buildout regression

software/gitlab/buildout.hash.cfg

@@ -54,15 +54,15 @@ md5sum = 056d7ed09e1bf20d022d3ef6b9363e00
 
 [instance-gitlab.cfg.in]
 _update_hash_filename_ = instance-gitlab.cfg.in
-md5sum = 6effe68f7ef240780d1027dd834a562c
+md5sum = 7a434e10a46bff72af2fb94c5dafa214
 
 [instance-gitlab-export.cfg.in]
 _update_hash_filename_ = instance-gitlab-export.cfg.in
-md5sum = 2af7dcf63f74e5edc53a3ff11fa4989b
+md5sum = 8f6920ab07a474be64b3cd320812c789
 
 [instance-gitlab-test.cfg.in]
 _update_hash_filename_ = instance-gitlab-test.cfg.in
-md5sum = a4ad76856db98e508af7e773d9ff78f9
+md5sum = 730fb8f10e37bbfbce1b13eaca11dbfa
 
 [macrolib.cfg.in]
 _update_hash_filename_ = macrolib.cfg.in

software/gitlab/instance-gitlab-export.cfg.in

@@ -3,11 +3,15 @@
 extends = {{ instance_gitlab_cfg }}
 # TODO + ${pbsready-export:output}
 
-parts +=
+export-parts =
     cron-entry-gitlab-backup
     resiliency-exclude-file
     gitlab-resiliency-restore-script
 
+parts =
+    ${:export-parts}
+    ${:default-parts}
+
 # -export specific instance parameters
 [instance-parameter]
 # cron frequency for gitlab backup (default: every 4h)

software/gitlab/instance-gitlab-test.cfg.in

@@ -2,8 +2,9 @@
 extends = 
   {{ instance_gitlab_cfg }}
   {{ instance_gitlab_export_cfg }}
-
-parts +=
+parts =
+  ${:export-parts}
+  ${:default-parts}
   install-demo-backup
 
 

software/gitlab/instance-gitlab.cfg.in

@@ -7,7 +7,8 @@
 extends =
   {{ gitlab_parameters_cfg }}
   {{ monitor_template }}
-parts =
+parts = ${:default-parts}
+default-parts =
     directory
     publish-instance-info
 

[Julien Muchembled]

why this line ?

[Jérome Perrin]

Sorry, I don't know why this line was here. It was not touched by the patch and I don't really know the details of this SR.

Also, to be clear, this patch was not made when understanding fully what was the problem. It's still at the "if I do this test no longer fail but I don't understand why" step.

[Jérome Perrin]

/cc @alain.takoudjou

[Julien Muchembled]

The commit message must shows the expected & wrong lists.

This is a workaround for what seem to be a buildout regression.

If it's the case, it must be reported.

But it's more probably an expected change of behaviour.

[Jérome Perrin]

If it's the case, it must be reported.

gitlab and kvm tests are failing since a few weeks, I think that's the reason

[Julien Muchembled]

Can you give a URL to a test result line showing such issue with KVM ?

/cc @tomo

[Jérome Perrin]

I thought that kvm test of SlapOS.SoftwareReleases.IntegrationTest-Master.Python2 was affected by such a problem, but looking more closely at KVM, it must be a different problem, because test is passing sometimes:

But gitlab tests are different, they started to fail between nexedi/slapos@8a1a673a...4aae0337 and never passed since.

instance step fails with

G0[gitlab-test]: Promise 'promise-unicorn.py' failed with output: ERROR '/srv/slapgrid/slappart8/srv/testnode/cvt/inst/test0-0/tmp/shared/curl/e617a3c549f9c2c5f284f996a1ce83c6/bin/curl --unix-socket /srv/slapgrid/slappart8/srv/testnode/cvt/inst/test0-0/tmp/inst/G0/srv/unicorn/unc.socket  http://localhost/' run with failure, output: "curl: (7) Couldn't connect to server\n"

This is because service-nginx is not started, because the part is not generated.

In installed.cfg we can see that we used instance-gitlab-test.cfg for software type, which is:

[buildout]
extends = 
  /srv/slapgrid/slappart8/srv/testnode/cvt/inst/test0-0/tmp/inst/G0/instance-gitlab.cfg
  /srv/slapgrid/slappart8/srv/testnode/cvt/inst/test0-0/tmp/inst/G0/instance-gitlab-export.cfg

parts +=
  install-demo-backup

instance-gitlab.cfg contains:

[buildout]
extends =
  /srv/slapgrid/slappart8/srv/testnode/cvt/inst/test0-0/tmp/soft/abe9e92f19caf84d2d5896b4a7b02b1f/gitlab-parameters.cfg
  /srv/slapgrid/slappart8/srv/testnode/cvt/inst/test0-0/tmp/soft/abe9e92f19caf84d2d5896b4a7b02b1f/template-monitor.cfg
parts =
    ...
    service-nginx
    ... 

so I guess with this configuration, service-nginx should be installed, but it's not in parts from .installed-switch-softwaretype.cfg.

I have not investigated more than this, in https://lab.nexedi.com/nexedi/slapos.buildout/compare/2.7.1+slapos005...2.7.1+slapos007 there's a change about += and extends so that's what made me suspicious for something in buildout.

But maybe it's not. Maybe gitlab SR was not using extends and += the right way. Today I understood that if it's a problem, it's only a problem in gitlab, so maybe gitlab SR is wrong.

Maybe we can finish this patch and apply it and say problem solved. += behavior in buildout is not really clear to me, if we write the profile in a more simple/stupid way it's easier. I gave up trying to fight against += :)

[Jérome Perrin]

but I don't mean Nexedi should give up, that was my understanding of this test failure, I have not looked at the actual buildout changes, just the changelog. If this we think it's a regression in buildout we can also fix it. I have this feeling that some uses of += in buildout are not really specified and maybe works different way in different buildout versions (because 10 years ago I got hit by bugs with this and don't want to trust this for some irrational reasons :))

Anyway, thanks for asking about this issue.

[Julien Muchembled]

There had a bug in slapos005...2.7.1+slapos007 and it was fixed in slapos005...2.7.1+slapos008. But overall, there's also an expected change of behaviour and I made a few changes about this in slapos.git

The best way to understand the issue is actually to compare the output of 'buildout annotate'.