software/kvm/test: Wait long when big images are downloaded

Some tests are checking deeply that bootstrap script works correctly inside
if prepared VM, but those prepared VMs are big chunks of data to be fetched,
so it's required to wait quite long.

software/kvm/test/test.py

@@ -249,6 +249,8 @@ class TestAccessDefaultAdditional(MonitorAccessMixin, InstanceTestCase):
 class TestAccessDefaultBootstrap(MonitorAccessMixin, InstanceTestCase):
   __partition_reference__ = 'adb'
   expected_partition_with_monitor_base_url_count = 1
+  # as few gigabytes are being downloaded, wait a bit longer
+  instance_max_retry = 40
 
   @classmethod
   def getInstanceParameterDict(cls):
@@ -344,6 +346,8 @@ class TestAccessKvmClusterAdditional(MonitorAccessMixin, InstanceTestCase):
 class TestAccessKvmClusterBootstrap(MonitorAccessMixin, InstanceTestCase):
   __partition_reference__ = 'akcb'
   expected_partition_with_monitor_base_url_count = 3
+  # as few gigabytes are being downloaded, wait a bit longer
+  instance_max_retry = 40
 
   @classmethod
   def getInstanceSoftwareType(cls):

[Łukasz Nowak]

@tomo I had to increase the wait time, as images introduced in 11349158 resulted with unstable test results, which are hard to track, like:

======================================================================
ERROR: setUpClass (test.TestAccessDefaultBootstrap)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/srv/slapgrid/slappart9/srv/testnode/cvr/soft/ae7ffc5e56b55ad22db74d88b36303a1/parts/slapos.core-repository/slapos/testing/testcase.py", line 521, in setUpClass
    max_retry=cls.instance_max_retry, debug=cls._debug)
  File "/srv/slapgrid/slappart9/srv/testnode/cvr/soft/ae7ffc5e56b55ad22db74d88b36303a1/parts/slapos.core-repository/slapos/slap/standalone.py", line 686, in waitForInstance
    error_lines=error_lines,
  File "/srv/slapgrid/slappart9/srv/testnode/cvr/soft/ae7ffc5e56b55ad22db74d88b36303a1/parts/slapos.core-repository/slapos/slap/standalone.py", line 756, in _runSlapOSCommand
    'exitstatus': process_info['exitstatus'],
slapos.slap.standalone.SlapOSNodeCommandError: SlapOSNodeCommandError exitstatus: 2 output:
2020-08-12 17:52:08 slapos[26041] INFO Updating set-default-interface.
2020-08-12 17:52:08 slapos[26041] INFO Requested start of adb0...
2020-08-12 17:52:08 slapos[26041] INFO Checking adb0 promises...
2020-08-12 17:52:08 slapos[26041] INFO Checking promise monitor-http-frontend.py...
2020-08-12 17:52:08 monitor-http-frontend.py[26041] INFO 'https://[2001:67c:1254:61:a915::6e78]:8026' is protected (returned 401).
2020-08-12 17:52:08 slapos[26041] INFO Checking promise check-free-disk-space.py...
2020-08-12 17:52:09 check-free-disk-space.py[26041] INFO No result from collector database: disk check skipped
2020-08-12 17:52:09 slapos[26041] INFO Checking promise monitor-httpd-listening-on-tcp.py...
2020-08-12 17:52:09 monitor-httpd-listening-on-tcp.py[26041] INFO 'https://[2001:67c:1254:61:a915::6e78]:8026' is protected (returned 401).
2020-08-12 17:52:09 slapos[26041] INFO Checking promise monitor-bootstrap-status.py...
2020-08-12 17:52:10 monitor-bootstrap-status.py[26041] INFO Bootstrap OK
2020-08-12 17:52:10 slapos[26041] INFO Checking promise buildout-adb0-status.py...
2020-08-12 17:52:10 buildout-adb0-status.py[26041] INFO buildout is OK
2020-08-12 17:52:10 slapos[26041] INFO Checking promise vnc_promise.py...
2020-08-12 17:52:10 vnc_promise.py[26041] ERROR ERROR while connecting to ('10.0.242.107', 5901): [Errno 111] Connection refused
2020-08-12 17:52:10 slapos[26041] ERROR ERROR while connecting to ('10.0.242.107', 5901): [Errno 111] Connection refused
2020-08-12 17:52:10 slapos[26041] INFO Checking promise kvm-disk-image-corruption.py...
2020-08-12 17:52:10 kvm-disk-image-corruption.py[26041] ERROR ERROR '/srv/slapgrid/slappart9/srv/testnode/cvr/inst/test0-0/tmp/inst/adb0/bin/kvm-disk-image-corruption' run with failure, output: b''
2020-08-12 17:52:10 slapos[26041] ERROR ERROR '/srv/slapgrid/slappart9/srv/testnode/cvr/inst/test0-0/tmp/inst/adb0/bin/kvm-disk-image-corruption' run with failure, output: b''
2020-08-12 17:52:10 slapos[26041] INFO Checking promise novnc_promise.py...
2020-08-12 17:52:11 novnc_promise.py[26041] INFO port connection OK (('2001:67c:1254:61:a915::6e78', 6080))
2020-08-12 17:52:11 slapos[26041] INFO Checking promise qemu-virtual-machine-is-ready.py...
2020-08-12 17:52:13 qemu-virtual-machine-is-ready.py[26041] INFO OK '/srv/slapgrid/slappart9/srv/testnode/cvr/inst/test0-0/tmp/inst/adb0/bin/qemu-virtual-machine-is-ready' run with success
2020-08-12 17:52:13 slapos[26041] INFO Checking promise frontend_promise.py...
2020-08-12 17:52:13 frontend_promise.py[26041] INFO 'https://[2001:67c:1254:61:a915::6e78]:6080/vnc.html?auto=1&encrypt=1&password=tsquzdwf' is available
2020-08-12 17:52:13 slapos[26041] ERROR Promise 'vnc_promise.py' failed with output: ERROR while connecting to ('10.0.242.107', 5901): [Errno 111] Connection refused
2020-08-12 17:52:14 slapos[26041] INFO Finished computer partitions.
2020-08-12 17:52:14 slapos[26041] INFO ================================================================================
2020-08-12 17:52:14 slapos[26041] INFO Error with promises for the following partitions:
2020-08-12 17:52:14 slapos[26041] INFO   adb0[RootSoftwareInstance]: Promise 'vnc_promise.py' failed with output: ERROR while connecting to ('10.0.242.107', 5901): [Errno 111] Connection refused

It's because KVM is not available until the image is present, but download can take a lot of time.

The another option would be to use much smaller image.

[Jérome Perrin]

is it this image ?

curl -OL http://shacache.org/shacache/ce07873dbab7fa8501d1bf5565c2737b2eed6c8b9361b4997b21daf5f5d1590972db9ac00131cc5b27d9aa353f2f94071e073f9980cc61badd6d2427f592e6e8
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  2 1827M    2 43.5M    0     0   9.7M      0  0:03:06  0:00:04  0:03:02  9.7M^C⏎   

would be nice if we could use a smaller one, assuming that any image is OK for this test

[Łukasz Nowak]

would be nice if we could use a smaller one, assuming that any image is OK for this test

Yep, as I said

The another option would be to use much smaller image.

But I prefer to wait for @tomo for this, maybe there are some image requirements or we can tweak the test a bit.