There is now only 2 kinds of softwares - ORS and generic, and their cfg.json
will diverge with ORS staying as it is now for backward compatibility, and
generic gaining more features and more kind of shared instances. That means
that it no longer makes sense to render those two from a common source.

-> Stop rendering them and leave previous rendered result in place for further manual edits.

Previously, e.g. for ORS it was:

    enb                 index=0
    gnb                 index=1
    core-network        index=2
    core-network-slave  index=3

which resulted in the following appearance order on SlapOS Master UI:

    gnb
    core-network
    core-network-slave
    enb

Note that enb comes last instead of being first as originally intended.

That happens because SlapOS Master treats index=0 as being unset and puts all
items with unset index in the end.

-> Fix this by starting explicitly-set indices from 1.

Now, for ORS, the order on UI is

    enb
    gnb
    core-network
    core-network-slave

As explained in the previous patch now once we have software.cfg and
software.cfg.json, json-schemas started to analyze them instead of skipping
software/ors-amarisoft/ completely, and it started to complain about missing
response schemas, for example:

    ERROR: test_ors-amarisoft_software_cfg_json (slapos.test.test_json_schema.TestJSONSchemaValidation)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 909, in resolve_from_url
        document = self.store[url]
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/_utils.py", line 28, in __getitem__
        return self.store[self.normalize(uri)]
    KeyError: 'file:///srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 1503, in open_local_file
        stats = os.stat(localfile)
    FileNotFoundError: [Errno 2] No such file or directory: '/srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 912, in resolve_from_url
        document = self.resolve_remote(url)
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 1018, in resolve_remote
        with urlopen(uri) as url:
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 214, in urlopen
        return opener.open(url, data, timeout)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 517, in open
        response = self._open(req, data)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 534, in _open
        result = self._call_chain(self.handle_open, protocol, protocol +
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 494, in _call_chain
        result = func(*args)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 1481, in file_open
        return self.open_local_file(req)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 1520, in open_local_file
        raise URLError(exp)
    urllib.error.URLError: <urlopen error [Errno 2] No such file or directory: '/srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'>

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/project/slapos/slapos/test/test_json_schema.py", line 72, in run
        resolver.resolve(software_type_definition['response'])
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 898, in resolve
        return url, self._remote_cache(url)
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 914, in resolve_from_url
        raise exceptions.RefResolutionError(exc)
    jsonschema.exceptions.RefResolutionError: <urlopen error [Errno 2] No such file or directory: '/srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'>

-> Fix this and all other similar errors by adding stub response schemas for all objects that is possible to instantiate.

Previously json-schemas tests were passing on software/ors-amarisoft but the
status was misleading: since json-schemas check only software.cfg.json, and
until recently - before the patch to switch RU type to be a runtime parameter,
there was _no_ such file in here, nothing about software-*.cfg.json was checked at all.

But after the switch we started to have software.cfg and software.cfg.json
files which json schemas checks and errors start to appear.

Here we fix the following error popping up due to extra blank lines around "ue" entry in software.cfg.json:

    FAIL: test_ors-amarisoft_software_cfg_json_format (slapos.test.test_json_schema.TestJSONSchemaValidation)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/project/slapos/slapos/test/test_json_schema.py", line 85, in run
        self.assertEqual(
    AssertionError: Lists differ: ['{',[145 chars]{', '    "ue": {', '      "title": "UE",', '  [1344 chars] '}'] != ['{',[145 chars]{', '', '    "ue": {', '      "title": "UE",',[1352 chars] '}']

    First differing element 5:
    '    "ue": {'
    ''

    Second list contains 2 additional elements.
    First extra element 48:
    '  }'

    Diff is 1703 characters long. Set self.maxDiff to None to see it.

Previously RU type was static parameter of particular software - it was
possible to instantiate cells only with selected RU type for particular template.

In MultiRU it will be possible to generally instantiate all kind of cells -
LTE/NR and TDD/FDD all at the same time and each served by a different kind of Radio Unit.

-> Switch RU to be runtime parameter as a preparatory step for that.

There is now only two software releases:

  - ORS (software-ors.cfg), and
  - generic (software.cfg).

ORS behaviour stays the same as before here by patch.

For generic each cell in cell_list now needs to explicitly specify which kind
of RU it wants to use and with which parameters. There are less defaults: for
example which CPRI board and SFP port to use now needs to be explicitly
specified.

Only ORS tests are left for now. The old test{LOPCOMM,M2RU,ANY} did not
exercised any driver-specific functionality and until recently were not run at
all due to '-' in their file name. We remove them for now and we will add tests
for generic in a soon follow-up patch after switching LTE/NR cell type to also
be a runtime parameter.

Backward compatibility:

  * nothing changes for ORS
  * for generic all RU-related parameters are now moved from cell_list to
    cell_list.ru, there are several renames and besides ru_type ru_link_type also
    needs to be specified. Please see new RU-related JSON schemas added into
    ru/ for details. Cell_list itself now also does not provide a default with
    one cell - if no cells are configured no cells are instantiated.

--------

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff -w config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 1b39f7044..89862f1d9 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -6,7 +6,7 @@

   // Radio Units
   rf_driver: {
-      // default-RU 2T2R  (ors)
+      // CELL-RU 2T2R  (sdr)
       name: "sdr",
       args: "dev0=/dev/sdr0",
       rx_antenna:"tx_rx",
@@ -27,7 +27,7 @@
   // LTE cells
   cell_list: [

-    // default  (default-RU)
+    // CELL  (CELL-RU)
     {
       rf_port:      0,
       n_antenna_dl: 2,
@@ -90,11 +90,11 @@
       srs_hopping_bandwidth: 0,
     },

-    drb_config: "default-drb.cfg",
+    drb_config: "CELL-drb.cfg",

     sib_sched_list: [
       {
-        filename: "default-sib23.asn",
+        filename: "CELL-sib23.asn",
         si_periodicity: 16,
       },
     ],
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 05deb5ada..1df699a22 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -6,7 +6,7 @@

   // Radio Units
   rf_driver: {
-      // default-RU 2T2R  (ors)
+      // CELL-RU 2T2R  (sdr)
       name: "sdr",
       args: "dev0=/dev/sdr0",
       rx_antenna:"tx_rx",
@@ -35,7 +35,7 @@
   // NR cells
   nr_cell_list: [

-      // default  (default-RU)
+      // CELL  (CELL-RU)
       {
         rf_port:      0,
         n_antenna_dl: 2,
@@ -136,7 +136,7 @@
           ],
         },

-        drb_config: "default-drb.cfg",
+        drb_config: "CELL-drb.cfg",
       },
   ],
```

Previously rf_mode was static parameter of particular software - it was
possible to instantiate cells only of the mode selected for particular template.

In MultiRU it will be possible to generally instantiate all kind of cells -
LTE/NR and TDD/FDD all at the same time.

-> Switch rf_mode to be runtime parameter as a preparatory step for that.

Software for ORS becomes just software-ors.cfg with software-tdd-ors.cfg
providing backward compatibility proxy as we have many ORS'es currently
deployed with that software-release URL. For other softwares backward
compatibility is not preserved including for software-fdd-ors.cfg because we
practically have very few deployments with those.

To show tdd_ul_dl_config only for TDD we use conditional feature of new JSON-editor.

Backward compatibility: nothing changes for ORS, for everything else rf_mode
now needs to be explicitly set in cell parameters if it is not TDD.

Rendered enb.cfg and gnb.cfg stay the same.

Similarly with cell_list, organize a loop in under nr_cell_list and emit
configuration for each NR cell we have.

Tests will be added later as full tests for generic MultiRU.

Only trivial space and comments changes in rendered gnb.cfg

Backward compatibility: no change for ORS and everything else.

So far CA was setup only for lte case and it is possible to simplify generation
of all pair of cells to be done without extra loop and state.

Tests will be added later as full tests for generic MultiRU.

Backward compatibility: no change for ORS, which has only one cell, and
rendered enb.cfg and gnb.cfg stay the same.

For uniformity with LTE where it already is.

We do not expose it in ORS schema because it will be exposed only in generic
LTE and NR schemas, but for ORS schemas I prefer to keep them intact myself.

Tests will be added later as full tests for generic MultiRU.

Backward compatibility: no change for ORS and everything else.

Diff for rendered configs:

$ ./pythonwitheggs slapos-render-config.py && git diff -w --no-index config/{old,out}
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 5849bb924..b8a385a71 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -56,6 +56,7 @@
     subcarrier_spacing: 30,
     ssb_pos_bitmap: "10000000",

+    root_sequence_index: 1,
     inactivity_timer: 10000,

       // Handover
@@ -176,7 +176,6 @@
     intra_freq_reselection: true,
     q_rx_lev_min: -70,
     q_qual_min: -20,
-    root_sequence_index: 1,
     sr_period: 40,
     dmrs_type_a_pos: 2,
     prach: {

As with Radio Units organize a registry of cells and start to handle that
registry generally everywhere. Make parameters that configure cells, for
example tdd_ul_dl_config, dl_earfcn, bandwidth, pci, etc to be per-cell.

Cell registry is still populated from cell_list and slapparameter_dict.

Tests will be added later as full tests for generic MultiRU.

Rendered enb.cfg and gnb.cfg stay practically the same.

Backward compatibility: no change for ORS; For everything else cell parameters
stop to be global and are moved to cell_list.

--------

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff -w config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 5c6743c21..1b39f7044 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -1,6 +1,3 @@
-
-
-
 {
   log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
   log_filename: "log/enb.log",
@@ -93,11 +90,11 @@
       srs_hopping_bandwidth: 0,
     },

-    drb_config: "drb",
+    drb_config: "default-drb.cfg",

     sib_sched_list: [
       {
-        filename: "sib",
+        filename: "default-sib23.asn",
         si_periodicity: 16,
       },
     ],
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 6b04559c6..1fa637925 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -1,6 +1,3 @@
-
-
-
 {
   log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
   log_filename: "log/enb.log",
@@ -77,7 +74,6 @@
     manual_ref_signal_power: true,
     ss_pbch_block_power: 8,

-
     tdd_ul_dl_config: {
       pattern1: {
          period: 5, /* in ms */
@@ -148,7 +144,7 @@
       ],
     },

-    drb_config: "drb",
+    drb_config: "default-drb.cfg",
   },
   ],
   nr_cell_default: {
```

Most were already so, but sdr_number and cpri_mult were not. We need to be able
to use different SDR boards and with different CPRI speed, so make those
parameters to be per-RU as well.

Backward compatibility: no change for ORS; For CPRI-based cases sdr_number and cpri_mult become per-RU.

Like e.g. txa0cc00_center_frequency, rxa0cc00_center_frequency_earfcn, etc -
because different units needs to use different parameters.

cron_schedule is renamed to -> reset_schedule to better reflect its meaning.

user-authorized-key remains to be global as this is per-eNB nor per-RU
property, but is likely unneded in the future. Corresponding FIXME is added to
highlight this.

Backward compatibility: no change for ORS; lopcomm parameters become per-cell and need adaptation.

Add code to organize a registry of Radio Units and handle that registry generally everywhere.

RU registry is still populated from cell_list and in practice there still can
be radio units only of the same type, but besides slaplte.load_ru_and_cell
which is aware of that, the rest of the code tries to do everything as if
RUs of different type could be present simultaneously.

Now it is not only lopcomm who can do multiCELL, but also sunwave and SDR as well.

gNB also starts to gain support for cell_list, because cell_list loading is
uniformly applied to both eNB and gNB. However, since enb.cfg is not yet
prepared to handle multiple NR cells yet, there is an assert that in case of NR
there is only one RU/cell present there. We will remove this limitation in a
follow-up patch.

Later we will also change the loading to load RU descriptions from shared instances
and they won't be constrained to be Radio Units of the same type. But we need
to prepare a lot to be able to do that.

One more step forward towards MultiRU.

Tests will be added later as full tests for generic MultiRU.

Backward compatibility: no change for ORS and practically no breaking change for everything else.

--------

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch:

```
$ ./pythonwitheggs slapos-render-config.py && git diff -w --no-index config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 884483b0a..cafdf42be 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -1,24 +1,22 @@

-
 {
   log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
   log_filename: "log/enb.log",

-
+  // Radio Units
   rf_driver: {
+      // default-RU 2T2R  (ors)
       name: "sdr",
       args: "dev0=/dev/sdr0",
-
       rx_antenna:"tx_rx",
       tdd_tx_mod: 1,
   },
-  tx_gain: 62,
-  rx_gain: 43,
-
+  tx_gain: [62, 62],
+  rx_gain: [43, 43],
   com_addr: "127.0.1.2:9001",
   // LTE core network
   mme_list: [
@@ -36,6 +34,8 @@

   // LTE cells
   cell_list: [
+
+    // default  (default-RU)
     {
       rf_port:      0,
       n_antenna_dl: 2,
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index fd57ca3dc..7818b4ea8 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -1,24 +1,22 @@

-
 {
   log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
   log_filename: "log/enb.log",

-
+  // Radio Units
   rf_driver: {
+      // default-RU 2T2R  (ors)
       name: "sdr",
       args: "dev0=/dev/sdr0",
-
       rx_antenna:"tx_rx",
       tdd_tx_mod: 1,
   },
-  tx_gain: 62,
-  rx_gain: 43,
-
+  tx_gain: [62, 62],
+  rx_gain: [43, 43],
   com_addr: "127.0.1.2:9001",
   // NR core network
   amf_list: [
```

For LTE tdd_ul_dl_config is

    enum [
         "[Configuration 2] 5ms 2UL 6DL (default)",
         "[Configuration 6] 5ms 5UL 3DL (maximum uplink)"
    ],

but default was outside of that enum. The code in enb.jinja2.cfg actually uses
the first option from the enum as the default.

-> Fix default in schema.

tdd_ul_dl_config in NR seems to be already correct.

There is cell_list.*.cpri_port_number that is used insted from the beginning.

-> Remove unused cpri_port from scheamas.

Previously type of emulated UE was static parameter of particular software
release - it was possible to simulate UE and attach to cells only of the RAT of
particular template.

In MultiRU it will be possible to generally emulate all kind of UEs - both LTE
and NR all at the same time, and attach them to LTE and NR cells
simultaneously.

-> Switch type of UE to be runtime parameter as a preparatory step for that.

URLs of software releases changes (we remove lte and nr in names), but here we
do not care about backward compatibility because currently there are just a few
UEsim deployments and migration should be easy.

Less code duplication

SIM card object is useful to have not only for Core Network - we will also use
it for UE configutation. Move SIM schema into common place as a preparatory step.

K and IMSI are orthogonal to RAT and, similarly to how we keep default
n_antenna_dl/ul independent of RAT we can do so with this default UE parameters.

ORS does not support UEsim as indicated by `if bbu != ors` in software.cfg.json.jinja2.

Amends f3f1cb46 (software/ors-amarisoft: Start to introduce ORS mode) because
that patch started to really reject instantiating ue-* on ORS.

Those were unused because software.cfg.json.jinja2 explicitly enables UEsim only for bbu != 'ors'.

Soon we are going to make rf_mode a runtime parameter and so there
won't be TDD and FDD in software names anymore.

Prepare to that and make --delete to remove the exact set of files that
render-template actually generates instead of becoming a noop after "runtime
rf_mode" restructuring.

We are currently querying it in instance.cfg and later, once again, in e.g. instance-enb.jinja2.cfg .

However ORS mode will need to adjust slapparameter_dict with ORS defaults
and pass it further to generic enb. So it won't work if we use
slapparameter_dict obtained the second time because the second query will
return unadjusted original slapparameter_dict.

In this patch we are only doing preparatory step - redo the code not query the
master from instance-{enb,ue}.jinja2.cfg and work with slapparameter_dict and
slap_configuration already queried by instance.cfg

For UE the change is trivial.

For eNB instance-enb.jinja2.cfg used to set the defaults for com, mme, amf and
gtp in the same section used for the second query. We rework those defaults to
be applied to slapparameter_dict via jinja2 - via the same way we are going to
later use in ORS mode.

Handling defaults for everything besides gtp_addr is straightforward. For
gtp_addr it has the semantic difference when explicitly given, and given only
implicitly. In the latter case the intent of original code is to autodetect
whether to use an address on loopback, or externally-visible address.

Original code used the check for emptiness of mme_list/amf_list as the
condition for "use loopback". Since now those lists, after applying their
defaults, are never empty we rework the code to see if core address is on the
loopback or not, and use auto-GTP-on-loopback only if core is also on loopback.
This should, hopefully, be more convenient as it also works ok out of the box
if core is on loopback, but its address was explicitly specified. Previously
for such cases gtp_addr was also needed to be specified, and now it should work
without that.

No change to rendered enb.cfg and gnb.cfg besides whitespace.

Adjust ipv6-random usage in core-network for consistency with enb as well.

For example it is the port that is setup in ru/dnsmasq.jinja2.cfg that is used,
not :port and there it is 5354, i.e. different. :ip is also not used anywhere.

-> Remove them to avoid confusion.

That data structure is really a dict, not list. It is more clear to name it appropriately.

-> Do the renaming but keep cell_list intact as described in JSON schemas so
   that external interface remains unchanged.

Rendered enb.cfg and gnb.cfg remain the same.

Not only inside rendered enb.jinja2.cfg and ue.jinja2.cfg because we will need
that information in slaplte and slaplte is imported by instance-enb and
instance-ue.

instance-enb and instance-ue currently setup that slap_configuration with
LTE/NR defaults to be accessible from enb.jinja2.cfg and ue.jinja2.cfg. But we
will soon need to have access to those defaults from slaplte.jinja2 as well,
and it will break if left as is because when slaplte is imported from e.g.
enb.jinja2 - it will work, but when slaplte is imported from instance-enb it
will break because parent of instance-enb (instance.cfg) does not setup
defaults in slap_configuration at that level.

The fix is to either duplicate slap_configuration at instance.cfg level, or to
switch access to the defaults to go via original default_* parameters.

We go the second way for simplicity.

software/ors-amarisoft: enb.jinja2.cfg: Switch internal ru.ru_type to be 'sunwave' for Sunwave M2RU Radio Unit

ru/libinstance and everything inside there already refer to that unit as 'sunwave'.
Do the same for uniformity in enb.cfg and because we already refer to Lopcomm
ORAN Radio Unit as just 'lopcomm'.

In the future, if we will need to distinguish different models of one
manufacturer, we could extend ru type to be e.g. manufacturer/model or do
something similar.

Template rendering is still done with ru='m2ru' coming from outside but
internally it is now ru.ru_type='sunwave' instead of ru.ru_type='m2ru'.

Even though I introduced cell object in 79370ebf (software/ors-amarisoft:
enb.jinja2.cfg: Stop using C Preprocessor and switch to Jinja2 completely) it
is currently only cell.bandwidth that is living there, while all other cell
parameters are still fetched from slapparameter_dict directly.

We will soon introduce cell variable - that will iterate over cell_dict, and
also keeping global cell won't work due to name shadowing. On the other hand we
are not yet ready to start migrating all cell parameters to be accessible via
cell object.

-> So inline code for lte bandwidth parameter until the time comes to rework
all cell parameters to be accessible via cell object.

Rendered enb.cfg and gnb.cfg stay the same.

To handle multiple radio units we will need to rework this code significantly.
Move this to a dedicated routine as a preparatory step. The code moves out of
enb.cfg because later it will be also used in UEsim to configure simulator
radio units as well.

No non-whitespace changes in rendered enb.cfg and gnb.cfg .

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus

To run tapsplit we use plone.recipe.command with both command and
update-command set to `tapsplit ...`. But tapsplit, when run, fully
recreates and reinitializes subtap interfaces, which leads to
interfering with running enb because subtap interfaces, that enb
started to use, are removed. This is not desirable behaviour.

What we need:

1) create subtap interfaces only once and keep them stable
2) until configuration changes which should lead to
   * subtaps recreated, and
   * enb restarted

Carefully reading plone.recipe.command documentation shows:

  command
    Command to run when the buildout part is installed.

  update-command
    Command to run when the buildout part is updated. This happens when
    buildout is run BUT THE CONFIGURATION FOR THIS BUILDOUT PART HAS NOT
    CHANGED.

  (emphasis mine)

So the fix looks to be to make update-command noop - this fulfills
requirement "1". For "2" - I've verified that when configuration
changes, e.g. number of RU changes, buildout reinstalls [vtap] section
from scratch, and it also should restart enb, because generated enb.cfg
changes.

So the fix is fully correct because it satifies all needed requirements.

Amends 49ce8ef5 (software/ors-amarisoft: Provide dedicated TAP interface for each Radio Unit)
/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus

In f3f1cb46 I've made a mistake in "inject ORS defaults" code - there it should
be RF.dl_earfcn and RF.dl_nr_arfcn instead of RF.earfcn and RF.nr_arfcn.

Test only ors-amarisoft and from kirr/slapos@knext + kirr/slapos.toolbox@next .

Don't repeat slapparameter_dict['ncell_list'][k] - we can introduce a name for
current neighbour cell, and use that name in the loop. And align entries to
that they read more clearly.

No non-whitespace changes in rendered enb.cfg and gnb.cfg .

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus
/proposed-for-review-on nexedi/slapos!1526
/reviewed-by TrustMe

We currently have LTE-specific handover configuration in under cell_list and
NR-specific handover configuration in under nr_cell_list. Those configuration
are different.

However in upcoming MultiRU we will need to handle LTE->NR handover, NR->LTE
handover, and also Intra-ENB handover in addition to Inter-ENB handover we
currently do.

-> Move handover code into common function as a preparatory step for that.

In the future the handover code for LTE and NR cells will be the same, so it
makes sense to move that code to common place to avoid duplication.

For rendered enb.cfg this unification introduces only space and trivial
changes as shown in the appendix.

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus
/proposed-for-review-on nexedi/slapos!1528
/reviewed-by TrustMe

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch

```
$ git diff -w --no-index config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 43301ee13..9dcca16c7 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -45,16 +45,18 @@
       root_sequence_index: 204,
       dl_earfcn: 36100,
       inactivity_timer: 10000,
+
       // Handover
       ncell_list: [
         // Inter-ENB HO
         {
+          rat: "eutra",
           n_id_cell: 35,
           dl_earfcn: 700,
           cell_id: 0x12345,
           tac: 123,
-        }],
-
+        },
+      ],

       // Carrier Aggregation
       scell_list: [
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 2127a2f6b..23b07d6e1 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -57,6 +57,7 @@
     ssb_pos_bitmap: "10000000",

     inactivity_timer: 10000,
+
       // Handover
       ncell_list: [
         // Inter-ENB HO
@@ -74,8 +75,8 @@
           ssb_period: 20,
           ssb_offset: 0,
           ssb_duration: 1,
-      }],
-
+        },
+      ],

     // tune NR parameters for the cell
     manual_ref_signal_power: true,
```

Useful to have while doing handover-related changes.

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus
/proposed-for-review-on nexedi/slapos!1528
/reviewed-by TrustMe

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch

```
$ git diff --no-index config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 1843e0f24..43301ee13 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -45,6 +45,16 @@
       root_sequence_index: 204,
       dl_earfcn: 36100,
       inactivity_timer: 10000,
+      // Handover
+      ncell_list: [
+        // Inter-ENB HO
+        {
+          n_id_cell: 35,
+          dl_earfcn: 700,
+          cell_id: 0x12345,
+          tac: 123,
+        }],
+

       // Carrier Aggregation
       scell_list: [
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index d76b45d3c..2127a2f6b 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -57,6 +57,25 @@
     ssb_pos_bitmap: "10000000",

     inactivity_timer: 10000,
+    // Handover
+    ncell_list: [
+      // Inter-ENB HO
+      {
+        rat: "nr",
+        dl_nr_arfcn: 520000,
+        ssb_nr_arfcn: ,
+        ul_nr_arfcn: 520000,
+        n_id_cell: 75,
+        gnb_id_bits: 22,
+        nr_cell_id: 0x77712,
+        tac: 321,
+        band: 38,
+        ssb_subcarrier_spacing: 30,
+        ssb_period: 20,
+        ssb_offset: 0,
+        ssb_duration: 1,
+      }],
+
```

1. Currently we have separate log_options for LTE and NR cases with listing
different subsystems in each. But in MultiRU one enb will be driving both LTE
and NR cells at the same time, so we will need to define both LTE- and
NR-related levels.

-> Merge all log settings into one log_options as a preparatory step

For current state it does not hurt for an LTE if we set e.g. ngap.level, and it
does not hurt for NR if we set e.g. s1ap.level - since those layers will be
unused. This way merging log settings for both LTE and NR subsystems is ok.

--------

2. Factorize log_phy_debug handling: instead of duplicating whole
log_options line and changing only phy.level settings there, construct the
log_options line programmatically and handle phy.level on its own.

--------

3. Use log/enb.log log_filename for both LTE and NR cases. In the upcoming
MultiRU there might be several cells activated at the same time and in general
it will be not possible to say are we doing "enb" or "gnb" now - for example if
there will be two cells - one LTE and one NR.

-> Use enb.log for log filename uniformly similarly to how the software is
named (lteenb) even though it can work as both enb and gnb.

For the reference we do the same with enb.xlog in nexedi/slapos!1522 .

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus
/proposed-for-review-on nexedi/slapos!1527
/reviewed-by TrustMe

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch

```
$ git diff --no-index config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 467bb6364..1843e0f24 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -3,9 +3,7 @@

 {
-
-  log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
-
+  log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
   log_filename: "log/enb.log",

diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 18523818a..d76b45d3c 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -3,10 +3,8 @@

 {
-
-  log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,rrc.level=debug,rrc.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
-
-  log_filename: "log/gnb.log",
+  log_options: "all.level=error,all.max_size=0,nas.level=debug,nas.max_size=1,s1ap.level=debug,s1ap.max_size=1,x2ap.level=debug,x2ap.max_size=1,rrc.level=debug,rrc.max_size=1,ngap.level=debug,ngap.max_size=1,xnap.level=debug,xnap.max_size=1,phy.level=info,file.rotate=1G,file.path=/dev/null",
+  log_filename: "log/enb.log",

   rf_driver: {
```

Current enb config is already quite complex and with MultiRU it will be growing
more - both with added features and with more sections emitted because there
will be multiple radio units, multiple cells and cross cell interactions. So
for clarity we will want to annotate with a comment to which cell or ru object
a section belongs, or to which cell-cell pair a particular interaction belongs.

Amarisoft supports C-style comments and preprocessor directives out of the box,
but if we use them in the configuration files, yaml.load, that we use in the
test to load generated configs, will break, because // and /* ... */ is not
valid YAML. It looks like Amarisoft does preprocessing as a separate step
before further loading given configuration via yaml.

So to be able to use the comments and still have tests working we need to do
the same - in the tests preprocess the files before feeding them to yaml loader.

-> Do that with the help of https://pypi.org/project/pcpp/

In my view that library has good quality and in my experience it worked
flawlessly. Anyway we need it to only handle comments, not sophisticated CPP
features, and for that it works just ok.

Add some comments to existing enb.cfg and ue.cfg to make sure it really works.
Those are simple comments, and in current state it they might seem as not 100%
necessary, but with more upcoming config changes it would be good to have those
descriptionary anchors present in the generated configs, so I suggest we add them.
Anyway they should not do any harm at all.

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus
/proposed-for-review-on nexedi/slapos!1526
/reviewed-by TrustMe

Currently on every test yaml.load issues a warning that using it without
explicitly specifying loader is deprecated, for example:

    test_enb_conf (testTDD.TestENBParameters) .../slapos/software/ors-amarisoft/test/test.py:29: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.

-> Fix the warning by using appropriate loader explicitly.

Use FullLoader since msg.pyyaml.org/load describes it as

   Loads the full YAML language. Avoids arbitrary code execution.
   This is currently (PyYAML 5.1+) the default loader called by yaml.load(input) (after issuing the warning).

i.e. we get support for full YAML feature set, do not get code execution and
make sure that the behaviour stays exactly the same as before, but without the
warning.

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus
/proposed-for-review-on nexedi/slapos!1526
/reviewed-by TrustMe

We are going to do fixes and enhancements to YAML loading processes. Move
corresponding code to one place before that as the preparatory step.

Place new utility in new test/test.py file - with MultiRU that will be the
place for tests of generic mode and test.jinja2.py, which will be renamed to
test_ors.py, will be using utilities from there. So place the utility into new
place from the start.

/cc @jhuge, @lu.xu, @tomo, @xavier_thompson, @Daetalus
/proposed-for-review-on !1526
/reviewed-by TrustMe