We already had CA for LTE+LTE case. Let's also setup it for NR+NR and LTE+NR
cases as well. Add tests for everything.

Currently when setting up cells we allow users to input

- dl_earfcn for LTE, and
- dl_nr_arfcn and nr_band for NR

and from that lteenb automatically computes

- ul_earfcn for LTE, and
- ul_nr_arfcn and ssb_nr_arfcn for NR

everything kind of works out of the box for eNB case when there is simple SDR Radio Unit.

Then there are also the following cases:

1. we also need to set UL frequency when configuring Lopcomm RU, or any other ORAN-based Radio Unit.
2. we also need to specify DL/UL frequencies in MHz when configuring Lopcomm RU.
3. when configuring NR peercell it is required to set ssb_nr_arfcn. It is also
   required to set both dl_nr_arfcn and ul_nr_arfcn, and so far we were setting
   only DL one and using it for both assuming TDD band.
4. when configuring UEsim we need to specify both dl_nr_arfcn and ssb_nr_arfcn for NR cells.

So the problem is that even though lteenb automatically computes UL and SSB
frequencies, there is no way for us to reuse results of that computation for
scenarios outside of "simple enb" case.

As the result we kind of workaround that currently with exposing additional
parameters and asking users to look into enb, probably its `cell phy` output,
with the following:

     ---- 8< ---- (from UEsim)
     "ssb_nr_arfcn": {
       "title": "SSB NR ARFCN",
       "description": "SSB NR ARFCN, you can retrieve from ENB/GNB side",

or use additional parameters just for ul_earfcn and frequency-in-MHz:

     ---- 8< ---- (from ru/lopcomm)
     "txa0cc00_center_frequency": {
       "title": "Lopcomm ORAN DL Center Frequency in MHz (TXA0CC00)",
       "description": "Lopcomm ORAN Center Frequency in MHz (TXA0CC00)",
       "type": "number",
       "default": 2140
     },
     "rxa0cc00_center_frequency_earfcn": {
       "title": "Lopcomm ORAN UL Center Frequency EARFCN (RXA0CC00)",
       "description": "Lopcomm ORAN Center Frequency EARFCN (RXA0CC00)",
       "type": "number",
       "default": 18300
     },
     "rxa0cc00_center_frequency": {
       "title": "Lopcomm ORAN UL Center Frequency in MHz (RXA0CC00)",
       "description": "Lopcomm ORAN Center Frequency in MHz (RXA0CC00)",
       "type": "number",
       "default": 1950
     },

which prevents automation, opens the door for inconsistencies and puts the load
to resolve all that on users.

The root cause of the problem is that there is no way to access at
instantiation time what lteenb would compute internally. But DL<->UL conversion
and DL->SSB conversion is not a difficult task and we can do that on our own.

-> Do that here and solve all the problems listed above in one go.

For frequency computations we use my xlte.earfcn and xlte.nrarfcn modules and
upstream nrarfcn egg. For this xlte is updated(*) to primarily pick up

kirr/xlte@6cb9d37f
kirr/xlte@b8065120

and the rest of the conversion is in slaplte to use corresponding dl2ul and
dl2ssb routines.

For consistency ul_earfcn and ul_nr_arfcn now become input parameters for LTE
and NR cells, but optional. If they are absent - they are computed with
defaults, but a user can now control them explicitly. The same applies for
ssb_nr_arfcn.

This patch does not convert UEsim yet, because UEsim does not use rulib yet and
will be handled all in one go in a follow-up step.

The computation routines are thoroughly tested. First they have unit tests
inside XLTE itself, then we also update our tests in generic test/test.py here
with explicitly checking that correct numbers are emitted for UL and SSB
frequencies, and third I've also verified SSB computation results with respect
to https://tech-academy.amarisoft.com/OutOfBox_UEsim_SA.html#Tips_SSB_Frequency .

All this creates a base to be sure that the computations are correct and we are
indeed safe to switch our frequencies computation modules.

(*) full upgrade brings kirr/xlte@e716ab51...8e606c64 ;
    nrarfcn egg: https://pypi.org/project/nrarfcn/

So far in the unit tests we were only generating enb.cfg without semantically
processing as running lteenb is not possible on testnodes due to licensing
restrictions. That allowed to use arbitrary numbers, even invalid, for
frequencies without hitting an error.

However in the next patch we are going to start handling frequencies ourselves,
and invalid frequencies will lead to instantiation failures like

    ... in dl2ul(dl_earfcn)
    KeyError: 'no band that corresponds to DL EARFCN=325320'

Avoid that by switching to valid LTE and NR frequency numbers in the test.

Radio Units like Lopcomm ORAN require explicit activation for the radio.
Generalize this and introduce a generic parameter for explicit activation of
the radio on any RU. Default it to _INACTIVE_ so that radio happens only when
it is explicitly asked to be on for real. But preserve backward compatibility
for ORS which used to do "radio on" by default. disable_sdr becomes
ORS-specific parameter.

Lopcomm-specific txa0cc00_active and rxa0cc00_active go away and are replaced
by txrx_active usage.

I.e. apply not only to SDR but to all Radio Units and expecting that they apply
to them with the same uniform semantic of controlling Tx/Rx amplifiers on on
the RU.

- For SDR nothing is changed.
- For Lopcomm txa0cc00_gain goes away and is replaced by tx_gain. For rx_gain
  there is a question as I could not find offhand how to control aplifier on RX
  circuit there.
- For Sunwave tx_gain/rx_gain, as with lots of more(*) is TODO.

For example by publishing on RU the list of CELLs that are using it.

While at it and as we are introducing per-RU publication, fix a long-standing
TODO in ru/lopcomm to publish RU-specific information in per-RU way instead of
global.

Instead of publishing it on enb with prefixing by cell reference. It is more
logical to have cell input parameters on the cell instance and also have output
parameters about that cell published on that same instance.

For now it is all only what was input. But in the next patches we'll extend set
of published information with also computed things, so it won't remain trivial.

For example if we need to do 4T4R but have SDR boards with only 2T2R ports, we
need to combine two such boards into single RF port. Allow to do that via
changing SDR schema from taking sdr_dev into accepting sdr_dev_list and
adjusting the code everywhere correspondingly.

For ORS full backward compatibility is preserved via adjusting "ORS mode" proxy
correspondingly.

One example in slapos-render-config.py is changed from 2T2R to be 4T2R on such
RF port combined from two SDR boards. For that example the diff for rendered
enb.cfg is as follows:

    --- a/config/old/enb/iRU1_SDR_tLTE2_tNR/enb.cfg
    +++ b/config/out/enb/iRU1_SDR_tLTE2_tNR/enb.cfg
    @@ -8,7 +8,7 @@
       rf_driver: {
           // __CELL__a__ru 4T2R  (sdr)
           name: "sdr",
    -      args: "dev0=/dev/sdr0",
    +      args: "dev0=/dev/sdr0,dev1=/dev/sdr1",
       },
       tx_gain: [51, 51, 51, 51],
       rx_gain: [52, 52],

All other cases, including ORS, remain without any change.

We currently have at least the following problem:

1. a malfored shared instance reference leads to instantiation failure and
   potential buildout code injection. For example if reference of shared
   instance contains space then buildout fails with something like

       zc.buildout.configparser.ParsingError: File contains parsing errors: .../instance-enb.cfg
           [line 45]: '[promise-testing partition 0.RU-sdr-busy]\n'

   since, when requesting, it is possible to use arbitrary characters for
   references, including newline, it also opens the door for code injection attacks.

2. we currently use `json` directive of slapos.recipe.template e.g. as

     extra-context =
         json iru_dict   {{ rulib.iru_dict   | tojson }}

   this also potentially leads to instantiation failure if JSON(rulib.iru_dict)
   contains buildout control characters.

Solve this problems.

For 1 we develop "buildout encoding" that encodes arbitrary string into form
that is safe to be used as a name of buildout section. This encoding never fails
and does not loose information, which means it is safe to be applied
unconditionanly and there is no chance for two string inputs to result in the
same encoded form. The encoding also has practical property that it leaves most
of the strings, that we use in our buildot profiles, intact. Which means that
the result of the encoding is readable and the encoding can be applied almost
unconditionally to all strings without hurting instantiation. Then this
encoding is applied wherever reference of a shared instance is used to form
e.g. name of buildout section or inside other codes. We apply this encoding
universally - in config files too to avoid breakage there - e.g. in enb.cfg to
protect from e.g. \n being inserted in the middle of comment resulting in YAML
breakage. See documentation of added xbuildout.py for details.

For 2 we switch from `json` to using `dumps` on an object and referring that
dumps result via `key` directive.

See the following links for preliminary discussions with Jérome and Vincent on
this topic:

kirr/slapos@d5d4f7c8
nexedi/slapos@bf6b2ef3

As promised add tests for generic enb that exercise enb instantiation with
multiple Radio Units and cells in all kind of combinations:

   FDD|TDD x LTE|NR x SDR|Lopcomm|Sunwave

and not only one particular case, but verifying instantiation with many
combinations at the same time - verifying how cells of different kinds are
handled and how radio units of different kinds are handled too. Using multiple
cells on top of several radio units of the same type is verified too.

For Lopcomm we also exercise generated cu_config.xml a bit.

See added tests for details.

Enb/generic tests are bootstrapped now.

From now one every fix to enb/generic and every semantic change will go on only
if it is accompanied test change or enhancement.

software/ors-amarisoft: ru: amarisoft-stats.jinja2.py: Don't hardcode n_antenna_ul=2 on testing=True

This usually matches ORS expectations, but for testing generic enb it is
inappropriate and work work good, because in those tests there are more uplink
channels.

-> Fix the code to detect how many uplink channels are there automatically.

As promised in indicated TODO.

It is useful for debugging and for being able to take all the rendered files,
copy them to a callbox, and to be able to run lteenb directly on them.

Besides enb.cfg we need to also have those drb and asn23, because enb.cfg
references them.

Extend slapos-render-config to simulate not only what ORS does, but also
simulate multiple radio units and/or multiple cells with RU being SDR or
CPRI-based Lopcomm. It is useful as an example on how to configure things in
new "shared instances" MultiRU, and to verify handling of all those combinations.

software/ors-amarisoft: enb/generic: Switch configuration of RU, cell, peer and peercell to shared instances

Previously we had cell_list, ncell_list and peers parameters and we now remove
them and rework the software release to accept configuration for said objects
via shared instances. In other words now it is possible to add Radio Units,
cells, peers and peer cells on on top of main eNB service.

Schema for parameters of those shared instances is based on what we just recently
generalized and established in ru/ cell/ peer/ and peer/cell/ input schemas. We
only add cell_kind=enb|enb_peer addition field to be able to distinguish a cell
from peercell object, and we add a way for cell to reference RU one way or another.

RU-CELL relation is no longer 1-1: one RU can be generally serving multiple
cells. For example transmission bandwidth of SDR100 board is ~ 100MHz while
bandwidth of one LTE cell is max 20MHz. This way it is possible to put several
cells whose frequencies are nearby each other to be run on the same SDR
board(*). And in general one Radio Unit can be serving transmission/reception in
multiple frequency ranges, thus providing ability for multiple cells to be
served by RU. This way RU is split off from CELL object and each CELL needs to
say which RU object it wants to use.

There are 3 ways to configure CELL->RU links:

1. CELL references a RU object. This is the most general.
2. CELL embeds definition of RU object. This variant is provided for simplicity
   of management for users that do not want to split CELL/RU relations.
3. CELL2 references another cell object and says to use the same RU as CELL1.
   Again, this variant is provided for simplicity of management when users want to
   add a cell on top of same RU when they already started with "2".

Full backward compatibility is provided for ORS. For this instance-ors-enb.jinja2.cfg
is adjusted to inject synthetic RU/CELL/PEER/PEERCELL shared instances instead
of translating to slapparameter_dict. As the result for ORS users it works
without visible change as parameters with original ORS schema are handled as expected.

Rendered files for ORS also stay practically the same as before this patch.

The main change is in slaplte.jinja2 - in the loading routines. The other
changes are mostly straightforward adaptation because details of how ru_dict,
cell_dict, peer_dict and peercell_dict are loaded were already localized to
slaplte and the rest of the code independent from that.

Now, once we switched to shared instances, we will be finally able to add tests
for updated enb. Please see a soon follow-up patch for that.

And for existing test_ors.py we also temporary workaround breakage of tests,
because currently generic code does not handle well special characters in
partition references. For example it currently breaks on spaces with buildout
complaining that sections with spaces in their name are invalid. We will fix
that in another soon-followup patch as well, but apply a workaround for now.

(*) see https://tech-academy.amarisoft.com/SDR_MultiCell_OneSdr.html for details.

--------

Appendix. Diff for rendered ORS enb.cfg and gnb.cfg before and after this patch:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff config/{old,out}
```

```diff
diff --git a/config/old/ors/enb/enb.cfg b/config/out/ors/enb/enb.cfg
index 15f3b68e9..6046d366e 100644
--- a/config/old/ors/enb/enb.cfg
+++ b/config/out/ors/enb/enb.cfg
@@ -6,7 +6,7 @@

   // Radio Units
   rf_driver: {
-      // CELL-RU 2T2R  (sdr)
+      // RU 2T2R  (sdr)
       name: "sdr",
       args: "dev0=/dev/sdr0",
       rx_antenna:"tx_rx",
@@ -28,7 +28,7 @@
   // LTE cells
   cell_list: [

-    // CELL  (CELL-RU)
+    // CELL  (RU)
     {
       rf_port:      0,
       n_antenna_dl: 2,
@@ -46,7 +46,7 @@
         // Inter-ENB HO
         {
           rat:          "eutra",
-          cell_id:      0x12345,  // -> 1
+          cell_id:      0x12345,  // -> PEERCELL1
           n_id_cell:    35,
           dl_earfcn:    700,
           tac:          123,
diff --git a/config/old/ors/gnb/enb.cfg b/config/out/ors/gnb/enb.cfg
index ac564db6c..9473f3207 100644
--- a/config/old/ors/gnb/enb.cfg
+++ b/config/out/ors/gnb/enb.cfg
@@ -6,7 +6,7 @@

   // Radio Units
   rf_driver: {
-      // CELL-RU 2T2R  (sdr)
+      // RU 2T2R  (sdr)
       name: "sdr",
       args: "dev0=/dev/sdr0",
       rx_antenna:"tx_rx",
@@ -35,7 +35,7 @@
   // NR cells
   nr_cell_list: [

-      // CELL  (CELL-RU)
+      // CELL  (RU)
       {
         rf_port:      0,
         n_antenna_dl: 2,
@@ -58,7 +58,7 @@
         // Inter-ENB HO
         {
           rat:          "nr",
-          nr_cell_id:   0x77712, // -> 1
+          nr_cell_id:   0x77712, // -> PEERCELL2
           gnb_id_bits:  22,
           n_id_cell:    75,
           dl_nr_arfcn:  520000,
```

We already have peer cells in the setup of generated enb/gnb, but there were no
X2/XN peers there. Add them to be able to see how handling of peers works.

Rendered files change as expected:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff config/{old,out}
```

```diff
diff --git a/config/old/ors/enb/enb.cfg b/config/out/ors/enb/enb.cfg
index 252c2e379..15f3b68e9 100644
--- a/config/old/ors/enb/enb.cfg
+++ b/config/out/ors/enb/enb.cfg
@@ -22,7 +22,7 @@
     },
   ],
   gtp_addr: "127.0.1.1",
-  x2_peers: [],
+  x2_peers: ["44.1.1.1", "44.1.1.2"],
   enb_id: 0x1A2D0,

   // LTE cells
diff --git a/config/old/ors/gnb/enb.cfg b/config/out/ors/gnb/enb.cfg
index b401f3513..ac564db6c 100644
--- a/config/old/ors/gnb/enb.cfg
+++ b/config/out/ors/gnb/enb.cfg
@@ -22,7 +22,7 @@
     },
   ],
   gtp_addr: "127.0.1.1",
-  xn_peers: [],
+  xn_peers: ["55.1.1.1", "55.1.1.2"],
   gnb_id_bits: 28,
   gnb_id: 0x12345,
   en_dc_support: true,
```

The procedures to generate those are currently very similar, and that's why we
employ a shared `do` function to render them. But enb and ue rendering will
become different from each other in the next step.

-> Prepare to that by pushing necessary bits of do into do_enb and do_ue insides.

No change in rendered files.

software/ors-amarisoft: slapos-render-config: Retarget currently generated enb.cfg and gnb.cfg to out/ors/

Besides ORS we will soon add other cases to enb.cfg generation. It will be
handy to clearly see what is what and to keep ORS case to be also alive and distinguished.

Prepare to that by emitting generated enb.cfg and gnb.cfg to out/ors/enb/ and
out/ors/gnb instead of just out. Name the file itself enb.cfg always to match
what happens for real in the SR.

There is no change to generated files - it is only renaming:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/ors/enb/enb.cfg
similarity index 100%
rename from config/old/enb.cfg
rename to config/out/ors/enb/enb.cfg
diff --git a/config/old/gnb.cfg b/config/out/ors/gnb/enb.cfg
similarity index 100%
rename from config/old/gnb.cfg
rename to config/out/ors/gnb/enb.cfg
```

In other words merge enb and gnb into sole enb that can handle both LTE and NR
cells at the same time and simultaneously.

Previously which type of Radio Access Technology to use - LTE or NR - was
static parameter of particular software - it was possible to instantiate cells
only with selected RAT for particular template.

In MultiRU it is possible to generally instantiate all kind of cells -
LTE/NR and TDD/FDD all at the same time and each served by a different kind of Radio Unit.

-> Switch cell type to be runtime parameter as the final step to be able to do that.

For generic cell definitions are now required to come with cell_type=lte|nr
parameter. In schemas cell_list now includes oneOf for lte|nr cell instead of
particular cell type. Similar adaption is done for ncell_list and
x2_peers/xn_peers are merged into just peers with similar oneOf.

Global gnb-specific parameters of went-away gnb are also added as global ones to enb:

  - gnb_id
  - gnb_id_bits
  - amf_list
  - plmn_list (as plmn_list_5g not to conflict with plmn_list)
  - nssai

They must be present if there are NR cells.

Full backward compatibility is preserved for ORS: now it injects cells to
generic with particular cell_type annotations and translates ORS-specific
x2_peers/xn_peers and gnb/plmn_list to proper generic parameters.

In ORS-gnb schema gnb_stats_fetch_period and gnb_drb_stats_enabled had to be
renamed to enb_stats_fetch_period and enb_drb_stats_enabled, but once again,
full backward compatibility is preserved because ORS mode already had the
following in its proxy:

    {#- backward compatibility: if ORS is running in gnb mode, and gnb_* parameters
        are present, replace their generic enb_* counterparts with gnb_* ones #}
    ...

The diff for rendered enb.cfg and gnb.cfg, that simulate ORS, before and after
hereby patch is also empty.

As with Radio Units and cells organize a registry of peer cells and start to handle
that registry generally everywhere. The registry is still populated with only
LTE cells on enb and only NR cells on gnb, but the only place that is aware of
that is registry loading routine.

In the old code defaults were noticed for tac to be 1 if not provided, and so
those defaults are pushed to live in ORS now. ORS schemas are also updated
because that defaults were not declared there.

Generic switches to use peer/cell/lte directly which means that now it accepts
e_cell_id instead of cell_id as the name for full E-UTRAN cell identity. Full
backward compatibility is provided for ORS which implements translation of its
original schema to the new generic one.

Tests will be added later as full tests for generic MultiRU.

Rendered enb.cfg and gnb.cfg change a bit, but the changes are mostly due to
reordering of fields. The only change with semantic value is

    -          ssb_nr_arfcn: ,
    +          ssb_nr_arfcn: 520090,

in gnb.cfg .

The old configuration was invalid anyway because if it is only

              ssb_nr_arfcn: ,

then enb does not start with the following error:

    # ../lteenb gnb.cfg
    gnb.cfg:119: unexpected token: ','

Like with many other handover-related fields
https://tech-academy.amarisoft.com/lteenb.doc#prop.ncell_list.ssb_nr_arfcn
documents that ssb_nr_arfcn "Must be present" for Inter-gNB HO case. The old
code was erroneously doing `ncell.get(name, '')` with yielding empty string in
case a parameter is missing, which is corrected here.

--------

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff -w config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 1a2c41b6b..252c2e379 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -46,9 +46,9 @@
         // Inter-ENB HO
         {
           rat:          "eutra",
+          cell_id:      0x12345,  // -> 1
           n_id_cell:    35,
           dl_earfcn:    700,
-          cell_id:      0x12345,
           tac:          123,
         },
       ],
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 635c4baf2..b401f3513 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -58,14 +58,14 @@
         // Inter-ENB HO
         {
           rat:          "nr",
+          nr_cell_id:   0x77712, // -> 1
+          gnb_id_bits:  22,
+          n_id_cell:    75,
           dl_nr_arfcn:  520000,
-          ssb_nr_arfcn: ,
+          band:         38,
+          ssb_nr_arfcn: 520090,
           ul_nr_arfcn:  520000,
-          n_id_cell:    75,
-          gnb_id_bits:  22,
-          nr_cell_id:   0x77712,
           tac:          321,
-          band:         38,
           ssb_subcarrier_spacing: 30,
           ssb_period: 20,
           ssb_offset: 0,
```

Move schema of peer cells into under peer/cell/lte and peer/cell/nr with
keeping in mind that peer/cell will become a separate entity in MultiRU.

NR deduplication is straightforward, but for LTE original ncell_list used
`cell_id` name for full E-UTRAN cell identity. peer/cell/lte uses e_cell_id
name for that and content of that peer/cell/lte cannot be applied directly to
ORS's ncell_list and ncell_list in current enb/generic. To keep things working
we pickup peer/cell/lte field by field with mapping e_cell_id into cell_id. For
the reference for NR both original ncell_list and new peer/cell/nr use
nr_cell_id for full 5G cell identity.

Backward compatibility: nothing practically changes for ORS and generic.

Add X2 peers support for symmetry with XN peers.

As with Radio Units and cells organize a registry of peers and start to handle
that registry generally everywhere. The registry is still populated with only
LTE peers on enb and only NR peers on gnb, but the only place that is aware of
that is registry loading routine.

Tests will be added later as full tests for generic MultiRU.

Rendered gnb.cfg stays the same, but there is a small change in rendered enb
but the config remains with the previous semantic as before hereby patch:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff -w config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 5842dfc22..1a2c41b6b 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -22,6 +22,7 @@
     },
   ],
   gtp_addr: "127.0.1.1",
+  x2_peers: [],
   enb_id: 0x1A2D0,

   // LTE cells
```

Use $ref to share definitions taken from either generic enb or cell object.

software/ors-amarisoft: Add notices to ORS and generic softwares what they do and intended usage area

Both software.cfg and software-ors.cfg became trivial wrappers over their bases
- so move bases to be directly software.

After previous patch this rendering became unneeded. And after we stop
render-templates becomes unneeded too, so say goodbye to it as well.

software/ors-amarisoft: enb+ue: Switch to JSON schemas as the primary source of defaults; stop rendering them

JSON schemas for eNB/gNB and UE specify defaults for many parameters and the
software release use those defaults when parameters are not explicitly
specified on the instantiation.

Some primary defaults - for bandwidth and n_antenna DL/UL - were setup in
render-templates and propagated through all places to avoid duplicating them.

Defaults for many other parameters were duplicated in both JSON schemas and in
the code that handles those parameters.

To avoid this duplication we either need to extend render-templates and put
defaults for all the other many parameters there, or just switch to JSON
schemas to be primary source of those defaults, and use the schemas on
instantiation by automatically loading them and extracting defaults for all
parameters from there.

I decided to go the second way because it avoids additional layer of rendering
and the need to keep on remembering not to put raw defaults in the JSON
schemas. Evidently this is the same approach that Rapid CDN is going to take
(see nexedi/slapos!1380 for details).

In this patch we stop rendering JSON schemas and keep their last rendered
result. In the later patches we will handle formed duplication there and go on
to further merge enb and gnb.

UE is handled brutally for now, because it will be much reworked after.

Actually loading JSON schemas is left as TODO as I do not have resources to
implement that now. Still this step is needed as precondition for further
patches. I hope bit of duplication for enb/gnb parameters is ok, given that
they are now centrally maintained and corresponding TODO warning is in place.

software/ors-amarisoft: enb/generic: Move LTE and NR cell definitions to schemas under cell/lte/ and cell/nr/

And reuse bits from there for ORS cell-related parameters.

Preparatory step to handle cell as separate object and a bit less code duplication.

A bit less duplication.

For uniformity with NR which already has bandwidth specified as number in MHz.

For ORS mode everything stays as is which inputs bandwidth as string with MHz
suffix, e.g. "20 MHz" instead of number `20`.

We currently have 2 kinds of software:

- ORS, whose intended usage is small private networks, and
- generic, whose intended usage is small-to-medium networks.

and currently they both share defaults for their parameters inherited from ORS.
But what is appropriate for small private networks, might be not so appropriate
for medium networks.

For example ORS has default enb_id/gnb_id, but in a network with 100 nodes, it
is better to force every node to be explicitly assigned an id and error
otherwise. Similarly for cells ORS has defaults for cell_id, but in network
with many nodes, cell_id needs to be explicitly assigned. With having a default
there is also a chance to misspell the parameter name, and do not notice it
because the software will instantiate without an error but work incorrectly.

So in this patch we tighten the defaults for generic:

- enb_id, mme_list become non-optional for LTE
- gnb_id, afm_list become non-optional for NR
- plmn_list becomes non-optional for both LTE and NR. The format is different
  in between RATs and so later we will use plmn_list and plmn_list_5g when
  merging enb and gnb.

- the following parameters of LTE cells now needs to be explicitly configured:
  rf_mode, dl_earfcn, bandwidth, cell_id, pci, tac.
- the following parameters of  NR cells now needs to be explicitly configured:
  rf_mode, dl_nr_arfcn, nr_band, bandwidth, cell_id, pci.

We rework ORS mode to translate its own set of parameters and defaults into
generic enb/gnb parameters.

We similarly pull some global generic defaults into instance-enb.cfg.jinja2
with the idea that they could be maintained in one central place. In the future
it would be good to automatically load them from JSON schemas to avoid
duplication.

This patch should be backward compatible for ORS, but it introduces the change
in cell_id and pci in rendered enb.cfg which become 1 instead of 0:

    --- a/config/old/enb.cfg
    +++ b/config/out/enb.cfg
    @@ -33,9 +33,9 @@
           n_antenna_dl: 2,
           n_antenna_ul: 2,

    -      cell_id:    0x00,
    +      cell_id:    0x01,
           tac:        0x0001,
    -      n_id_cell:  0,
    +      n_id_cell:  1,
           root_sequence_index: 204,
           dl_earfcn:  36100,
           inactivity_timer: 10000,

The defaults for those parameters according to instance-enb-input-schema.json
are 0x01 and 1. And it looks like enb.jinja2.cfg was changed to emit them starting from
0 instead of 1 in c4d0958e due to probable thinko in that patch because before
that patch those parameters were emitted as 0x01 and 1:

    c4d0958e

and, once again, the schema says their defaults should be 0x01 and 1 as well.

The rest of the changes in rendered enb.cfg and gnb.cfg should not introduce
any semantic difference. Please see the appendix for full details.

--------

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff -w config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 89862f1d9..5842dfc22 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -33,9 +33,9 @@
       n_antenna_dl: 2,
       n_antenna_ul: 2,

-      cell_id:    0x00,
+      cell_id:    0x01,
       tac:        0x0001,
-      n_id_cell:  0,
+      n_id_cell:  1,
       root_sequence_index: 204,
       dl_earfcn:  36100,
       inactivity_timer: 10000,
@@ -102,8 +102,11 @@
   ],
   cell_default: {
     plmn_list: [
-      "00101",
-
+      {
+        plmn: "00101",
+        reserved: false,
+        attach_without_pdn: false,
+      },
     ],
     cyclic_prefix: "normal",

diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 1df699a22..635c4baf2 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -22,7 +22,7 @@
     },
   ],
   gtp_addr: "127.0.1.1",
-
+  xn_peers: [],
   gnb_id_bits: 28,
   gnb_id: 0x12345,
   en_dc_support: true,
@@ -142,6 +142,7 @@

   nr_cell_default: {
     ssb_period: 20,
+
     plmn_list: [
       {
         plmn: "00101",
@@ -151,10 +152,10 @@
           {
             sst: 1,
           },
-
         ],
       },
     ],
+
     si_window_length: 40,
     cell_barred: false,
     intra_freq_reselection: true,
```

Else enb treats args="" unset and implicitly uses /dev/sdr0 by default.

We do not want implicit default behaviour -> we want to see an error.

We currently have support for a3_offset and a3_time_to_trigger only, while
there is also a3_hysteresis plus at least a1_*, a2_* and probably other parameters.

Simple A3-only based meas_config tuning might be appropriate for ORS, but for
generic enb it is better to expose full control which I do not currently feel
good to implement right now. So do not expose it at all on generic to avoid
establishing a public API yet.

enb.jinja2.cfg already has corresponding comments:

  {#  TODO fully expose lte meas_config_desc in generic SR #}
    meas_config_desc: {
    ...

  {#  TODO fully expose nr meas_config_desc in generic SR #}
    meas_config_desc: {

- Indicate that it is not for ORS, but for generic case.
- Move ue to the end, to where it logically belongs.
- Indicate shared instances clearly with clear visually appearing indent.
  For now it is only about SIM card, but later we will be adding more kinds of
  shared instances, so it will be needed to clearly see on the UR if an item is
  a kind of service, or a shared instance of a service.

There is now only 2 kinds of softwares - ORS and generic, and their cfg.json
will diverge with ORS staying as it is now for backward compatibility, and
generic gaining more features and more kind of shared instances. That means
that it no longer makes sense to render those two from a common source.

-> Stop rendering them and leave previous rendered result in place for further manual edits.

Previously, e.g. for ORS it was:

    enb                 index=0
    gnb                 index=1
    core-network        index=2
    core-network-slave  index=3

which resulted in the following appearance order on SlapOS Master UI:

    gnb
    core-network
    core-network-slave
    enb

Note that enb comes last instead of being first as originally intended.

That happens because SlapOS Master treats index=0 as being unset and puts all
items with unset index in the end.

-> Fix this by starting explicitly-set indices from 1.

Now, for ORS, the order on UI is

    enb
    gnb
    core-network
    core-network-slave

As explained in the previous patch now once we have software.cfg and
software.cfg.json, json-schemas started to analyze them instead of skipping
software/ors-amarisoft/ completely, and it started to complain about missing
response schemas, for example:

    ERROR: test_ors-amarisoft_software_cfg_json (slapos.test.test_json_schema.TestJSONSchemaValidation)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 909, in resolve_from_url
        document = self.store[url]
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/_utils.py", line 28, in __getitem__
        return self.store[self.normalize(uri)]
    KeyError: 'file:///srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 1503, in open_local_file
        stats = os.stat(localfile)
    FileNotFoundError: [Errno 2] No such file or directory: '/srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 912, in resolve_from_url
        document = self.resolve_remote(url)
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 1018, in resolve_remote
        with urlopen(uri) as url:
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 214, in urlopen
        return opener.open(url, data, timeout)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 517, in open
        response = self._open(req, data)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 534, in _open
        result = self._call_chain(self.handle_open, protocol, protocol +
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 494, in _call_chain
        result = func(*args)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 1481, in file_open
        return self.open_local_file(req)
      File "/srv/slapgrid/slappart91/srv/runner/shared/python3/dd8479ce2a2b9dff51f7185535dd0403/lib/python3.9/urllib/request.py", line 1520, in open_local_file
        raise URLError(exp)
    urllib.error.URLError: <urlopen error [Errno 2] No such file or directory: '/srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'>

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/project/slapos/slapos/test/test_json_schema.py", line 72, in run
        resolver.resolve(software_type_definition['response'])
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 898, in resolve
        return url, self._remote_cache(url)
      File "/srv/slapgrid/slappart91/srv/runner/software/45a1e838784afb18daf420009d925c78/eggs/jsonschema-4.17.3-py3.9.egg/jsonschema/validators.py", line 914, in resolve_from_url
        raise exceptions.RefResolutionError(exc)
    jsonschema.exceptions.RefResolutionError: <urlopen error [Errno 2] No such file or directory: '/srv/slapgrid/slappart91/srv/project/slapos/software/ors-amarisoft/instance-ue-schema.json'>

-> Fix this and all other similar errors by adding stub response schemas for all objects that is possible to instantiate.

Previously json-schemas tests were passing on software/ors-amarisoft but the
status was misleading: since json-schemas check only software.cfg.json, and
until recently - before the patch to switch RU type to be a runtime parameter,
there was _no_ such file in here, nothing about software-*.cfg.json was checked at all.

But after the switch we started to have software.cfg and software.cfg.json
files which json schemas checks and errors start to appear.

Here we fix the following error popping up due to extra blank lines around "ue" entry in software.cfg.json:

    FAIL: test_ors-amarisoft_software_cfg_json_format (slapos.test.test_json_schema.TestJSONSchemaValidation)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/srv/slapgrid/slappart91/srv/project/slapos/slapos/test/test_json_schema.py", line 85, in run
        self.assertEqual(
    AssertionError: Lists differ: ['{',[145 chars]{', '    "ue": {', '      "title": "UE",', '  [1344 chars] '}'] != ['{',[145 chars]{', '', '    "ue": {', '      "title": "UE",',[1352 chars] '}']

    First differing element 5:
    '    "ue": {'
    ''

    Second list contains 2 additional elements.
    First extra element 48:
    '  }'

    Diff is 1703 characters long. Set self.maxDiff to None to see it.

Previously RU type was static parameter of particular software - it was
possible to instantiate cells only with selected RU type for particular template.

In MultiRU it will be possible to generally instantiate all kind of cells -
LTE/NR and TDD/FDD all at the same time and each served by a different kind of Radio Unit.

-> Switch RU to be runtime parameter as a preparatory step for that.

There is now only two software releases:

  - ORS (software-ors.cfg), and
  - generic (software.cfg).

ORS behaviour stays the same as before here by patch.

For generic each cell in cell_list now needs to explicitly specify which kind
of RU it wants to use and with which parameters. There are less defaults: for
example which CPRI board and SFP port to use now needs to be explicitly
specified.

Only ORS tests are left for now. The old test{LOPCOMM,M2RU,ANY} did not
exercised any driver-specific functionality and until recently were not run at
all due to '-' in their file name. We remove them for now and we will add tests
for generic in a soon follow-up patch after switching LTE/NR cell type to also
be a runtime parameter.

Backward compatibility:

  * nothing changes for ORS
  * for generic all RU-related parameters are now moved from cell_list to
    cell_list.ru, there are several renames and besides ru_type ru_link_type also
    needs to be specified. Please see new RU-related JSON schemas added into
    ru/ for details. Cell_list itself now also does not provide a default with
    one cell - if no cells are configured no cells are instantiated.

--------

Appendix. Diff for rendered enb.cfg and gnb.cfg before and after this patch:

```
$ ./pythonwitheggs slapos-render-config.py && xdiff -w config/{old,out}
```

```diff
diff --git a/config/old/enb.cfg b/config/out/enb.cfg
index 1b39f7044..89862f1d9 100644
--- a/config/old/enb.cfg
+++ b/config/out/enb.cfg
@@ -6,7 +6,7 @@

   // Radio Units
   rf_driver: {
-      // default-RU 2T2R  (ors)
+      // CELL-RU 2T2R  (sdr)
       name: "sdr",
       args: "dev0=/dev/sdr0",
       rx_antenna:"tx_rx",
@@ -27,7 +27,7 @@
   // LTE cells
   cell_list: [

-    // default  (default-RU)
+    // CELL  (CELL-RU)
     {
       rf_port:      0,
       n_antenna_dl: 2,
@@ -90,11 +90,11 @@
       srs_hopping_bandwidth: 0,
     },

-    drb_config: "default-drb.cfg",
+    drb_config: "CELL-drb.cfg",

     sib_sched_list: [
       {
-        filename: "default-sib23.asn",
+        filename: "CELL-sib23.asn",
         si_periodicity: 16,
       },
     ],
diff --git a/config/old/gnb.cfg b/config/out/gnb.cfg
index 05deb5ada..1df699a22 100644
--- a/config/old/gnb.cfg
+++ b/config/out/gnb.cfg
@@ -6,7 +6,7 @@

   // Radio Units
   rf_driver: {
-      // default-RU 2T2R  (ors)
+      // CELL-RU 2T2R  (sdr)
       name: "sdr",
       args: "dev0=/dev/sdr0",
       rx_antenna:"tx_rx",
@@ -35,7 +35,7 @@
   // NR cells
   nr_cell_list: [

-      // default  (default-RU)
+      // CELL  (CELL-RU)
       {
         rf_port:      0,
         n_antenna_dl: 2,
@@ -136,7 +136,7 @@
           ],
         },

-        drb_config: "default-drb.cfg",
+        drb_config: "CELL-drb.cfg",
       },
   ],
```

Previously rf_mode was static parameter of particular software - it was
possible to instantiate cells only of the mode selected for particular template.

In MultiRU it will be possible to generally instantiate all kind of cells -
LTE/NR and TDD/FDD all at the same time.

-> Switch rf_mode to be runtime parameter as a preparatory step for that.

Software for ORS becomes just software-ors.cfg with software-tdd-ors.cfg
providing backward compatibility proxy as we have many ORS'es currently
deployed with that software-release URL. For other softwares backward
compatibility is not preserved including for software-fdd-ors.cfg because we
practically have very few deployments with those.

To show tdd_ul_dl_config only for TDD we use conditional feature of new JSON-editor.

Backward compatibility: nothing changes for ORS, for everything else rf_mode
now needs to be explicitly set in cell parameters if it is not TDD.

Rendered enb.cfg and gnb.cfg stay the same.