Commit cd75648d authored by Kirill Smelkov's avatar Kirill Smelkov

format-json, test: Don't force ASCII

Currently, due to ensure_ascii=True default of json.dumps, we are
insisting on our JSON schemas to be ascii-only and all other characters
to be represented by \uxxxx escapes. So far this was not problematic as
all our schemas contains only ASCII characters, but upcoming
ors-amarisoft changes want to use e.g. "→" symbol:

    https://lab.nexedi.com/kirr/slapos/blob/b51f5523/software/ors-amarisoft/software.cfg.json#L15

which currently results in failure of json-schema test:

    FAIL: test_ors-amarisoft_software_cfg_json_format (slapos.test.test_json_schema.TestJSONSchemaValidation)
    ...

    First differing element 14:
    '      "title": "\\u2192  eNB/gNB | Radio Unit",'
    '      "title": "→  eNB/gNB | Radio Unit",'

And in general, in 2023 I think there is no reason to insist on our schemas to
be ASCII-only: say if one wants to describe something about "α" parameter. It
would be good to use that α character directly and seeing it in the editor,
instead of using escapes all the time.

As indicated by below stackoverflow answer "JSON spec requires UTF-8 support by
decoders": https://stackoverflow.com/a/594881/9456786 , and indeed checking
JSON specification also confirms that by default JSON decoders shall use UTF-8:

    https://datatracker.ietf.org/doc/html/rfc7159#section-8.1

This way, I think, we can switch to UTF-8 safely.

/reviewed-by @jerome, @lu.xu
/reviewed-on nexedi/slapos!1498
parent a12dac70
Pipeline #32130 passed with stage
...@@ -28,7 +28,7 @@ def main(): ...@@ -28,7 +28,7 @@ def main():
print(e, file=sys.stderr) print(e, file=sys.stderr)
else: else:
with open(f, 'w') as outfile: with open(f, 'w') as outfile:
json.dump(obj, outfile, sort_keys=False, indent=2, separators=(',', ': ')) json.dump(obj, outfile, ensure_ascii=False, sort_keys=False, indent=2, separators=(',', ': '))
outfile.write('\n') outfile.write('\n')
sys.exit(exit_code) sys.exit(exit_code)
......
...@@ -85,6 +85,7 @@ def createFormatTest(path): ...@@ -85,6 +85,7 @@ def createFormatTest(path):
self.assertEqual( self.assertEqual(
(json.dumps( (json.dumps(
json.loads(content, object_pairs_hook=collections.OrderedDict), json.loads(content, object_pairs_hook=collections.OrderedDict),
ensure_ascii=False,
sort_keys=False, sort_keys=False,
indent=2, indent=2,
separators=(',', ': ')) + "\n").splitlines(), separators=(',', ': ')) + "\n").splitlines(),
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment