Commit 3fa0108e authored by Kirill Smelkov's avatar Kirill Smelkov

Add custom Dict that mirrors Python dict behaviour

Ogórek currently represents unpickled dict via map[any]any, which is
logical, but also exhibits issues because builtin Go map behaviour is
different from Python's dict behaviour. For example:

- Python's dict allows tuples to be used in keys, while Go map
  does not (https://github.com/kisielk/og-rek/issues/50),

- Python's dict allows both long and int to be used interchangeable as
  keys, while Go map does not handle *big.Int as key with the same
  semantic (https://github.com/kisielk/og-rek/issues/55)

- Python's dict allows to use numbers interchangeable in keys - all int
  and float, but on Go side int(1) and float64(1.0) are considered by
  builtin map as different keys.

- In Python world bytestring (str from py2) is considered to be related
  to both unicode (str on py3) and bytes, but builtin map considers all
  string, Bytes and ByteString as different keys.

- etc...

All in all there are many differences in behaviour in builtin Python
dict and Go map that result in generally different semantics when
decoding pickled data. Those differences can be fixed only if we add
custom dict implementation that mirrors what Python does.

-> Do that: add custom Dict that implements key -> value mapping with
   mirroring Python behaviour.

For now we are only adding the Dict class itself and its tests.
Later we will use this new Dict to handle decoding dictionaries from the pickles.

For the implementation we use github.com/aristanetworks/gomap which
provides extraction of builtin go map code wrapped into generic type
Map[Key,Value] that accepts custom equal and hash functions. And it is
those equal and hash functions via which we define equality behaviour to
be the same as on Python side.

Please see documentation in the added code and tests for details.
parent 78a0a4b9
This diff is collapsed.
This diff is collapsed.
module github.com/kisielk/og-rek
go 1.18
require github.com/aristanetworks/gomap v0.0.0-20230726210543-f4e41046dced
require golang.org/x/exp v0.0.0-20230725093048-515e97ebf090 // indirect
github.com/aristanetworks/gomap v0.0.0-20230726210543-f4e41046dced h1:HxlRMDx/VeRqzj3nvqX9k4tjeBcEIkoNHDJPsS389hs=
github.com/aristanetworks/gomap v0.0.0-20230726210543-f4e41046dced/go.mod h1:p7lmI+ecoe1RTyD11SPXWsSQ3H+pJ4cp5y7vtKW4QdM=
golang.org/x/exp v0.0.0-20230725093048-515e97ebf090 h1:Di6/M8l0O2lCLc6VVRWhgCiApHV8MnQurBnFSHsQtNY=
golang.org/x/exp v0.0.0-20230725093048-515e97ebf090/go.mod h1:FXUEEKJgO7OQYeo8N01OfiKP8RXMtf6e8aTskBGqWdc=
//go:build !go1.21
package ogórek
import (
"math/big"
)
func bigInt_Float64(b *big.Int) (float64, big.Accuracy) {
return new(big.Float).SetInt(b).Float64()
}
//go:build go1.21
package ogórek
import (
"math/big"
)
func bigInt_Float64(b *big.Int) (float64, big.Accuracy) {
return b.Float64()
}
//go:build !go1.19
package ogórek
import (
"hash/maphash"
)
func maphash_String(seed maphash.Seed, s string) uint64 {
var h maphash.Hash
h.SetSeed(seed)
h.WriteString(s)
return h.Sum64()
}
//go:build go1.19
package ogórek
import (
"hash/maphash"
)
func maphash_String(seed maphash.Seed, s string) uint64 {
return maphash.String(seed, s)
}
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment