-
Kirill Smelkov authored
Add to neotest zbench-local and zbench-cluster commands that perform ZODB benchmarks on FileStorage, ZEO and NEO with Python and Go clients either locally, or with a server and client running on 2 different nodes. There are 2 client programs: tzodb.py and tzodb.go which for now compute hash of whole latest objects stream in a ZODB database. On server side neotest is taught to launch ZEO and various NEO clusters and to execute client load on them. Two test datasets are used: wczblk1-8 - the dataset with wendelin.core ZBlk1 objects covering 8M array, and prod1-1024 - synthethic dataset that tries to represent regular ERP5 instance. Both datasets are very small and so we can assume they reside completely in server disk cache while running benchmarks. Benchmark timings will thus give pure storage software processing latency, as pagecache hit time is on par, or less, to 1µs. Example output: x/src/lab.nexedi.com/kirr/neo/go/neo/t$ ./neotest zbench-local dataset: wczblk1-8 node: cluster: deco *** generating fs1 data... I: RAM: 7.47GB I: WORK: 0.01GB gen signal t=0...1.05e+06 float64 (= 0.01GB) gen signal blk [0:1048576] (100.0%) VIRT: 297 MB RSS: 48MB *** generating sqlite data... I: RAM: 7.47GB I: WORK: 0.01GB gen signal t=0...1.05e+06 float64 (= 0.01GB) gen signal blk [0:1048576] (100.0%) VIRT: 386 MB RSS: 58MB 2018-07-10 19:57:35.7065 ERROR NEO [ app: 91] primary master is down Cluster state changed *** generating sql data... 2018-07-10 19:57:35 140115116649600 [Note] /usr/sbin/mysqld (mysqld 10.1.29-MariaDB-6+b1) starting as process 27574 ... 2018-07-10 19:57:39 140205509999744 [Note] /usr/sbin/mysqld (mysqld 10.1.29-MariaDB-6+b1) starting as process 27603 ... 2018-07-10 19:57:42 139692109810816 [Note] /usr/sbin/mysqld (mysqld 10.1.29-MariaDB-6+b1) starting as process 27633 ... 2018-07-10 19:57:45 139759221546112 [Note] mysqld (mysqld 10.1.29-MariaDB-6+b1) starting as process 27662 ... I: RAM: 7.47GB I: WORK: 0.01GB gen signal t=0...1.05e+06 float64 (= 0.01GB) gen signal blk [0:1048576] (100.0%) VIRT: 387 MB RSS: 59MB 2018-07-10 19:57:48.2565 ERROR NEO [ app: 91] primary master is down Cluster state changed *** FileStorage Benchmarkfs1-zhash.py 2127 16.3 µs/object # crc32:14640593 nread=8540363 t=0.035s # POLL·2 C1·73 C1E·38 C3·12 C6·36 C7s·0 C8·112 C9·0 C10·62 Benchmarkfs1-zhash.py 2127 16.6 µs/object # crc32:14640593 nread=8540363 t=0.035s # POLL·0 C1·113 C1E·21 C3·16 C6·56 C7s·0 C8·136 C9·0 C10·41 Benchmarkfs1-zhash.py 2127 15.9 µs/object # crc32:14640593 nread=8540363 t=0.034s # POLL·0 C1·71 C1E·36 C3·22 C6·50 C7s·0 C8·167 C9·0 C10·47 Benchmarkfs1-zhash.py 2127 15.9 µs/object # crc32:14640593 nread=8540363 t=0.034s # POLL·0 C1·77 C1E·32 C3·11 C6·55 C7s·0 C8·184 C9·0 C10·31 Benchmarkfs1-zhash.py 2127 16.0 µs/object # crc32:14640593 nread=8540363 t=0.034s # POLL·0 C1·78 C1E·15 C3·12 C6·51 C7s·0 C8·140 C9·0 C10·44 # 16 clients in parallel Benchmarkfs1-zhash.py·P16 2127 129.0 µs/object # crc32:14640593 nread=8540363 t=0.274s Benchmarkfs1-zhash.py·P16 2127 132.6 µs/object # crc32:14640593 nread=8540363 t=0.282s Benchmarkfs1-zhash.py·P16 2127 135.0 µs/object # crc32:14640593 nread=8540363 t=0.287s Benchmarkfs1-zhash.py·P16 2127 135.3 µs/object # crc32:14640593 nread=8540363 t=0.288s Benchmarkfs1-zhash.py·P16 2127 136.6 µs/object # crc32:14640593 nread=8540363 t=0.291s Benchmarkfs1-zhash.py·P16 2127 122.8 µs/object # crc32:14640593 nread=8540363 t=0.261s Benchmarkfs1-zhash.py·P16 2127 130.9 µs/object # crc32:14640593 nread=8540363 t=0.279s Benchmarkfs1-zhash.py·P16 2127 126.4 µs/object # crc32:14640593 nread=8540363 t=0.269s Benchmarkfs1-zhash.py·P16 2127 125.8 µs/object # crc32:14640593 nread=8540363 t=0.268s Benchmarkfs1-zhash.py·P16 2127 108.3 µs/object # crc32:14640593 nread=8540363 t=0.230s Benchmarkfs1-zhash.py·P16 2127 131.0 µs/object # crc32:14640593 nread=8540363 t=0.279s Benchmarkfs1-zhash.py·P16 2127 124.1 µs/object # crc32:14640593 nread=8540363 t=0.264s Benchmarkfs1-zhash.py·P16 2127 129.3 µs/object # crc32:14640593 nread=8540363 t=0.275s Benchmarkfs1-zhash.py·P16 2127 125.0 µs/object # crc32:14640593 nread=8540363 t=0.266s Benchmarkfs1-zhash.py·P16 2127 131.5 µs/object # crc32:14640593 nread=8540363 t=0.280s Benchmarkfs1-zhash.py·P16 2127 131.4 µs/object # crc32:14640593 nread=8540363 t=0.280s # POLL·0 C1·4 C1E·13 C3·11 C6·79 C7s·0 C8·14 C9·0 C10·0 ... And its summary via benchstat: x/src/lab.nexedi.com/kirr/neo/go/neo/t$ benchstat -split node,cluster,dataset x.log name time/object cluster:deco dataset:wczblk1-8 fs1-zhash.py 16.1µs ± 3% fs1-zhash.py·P16 130µs ± 5% fs1-zhash.go 3.00µs ±10% fs1-zhash.go+prefetch128 3.40µs ±18% fs1-zhash.go·P16 10.2µs ±71% zeo/py/fs1-zhash.py 336µs ± 3% zeo/py/fs1-zhash.py·P16 3.22ms ± 6% zeo/py/fs1-zhash.go 112µs ± 2% zeo/py/fs1-zhash.go+prefetch128 60.9µs ± 1% zeo/py/fs1-zhash.go·P16 1.07ms ± 5% neo/py(!log)/sqlite·P1-zhash.py 291µs ± 2% neo/py(!log)/sqlite·P1-zhash.py·P16 2.86ms ± 1% neo/py(!log)/sql·P1-zhash.py 318µs ± 4% neo/py(!log)/sql·P1-zhash.py·P16 3.99ms ± 0% cluster:deco dataset:prod1-1024 fs1-zhash.py 12.3µs ± 1% fs1-zhash.py·P16 106µs ±10% fs1-zhash.go 2.56µs ±10% fs1-zhash.go+prefetch128 2.68µs ± 8% fs1-zhash.go·P16 9.48µs ±43% zeo/py/fs1-zhash.py 319µs ± 3% zeo/py/fs1-zhash.py·P16 3.13ms ± 3% zeo/py/fs1-zhash.go 101µs ± 5% zeo/py/fs1-zhash.go+prefetch128 56.9µs ± 1% zeo/py/fs1-zhash.go·P16 1.19ms ± 4% neo/py(!log)/sqlite·P1-zhash.py 281µs ± 3% neo/py(!log)/sqlite·P1-zhash.py·P16 2.80ms ± 1% neo/py(!log)/sql·P1-zhash.py 316µs ± 1% neo/py(!log)/sql·P1-zhash.py·P16 3.91ms ± 1% Since there is no NEO/go support yet, corresponding neotest parts are merged, but commented-out with appropriate remark. Parallel access is simulated with spawning many OS processes for now. This will change in the nearby followup patch to zwrk. Results of ZODB benchmarking were discussed in http://navytux.spb.ru/~kirr/neo.html#performance-tests , and http://navytux.spb.ru/~kirr/neo.html#results-and-discussion Some draft history related to this patch: lab.nexedi.com/kirr/neo/commit/e0d875bc X neotest: Teach it to benchmark NEO with storage partitioned to several nodes lab.nexedi.com/kirr/neo/commit/590f0a46 X neo/py uses n(replica) as n(real-replica) - 1 lab.nexedi.com/kirr/neo/commit/b655da26 X save time not benchmarking things we do not show lab.nexedi.com/kirr/neo/commit/f834f40d X zhash: Show N(obj) read, not 1, in place of N(iter) lab.nexedi.com/kirr/neo/commit/a16e8d52 X teach golang to access ZEO lab.nexedi.com/kirr/neo/commit/b9827725 X switch to using no compression, because this way it is more fair for comparing storage latencies lab.nexedi.com/kirr/neo/commit/c0067335 X neotest: Don't depend on killall lab.nexedi.com/kirr/neo/commit/2bcd6ebb X neotest: add zbench-local & zbench-cluster subcomands lab.nexedi.com/kirr/neo/commit/fb165ad9 X neotest: Also benchmark NEO/py with logging disabled lab.nexedi.com/kirr/neo/commit/2118ba38 X neotest: Help mysqlk_install_db find its basedir under SlapOS lab.nexedi.com/kirr/neo/commit/80eaa05e X zgenprod1 tool lab.nexedi.com/kirr/neo/commit/eb0e516f X check hash result and error if mismatch (zhash.* part); neotest part pending lab.nexedi.com/kirr/neo/commit/046370db X benchify rest of bench-cluster lab.nexedi.com/kirr/neo/commit/2d13818e X bench-local + zhash: Add output in std bench format lab.nexedi.com/kirr/neo/commit/1d692a3b X add NEO/go with SHA1 disabled (both Sgo and Cgo to regular benchmarks)
3f578560