1. 03 Nov, 2020 5 commits
  2. 01 Nov, 2020 35 commits
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · a179fa40
      Kirill Smelkov authored
      * master:
      a179fa40
    • Kirill Smelkov's avatar
      Sync with NEO/py v1.12-13-gf2ea4be2 (oldproto branch) · d14040ac
      Kirill Smelkov authored
      * origin/old-proto:
        qa: skip broken ZODB test
        client: fix race with invalidations when starting a new transaction on ZODB 5
        Code clean-up, comment fixes
        master: fix crash in STARTING_BACKUP when connecting to an upstream secondary master
        mysql: workaround for MDEV-20693
        client: inline Application._loadFromCache
        client: replace global load lock by a per-oid one
        client: unindent code
        client: remove load lock in tpc_finish
        qa: check cache in testExternalInvalidation
        qa: comment testExternalInvalidation2
      d14040ac
    • Kirill Smelkov's avatar
      Sync with NEO/py v1.12-13-gf2ea4be2 (oldproto branch) · 3a6a3475
      Kirill Smelkov authored
      * origin/old-proto:
        qa: skip broken ZODB test
        client: fix race with invalidations when starting a new transaction on ZODB 5
        Code clean-up, comment fixes
        master: fix crash in STARTING_BACKUP when connecting to an upstream secondary master
        mysql: workaround for MDEV-20693
        client: inline Application._loadFromCache
        client: replace global load lock by a per-oid one
        client: unindent code
        client: remove load lock in tpc_finish
        qa: check cache in testExternalInvalidation
        qa: comment testExternalInvalidation2
      3a6a3475
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · a4bf053e
      Kirill Smelkov authored
      * master:
        go/neo/proto: Version 5 -> 6
        go/neo/proto: tweak += dry-run
        go/neo/proto: ErrorCode += DENIED
        go/neo/proto: AnswerPartitionList += NumReplicas
        go/neo/proto: Sync with NEO/py on changes to make the number of replicas modifiable at runtime
        go/neo/proto: RequestIdentification += NewNID; Move .DevPath
      a4bf053e
    • Kirill Smelkov's avatar
      Sync with NEO/py v1.12 · 8ea85c71
      Kirill Smelkov authored
      8ea85c71
    • Kirill Smelkov's avatar
      go/neo/proto: Version 5 -> 6 · d4c2253f
      Kirill Smelkov authored
      This protocol version corresponds to protocol version used by NEO/py
      v1.12 and was set in NEO/py commit c6453626 (Bump protocol version).
      
      The protocol definition was updated to match that NEO/py release in
      the previous patches.
      d4c2253f
    • Kirill Smelkov's avatar
      go/neo/proto: tweak += dry-run · 595052c2
      Kirill Smelkov authored
      This corresponds to NEO/py commit 2a27239d (tweak: add option to
      simulate).
      595052c2
    • Kirill Smelkov's avatar
      go/neo/proto: ErrorCode += DENIED · 484c34a5
      Kirill Smelkov authored
      This corresponds to NEO/py commit c2c9e99d (Better error reporting from
      the master to neoctl for denied requests).
      484c34a5
    • Kirill Smelkov's avatar
      go/neo/proto: AnswerPartitionList += NumReplicas · 56835167
      Kirill Smelkov authored
      This corresponds to NEO/py commit 21190ee7 (Make 'neoctl print pt'
      report the number of replicas).
      56835167
    • Kirill Smelkov's avatar
      go/neo/proto: Sync with NEO/py on changes to make the number of replicas modifiable at runtime · 0d75e144
      Kirill Smelkov authored
      This corresponds to NEO/py commit ef5fc508 (Make the number of replicas
      modifiable when the cluster is running).
      
      One important change in the protocol is that Client no longer queries
      Master for partition table - instead M pushed partTab to C right after
      identification (after pushing nodeTab).
      
      See also: https://neo.nexedi.com/P-NEO-Protocol.Specification.2019?portal_skin=CI_slideshow#/9/5
      0d75e144
    • Kirill Smelkov's avatar
      go/neo/proto: RequestIdentification += NewNID; Move .DevPath · 98869a9b
      Kirill Smelkov authored
      This corresponds to NEO/py commit 27e3f620 (New --new-nid storage option
      for fast cloning).
      98869a9b
    • Kirill Smelkov's avatar
      Merge tag v1.12 · c85756a0
      Kirill Smelkov authored
      NEO 1.12
      
      * tag 'v1.12': (28 commits)
        Release version 1.12
        master: reject drop/tweak ctl commands that could lead to unwanted status
        qa: extend test reproducing the migration of a big ZODB to NEO
        neoctl: better display of full partition tables
        Bump protocol version
        tweak: add option to simulate
        tweak: do not crash when trying to remove all nodes
        tweak: do not touch cells of nodes that are intended to be dropped
        Better error reporting from the master to neoctl for denied requests
        Make 'neoctl print pt' report the number of replicas
        Make the number of replicas modifiable when the cluster is running
        New --new-nid storage option for fast cloning
        qa: fix 2 tests with ZODB5
        qa: new tools/stress options to evaluate MySQL engines
        qa: provide a way to let tests start 1 mysqld per storage node
        mysql: make 'user' actually optional in the DB connection string
        mysql: specify column families for RocksDB
        qa: add testIncremental (testImporter) test
        importer: fix hidden "maximum recursion depth exceeded" at startup
        importer: fix closure of ZODB, and also do it when the import is finished
        sqlite: fix resumption of migration to NEO with Importer
        qa: fix a random failure in threaded tests
        importer: speed up startup when the import is already finished
        importer: fix replication (as source) once import is finished
        storage: fix DatabaseManager.getLastTID with max_tid
        qa: remove 2 useless unit tests
        storage: allow the master to change our node id
        Rename --uuid command-line options into --nid
        importer: fix possible data loss on writeback
      c85756a0
    • Kirill Smelkov's avatar
      Sync with NEO/py v1.12 · f4b87d77
      Kirill Smelkov authored
      f4b87d77
    • Kirill Smelkov's avatar
      go/neo/proto: Version 5 -> 6 · 837f49d1
      Kirill Smelkov authored
      This protocol version corresponds to protocol version used by NEO/py
      v1.12 and was set in NEO/py commit c6453626 (Bump protocol version).
      
      The protocol definition was updated to match that NEO/py release in
      the previous patches.
      837f49d1
    • Kirill Smelkov's avatar
      go/neo/proto: tweak += dry-run · 20cacd14
      Kirill Smelkov authored
      This corresponds to NEO/py commit 2a27239d (tweak: add option to
      simulate).
      20cacd14
    • Kirill Smelkov's avatar
      go/neo/proto: ErrorCode += DENIED · 789c16d3
      Kirill Smelkov authored
      This corresponds to NEO/py commit c2c9e99d (Better error reporting from
      the master to neoctl for denied requests).
      789c16d3
    • Kirill Smelkov's avatar
      go/neo/proto: AnswerPartitionList += NumReplicas · f5390f37
      Kirill Smelkov authored
      This corresponds to NEO/py commit 21190ee7 (Make 'neoctl print pt'
      report the number of replicas).
      f5390f37
    • Kirill Smelkov's avatar
      go/neo/*: Sync with NEO/py on changes to make the number of replicas modifiable at runtime · bf240897
      Kirill Smelkov authored
      This corresponds to NEO/py commit ef5fc508 (Make the number of replicas
      modifiable when the cluster is running).
      
      One important change in the protocol is that Client no longer queries
      Master for partition table - instead M pushed partTab to C right after
      identification (after pushing nodeTab).
      
      See also: https://neo.nexedi.com/P-NEO-Protocol.Specification.2019?portal_skin=CI_slideshow#/9/5
      bf240897
    • Kirill Smelkov's avatar
      go/neo/proto: RequestIdentification += NewNID; Move .DevPath · f609d6df
      Kirill Smelkov authored
      This corresponds to NEO/py commit 27e3f620 (New --new-nid storage option
      for fast cloning).
      f609d6df
    • Kirill Smelkov's avatar
      Merge tag v1.12 · 4efd2626
      Kirill Smelkov authored
      NEO 1.12
      
      * tag 'v1.12': (28 commits)
        Release version 1.12
        master: reject drop/tweak ctl commands that could lead to unwanted status
        qa: extend test reproducing the migration of a big ZODB to NEO
        neoctl: better display of full partition tables
        Bump protocol version
        tweak: add option to simulate
        tweak: do not crash when trying to remove all nodes
        tweak: do not touch cells of nodes that are intended to be dropped
        Better error reporting from the master to neoctl for denied requests
        Make 'neoctl print pt' report the number of replicas
        Make the number of replicas modifiable when the cluster is running
        New --new-nid storage option for fast cloning
        qa: fix 2 tests with ZODB5
        qa: new tools/stress options to evaluate MySQL engines
        qa: provide a way to let tests start 1 mysqld per storage node
        mysql: make 'user' actually optional in the DB connection string
        mysql: specify column families for RocksDB
        qa: add testIncremental (testImporter) test
        importer: fix hidden "maximum recursion depth exceeded" at startup
        importer: fix closure of ZODB, and also do it when the import is finished
        sqlite: fix resumption of migration to NEO with Importer
        qa: fix a random failure in threaded tests
        importer: speed up startup when the import is already finished
        importer: fix replication (as source) once import is finished
        storage: fix DatabaseManager.getLastTID with max_tid
        qa: remove 2 useless unit tests
        storage: allow the master to change our node id
        Rename --uuid command-line options into --nid
        importer: fix possible data loss on writeback
      4efd2626
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · 0e0820ab
      Kirill Smelkov authored
      * master:
        go/neo/proto: Version 4 -> 5
        go/neo/proto += FlushLog message
      0e0820ab
    • Kirill Smelkov's avatar
      Sync with NEO/py v1.11 · a25e62e7
      Kirill Smelkov authored
      a25e62e7
    • Kirill Smelkov's avatar
      go/neo/proto: Version 4 -> 5 · 5d9f5fb2
      Kirill Smelkov authored
      This protocol version corresponds to protocol version used by NEO/py
      v1.11 and was set in NEO/py commit 9a5b46dd (Bump protocol version).
      
      The protocol definition was updated to match that NEO/py release in
      the previous patch.
      5d9f5fb2
    • Kirill Smelkov's avatar
      go/neo/proto += FlushLog message · e0b19eb9
      Kirill Smelkov authored
      This corresponds to NEO/py commit 64826794 (New neoctl command to flush
      the logs of all nodes in the cluster).
      e0b19eb9
    • Kirill Smelkov's avatar
      Merge tag v1.11 · c1538a3d
      Kirill Smelkov authored
      NEO 1.11
      
      * tag 'v1.11': (52 commits)
        Release version 1.11
        Fix short descriptions of neoctl & neomigrate in their headers
        Update copyright year
        qa: new tool to stress-test NEO
        master: fix typo in comment
        Fix error handling when setting up a listening connector
        Fix incomplete/incorrect mapping of node ids in logs
        Fix log corruption on rotation in multi-threaded applications (e.g. client)
        sqlite: optimize storage of metadata
        neolog: do not die when a table is corrupted
        neolog: add support for zstd-compressed logs
        neolog: do not hardcode default value of -L option in help message
        fixup! New log format to show node id (and optionally cluster name) in node column
        New log format to show node id (and optionally cluster name) in node column
        fixup! client: discard late answers to lockless writes
        client: fix race condition between Storage.load() and invalidations
        client: fix race condition in refcounting dispatched answer packets
        More RTMIN+2 (log) information for clients and connections
        storage: check for conflicts when notifying that the a partition is replicated
        storage: clarify several assertions
        qa: new expectedFailure testcase method
        client: merge ConnectionPool inside Application
        client: prepare merge of ConnectionPool inside Application
        client: fix AssertionError when trying to reconnect too quickly after an error
        qa: fix attributeTracker
        storage: fix storage leak when an oid is stored several times within a transaction
        client: discard late answers to lockless writes
        qa: in threaded tests, log queued packets when "tic is looping forever"
        In logs, dump the partition table in a more compact and readable way
        storage: fix write-locking bug when a deadlock happens at the end of a replication
        client: log_flush most exceptions raised from Application to ZODB
        client: fix assertion failure in case of conflict + storage disconnection
        client: simplify connection management in transaction contexts
        client: also vote to nodes that only check serials
        qa: deindent code
        Bump protocol version
        client: fix undetected disconnections to storage nodes during commit
        Fix data corruption due to undetected conflicts after storage failures
        master: notify replicating nodes of aborted watched transactions
        New neoctl command to flush the logs of all nodes in the cluster
        storage: fix premature write-locking during rebase when replication ends
        client: fix race condition when a storage connection is closed just after identification
        storage: relax assertion
        comments, unused import
        storage: fix write-lock leak
        client: fix possible corruption in case of network failure with a storage
        qa: comment about potential freeze when a functional test ends
        storage: fix assertion failure in case of connection reset with a client node
        qa: document a rare random failure in testExport
        debug: add script to trace all accesses to the client cache
        Use argparse instead of optparse
        neolog: use argparse instead of optparse
        Add comment about dormant bug when sending a lot of data to a slow node
        client: make clearer that max_size attribute is used from outside ClientCache
      c1538a3d
    • Kirill Smelkov's avatar
      Sync with NEO/py v1.11 · 2e98a411
      Kirill Smelkov authored
      2e98a411
    • Kirill Smelkov's avatar
      go/neo/sqlite: Sync with NEO/py v1.11 · e7663c22
      Kirill Smelkov authored
      ROWID was dropped from trans and obj tables.
      
      This corresponds to NEO/py commit 243c1a0f (sqlite: optimize storage of
      metadata).
      e7663c22
    • Kirill Smelkov's avatar
      X go/neo: tests: Fixup neomigrate invocation · 23d8c235
      Kirill Smelkov authored
      After NEO/py commit 9f1e4eef (Use argparse instead of optparse)
      neomigrate signature changed:
      
      * -s and -d were removed: src and dst are specified as just arguments;
      * -q was added to silence importer-related warnings.
      23d8c235
    • Kirill Smelkov's avatar
      go/neo/proto: Version 4 -> 5 · 769787fc
      Kirill Smelkov authored
      This protocol version corresponds to protocol version used by NEO/py
      v1.11 and was set in NEO/py commit 9a5b46dd (Bump protocol version).
      
      The protocol definition was updated to match that NEO/py release in
      the previous patch.
      769787fc
    • Kirill Smelkov's avatar
      go/neo/proto += FlushLog message · 5e490b43
      Kirill Smelkov authored
      This corresponds to NEO/py commit 64826794 (New neoctl command to flush
      the logs of all nodes in the cluster).
      5e490b43
    • Kirill Smelkov's avatar
      Merge tag 'v1.11' · f9a0c181
      Kirill Smelkov authored
      NEO 1.11
      
      * tag 'v1.11': (52 commits)
        Release version 1.11
        Fix short descriptions of neoctl & neomigrate in their headers
        Update copyright year
        qa: new tool to stress-test NEO
        master: fix typo in comment
        Fix error handling when setting up a listening connector
        Fix incomplete/incorrect mapping of node ids in logs
        Fix log corruption on rotation in multi-threaded applications (e.g. client)
        sqlite: optimize storage of metadata
        neolog: do not die when a table is corrupted
        neolog: add support for zstd-compressed logs
        neolog: do not hardcode default value of -L option in help message
        fixup! New log format to show node id (and optionally cluster name) in node column
        New log format to show node id (and optionally cluster name) in node column
        fixup! client: discard late answers to lockless writes
        client: fix race condition between Storage.load() and invalidations
        client: fix race condition in refcounting dispatched answer packets
        More RTMIN+2 (log) information for clients and connections
        storage: check for conflicts when notifying that the a partition is replicated
        storage: clarify several assertions
        qa: new expectedFailure testcase method
        client: merge ConnectionPool inside Application
        client: prepare merge of ConnectionPool inside Application
        client: fix AssertionError when trying to reconnect too quickly after an error
        qa: fix attributeTracker
        storage: fix storage leak when an oid is stored several times within a transaction
        client: discard late answers to lockless writes
        qa: in threaded tests, log queued packets when "tic is looping forever"
        In logs, dump the partition table in a more compact and readable way
        storage: fix write-locking bug when a deadlock happens at the end of a replication
        client: log_flush most exceptions raised from Application to ZODB
        client: fix assertion failure in case of conflict + storage disconnection
        client: simplify connection management in transaction contexts
        client: also vote to nodes that only check serials
        qa: deindent code
        Bump protocol version
        client: fix undetected disconnections to storage nodes during commit
        Fix data corruption due to undetected conflicts after storage failures
        master: notify replicating nodes of aborted watched transactions
        New neoctl command to flush the logs of all nodes in the cluster
        storage: fix premature write-locking during rebase when replication ends
        client: fix race condition when a storage connection is closed just after identification
        storage: relax assertion
        comments, unused import
        storage: fix write-lock leak
        client: fix possible corruption in case of network failure with a storage
        qa: comment about potential freeze when a functional test ends
        storage: fix assertion failure in case of connection reset with a client node
        qa: document a rare random failure in testExport
        debug: add script to trace all accesses to the client cache
        Use argparse instead of optparse
        neolog: use argparse instead of optparse
        Add comment about dormant bug when sending a lot of data to a slow node
        client: make clearer that max_size attribute is used from outside ClientCache
      f9a0c181
    • Kirill Smelkov's avatar
      Merge branch 'master' into t · b3e24d58
      Kirill Smelkov authored
      * master:
        go/neo/proto: Version 1 -> 4
        go/neo/proto: RequestIdentification += DevPath
        go/neo/proto: Sync GetObject and docstrings with NEO/py
        go/neo/proto: Switch enum encoding from int32 to int8
        go/neo/proto: Sync message type codes with NEO/py
        go/neo/proto: Sync CellState enums order with NEO/py
        go/neo/proto: Sync OUT_OF_DATE text with NEO/py
      b3e24d58
    • Kirill Smelkov's avatar
      Sync with NEO/py v1.10 · 1fb86fb3
      Kirill Smelkov authored
      1fb86fb3
    • Kirill Smelkov's avatar
      go/neo/proto: Version 1 -> 4 · 9efe98bb
      Kirill Smelkov authored
      This protocol version corresponds to protocol version used by NEO/py v1.10.
      
      The protocol definition was updated to match that NEO/py release in
      the previous patches.
      9efe98bb
    • Kirill Smelkov's avatar
      go/neo/proto: RequestIdentification += DevPath · db2062f5
      Kirill Smelkov authored
      This corresponds to NEO/py commit 97af23cc (Maximize resiliency by
      taking into account the topology of storage nodes).
      db2062f5