GitLab

Following errors model in Go and fd95c88a (golang, errors, fmt: Error
chaining (C++/Pyx)) let's add support at Python-level for errors to wrap
each other and to be inspected/unwrapped:

- an error can additionally provide way to unwrap itself, if it
  provides .Unwrap() method. .__cause__ is not taken into account yet,
  but will be in a follow-up patch;
- errors.Is(err) tests whether an item in error's chain matches target;
- `fmt.Errorf("... : %w", ... err)` is similar to `"... : %s" % (..., err)`
  but resulting error, when unwrapped, will return err.
- errors.Unwrap is not exposed as chaining through both .Unwrap() and
  .__cause__ will need more than just "current element" as unwrapping
  state (i.e. errors.Unwrap API is insufficient - see next patch), and
  in practice users of errors.Unwrap() are very seldom.

Support for error chaining through .__cause__ will follow in the next
patch.

Top-level documentation is TODO.

See https://blog.golang.org/go1.13-errors for error chaining overview.

The first step to expose errors and error chaining to Python:

- Add pyerror that wraps a pyx/nogil C-level error and is exposed as golang.error at py level.
- py errors must be compared by ==, not by "is"
- Add (py) errors.New to create a new error from text.
- a C-level error that has .Unwrap, is exposed with .Unwrap at py level,
  but full py-level chaining will be implemented in a follow-up patch.
- py error does not support inheritance yet.

Top-level documentation is TODO.

Package cxx was added in 9785f2d3 (cxx: New package), but the interface
that cxx:dict provided turned out to be not optimal:

    dict.get  was returning (v, ok), and
    dict.pop  ----//---

Correct dict.get and dict.pop to return just value, and, similarly to
channels API, provide additional dict.get_ and dict.pop_ - extended
versions that also return ok:

    dict.get(k)  -> v
    dict.pop(k)  -> v
    dict.get_(k) -> (v, ok)
    dict.pop_(k) -> (v, ok)

This time add tests.

errors.New was added in a245ab56 (errors: New package) without test.

Projects that use pyx.build (ex. wendelin.core) need recent
setuptools_dso fixes:

https://github.com/mdavidsaver/setuptools_dso/issues/5
https://github.com/mdavidsaver/setuptools_dso/commit/67d717a6
https://github.com/mdavidsaver/setuptools_dso/commit/e40f5883
https://github.com/mdavidsaver/setuptools_dso/commit/40b492ab

Increase setuptools_dso version in pygolang's build requirement for
uniformity as well.

Provide string utilities to verify whether string has suffix/prefix,
trim it, and split string by a delimiter. The code originated in wcfs
codebase in wendelin.core .

Pyx/nogil only.

With fmt.sprintf() and fmt.errorf() to format strings and errors. The
code was extracted from wcfs in wendelin.core .

Pyx/nogil only.

It was all working on Debian 10, but running tests on Ubuntu 16.04
turned out to fail:

    $ python -m pytest
    ================================== test session starts ===================================
    platform linux2 -- Python 2.7.12, pytest-4.6.6, py-1.8.0, pluggy-0.13.0
    rootdir: /home/kirr/src/pygolang
    collected 0 items / 10 errors

    ========================================= ERRORS =========================================
    ________________________ ERROR collecting golang/_gopath_test.py _________________________
    ImportError while importing test module '/home/kirr/src/pygolang/golang/_gopath_test.py'.
    Hint: make sure your test modules/packages have valid Python names.
    Traceback:
    golang/__init__.py:42: in <module>
        from golang._golang import _pysys_exc_clear as _sys_exc_clear
    golang/_golang.pyx:35: in init golang._golang
        _init_libpyxruntime()
    golang/_golang.pyx:532: in golang._golang._init_libpyxruntime
        import golang.pyx.runtime
    E   ImportError: /home/kirr/src/pygolang/golang/pyx/../runtime/liblibpyxruntime.so.0.1: undefined symbol: _ZN6golang4sync9WaitGroup3addEi

    $ c++filt _ZN6golang4sync9WaitGroup3addEi
    golang::sync::WaitGroup::add(int)

So since libpyxruntime is using sync.WaitGroup (and sync.Mutex etc) it should
be linking to libgolang where those symbols are provided. Fix it.

Fixes: 4fc6e49c (time: Factor-out PyFunc into shared library libpyxruntime.so)

Provide sync.WorkGroup that can be used directly from C++ and Pyx/nogil codes.
Python-level sync.WorkGroup becomes a wrapper around pyx/nogil one.

Like with context (2a359791 "context: Move/Port context package to
C++/Pyx nogil"), timers (b073f6df "time: Move/Port timers to C++/Pyx
nogil") and interfaces (5a99b769 "libgolang: Start providing
interfaces") memory for on-heap allocated WorkGroup is managed
automatically.

Python-level tests should be enough to cover C++/Pyx functionality at
zero-level approximation.

pygil_ensure() was taking pyexitedMu and then GIL, which can deadlock
via AB BA scenario if pygil_ensure() is also called in another thread
with GIL already held.

-> Fix it by always taking GIL with pyexitedMu released.

Without the fix the test added always hangs with thread backend, e.g.
like this:

    golang/pyx/runtime_test.py::test_pyx_pyfunc_vs_gil_deadlock
    STUCK
    terminate called after throwing an instance of 'golang::PanicError'
      what():  STUCK
    Fatal Python error: Aborted

The bug was introduced in b073f6df (time: Move/Port timers to C++/Pyx nogil).

Commit b073f6df (time: Move/Port timers to C++/Pyx nogil) added C++
level PyFunc as a way to call Python function from under nogil code.
We will soon need this functionality for sync.WorkGroup too, so move it
into shared place.

We move into regular library - not a pyx module (e.g. golang.pyx) -
because for pyx modules linking is done at import time by specially
generated code, while we need the linker support to automatically
resolve e.g. calls to PyFunc::operator() from inside std::function.

Provide context-related functionality that can be used directly from C++
and Pyx/nogil codes. Python-level classes and functions become small
wrappers around pyx/nogil ones.

Like with timers (b073f6df "time: Move/Port timers to C++/Pyx nogil")
and interfaces (5a99b769 "libgolang: Start providing interfaces")
memory for objects dynamically allocated on heap is managed
automatically.

Start new package that provides C++ amendments to be used by libgolang
and its users. Current functionality: dict and set that wrap
std::unordered_map and std::unordered_set into ergonomic interface.

The code originates from wendelin.core:
https://lab.nexedi.com/kirr/wendelin.core/blob/5a045ed1/wcfs/internal/wcfs_misc.h#L214-257

Pyx/nogil only.

With just one function - errors.New() - to create new error with given
text.

This will be soon used in Pyx/nogil context package to create context.canceled
and context.deadlineExceeded errors.

Pyx/nogil only for now.

We already have some sync functionality implemented in C++ (e.g.
sync.Once, sync.WaitGroup) and we are going to add more and also move
timers implementation to C++. It is getting crowded for that
functionality to still live in libgolang.{h,cpp}

-> Split sync & time functionality into their own C++ packages (still
built-into libgolang.so)

In preparation to start migrating, at least partially, context
functionality to nogil mode, first, similarly to package time (see
32f34607 "time: Move code to pyx"), move the code from context.py to
_context.pyx.

This is straight code movement + associated  setup.py & co adjusts.

We don't move just to context.pyx (note no _ prefix), since we will need
to continue distinguishing pyx/nogil from py objects/functions.

Python3 chains exceptions, so that e.g. if exc1 is raised and, while it
was not handled, another exc2 is raised, exc2 will be linked to exc1 via
exc2.__context__ attribute and exc1 will be included into exc2 traceback
printout. However many projects still use Python2 and there is no
similar chaining functionality there. This way exc1 is completely lost.

Since defer code is in our hands, we can teach it to implement exception
chaining even on Python2 by carefully analyzing what happens in
_GoFrame.__exit__().

Implementing chaining itself is relatively easy, but is only part of the
story. Even if an exception is chained with its cause, but exception
dump does not show the cause, the chaining will be practically useless.
With this in mind this patches settles not only on implementing chaining
itself, but on also giving a promise that chained cause exceptions will
be included into traceback dumps as well.

To realize this promise we adjust all exception dumping funcitons in
traceback module and carefully install adjusted
traceback.print_exception() into sys.excepthook. This amends python
interactive sessions and programs run by python interpreter to include
causes in exception dumps. "Careful" here means that we don't change
sys.excepthook if on golang module load we see that sys.excepthook was already
changed by some other module - e.g. due to IPython session running
because IPython installs its own sys.excepthook. In such cases we don't
install our sys.excepthook, but we also provide integration patches that
add exception chaining support for traceback dump functionality in
popular third-party software. The patches (currently for IPython and
Pytest) are activated automatically, but only when/if corresponding
software is imported and actually used. This should give practically
good implementation of the promise - a user can now rely on seeing
exception cause in traceback dump whatever way python programs are run.

The implementation takes https://pypi.org/project/pep3134/ experience
into account [1]. peak.utils.imports [2,3] is used to be notified when/if
third-party module is imported.

[1] https://github.com/9seconds/pep3134/
[2] https://pypi.org/project/Importing/
[3] http://peak.telecommunity.com/DevCenter/Importing

This patch originally started as hacky workaround in wendelin.core
because in wcfs tests I was frequently hitting situations, where
exception raised by an assert was hidden by another exception raised in
further generic teardown check. For example wcfs tests check that wcfs
is unmounted after every test run [4] and if that fails it was hiding
problems raised by an assert. As the result I was constantly guessing
and adding code like [5] to find what was actually breaking. At some
point I added hacky workaround for defer to print cause exception not to
loose it [6]. [7] has more context and background discussion on this topic.

[4] https://lab.nexedi.com/kirr/wendelin.core/blob/49e73a6d/wcfs/wcfs_test.py#L70
[5] https://lab.nexedi.com/kirr/wendelin.core/blob/49e73a6d/wcfs/wcfs_test.py#L853-857
[6] kirr/wendelin.core@c00d94c7
[7] nexedi/zodbtools!13 (comment 81553)

After this patch, on Python2

    defer(cleanup1)
    defer(cleanup2)
    defer(cleanup3)
    ...

is no longer just a syntatic sugar for

    try:
        try:
            try:
                ...
            finally:
                cleanup3()
        finally:
            cleanup2()
    finally:
        cleanup1()

Until now libgolang had semaphore and mutex functionality, but
implemented only internally and not exposed to users. Since for its
pinner thread wendelin.core v2 needs not only nogil channels, but also
nogil mutexes, timers and so on, it is now time to start providing that.

We start with mutexes:

- Expose Mutex from insides of libgolang.cpp to public API in
  libgolang.h . We actually expose both Mutex and Sema because
  semaphores are sometimes also useful for low-level synchronization,
  and because it is easier to export Mutex and Sema both at the same time.

- Provide pyx/nogil API to those as sync.Mutex and sync.Sema.

- Provide corresponding python wrappers.

- Add Pyx-level test for semaphore wakeup when wakeup is done not by the
  same thread which did the original acquire. This is the test that was
  promised in 5142460d (libgolang/thread: Add links to upstream
  PyThread_release_lock bug), and it used to corrupt memory and deadlock
  on macOS due to CPython & PyPy runtime bugs:

    https://bugs.python.org/issue38106
      -> https://github.com/python/cpython/pull/16047
    https://bitbucket.org/pypy/pypy/issues/3072

Note about python wrappers: At present, due to
https://github.com/cython/cython/issues/3165, C-level panic is not
properly translated into Py-level exception in (Py)Sema/Mutex constructors.
This should not pose a real problem in practice though.

This release is bugfix-only. Compared to pygolang v0.0.3 (4ca65816)
the change in speed is likely within noise:

     (on i7@2.6GHz)

thread runtime:

    name             old time/op  new time/op  delta
    go               18.3µs ± 1%  18.3µs ± 1%    ~     (p=0.218 n=10+10)
    chan             2.97µs ± 5%  2.97µs ± 8%    ~     (p=0.781 n=10+10)
    select           3.59µs ± 2%  3.55µs ± 5%    ~     (p=0.447 n=9+10)
    def              56.0ns ± 0%  55.0ns ± 0%  -1.79%  (p=0.000 n=10+10)
    func_def         43.7µs ± 1%  43.8µs ± 1%  +0.35%  (p=0.029 n=10+10)
    call             65.0ns ± 0%  62.3ns ± 1%  -4.15%  (p=0.000 n=10+10)
    func_call        1.06µs ± 1%  1.04µs ± 0%  -1.26%  (p=0.000 n=10+8)
    try_finally       137ns ± 1%   137ns ± 0%    ~     (p=1.000 n=10+10)
    defer            2.32µs ± 0%  2.33µs ± 1%  +0.43%  (p=0.000 n=9+10)
    workgroup_empty  37.6µs ± 1%  37.1µs ± 2%  -1.29%  (p=0.003 n=10+10)
    workgroup_raise  47.9µs ± 1%  47.6µs ± 0%  -0.63%  (p=0.001 n=9+9)

gevent runtime:

    name             old time/op  new time/op  delta
    go               15.8µs ± 0%  16.1µs ± 1%  +2.18%  (p=0.000 n=9+10)
    chan             7.36µs ± 0%  7.21µs ± 0%  -1.97%  (p=0.000 n=8+10)
    select           10.4µs ± 0%  10.5µs ± 0%  +0.71%  (p=0.000 n=10+10)
    def              57.0ns ± 0%  55.0ns ± 0%  -3.51%  (p=0.000 n=10+10)
    func_def         43.3µs ± 1%  44.1µs ± 2%  +1.81%  (p=0.000 n=10+10)
    call             66.0ns ± 0%  65.0ns ± 0%  -1.52%  (p=0.000 n=10+10)
    func_call        1.04µs ± 1%  1.06µs ± 1%  +1.48%  (p=0.000 n=10+10)
    try_finally       137ns ± 1%   136ns ± 0%  -1.31%  (p=0.000 n=10+10)
    defer            2.32µs ± 0%  2.31µs ± 1%    ~     (p=0.472 n=8+10)
    workgroup_empty  56.0µs ± 0%  55.7µs ± 0%  -0.49%  (p=0.000 n=10+10)
    workgroup_raise  71.3µs ± 1%  71.7µs ± 1%  +0.62%  (p=0.001 n=10+10)

- ThreadSanitizer helps to detect races and some memory errors,
- AddressSanitizer helps to detect memory errors,
- Python debug builds help to detect e.g reference counting errors.

Adding all those tools to testing coverage discovers e.g. the following
bugs (not a full list):

---- 8< ----

py27-thread-tsan:
WARNING: ThreadSanitizer: data race (pid=7143)
  Write of size 8 at 0x7b1400000650 by main thread:
    #0 free ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:649 (libtsan.so.0+0x2b46a)
    #1 free ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:643 (libtsan.so.0+0x2b46a)
    #2 golang::_chan::decref() golang/runtime/libgolang.cpp:470 (liblibgolang.so.0.1+0x47f2)
    #3 _chanxdecref golang/runtime/libgolang.cpp:452 (liblibgolang.so.0.1+0x484a)
    #4 _test_go_c golang/runtime/libgolang_test_c.c:86 (_golang_test.so+0x13a2e)
    #5 __pyx_pf_6golang_12_golang_test_12test_go_c golang/_golang_test.cpp:3340 (_golang_test.so+0xcbaa)
    #6 __pyx_pw_6golang_12_golang_test_13test_go_c golang/_golang_test.cpp:3305 (_golang_test.so+0xcbaa)
    #7 PyEval_EvalFrameEx <null> (python2.7+0xf68b4)

  Previous read of size 8 at 0x7b1400000650 by thread T8:
    #0 golang::Sema::acquire() golang/runtime/libgolang.cpp:164 (liblibgolang.so.0.1+0x410a)
    #1 golang::Mutex::lock() golang/runtime/libgolang.cpp:175 (liblibgolang.so.0.1+0x4c82)
    #2 golang::_chan::close() golang/runtime/libgolang.cpp:754 (liblibgolang.so.0.1+0x4c82)
    #3 _chanclose golang/runtime/libgolang.cpp:732 (liblibgolang.so.0.1+0x4d1a)
    #4 _work golang/runtime/libgolang_test_c.c:92 (_golang_test.so+0x136cc)
    #5 <null> <null> (python2.7+0x1929e3)

  Thread T8 (tid=7311, finished) created by main thread at:
    #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:915 (libtsan.so.0+0x2be1b)
    #1 PyThread_start_new_thread <null> (python2.7+0x19299f)
    #2 _taskgo golang/runtime/libgolang.cpp:119 (liblibgolang.so.0.1+0x3f68)
    #3 _test_go_c golang/runtime/libgolang_test_c.c:84 (_golang_test.so+0x13a1c)
    #4 __pyx_pf_6golang_12_golang_test_12test_go_c golang/_golang_test.cpp:3340 (_golang_test.so+0xcbaa)
    #5 __pyx_pw_6golang_12_golang_test_13test_go_c golang/_golang_test.cpp:3305 (_golang_test.so+0xcbaa)
    #6 PyEval_EvalFrameEx <null> (python2.7+0xf68b4)

py37-thread-asan:
==22205==ERROR: AddressSanitizer: heap-use-after-free on address 0x607000002cd0 at pc 0x7fd3732a7679 bp 0x7fd3723c8c50 sp 0x7fd3723c8c48
READ of size 8 at 0x607000002cd0 thread T7
    #0 0x7fd3732a7678 in golang::Sema::acquire() golang/runtime/libgolang.cpp:164
    #1 0x7fd3732a8644 in golang::Mutex::lock() golang/runtime/libgolang.cpp:175
    #2 0x7fd3732a8644 in golang::_chan::close() golang/runtime/libgolang.cpp:754
    #3 0x7fd3724004b2 in golang::chan<golang::structZ>::close() const golang/libgolang.h:323
    #4 0x7fd3724004b2 in operator() golang/runtime/libgolang_test.cpp:262
    #5 0x7fd3724004b2 in __invoke_impl<void, _test_chan_vs_stackdeadwhileparked()::<lambda()>&> /usr/include/c++/8/bits/invoke.h:60
    #6 0x7fd3724004b2 in __invoke<_test_chan_vs_stackdeadwhileparked()::<lambda()>&> /usr/include/c++/8/bits/invoke.h:95
    #7 0x7fd3724004b2 in __call<void> /usr/include/c++/8/functional:400
    #8 0x7fd3724004b2 in operator()<> /usr/include/c++/8/functional:484
    #9 0x7fd3724004b2 in _M_invoke /usr/include/c++/8/bits/std_function.h:297
    #10 0x7fd3723fdc6e in std::function<void ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687
    #11 0x7fd3723fdc6e in operator() golang/libgolang.h:273
    #12 0x7fd3723fdc6e in _FUN golang/libgolang.h:271
    #13 0x62ddf3  (/home/kirr/src/tools/go/pygolang-master/.tox/py37-thread-asan/bin/python3+0x62ddf3)
    #14 0x7fd377393fa2 in start_thread /build/glibc-vjB4T1/glibc-2.28/nptl/pthread_create.c:486
    #15 0x7fd376eda4ce in clone (/lib/x86_64-linux-gnu/libc.so.6+0xf94ce)

0x607000002cd0 is located 16 bytes inside of 72-byte region [0x607000002cc0,0x607000002d08)
freed by thread T0 here:
    #0 0x7fd377519fb0 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe8fb0)
    #1 0x7fd372401335 in golang::chan<golang::structZ>::~chan() golang/libgolang.h:292
    #2 0x7fd372401335 in _test_chan_vs_stackdeadwhileparked() golang/runtime/libgolang_test.cpp:222

previously allocated by thread T0 here:
    #0 0x7fd37751a518 in calloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0xe9518)
    #1 0x7fd3732a7d0b in zalloc golang/runtime/libgolang.cpp:1185
    #2 0x7fd3732a7d0b in _makechan golang/runtime/libgolang.cpp:413

Thread T7 created by T0 here:
    #0 0x7fd377481db0 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x50db0)
    #1 0x62df39 in PyThread_start_new_thread (/home/kirr/src/tools/go/pygolang-master/.tox/py37-thread-asan/bin/python3+0x62df39)

SUMMARY: AddressSanitizer: heap-use-after-free golang/runtime/libgolang.cpp:164 in golang::Sema::acquire()
Shadow bytes around the buggy address:
  0x0c0e7fff8540: fa fa fa fa fd fd fd fd fd fd fd fd fd fa fa fa
  0x0c0e7fff8550: fa fa fd fd fd fd fd fd fd fd fd fa fa fa fa fa
  0x0c0e7fff8560: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fd fd
  0x0c0e7fff8570: fd fd fd fd fd fd fd fa fa fa fa fa fd fd fd fd
  0x0c0e7fff8580: fd fd fd fd fd fa fa fa fa fa fd fd fd fd fd fd
=>0x0c0e7fff8590: fd fd fd fa fa fa fa fa fd fd[fd]fd fd fd fd fd
  0x0c0e7fff85a0: fd fa fa fa fa fa 00 00 00 00 00 00 00 00 00 fa
  0x0c0e7fff85b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e7fff85c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e7fff85d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c0e7fff85e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb

---- 8< ----

The bugs will be addressed in the followup patches.

Copy linux/list.h from wendelin.core which copied it from
util-linux.git which took it from linux.git at LGPL license state.

Here are corresponding links for wendelin.core and util-linux:

    nexedi/wendelin.core@a0f940ad
    https://git.kernel.org/cgit/utils/util-linux/util-linux.git/tree/include/list.h?id=v2.25-165-g9138d6f

Linked lists will be used in channels implementation in the next patch.

- Add go functionality to libgolang runtime;
- Implement go for thread and gevent runtimes.

  * Thread runtime uses PyThread_start_new_thread which, if used
    carefully, does not depend on Python GIL and on e.g. POSIX reduces to
    tiny wrapper around pthread_create.

  * Gevent runtime uses gevent's Greenlet in Pyx mode. This turns
    gevent to be a build-time dependency.

- Provide low-level _taskgo in C client API;
- Provide type-safe C++-level go wrapper over _taskgo;
- Switch golang.go from py implementation into Pyx wrapper over
  Pyx/nogil API.

This is the first patch that adds Pyx/C++/C-level unit tests and hooks
them into golang_test.py .

This patch brings the following speedup to Python-level interface:

 (on i7@2.6GHz)

thread runtime:

    name             old time/op  new time/op  delta
    go               93.0µs ± 1%  26.6µs ± 1%  -71.41%  (p=0.000 n=10+10)
    chan             13.6µs ± 2%  13.7µs ± 1%     ~     (p=0.280 n=10+10)
    select           29.9µs ± 4%  29.3µs ± 2%   -1.89%  (p=0.017 n=10+9)
    def              61.0ns ± 0%  55.0ns ± 0%   -9.84%  (p=0.000 n=10+10)
    func_def         43.8µs ± 1%  44.0µs ± 1%   +0.66%  (p=0.006 n=10+10)
    call             62.5ns ± 1%  64.0ns ± 0%   +2.40%  (p=0.000 n=10+8)
    func_call        1.06µs ± 1%  1.09µs ± 1%   +2.72%  (p=0.000 n=10+10)
    try_finally       137ns ± 0%   139ns ± 2%   +1.17%  (p=0.033 n=10+10)
    defer            2.34µs ± 1%  2.36µs ± 1%   +0.84%  (p=0.015 n=10+10)
    workgroup_empty  96.1µs ± 1%  58.1µs ± 1%  -39.55%  (p=0.000 n=10+10)
    workgroup_raise   135µs ± 1%    73µs ± 1%  -45.97%  (p=0.000 n=10+10)

gevent runtime:

    name             old time/op  new time/op  delta
    go               68.8µs ± 1%  28.6µs ± 0%  -58.47%  (p=0.000 n=10+8)
    chan             14.8µs ± 1%  15.8µs ± 1%   +6.19%  (p=0.000 n=10+10)
    select           32.0µs ± 0%  33.1µs ± 1%   +3.25%  (p=0.000 n=10+10)
    def              58.0ns ± 0%  55.0ns ± 0%   -5.17%  (p=0.000 n=10+10)
    func_def         43.9µs ± 1%  44.4µs ± 2%   +1.20%  (p=0.007 n=10+10)
    call             63.5ns ± 1%  64.0ns ± 2%     ~     (p=0.307 n=10+10)
    func_call        1.08µs ± 1%  1.06µs ± 0%   -2.55%  (p=0.000 n=10+8)
    try_finally       142ns ± 0%   136ns ± 0%   -4.23%  (p=0.000 n=10+9)
    defer            2.32µs ± 1%  2.29µs ± 1%   -0.96%  (p=0.000 n=10+10)
    workgroup_empty  90.3µs ± 0%  73.8µs ± 1%  -18.29%  (p=0.000 n=10+10)
    workgroup_raise   108µs ± 1%    94µs ± 0%  -13.29%  (p=0.000 n=10+10)

(small changes are probably within noise; "go" and "workgroup_*" should be
representative)

- Add sleep functionality to libgolang runtime;
- Implement sleep for thread and gevent runtimes. Thread runtime
  implements sleep independently of GIL, but only for POSIX for now;
- Switch golang.time py module into using golang.time pyx module.

As we are adding sleep, related functionality to query system about
"what is current time?" is also added.

Libgolang will be generic library and it will adapt itself to particular
execution environment by way of runtime plugin. This commit introduces
stubs for two such runtimes - "thread" and "gevent" - and initializes
libgolang with particular runtime depending on whether golang is
imported with gevent preactivated or not.

The runtimes themselves are empty at this step. We'll be adding runtime
functionality in the following patches.

Start transforming Pygolang from being pure Python project into project
that uses both Python and Cython and provides both Python and Cython API
to its users.

For Cython API there is a goal for it to be usable independently of
Python GIL - i.e. independently of Python runtime and limitations
imposed by Python's global interpreter lock. The motivation for this
work is wendelin.core v2, which, due to its design, would deadlock if it
tries to take the GIL in its pinner thread.

This commit brings bare minimum establishing libraries and build system.
It:

- Introduces C/C++ library libgolang. Libgolang will be serving most of
  nogil functionality and Pyx API will be a small wrapper over it. At
  this step Libgolang comes only with C-level panic;

- Introduces Pyx package golang that is aliased to Pyx package
  golang._golang . Users can do `from golang cimport ...` similarly to how
  they do py-level `from golang import ...`. At this step golang.pyx
  only wraps panic and provides way to transform C-level panic into Python
  exception;

- Introduces Py package golang.pyx.build . This package, similarly to
  golang.org/pkg/go/build, serves as the system to build Pyx packages -
  both external packages that use Pygolang in pyx mode, and to build
  Pygolang itself. The build, internally, is served by setuptools_dso
  and cython. An example how to use golang.pyx.build from external
  project comes in golang/pyx/testprog/golang_pyx_user/ and is used by
  tests to verify golang.pyx.build functionality;

- Introduces Pygolang build dependency to cython and setuptools_dso.

... and everything that whole pygolang depends on in pygolang[all].

The reason we do this: currently golang.perf.benchlib needs NumPy, but
it would be not ok to depend whole pygolang on numpy. Similarly pybench
needs pytest, but it would be not ok to require pytest unconditionally.

Thus allow users to specify what they need from pygolang via extras,
like pygolang[x.perf.benchlib] or pygolang[cmd.pybench], with
pygolang[all] serving as merge point for all subpackages dependencies.

TODO in the future we'll, hopefully, also provide accumulation points,
like pygolang[x.perf] which contains everything pygolang[x.perf.*]
depends on.

Note: core functionality and gpython are provided always unconditionally.

Provide gpython interpreter, that sets UTF-8 as default encoding,
integrates gevent and puts go, chan, select etc into builtin namespace.
Please see details in the documentation added by this patch.

pymain was taken from kirr/go123@7a476082
(go123: tracing/pyruntraced: New tool to run Python code with tracepoints activated (draft))

We describe golang/testdata/ in MANIFEST.in, and that makes testdata to
be included into sdist. However testdata does not get installed.

Since, we want to be able to run pygolang tests, even on installed
package, we need the testdata files to be present there as well.

Fix it.

See https://setuptools.readthedocs.io/en/latest/setuptools.html#including-data-files
for setuptools-related details.

Move version definition from setup.py to golang module so that it is
available at runtime with plain golang.__version__ .

`defer` allows to schedule a cleanup to be executed when current function
returns. It is similar to `try`/`finally` but does not force the cleanup part
to be far away in the end. For example::

   wc = wcfs.join(zurl)    │     wc = wcfs.join(zurl)
   defer(wc.close)         │     try:
                           │        ...
   ...                     │        ...
   ...                     │        ...
   ...                     │     finally:
                           │        wc.close()

For completeness there is `recover` and `panic` that allow to program with
Go-style error handling, for example::

   def _():
      r = recover()
      if r is not None:
         print("recovered. error was: %s" % (r,))
   defer(_)

   ...

   panic("aaa")

But `recover` and `panic` are probably of less utility since they can be
practically natively modelled with `try`/`except`.

If `defer` is used, the function that uses it must be wrapped with `@func` or
`@method` decorators.

The implementation is partly inspired by work of Denis Kolodin:

- https://habr.com/post/191786
- https://stackoverflow.com/a/43028386/9456786

+ corresponding bits in golang.testing package.

py.bench description follows:

"""
py.bench, similarly to py.test, discovers bench_* functions and Bench* classes
and then runs each discovered benchmark via golang.testing.benchmark. Similarly
to `go test -bench`, benchmarking results are printed in Go benchmark format.

For example, running py.bench on the following code::

    def bench_add(b):
        x, y = 1, 2
        for i in xrange(b.N):
            x + y

gives something like::

    $ py.bench --count=3 x.py
    ...
    pymod: bench_add.py
    Benchmarkadd    50000000        0.020 µs/op
    Benchmarkadd    50000000        0.020 µs/op
    Benchmarkadd    50000000        0.020 µs/op
"""

The implementation is based on t/py.bench from Wendelin.core - see
following commits for its history:

lab.nexedi.com/nexedi/wendelin.core/commit/51f252d4
lab.nexedi.com/nexedi/wendelin.core/commit/074ce24d
lab.nexedi.com/nexedi/wendelin.core/commit/ed13c3f9
lab.nexedi.com/nexedi/wendelin.core/commit/fc08766d
lab.nexedi.com/nexedi/wendelin.core/commit/5a1ed45a
lab.nexedi.com/nexedi/wendelin.core/commit/bcab1246