1. 25 Nov, 2019 2 commits
    • Kirill Smelkov's avatar
      fixup! sync: Move/Port sync.WorkGroup to C++/Pyx nogil · cf96559c
      Kirill Smelkov authored
      This ammends 33cf3113: Kill now unused imports.
      cf96559c
    • Kirill Smelkov's avatar
      libpyxruntime: It should link to libgolang · 6d94fccf
      Kirill Smelkov authored
      It was all working on Debian 10, but running tests on Ubuntu 16.04
      turned out to fail:
      
          $ python -m pytest
          ================================== test session starts ===================================
          platform linux2 -- Python 2.7.12, pytest-4.6.6, py-1.8.0, pluggy-0.13.0
          rootdir: /home/kirr/src/pygolang
          collected 0 items / 10 errors
      
          ========================================= ERRORS =========================================
          ________________________ ERROR collecting golang/_gopath_test.py _________________________
          ImportError while importing test module '/home/kirr/src/pygolang/golang/_gopath_test.py'.
          Hint: make sure your test modules/packages have valid Python names.
          Traceback:
          golang/__init__.py:42: in <module>
              from golang._golang import _pysys_exc_clear as _sys_exc_clear
          golang/_golang.pyx:35: in init golang._golang
              _init_libpyxruntime()
          golang/_golang.pyx:532: in golang._golang._init_libpyxruntime
              import golang.pyx.runtime
          E   ImportError: /home/kirr/src/pygolang/golang/pyx/../runtime/liblibpyxruntime.so.0.1: undefined symbol: _ZN6golang4sync9WaitGroup3addEi
      
          $ c++filt _ZN6golang4sync9WaitGroup3addEi
          golang::sync::WaitGroup::add(int)
      
      So since libpyxruntime is using sync.WaitGroup (and sync.Mutex etc) it should
      be linking to libgolang where those symbols are provided. Fix it.
      
      Fixes: 4fc6e49c (time: Factor-out PyFunc into shared library libpyxruntime.so)
      6d94fccf
  2. 22 Nov, 2019 5 commits
    • Kirill Smelkov's avatar
      golang: Fix defer exception chaining in the presence of subchains · 1477dd02
      Kirill Smelkov authored
      A deferred function can raise exception and this exception itself can
      have .__context__ - consider for example if B1 is raised and its chain
      is B1->B2->B3. Before calling that deferred function, we save
      then-current exception A1 into GoFrame.exc_ctx and link-to .exc_ctx after
      the call. We were previously trying to link-to .exc_ctx from raised
      exception itself - i.e. B1 in the above example which is not correct:
      B1 was raised while B2 was being raised ... etc and it was B3 who was
      raised after A1. The consequence was that A1 was list, since B1 already
      had non-empty .__context__
      
      -> Fix it by linking-to A1 from B3, not from B1. In other words by
      linking-to .exc_ctx chain from tail of exception chain of raised
      exception.
      
      We can be sure that updated test is correct because it passes under
      Python3 where exception chaining is implemented natively.
      
      Fixes: bb9a94c3 (golang: Teach defer to chain exceptions (PEP 3134) even on Python2)
      1477dd02
    • Kirill Smelkov's avatar
      golang: Fix defer exception chaining for regular exceptions raised from under @func · 06cac90b
      Kirill Smelkov authored
      Defer support for exception chaining on Python2 (see bb9a94c3 "golang:
      Teach defer to chain exceptions (PEP 3134) even on Python2") simulates
      PEP 3134 by manually setting and chaining exc .__context__, .__cause__
      and .__traceback__: if exceptions are thrown by several deferred
      functions they are linked into exception chain via .__context__
      attribute. Defer support in @func also makes sure that any exception
      that comes out of a function has all those PEP 3134 attributes and
      presets them to their default values if exception object did not have
      them initially. In particular .__context__ is preset to None for first
      raised exception.
      
      There was a logic error in chaining handling: .__context__ was glued to
      previously raised exception only if current exception object did not
      have .__context__ attribute at all. And this was failing to chain them
      if current exception was raised from under another function wrapped in
      @func, because GoFrame wrapping that function makes sure to set
      curexc.__context__=None and oops - it was not changed later.
      
      -> Fix chaining implementation by gluing .__context__ either if there is
      no such attribute in current exception, or if .__context__ is None.
      
      This is correct to do since .__context__, by its definition, represents
      implicitly / automatically chained exceptions - contrary to .__cause__
      which is explicitly set and would be incorrect to automatically change
      from None to something. We also know that the end result is the same as
      Python3 behaviour since updated tests (see bellow) pass OK also when run
      under Python3 where exceptions chaining is implemented by Python runtime
      natively.
      
      Update test_defer_excchain() in golang_test.py, which verifies how
      raised exceptions are chained.
      
      No need to update testprog/golang_test_defer_excchain.{py,txt} since the
      test there verifies that our traceback dumper is correctly hooked into
      Python interpreter - where we know that exceptions are already chained
      correctly and we verify only that automatic traceback dump takes this
      chaining into account.
      
      Fixes: bb9a94c3 (golang: Teach defer to chain exceptions (PEP 3134) even on Python2)
      06cac90b
    • Kirill Smelkov's avatar
      golang: Fix defer traceback tests when run somewhere under symlink · 7faaecbc
      Kirill Smelkov authored
      Since realpath was used to find out pygolang dir, it will give different
      prefix compared to what actual prefix of __file__ is if __file__ was
      imported through some symlinked directory. And it is the actual
      __file__ that is included into traceback - not its realpath'ed version.
      
      This was breaking for example like this on py27-gevent tox tests:
      
          __file__:          '.../pygolang/.tox/py27-gevent/local/lib/python2.7/site-packages/golang/golang_test.py'
          dirname(__file__): '.../pygolang/.tox/py27-gevent/local/lib/python2.7/site-packages/golang'
          PYGOLANG:          '.../pygolang/.tox/py27-gevent/lib/python2.7/site-packages'
      
          E           Failed: not equal:
          E           Differences (unified diff with -expected +actual):
          E               @@ -1,8 +1,8 @@
          E                Traceback (most recent call last):
          E               -  File "PYGOLANG/golang/golang_test.py", line ..., in test_defer_excchain_traceback
          E               +  File ".../pygolang/.tox/py27-gevent/local/lib/python2.7/site-packages/golang/golang_test.py", line 1360, in test_defer_excchain_traceback
          E                    alpha()
          E               -  File "PYGOLANG/golang/golang_test.py", line ..., in alpha
          E               +  File ".../pygolang/.tox/py27-gevent/local/lib/python2.7/site-packages/golang/golang_test.py", line 1357, in alpha
          E                    beta()
          E               -  File "PYGOLANG/golang/golang_test.py", line ..., in beta
          E               +  File ".../pygolang/.tox/py27-gevent/local/lib/python2.7/site-packages/golang/golang_test.py", line 1356, in beta
          E                    raise RuntimeError("gamma")
          E                RuntimeError: gamma
      
      -> Fix it by not doing realpath'ification when detecting PYGOLANG
      prefix.
      
      Fixes: bb9a94c3 (golang: Teach defer to chain exceptions (PEP 3134) even on Python2)
      7faaecbc
    • Kirill Smelkov's avatar
      libgolang: Fix globals atexit race condition of ~refptr vs access from another thread · fd2a6fab
      Kirill Smelkov authored
      Until now we were defining errors like this:
      
          const error ErrSomething  = errors::New("abc");
      
      However there is a problem here: the runtime will call ErrSomething
      destructor on program exit, and for refptr<T> the destructor does 2
      things: a) decrefs the object, and b) switches embedded pointer to NULL;
      
      If another thread is still running both "a" and "b" can become race
      conditions if that T2 uses ErrSomething via e.g. just
      
          ...
          if (err == ErrSomething)
              ...
      
      Here is, for example TSAN report about ErrPyStopped usage when accessed by timer thread:
      
          WARNING: ThreadSanitizer: data race (pid=4224)
           Read of size 8 at 0x7f0d6840e150 by thread T92:
             #0 golang::refptr<golang::_error>::refptr(golang::refptr<golang::_error> const&) golang/libgolang.h:525 (liblibpyxruntime.so.0.1+0x4edc)
             #1 golang::pyx::runtime::PyFunc::operator()() const golang/runtime/libpyxruntime.cpp:222 (liblibpyxruntime.so.0.1+0x423d)
             #2 std::_Function_handler<void (), golang::pyx::runtime::PyFunc>::_M_invoke(std::_Any_data const&) /usr/include/c++/8/bits/std_function.h:297 (_time.so+0x1382b)
             #3 std::function<void ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687 (liblibgolang.so.0.1+0x2ede6)
             #4 golang::time::_Timer::_fire(double, int) golang/time.cpp:197 (liblibgolang.so.0.1+0x48cbe)
             #5 operator() golang/time.cpp:171 (liblibgolang.so.0.1+0x4894c)
             #6 __invoke_impl<void, golang::time::_Timer::reset(double)::<lambda(int)>&, int&> /usr/include/c++/8/bits/invoke.h:60 (liblibgolang.so.0.1+0x4a142)
             #7 __invoke<golang::time::_Timer::reset(double)::<lambda(int)>&, int&> /usr/include/c++/8/bits/invoke.h:95 (liblibgolang.so.0.1+0x4a080)
             #8 __call<void, 0> /usr/include/c++/8/functional:400 (liblibgolang.so.0.1+0x49fcc)
             #9 operator()<> /usr/include/c++/8/functional:484 (liblibgolang.so.0.1+0x49cee)
             #10 _M_invoke /usr/include/c++/8/bits/std_function.h:297 (liblibgolang.so.0.1+0x49858)
             #11 std::function<void ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687 (liblibgolang.so.0.1+0x2ede6)
             #12 operator() golang/libgolang.h:326 (liblibgolang.so.0.1+0x48fa1)
             #13 _FUN golang/libgolang.h:324 (liblibgolang.so.0.1+0x49004)
             #14 <null> <null> (python2+0x194d13)
      
           Previous write of size 8 at 0x7f0d6840e150 by main thread:
             #0 golang::refptr<golang::_error>::~refptr() golang/libgolang.h:510 (liblibpyxruntime.so.0.1+0x4d39)
             #1 at_exit_wrapper ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:389 (libtsan.so.0+0x28693)
      
           As if synchronized via sleep:
             #0 nanosleep ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:366 (libtsan.so.0+0x49960)
             #1 __pyx_f_6golang_7runtime_15_runtime_thread_nanosleep golang/runtime/_runtime_thread.c:1702 (_runtime_thread.so+0x48f2)
             #2 _tasknanosleep golang/runtime/libgolang.cpp:1237 (liblibgolang.so.0.1+0x2cca7)
             #3 golang::time::sleep(double) golang/runtime/libgolang.cpp:1251 (liblibgolang.so.0.1+0x2ce0b)
             #4 golang::time::_Timer::_fire(double, int) golang/time.cpp:179 (liblibgolang.so.0.1+0x48bd6)
             #5 operator() golang/time.cpp:171 (liblibgolang.so.0.1+0x4894c)
             #6 __invoke_impl<void, golang::time::_Timer::reset(double)::<lambda(int)>&, int&> /usr/include/c++/8/bits/invoke.h:60 (liblibgolang.so.0.1+0x4a142)
             #7 __invoke<golang::time::_Timer::reset(double)::<lambda(int)>&, int&> /usr/include/c++/8/bits/invoke.h:95 (liblibgolang.so.0.1+0x4a080)
             #8 __call<void, 0> /usr/include/c++/8/functional:400 (liblibgolang.so.0.1+0x49fcc)
             #9 operator()<> /usr/include/c++/8/functional:484 (liblibgolang.so.0.1+0x49cee)
             #10 _M_invoke /usr/include/c++/8/bits/std_function.h:297 (liblibgolang.so.0.1+0x49858)
             #11 std::function<void ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687 (liblibgolang.so.0.1+0x2ede6)
             #12 operator() golang/libgolang.h:326 (liblibgolang.so.0.1+0x48fa1)
             #13 _FUN golang/libgolang.h:324 (liblibgolang.so.0.1+0x49004)
             #14 <null> <null> (python2+0x194d13)
      
           Location is global 'golang::pyx::runtime::ErrPyStopped' of size 8 at 0x7f0d6840e150 (liblibpyxruntime.so.0.1+0x000000009150)
      
           Thread T92 (tid=7834, running) created by thread T15 at:
             #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors.cc:915 (libtsan.so.0+0x2be1b)
             #1 PyThread_start_new_thread <null> (python2+0x194ccf)
             #2 _taskgo golang/runtime/libgolang.cpp:124 (liblibgolang.so.0.1+0x288e7)
             #3 go<golang::time::_Timer::reset(double)::<lambda(int)>, int> golang/libgolang.h:324 (liblibgolang.so.0.1+0x490b7)
             #4 golang::time::_Timer::reset(double) golang/time.cpp:170 (liblibgolang.so.0.1+0x48b36)
             #5 __pyx_f_6golang_5_time_timer_reset_pyexc golang/_time.cpp:3271 (_time.so+0xa506)
             #6 __pyx_pf_6golang_5_time_7PyTimer_6reset golang/_time.cpp:2824 (_time.so+0x97d8)
             #7 __pyx_pw_6golang_5_time_7PyTimer_7reset golang/_time.cpp:2789 (_time.so+0x9751)
             #8 PyEval_EvalFrameEx <null> (python2+0xf27f7)
             #9 std::_Function_handler<void (), golang::pyx::runtime::PyFunc>::_M_invoke(std::_Any_data const&) /usr/include/c++/8/bits/std_function.h:297 (_time.so+0x1382b)
             #10 std::function<void ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687 (liblibgolang.so.0.1+0x2ede6)
             #11 golang::time::_Timer::_fire(double, int) golang/time.cpp:197 (liblibgolang.so.0.1+0x48cbe)
             #12 operator() golang/time.cpp:171 (liblibgolang.so.0.1+0x4894c)
             #13 __invoke_impl<void, golang::time::_Timer::reset(double)::<lambda(int)>&, int&> /usr/include/c++/8/bits/invoke.h:60 (liblibgolang.so.0.1+0x4a142)
             #14 __invoke<golang::time::_Timer::reset(double)::<lambda(int)>&, int&> /usr/include/c++/8/bits/invoke.h:95 (liblibgolang.so.0.1+0x4a080)
             #15 __call<void, 0> /usr/include/c++/8/functional:400 (liblibgolang.so.0.1+0x49fcc)
             #16 operator()<> /usr/include/c++/8/functional:484 (liblibgolang.so.0.1+0x49cee)
             #17 _M_invoke /usr/include/c++/8/bits/std_function.h:297 (liblibgolang.so.0.1+0x49858)
             #18 std::function<void ()>::operator()() const /usr/include/c++/8/bits/std_function.h:687 (liblibgolang.so.0.1+0x2ede6)
             #19 operator() golang/libgolang.h:326 (liblibgolang.so.0.1+0x48fa1)
             #20 _FUN golang/libgolang.h:324 (liblibgolang.so.0.1+0x49004)
             #21 <null> <null> (python2+0x194d13)
      
          SUMMARY: ThreadSanitizer: data race golang/libgolang.h:525 in golang::refptr<golang::_error>::refptr(golang::refptr<golang::_error> const&)
      
      -> Fix it by not deallocating the object, nor clearing the pointer on smart pointer destruction.
      
      To do so in ergonomic way introduce another template type global<X>,
      which can be used instead of X and change types of all global error
      variables from `const error` to `const global<error>`. Don't change pxd
      to offload pyx users from thinking about global/not-global aspect.
      
      Top-level documentation is TODO.
      fd2a6fab
    • Kirill Smelkov's avatar
      sync: Move/Port sync.WorkGroup to C++/Pyx nogil · 33cf3113
      Kirill Smelkov authored
      Provide sync.WorkGroup that can be used directly from C++ and Pyx/nogil codes.
      Python-level sync.WorkGroup becomes a wrapper around pyx/nogil one.
      
      Like with context (2a359791 "context: Move/Port context package to
      C++/Pyx nogil"), timers (b073f6df "time: Move/Port timers to C++/Pyx
      nogil") and interfaces (5a99b769 "libgolang: Start providing
      interfaces") memory for on-heap allocated WorkGroup is managed
      automatically.
      
      Python-level tests should be enough to cover C++/Pyx functionality at
      zero-level approximation.
      33cf3113
  3. 20 Nov, 2019 3 commits
  4. 19 Nov, 2019 5 commits
    • Kirill Smelkov's avatar
      context: Provide way to create PyContext from raw Context · 4fb9b51c
      Kirill Smelkov authored
      sync.PyWorkGroup will need to wrap raw Context into PyContext for
      spawned workers.
      4fb9b51c
    • Kirill Smelkov's avatar
      libpyxruntime: Teach PyFunc to report Python exception after the call · a36efe6d
      Kirill Smelkov authored
      - Add PyError class that wraps Python exception;
      - change PyFunc::operator() return from void -> to error, which might be
        either PyError or special error indicating that Python interpreter is
        stopped.
      - Provide way to reraise that Python exception.
      
      We will need to see which exception, if any, PyFunc call raised in
      sync.PyWorkGroup, where this python-level exception will need to be
      propagated to PyWorkGroup.wait() and reraised from there.
      a36efe6d
    • Kirill Smelkov's avatar
      libpyxruntime: Fix deadlock in pygil_ensure() · 73182038
      Kirill Smelkov authored
      pygil_ensure() was taking pyexitedMu and then GIL, which can deadlock
      via AB BA scenario if pygil_ensure() is also called in another thread
      with GIL already held.
      
      -> Fix it by always taking GIL with pyexitedMu released.
      
      Without the fix the test added always hangs with thread backend, e.g.
      like this:
      
          golang/pyx/runtime_test.py::test_pyx_pyfunc_vs_gil_deadlock
          STUCK
          terminate called after throwing an instance of 'golang::PanicError'
            what():  STUCK
          Fatal Python error: Aborted
      
      The bug was introduced in b073f6df (time: Move/Port timers to C++/Pyx nogil).
      73182038
    • Kirill Smelkov's avatar
      time: Factor-out PyFunc into shared library libpyxruntime.so · 4fc6e49c
      Kirill Smelkov authored
      Commit b073f6df (time: Move/Port timers to C++/Pyx nogil) added C++
      level PyFunc as a way to call Python function from under nogil code.
      We will soon need this functionality for sync.WorkGroup too, so move it
      into shared place.
      
      We move into regular library - not a pyx module (e.g. golang.pyx) -
      because for pyx modules linking is done at import time by specially
      generated code, while we need the linker support to automatically
      resolve e.g. calls to PyFunc::operator() from inside std::function.
      4fc6e49c
    • Kirill Smelkov's avatar
      libgolang: Provide DSO_EXPORT/DSO_IMPORT macros for other libraries · 5146a416
      Kirill Smelkov authored
      In the next patch we are going to move PyFunc from time.pyx into shared
      library. Before doing that let's provide DSO visibility macros from
      libgolang.h to offload other libraries from duplicating dance on how to
      mark symbols visibility properly depending on compiler/OS.
      5146a416
  5. 18 Nov, 2019 2 commits
    • Kirill Smelkov's avatar
      libgolang/{gevent,thread}: Preserve Python-level exception in runtime calls · 47fac0a9
      Kirill Smelkov authored
      Gevent runtime uses python-level calls internally which might interfere
      with current python state. For example if current python exception is
      set, and e.g. go or makesema runtime call is made, the following
      happens:
      
          golang/golang_test.py::test_pyx_runtime_vs_pyexc RuntimeError: abc
      
          The above exception was the direct cause of the following exception:
      
          SystemError: <class 'gevent.__semaphore.Semaphore'> returned a result with an error set
          Exception ignored in: 'golang.runtime._runtime_gevent._sema_alloc'
          SystemError: <class 'gevent.__semaphore.Semaphore'> returned a result with an error set
          terminate called after throwing an instance of 'golang::PanicError'
            what():  makesema: alloc failed
          Fatal Python error: Aborted
      
      -> Fix all functions in the runtimes that work at Python level to
         save/restore Python-level exception on entry/exit.
      
      This is mostly gevent runtime, but also a couple of non-posix fallbacks in
      thread runtime.
      
      The bug was there from day 1 of runtimes - from ce8152a2 (pyx api:
      Provide sleep), f971a2a8 (pyx api: Provide go) and 69db91bf (libgolang:
      Add internal semaphores).
      47fac0a9
    • Kirill Smelkov's avatar
      libgolang/{gevent,thread}: Restructure to keep functions that access Python localized · 689dc862
      Kirill Smelkov authored
      In the next patch we will fix runtime functions that work at Python
      level to preserve user Python exception state. Before that, as a
      preparatory step, restructure runtimes code, so that all such
      Python-accessing functions are grouped together.
      689dc862
  6. 15 Nov, 2019 1 commit
    • Kirill Smelkov's avatar
      pyx.build: Allow to use custom build_ext · b4feee6f
      Kirill Smelkov authored
      setuptools_dso hardcodes to use its own build_ext in its setup.
      We amend that build_ext and inject our own version carefully while
      setup() is run - see 7ae8c4f3 (pyx.build: Allow to combine C and C++
      sources in one extension). However, currently, if user uses cmdclass =
      {'build_ext': ...} in his own setup, this will be ignored and
      overwritten by golang.pyx.build.build_ext . One example where this
      breaks things is wendelin.core which hooks into build_ext to first
      configure/build CCAN before any extension:
      
      https://lab.nexedi.com/nexedi/wendelin.core/blob/b26ba558/setup.py#L147-153
      
      -> Fix it by taking into account what user could put into cmdclass.
      b4feee6f
  7. 14 Nov, 2019 17 commits
    • Kirill Smelkov's avatar
      libgolang: Fix double C++ defer · 14a249cb
      Kirill Smelkov authored
      If defer was used multiple times in one function it was failing to
      compile. For example added test is failing like this:
      
          In file included from golang/runtime/libgolang_test.cpp:22:
          golang/runtime/libgolang_test.cpp: In function ‘void __test_defer(int*)’:
          ./golang/libgolang.h:457:59: error: redeclaration of ‘golang::_deferred _defer___COUNTER__’
           #define defer(f) golang::_deferred _defer_ ## __COUNTER__ (f)
                                                                     ^
          golang/runtime/libgolang_test.cpp:473:5: note: in expansion of macro ‘defer’
               defer([&]() {
               ^~~~~
          ./golang/libgolang.h:457:36: note: ‘golang::_deferred _defer___COUNTER__’ previously declared here
           #define defer(f) golang::_deferred _defer_ ## __COUNTER__ (f)
                                              ^~~~~~~
          golang/runtime/libgolang_test.cpp:470:5: note: in expansion of macro ‘defer’
               defer([&]() {
               ^~~~~
      
      It turns out the __COUNTER__ was not expanded and they say you have to
      use 3 level of macro calls. Oh well...
      
      https://stackoverflow.com/a/27611869/9456786
      14a249cb
    • Kirill Smelkov's avatar
      libgolang: Provide func as alias for std::function · de269272
      Kirill Smelkov authored
      std::function is frequently too long to type while func is more native to Go.
      de269272
    • Kirill Smelkov's avatar
      context: Export PyContext for cimport · a6c1c984
      Kirill Smelkov authored
      For example wendelin.core wants to accept Python-level context,
      transform it into C-level context and pass to pyx/nogil code.
      a6c1c984
    • Kirill Smelkov's avatar
      context: Move/Port context package to C++/Pyx nogil · 2a359791
      Kirill Smelkov authored
      Provide context-related functionality that can be used directly from C++
      and Pyx/nogil codes. Python-level classes and functions become small
      wrappers around pyx/nogil ones.
      
      Like with timers (b073f6df "time: Move/Port timers to C++/Pyx nogil")
      and interfaces (5a99b769 "libgolang: Start providing interfaces")
      memory for objects dynamically allocated on heap is managed
      automatically.
      2a359791
    • Kirill Smelkov's avatar
      cxx: New package · 9785f2d3
      Kirill Smelkov authored
      Start new package that provides C++ amendments to be used by libgolang
      and its users. Current functionality: dict and set that wrap
      std::unordered_map and std::unordered_set into ergonomic interface.
      
      The code originates from wendelin.core:
      https://lab.nexedi.com/kirr/wendelin.core/blob/5a045ed1/wcfs/internal/wcfs_misc.h#L214-257
      
      Pyx/nogil only.
      9785f2d3
    • Kirill Smelkov's avatar
      errors: New package · a245ab56
      Kirill Smelkov authored
      With just one function - errors.New() - to create new error with given
      text.
      
      This will be soon used in Pyx/nogil context package to create context.canceled
      and context.deadlineExceeded errors.
      
      Pyx/nogil only for now.
      a245ab56
    • Kirill Smelkov's avatar
      libgolang: Start providing interfaces · 5a99b769
      Kirill Smelkov authored
      - interface is analog of interface{} in Go.
      - error is analog of error in Go.
      
      For now the interfaces are implemented via classic C++ scheme via
      inheritance and additional vtable field in pointed object.
      
      In the future we might want to consider reworking that to Go-style
      interfaces without predeclaring which interfaces a class implements.
      
      Top-level documentation is TODO.
      
      Interfaces will be soon needed to describe Context interface in C
      context package.
      5a99b769
    • Kirill Smelkov's avatar
      golang.pxd: Expose missing memory-management bits · 274afa3f
      Kirill Smelkov authored
      - refptr==refptr  (e.g. to compare whether returned error object is
        something particular)
      - adoptref() and newref()
      - object (exposed as gobject not to be confused with builtin pyx
        "object" referring to python object)
      
      All this will be soon used in C version of context package.
      274afa3f
    • Kirill Smelkov's avatar
      context: Provide package documentation · 9216e2db
      Kirill Smelkov authored
      Briefly describe provided functionality instead of only referring to
      https://golang.org/pkg/context and https://blog.golang.org/context.
      9216e2db
    • Kirill Smelkov's avatar
      libgolang: Rename refobj -> object · b2253abf
      Kirill Smelkov authored
      Thinking a bit more after e82b4fab (libgolang: Objects refcounting
      (initial draft)) and working on interfaces as part of context pyx -> C
      move, I've came to conclusion that the better name for on-heap objects
      managed by libgolang is just "object" without "ref" prefix.
      
      -> Rename refobj -> object and amend its documentation correspondingly.
      b2253abf
    • Kirill Smelkov's avatar
      libgolang: Provide std::hash<refptr> · af4a8d80
      Kirill Smelkov authored
      So that refptr<T> could be used as keys in e.g. dict (unordered_map) or
      set (unordered_set). set<refptr> will soon be used by C version of
      context package.
      af4a8d80
    • Kirill Smelkov's avatar
      libgolang: Provide select(vector<_selcase>) · 66e1e756
      Kirill Smelkov authored
      We already provide select for array and std::initializer_list. However
      when cases are constructed at runtime dynamically both array and
      initializer_list are not convenient to use.
      
      -> Provide select() overload that accepts vector<_selcase>.
      
      This functionality will soon be used in C version of context package.
      66e1e756
    • Kirill Smelkov's avatar
      sync: Fixup package documentation · 85df1e40
      Kirill Smelkov authored
      - indicated that WorkGroup is not provided in Go version (amends
        40f3b90c "sync: Provide package documentation").
      - fix typo in sync.pxd subject (amends 34b7a1f4 "golang: Expose Sema and
        Mutex as public Python and Cython/nogil API")
      85df1e40
    • Kirill Smelkov's avatar
      golang: tests: Silence inspect deprecation warning · 45f4df9e
      Kirill Smelkov authored
      Every time tests are run under Python3 the following warnings are printed:
      
        golang/golang_test.py::test_func
          .../golang/golang_test.py:990: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
            assert inspect.formatargspec(*inspect.getargspec(MyClass.zzz)) == '(self, v, x=2, **kkkkwww)'
      
        golang/golang_test.py::test_func
          .../golang/golang_test.py:990: DeprecationWarning: `formatargspec` is deprecated since Python 3.5. Use `signature` and the `Signature` object directly
            assert inspect.formatargspec(*inspect.getargspec(MyClass.zzz)) == '(self, v, x=2, **kkkkwww)'
      
      However since we are not going to drop Python2 support soon, and there is no
      reffered "use-instead" functionality on Python2, let's simply silence the
      warning for now.
      45f4df9e
    • Kirill Smelkov's avatar
      context: Switch to comparing value keys by identity instead of equality · 281defb2
      Kirill Smelkov authored
      It will be some extra work to define general equality when moving
      context code into C. However that is unnecessary as usually a key is
      specific to one particular package, and value associated with that key
      is struct, or some other aggregate, that is holding all package-specific
      context values.
      
      Switching to compare keys by identity preserves that use-case while
      making transition to C easier.
      281defb2
    • Kirill Smelkov's avatar
      context: Don't use dict in _ValueCtx · f76c11f3
      Kirill Smelkov authored
      We always store just one key in there.
      This will simplify transition to C.
      f76c11f3
    • Kirill Smelkov's avatar
      context: tests: Don't compare .done() by "is" · 20761c55
      Kirill Smelkov authored
      This continues 2c8063f4 (*: Channels must be compared by ==, not by "is"
      even for nilchan) and changes comparision of .done() for different
      contexts from "is" to ==.
      
      The reason is that soon we are going to move contexts into C, and this
      way in context tree there potentially can be PyCtx -> CCtx -> PyCtx
      chains, and while all those contexts could have the same C-level channel,
      it would be difficult to propagate the knowledge that Python-level
      pychan wrapper should be also exactly the same.
      
      We already treat channel objects as pointers and require to compare them
      by ==, so there should be no loss here. However to keep performance of
      repeated
      
      	select(
      	    ctx.done(),
      	    ...
      	)
      
      we continue to require that .done() for a particular python context
      object returns exactly the same channel object every time.
      20761c55
  8. 13 Nov, 2019 1 commit
  9. 11 Nov, 2019 2 commits
  10. 07 Nov, 2019 2 commits
    • Kirill Smelkov's avatar
      *: Suppress DeprecationWarning on `import imp` · ec2e882b
      Kirill Smelkov authored
      As of 20191107 imp is present in Python 3.8 source tree and is _used_
      there - see e.g.
      
          https://github.com/python/cpython/blob/v3.8.0-66-g7c20888e713/Lib/pkgutil.py#L188-L192
      
      And they say imp will be there until at least Python 4 - see e.g.
      
          https://github.com/python/cpython/commit/e4f41deccf94
      
      This fixes test_defer_excchain_dump failure under python3-dbg, where
      
      E           Failed: not equal:
      E           Differences (unified diff with -expected +actual):
      E               @@ -1,4 +1,6 @@
      E               +PYGOLANG/golang/_gopath.py:37: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
      E               +  import imp
      
      was added into traceback printed on stderr.
      
      Fixes: bb9a94c3 (golang: Teach defer to chain exceptions (PEP 3134) even on Python2)
      ec2e882b
    • Kirill Smelkov's avatar
      time: Move/Port timers to C++/Pyx nogil · b073f6df
      Kirill Smelkov authored
      Provide time.Ticker, time.Timer and friends that can be used directly
      from C++ and Pyx/nogil codes. Python-level classes become small wrapper
      around pyx/nogil ones.
      
      This is the first patch that moves to Pyx/nogil classes that are
      dynamically allocated on heap. refptr<T> is used to automatically manage
      lifetime of such objects. At Pyx level exposed API is very similar to
      Python-one, while internally it uses refptr<T> and friends.
      b073f6df