Skip to content

G
gevent
Commit bdd08a15 authored by Jason Madden's avatar Jason Madden
Browse files
Options

Update change docs [skip ci]

parent 5646ae7f
Show whitespace changes
Inline Side-by-side
Showing with 71 additions and 285 deletions
changelog.rst
View file @ bdd08a15
......@@ -4,8 +4,8 @@
.. currentmodule:: gevent
1.2a1 (unreleased)
==================
1.2a1 (Oct 26, 2016)
====================
Incompatible Changes
--------------------
......@@ -17,23 +17,24 @@ Incompatible Changes
``gevent.core`` instead; this has always been the only documented place to
import from.
Libraries
---------
Libraries and Installation
--------------------------
- Update libev to version 4.22 (was 4.20).
- Update tblib to 1.3.0.
- Update Cython to 0.25 (was 0.24).
- Update Cython to 0.25 (was 0.23.5).
- Update c-ares to version 1.12.0 (was 1.10.0) (`release notes <https://c-ares.haxx.se/changelog.html>`_).
- For the benefit of downstream package maintainers, gevent is now
tested with c-ares and libev linked dynamically and not embedded
(i.e., using the system libraries). However, only the versions
shipped with gevent are tested and known to work.
- The repository directory layout has been changed to make it easier
to include third-party dependencies. Likewise, the setup.py script
has been split to make it easier to build third-party dependencies.
- PyPy/CFFI: The corecffi native extension is now only built at
installation time. Previously, if it wasn't available, a build was
attempted at every import. This could lead to scattered "gevent"
directories and undependable results.
- Update Cython to 0.24. Cython 0.25 beta is known to work and will
probably be used by a future 1.2 release.
- setuptools is now required at build time on all platforms.
Previously it was only required for Windows and PyPy.
- POSIX: Don't hardcode ``/bin/sh`` into the configuration command
......@@ -67,7 +68,7 @@ Platforms
this parameter isn't useful prior to 3.6.
* SSLSocket.recv(0) or read(0) returns an empty byte string. This is
a fix for `Python bug #23804 <http://bugs.python.org/issue23804>`_
which has been merged into Python 2.7 and Python 3.
which has also been merged into Python 2.7 and Python 3.5.
- PyPy3 5.5.0 *alpha* (supporting Python 3.3.5) is now tested and passes the
test suite. Thanks to btegs for :issue:`866`, and Fabio Utzig for :pr:`826`.
Note that PyPy3 is not optimized for performance either by the PyPy
......
doc/whatsnew_1_2.rst
View file @ bdd08a15
......@@ -2,321 +2,106 @@
What's new in gevent 1.2
==========================
Detailed information an what has changed is available in the
Detailed information on what has changed is available in the
:doc:`changelog`. This document summarizes the most important changes
since :doc:`gevent 1.1 <whatsnew_1_1>`.
.. warning:: This is a placeholder document.
In general, gevent 1.2 is a smaller update than gevent 1.1, focusing
on platform support, standard library compatibility, security, bug
fixes and consistency.
Broader Platform Support
Platform Support
========================
gevent 1.1 supports Python 2.6, 2.7, 3.3, and 3.4 on the CPython
(`python.org`_) interpreter. It also supports `PyPy`_ 2.6.1 and above
(PyPy 4.0.1 or higher is recommended); PyPy3 is not supported.
gevent 1.2 supports Python 2.7, 3.4, 3.5 and 3.6 on the CPython
(`python.org`_) interpreter. It also supports `PyPy2`_ 4.0.1 and above
(PyPy2 5.2 or higher is recommended) and PyPy3 5.5.0.
Support for Python 2.5 was removed when support for Python 3 was
added. Any further releases in the 1.0.x line will maintain support
for Python 2.5.
.. note:: Version 1.1.x will be the last series of gevent releases
to support Python 2.6. The next major release will only
support Python 2.7 and above.
Python 3.5 has preliminary support, which means that gevent is
expected to generally run and function with the same level of support
as on Python 3.4, but new features and APIs introduced in 3.5 may not
be properly supported (e.g., `DevpollSelector`_) and due to the recent
arrival of Python 3.5, the level of testing it has received is lower.
For ease of installation on Windows and OS X, gevent 1.1 is
distributed as pre-compiled binary wheels, in addition to source code.
.. _python.org: http://www.python.org/downloads/
.. _PyPy: http://pypy.org
.. _DevpollSelector: https://docs.python.org/3.5/whatsnew/3.5.html#selectors
PyPy Notes
----------
PyPy has been tested on OS X and 64-bit Linux from version 2.6.1
through 4.0.0 and 4.0.1, and on 32-bit ARM on Raspbian with version 4.0.1.
.. caution:: Support for Python 2.6 was removed. Support for Python 3.3 is only
tested on PyPy3.
.. note:: PyPy is not supported on Windows. (gevent's CFFI backend is not
available on Windows.)
- Version 4.0.1 or above is **highly recommended** due to its extensive
bug fixes relative to earlier versions.
- Version 2.6.1 or above is **required** for proper signal handling. Prior
to 2.6.1 and its inclusion of `cffi 1.3.0`_, signals could be
delivered incorrectly or fail to be delivered during a blocking
operation. (PyPy 2.5.0 includes CFFI 0.8.6 while 2.6.0 has 1.1.0;
the necessary feature was added in `1.2.0`_ which is not itself
directly present in any PyPy release.) CFFI 1.3.0 also allows using
the CFFI backend on CPython.
- Overall performance seems to be quite acceptable with newer versions
of PyPy. The benchmarks distributed with gevent typically perform as
well or better on PyPy than on CPython at least on some platforms.
Things that are known or expected to be (relatively) slower under
PyPy include the :mod:`c-ares resolver <gevent.resolver_ares>` and
:class:`~gevent.lock.Semaphore`. Whether or not these matter will
depend on the workload of each application (:pr:`708` mentions
some specific benchmarks for ``Semaphore``).
.. caution:: The ``c-ares`` resolver is considered highly experimental
under PyPy and is not recommended for production use.
Released versions of PyPy through at least 4.0.1 have `a
bug`_ that can cause a memory leak when subclassing
objects that are implemented in Cython, as is the c-ares
resolver. In addition, thanks to reports like
:issue:`704`, we know that the PyPy garbage collector can
interact badly with Cython-compiled code, leading to
crashes. While the intended use of the ares resolver has
been loosely audited for these issues, no guarantees are made.
.. note:: PyPy 4.0.x on Linux is known to *rarely* (once per 24 hours)
encounter crashes when running heavily loaded, heavily
networked gevent programs (even without ``c-ares``). The
exact cause is unknown and is being tracked in :issue:`677`.
.. _cffi 1.3.0: https://bitbucket.org/cffi/cffi/src/ad3140a30a7b0ca912185ef500546a9fb5525ece/doc/source/whatsnew.rst?at=default
.. _1.2.0: https://cffi.readthedocs.io/en/latest/whatsnew.html#v1-2-0
.. _a bug: https://bitbucket.org/pypy/pypy/issues/2149/memory-leak-for-python-subclass-of-cpyext
Python 3.6 is currently in beta and is supported at the same level as 3.5.
.. _operating_systems_label:
Operating Systems
-----------------
gevent is regularly built and tested on Mac OS X, Ubuntu Linux, and
Windows, in both 32- and 64-bit configurations. All three platforms
are primarily tested on the x86/amd64 architecture, while Linux is
also occasionally tested on Raspian on ARM.
In general, gevent should work on any platform that both Python and
`libev support`_. However, some less commonly used platforms may
require tweaks to the gevent source code or user environment to
compile (e.g., `SmartOS`_). Also, due to differences in
things such as timing, some platforms may not be able to fully pass gevent's
extensive test suite (e.g., `OpenBSD`_).
For ease of installation on Windows and OS X, gevent 1.2 is
distributed as pre-compiled binary wheels, in addition to source code.
.. _libev support: http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod#PORTABILITY_NOTES
.. _SmartOS: https://github.com/gevent/gevent/pull/711
.. _OpenBSD: https://github.com/gevent/gevent/issues/737
.. _python.org: http://www.python.org/downloads/
.. _PyPy2: http://pypy.org
Bug Fixes
=========
Since 1.0.2, gevent 1.1 contains over 600 commits from nearly two
dozen contributors. Over 200 issues were closed, and over 50 pull
requests were merged.
Since 1.1.2, gevent 1.2 contains over 240 commits from nine different
dozen contributors. About two dozen pull requests were merged.
Improved subprocess support
===========================
In gevent 1.0, support and monkey patching for the :mod:`subprocess`
module was added. Monkey patching this module was off by default.
In 1.1, monkey patching ``subprocess`` is on by default due to
improvements in handling child processes and requirements by
downstream libraries, notably `gunicorn`_.
- :func:`gevent.os.fork`, which is monkey patched by default (and
should be used to fork a gevent-aware process that expects to use
gevent in the child process) has been improved and cooperates with
:func:`gevent.os.waitpid` (again monkey patched by default) and
:func:`gevent.signal.signal` (which is monkey patched only for the
:data:`signal.SIGCHLD` case). The latter two patches are new in 1.1.
- In gevent 1.0, use of libev child watchers (which are used
internally by ``gevent.subprocess``) had race conditions with
user-provided ``SIGCHLD`` handlers, causing many types of
unpredictable breakage. The two new APIs described above are
intended to rectify this.
- Fork-watchers will be called, even in multi-threaded programs
(except on Windows).
- The default threadpool and threaded resolver work in child
processes.
- File descriptors are no longer leaked if
:class:`gevent.subprocess.Popen` fails to start the child.
In addition, simple use of :class:`multiprocessing.Process` is now
possible in a monkey patched system, at least on POSIX platforms.
.. caution:: Use of :class:`multiprocessing.Queue` when :mod:`thread`
has been monkey-patched will lead to a hang due to
``Queue``'s internal use of a blocking pipe and threads. For the same
reason, :class:`concurrent.futures.ProcessPoolExecutor`,
which internally uses a ``Queue``, will hang.
.. caution:: It is not possible to use :mod:`gevent.subprocess` from
native threads. See :mod:`gevent.subprocess` for details.
.. note:: If the ``SIGCHLD`` signal is to be handled, it is important
to monkey patch (or directly use) both :mod:`os` and
:mod:`signal`; this is the default for
:func:`~gevent.monkey.patch_all`. Failure to do so can
result in the ``SIGCHLD`` signal being lost.
.. tip:: All of the above entail forking a child process. Forking
a child process that uses gevent, greenlets, and libev
can have some unexpected consequences if the child
doesn't immediately ``exec`` a new binary. Be sure you
understand these consequences before using this
functionality, especially late in a program's lifecycle.
For a more robust solution to certain uses of child
process, consider `gipc`_.
.. _gunicorn: http://gunicorn.org
.. _gipc: https://gehrcke.de/gipc/
Monkey patching
===============
Monkey patching is more robust, especially if the standard library
:mod:`threading` or :mod:`logging` modules had been imported before
applying the patch. In addition, there are now supported ways to
determine if something has been monkey patched.
In gevent 1.1, subprocess monkey-patching was on by default for the
first time. Over time this led to discovery of a few issues and corner
cases that have been fixed in 1.2.
- Setting SIGCHLD to SIG_IGN or SIG_DFL after :mod:`gevent.subprocess`
had been used previously could not be reversed, causing
``Popen.wait`` and other calls to hang. Now, if SIGCHLD has been
ignored, the next time :mod:`gevent.subprocess` is used this will be
detected and corrected automatically. (This potentially leads to
issues with :func:`os.popen` on Python 2, but the signal can always
be reset again. Mixing the low-level process handling calls,
low-level signal management and high-level use of
:mod:`gevent.subprocess` is tricky.) Reported in :issue:`857` by
Chris Utz.
- ``Popen.kill`` and ``send_signal`` no longer attempt to send signals
to processes that are known to be exited.
- The :func:`gevent.os.waitpid` function is cooperative in more
circumstances. Reported in :issue:`878` by Heungsub Lee.
API Additions
=============
Numerous APIs offer slightly expanded functionality in this version.
Look for "changed in version 1.1" or "added in version 1.1" throughout
the documentation for specifics. Highlights include:
Look for "changed in version 1.2" or "added in version 1.2" throughout
the documentation for specifics.
- A gevent-friendly version of :obj:`select.poll` (on platforms that
implement it).
- :class:`~gevent.fileobject.FileObjectPosix` uses the :mod:`io`
package on both Python 2 and Python 3, increasing its functionality,
correctness, and performance. (Previously, the Python 2 implementation used the
undocumented class :class:`socket._fileobject`.)
- Locks raise the same error as standard library locks if they are
over-released. Likewise, SSL sockets raise the same errors as their
bundled counterparts if they are read or written after being closed.
- :meth:`ThreadPool.apply <gevent.threadpool.ThreadPool.apply>` can
now be used recursively.
- The various pool objects (:class:`~gevent.pool.Group`,
:class:`~gevent.pool.Pool`, :class:`~gevent.threadpool.ThreadPool`)
support the same improved APIs: :meth:`imap <gevent.pool.Group.imap>`
and :meth:`imap_unordered <gevent.pool.Group.imap_unordered>` accept
multiple iterables, :meth:`apply <gevent.pool.Group.apply>` raises any exception raised by the
target callable, etc.
- Killing a greenlet (with :func:`gevent.kill` or
:meth:`Greenlet.kill <gevent.Greenlet.kill>`) before it is actually started and
switched to now prevents the greenlet from ever running, instead of
raising an exception when it is later switched to. Attempting to
spawn a greenlet with an invalid target now immediately produces
a useful :exc:`TypeError`, instead of spawning a greenlet that would
(usually) immediately die the first time it was switched to.
- Almost anywhere that gevent raises an exception from one greenlet to
another (e.g., :meth:`Greenlet.get <gevent.Greenlet.get>`),
the original traceback is preserved and raised.
- Various logging/debugging outputs have been cleaned up.
- The WSGI server found in :mod:`gevent.pywsgi` is more robust against
errors in either the client or the WSGI application, fixing several
hangs or HTTP protocol violations. It also supports new
functionality such as configurable error handling and logging.
- Documentation has been expanded and clarified.
Of particular note, several backwards compatible updates to the
subprocess module have been backported from Python 3 to Python 2,
making :mod:`gevent.subprocess` smaller, easier to maintain and in
some cases safer, while letting gevent clients use the updated APIs
even on older versions of Python.
.. _library_updates_label:
If ``concurrent.futures`` is available (Python 3, or if the Python 2
backport has been installed), then the class
:class:`gevent.threadpool.ThreadPoolExecutor` is defined to create an
executor that always uses native threads, even when the system is
monkey-patched.
Library Updates
===============
The two C libraries that are bundled with gevent have been updated.
libev has been updated from 4.19 to 4.20 (`libev release notes`_) and
c-ares has been updated from 1.9.1 to 1.10.0 (`c-ares release notes`_).
libev has been updated from 4.20 to 4.22 (`libev release notes`_) and
c-ares has been updated from 1.10.0 to 1.12.0 (`c-ares release notes`_).
.. caution:: The c-ares ``configure`` script is now *much* stricter
about the contents of compilation environment variables
such as ``$CFLAGS`` and ``$LDFLAGS``. For example,
``$CFLAGS`` is no longer allowed to contain ``-I``
directives; instead, these must be placed in
``$CPPFLAGS``. That's one common cause of an error
like the following when compiling from scratch on a POSIX
platform::
Running '(cd "/tmp/easy_install-NT921u/gevent-1.1b2/c-ares" && if [ -e ares_build.h ]; then cp ares_build.h ares_build.h.orig; fi && /bin/sh ./configure CONFIG_COMMANDS= CONFIG_FILES= && cp ares_config.h ares_build.h "$OLDPWD" && mv ares_build.h.orig ares_build.h) > configure-output.txt' in /tmp/easy_install-NT921u/gevent-1.1b2/build/temp.linux-x86_64-2.7/c-ares
configure: error: Can not continue. Fix errors mentioned immediately above this line.
.. _libev release notes: https://github.com/gevent/gevent/blob/master/libev/Changes#L17
.. _c-ares release notes: https://raw.githubusercontent.com/bagder/c-ares/cares-1_10_0/RELEASE-NOTES
.. _libev release notes: https://github.com/gevent/gevent/blob/master/deps/libev/Changes#L17
.. _c-ares release notes: https://c-ares.haxx.se/changelog.html
Compatibility
=============
This release is intended to be compatible with 1.0.x with minimal or
no changes to client source code. However, there are a few changes to
be aware of that might affect some applications. Most of these changes
are due to the increased platform support of Python 3 and PyPy and
reduce the cases of undocumented or non-standard behaviour.
- :class:`gevent.baseserver.BaseServer` deterministically
`closes its sockets <https://github.com/gevent/gevent/issues/248#issuecomment-82467350>`_.
As soon as a request completes (the request handler returns),
the ``BaseServer`` and its subclasses including
:class:`gevent.server.StreamServer` and
:class:`gevent.pywsgi.WSGIServer` close the client socket.
In gevent 1.0, the client socket was left to the mercies of the
garbage collector (this was undocumented). In the typical case, the
socket would still be closed as soon as the request handler returned
due to CPython's reference-counting garbage collector. But this
meant that a reference cycle could leave a socket dangling open for
an indeterminate amount of time, and a reference leak would result
in it never being closed. It also meant that Python 3 would produce
ResourceWarnings, and PyPy (which, unlike CPython, `does not use a
reference-counted GC`_) would only close (and flush!) the socket at
an arbitrary time in the future.
If your application relied on the socket not being closed when the
request handler returned (e.g., you spawned a greenlet that
continued to use the socket) you will need to keep the request
handler from returning (e.g., ``join`` the greenlet). If for some
reason that isn't possible, you may subclass the server to prevent
it from closing the socket, at which point the responsibility for
closing and flushing the socket is now yours; *but* the former
approach is strongly preferred, and subclassing the server for this
reason may not be supported in the future.
.. _does not use a reference-counted GC: http://doc.pypy.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies
- :class:`gevent.pywsgi.WSGIServer` ensures that headers (names and values) and the
status line set by the application can be encoded in the ISO-8859-1
(Latin-1) charset and are of the *native string type*.
Under gevent 1.0, non-``bytes`` headers (that is, ``unicode``, since
gevent 1.0 only ran on Python 2, although objects like ``int`` were
also allowed) were encoded according to the current default Python
encoding. In some cases, this could allow non-Latin-1 characters to
be sent in the headers, but this violated the HTTP specification,
and their interpretation by the recipient is unknown. In other
cases, gevent could send malformed partial HTTP responses. Now, a
:exc:`UnicodeError` will be raised proactively.
Most applications that adhered to the WSGI PEP, :pep:`3333`, will not
need to make any changes. See :issue:`614` for more discussion.
- Under Python 2, the previously undocumented ``timeout`` parameter to
:meth:`Popen.wait <gevent.subprocess.Popen.wait>` (a gevent extension
) now throws an exception, just like the documented parameter to the
same stdlib method in Python 3.
This release is intended to be compatible with 1.1.x with no changes
to client source code, so long as only non-deprecated and supported
interfaces were used (as always, internal, non-documented
implementation details may have changed).
- Under Python 3, several standard library methods added ``timeout``
parameters. These often default to -1 to mean "no timeout", whereas
gevent uses a default of ``None`` to mean the same thing,
potentially leading to great confusion and bugs in portable code. In
gevent, using a negative value has always been ill-defined and hard
to reason about. Because of those two things, as of this release,
negative ``timeout`` values should be considered deprecated (unless
otherwise documented). The current ill-defined behaviour is
maintained, but future releases may choose to treat it the same as
``None`` or raise an error. No runtime warnings are issued for this
change for performance reasons.
In particular the deprecated ``gevent.coros`` module has been removed
and ``gevent.corecext`` and ``gevent.corecffi`` have also been removed.
- The previously undocumented class
``gevent.fileobject.SocketAdapter`` has been removed, as have the
internal ``gevent._util`` module and some internal implementation modules
found in early pre-releases of 1.1.
For security, ``gevent.pywsgi`` no longer accepts incoming headers
containing an underscore, and header values passed to
``start_response`` cannot contain a carriage return or newline. See
:issue:`819` and :issue:`775`, respectively.
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7