Merge pull request #2483 from gabrieldemarmiesse/move_compilation_3

Moving compilation from the reference guide to the userguide Part 3

docs/src/userguide/source_files_and_compilation.rst

@@ -67,6 +67,164 @@ current directory use:
 
     $ python setup.py build_ext --inplace
 
+Configuring the C-Build
+------------------------
+
+If you have include files in non-standard places you can pass an
+``include_path`` parameter to ``cythonize``::
+
+    from distutils.core import setup
+    from Cython.Build import cythonize
+
+    setup(
+        name="My hello app",
+        ext_modules=cythonize("src/*.pyx", include_path=[...]),
+    )
+
+Often, Python packages that offer a C-level API provide a way to find
+the necessary include files, e.g. for NumPy::
+
+    include_path = [numpy.get_include()]
+
+.. note::
+
+    Using memoryviews or importing NumPy with ``import numpy`` does not mean that
+    you have to add the path to NumPy include files. You need to add this path only
+    if you use ``cimport numpy``.
+
+Despite this, you will still get warnings like the
+following from the compiler, because Cython is using a deprecated Numpy API::
+
+   .../include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
+
+For the time being, it is just a warning that you can ignore.
+
+If you need to specify compiler options, libraries to link with or other
+linker options you will need to create ``Extension`` instances manually
+(note that glob syntax can still be used to specify multiple extensions
+in one line)::
+
+    from distutils.core import setup
+    from distutils.extension import Extension
+    from Cython.Build import cythonize
+
+    extensions = [
+        Extension("primes", ["primes.pyx"],
+            include_dirs=[...],
+            libraries=[...],
+            library_dirs=[...]),
+        # Everything but primes.pyx is included here.
+        Extension("*", ["*.pyx"],
+            include_dirs=[...],
+            libraries=[...],
+            library_dirs=[...]),
+    ]
+    setup(
+        name="My hello app",
+        ext_modules=cythonize(extensions),
+    )
+
+Note that when using setuptools, you should import it before Cython as
+setuptools may replace the ``Extension`` class in distutils.  Otherwise,
+both might disagree about the class to use here.
+
+Note also that if you use setuptools instead of distutils, the default
+action when running ``python setup.py install`` is to create a zipped
+``egg`` file which will not work with ``cimport`` for ``pxd`` files
+when you try to use them from a dependent package.
+To prevent this, include ``zip_safe=False`` in the arguments to ``setup()``.
+
+If your options are static (for example you do not need to call a tool like
+``pkg-config`` to determine them) you can also provide them directly in your
+.pyx or .pxd source file using a special comment block at the start of the file::
+
+    # distutils: libraries = spam eggs
+    # distutils: include_dirs = /opt/food/include
+
+If you cimport multiple .pxd files defining libraries, then Cython
+merges the list of libraries, so this works as expected (similarly
+with other options, like ``include_dirs`` above).
+
+If you have some C files that have been wrapped with Cython and you want to
+compile them into your extension, you can define the distutils ``sources``
+parameter::
+
+    # distutils: sources = helper.c, another_helper.c
+
+Note that these sources are added to the list of sources of the current
+extension module.  Spelling this out in the :file:`setup.py` file looks
+as follows::
+
+    from distutils.core import setup
+    from Cython.Build import cythonize
+    from distutils.extension import Extension
+
+    sourcefiles = ['example.pyx', 'helper.c', 'another_helper.c']
+
+    extensions = [Extension("example", sourcefiles)]
+
+    setup(
+        ext_modules=cythonize(extensions)
+    )
+
+The :class:`Extension` class takes many options, and a fuller explanation can
+be found in the `distutils documentation`_. Some useful options to know about
+are ``include_dirs``, ``libraries``, and ``library_dirs`` which specify where
+to find the ``.h`` and library files when linking to external libraries.
+
+.. _distutils documentation: https://docs.python.org/extending/building.html
+
+Sometimes this is not enough and you need finer customization of the
+distutils :class:`Extension`.
+To do this, you can provide a custom function ``create_extension``
+to create the final :class:`Extension` object after Cython has processed
+the sources, dependencies and ``# distutils`` directives but before the
+file is actually Cythonized.
+This function takes 2 arguments ``template`` and ``kwds``, where
+``template`` is the :class:`Extension` object given as input to Cython
+and ``kwds`` is a :class:`dict` with all keywords which should be used
+to create the :class:`Extension`.
+The function ``create_extension`` must return a 2-tuple
+``(extension, metadata)``, where ``extension`` is the created
+:class:`Extension` and ``metadata`` is metadata which will be written
+as JSON at the top of the generated C files. This metadata is only used
+for debugging purposes, so you can put whatever you want in there
+(as long as it can be converted to JSON).
+The default function (defined in ``Cython.Build.Dependencies``) is::
+
+    def default_create_extension(template, kwds):
+        if 'depends' in kwds:
+            include_dirs = kwds.get('include_dirs', []) + ["."]
+            depends = resolve_depends(kwds['depends'], include_dirs)
+            kwds['depends'] = sorted(set(depends + template.depends))
+
+        t = template.__class__
+        ext = t(**kwds)
+        metadata = dict(distutils=kwds, module_name=kwds['name'])
+        return ext, metadata
+
+In case that you pass a string instead of an :class:`Extension` to
+``cythonize()``, the ``template`` will be an :class:`Extension` without
+sources. For example, if you do ``cythonize("*.pyx")``,
+the ``template`` will be ``Extension(name="*.pyx", sources=[])``.
+
+Just as an example, this adds ``mylib`` as library to every extension::
+
+    from Cython.Build.Dependencies import default_create_extension
+
+    def my_create_extension(template, kwds):
+        libs = kwds.get('libraries', []) + ["mylib"]
+        kwds['libraries'] = libs
+        return default_create_extension(template, kwds)
+
+    ext_modules = cythonize(..., create_extension=my_create_extension)
+
+.. note::
+
+    If you Cythonize in parallel (using the ``nthreads`` argument),
+    then the argument to ``create_extension`` must be pickleable.
+    In particular, it cannot be a lambda function.
+
 
 Multiple Cython Files in a Package
 ===================================