Software contribution and development guide

Mixed Coding Arts

C/C++ in Python

We encourage three main methods of incorporating C/C++ into python:

These both produce reliable results without auto-generating code in unpredictable fashion. For a good example on how to properly wrap a large C-project in python see rebound.

ctypes

Below is an example of how to wrap some C-code that also uses complex numbers and can pass those complex numbers back and forth between C and Python seamlessly. First we define our c-code in the file foo.c:

#include <stdio.h>
#include <complex.h>


void foo(double* in_array, double* out_array, int length) {
    int i;
    for(i = 0; i<length; i++) {
        printf("%f\n", in_array[i]);
        out_array[i] = in_array[i]*2;
    }
}

void cfoo(float complex* in_array, float complex* out_array, int length) {
    int i;
    for(i = 0; i<length; i++) {
        printf("%f + i%f\n", creal(in_array[i]), cimag(in_array[i]));
        out_array[i] = in_array[i]*2;
    }
}

We then need to compile this code with gcc -lm foo.c -fPIC -shared -o foo.so. For more information on the specific flags used see gcc docs.

If the python code is a package this compilation will later be done automatically by setuptools using the setuptools.Extension functionality so that the code is automatically compiled during a pip install. The below code uses ctypes to load the shared c-library, declare the c-interface to the functions within that library and then call those functions. The below code is what can be called the “pythonification” of the c-code by wrapping the c-library usage with in python equivalent functions.

import ctypes

import numpy as np
import numpy.ctypeslib as npct

lib = ctypes.cdll.LoadLibrary('./foo.so')

# Define the C-interface
lib.foo.restype = None
lib.foo.argtypes = [
    npct.ndpointer(np.float64, ndim=1, flags='aligned, contiguous'),
    npct.ndpointer(np.float64, ndim=1, flags='aligned, contiguous, writeable'),
    ctypes.c_int,
]

lib.cfoo.restype = None
lib.cfoo.argtypes = [
    npct.ndpointer(np.complex64, ndim=1, flags='aligned, contiguous'),
    npct.ndpointer(np.complex64, ndim=1, flags='aligned, contiguous, writeable'),
    ctypes.c_int
]


def foo(values):
    '''Double the values of the input numpy array and returns a new array.
    Supports float64 and complex64 dtypes.
    '''
    result = np.empty_like(values)
    length = ctypes.c_int(len(values))

    c_func = lib.cfoo if np.iscomplexobj(values) else lib.foo

    c_func(values, result, length)

    return ret



if __name__ == '__main__':
    num = 10

    print('RUNNING FOO')
    x = np.arange(num, dtype=np.float64)

    y = foo(x)

    print(f'INPUT : {x}')
    print(f'OUTPUT: {y}')


    print('\nRUNNING COMPLEX-FOO')
    x = (np.arange(num) + 1j*np.arange(num)[::-1]).astype(np.complex64)

    y = foo(x)

    print(f'INPUT : {x}')
    print(f'OUTPUT: {y}')

"Python.h" header

In the previous example, if we would have tried to import foo rather than ctypes.cdll.LoadLibrary('./foo.so') we would have encountered the following error.

Traceback (most recent call last):
  File "python_foo.py", line 7, in <module>
    import foo as lib
ImportError: dynamic module does not define module export function (PyInit_foo)

This leads us to the topic of building C/C++ code with the specific purpose of being used in python. For extensive information on the topic see the official documentation.

To summarize

  1. We first need a C-file which defines a Python module, an example file can be found here.
  2. Then we need to compile the library with the correct c-flags (it needs to be a shared library), libraries and includes. This can be done by calling the python3-config command. Here we have made a simple makefile to do this and the compiling for us.
  3. Lastly we import our shared module into a python file and run a C-function from it.

Other options

In rare cases one might need to automatically convert code to interface with python. In these situations CFFI might be an option.

FORTRAN in Python

We recommend using https://numpy.org/doc/stable/f2py/ for this purpose. If this is not an options, one can always wrap FORTRAN in C-code and then compile the C-code as an loadable library.

Rust in Python and Python in Rust

Rust has existing solutions for wrapping found in pyo3.

Video tutorials