Writing C Code in Cython

Original author: Matthew Honnibal
  • Transfer
For the past two years, I have been solving all problems exclusively on Cython . This does not mean at all that I write in Python, and then “Sitonize” it using various type declarations, no, I just write in Cython. I use raw C structures and arrays (and sometimes C ++ vectors) and a small wrapper around malloc / free, which I wrote myself. The code runs almost as fast as C / C ++, because that is the C / C ++ code decorated with syntactic sugar. This is C / C ++ code with Python functionality exactly where I need it and where I want it.

In fact, this is the opposite version of the standard application of languages ​​similar to Python: you write the entire application in Python, optimize important places in C and ... Profit! Speed ​​C, Python convenience, sheep safe, wolves full.

In theory, this always looks better than in practice. In practice, your data structures have a huge impact on the efficiency of your code and the complexity of writing it. Working with arrays is always a pain, but they are fast. Lists are extremely convenient, but very slow. Loops and function calls in Python are always slow, so the part of the application that you write in C tends to grow and grow until almost all of your application is written in C.

A post about writing C extensions for Python was recently published . The author wrote an implementation of the algorithm in pure Python and C using the Numpy C API. I decided that this was a good opportunity to demonstrate the differences, and, for comparison, I wrote my version in Cython:

import random
from cymem.cymem cimport Pool
from libc.math cimport sqrt
cimport cython
cdef struct Point:
    double x
    double y
cdef class World:
    cdef Pool mem
    cdef int N
    cdef double* m
    cdef Point* r
    cdef Point* v
    cdef Point* F
    cdef readonly double dt
    def __init__(self, N, threads=1, m_min=1, m_max=30.0, r_max=50.0, v_max=4.0, dt=1e-3):
        self.mem = Pool()
        self.N = N
        self.m = self.mem.alloc(N, sizeof(double))
        self.r = self.mem.alloc(N, sizeof(Point))
        self.v = self.mem.alloc(N, sizeof(Point))
        self.F = self.mem.alloc(N, sizeof(Point))
        for i in range(N):
            self.m[i] = random.uniform(m_min, m_max)
            self.r[i].x = random.uniform(-r_max, r_max)
            self.r[i].y = random.uniform(-r_max, r_max)
            self.v[i].x = random.uniform(-v_max, v_max)
            self.v[i].y = random.uniform(-v_max, v_max)
            self.F[i].x = 0
            self.F[i].y = 0
        self.dt = dt
@cython.cdivision(True)
def compute_F(World w):
    """Compute the force on each body in the world, w."""
    cdef int i, j
    cdef double s3, tmp
    cdef Point s
    cdef Point F
    for i in range(w.N):
        # Set all forces to zero. 
        w.F[i].x = 0
        w.F[i].y = 0
        for j in range(i+1, w.N):
            s.x = w.r[j].x - w.r[i].x
            s.y = w.r[j].y - w.r[i].y
            s3 = sqrt(s.x * s.x + s.y * s.y)
            s3 *= s3 * s3;
            tmp = w.m[i] * w.m[j] / s3
            F.x = tmp * s.x
            F.y = tmp * s.y
            w.F[i].x += F.x
            w.F[i].y += F.y
            w.F[j].x -= F.x
            w.F[j].y -= F.y
@cython.cdivision(True)
def evolve(World w, int steps):
    """Evolve the world, w, through the given number of steps."""
    cdef int _, i
    for _ in range(steps):
        compute_F(w)
        for i in range(w.N):
            w.v[i].x += w.F[i].x * w.dt / w.m[i]
            w.v[i].y += w.F[i].y * w.dt / w.m[i]
            w.r[i].x += w.v[i].x * w.dt
            w.r[i].y += w.v[i].y * w.dt

This version in Cython was written in 30 minutes, and it is as fast as the code in C. Actually, why not, because this is the code in C, just written using syntactic sugar. And you don’t even have to think about the complex and hostile C API and learn it, you just ... just write C or C ++ code. Both versions, C and Cython, are about 70 times faster than the pure Python version, given that it uses Numpy arrays.

Only one difference from C: I use a small wrapper for malloc / free, which I wrote myself - cymem . It remembers the used memory addresses, and when the garbage collector fires, it simply frees up unnecessary memory. Since I started using this wrapper, I have never had problems with memory leaks.

An intermediate option to write in Cython is to use typed memory-views, which allows you to work with multidimensional Numpy arrays. However, for me it looks more complicated. I usually work with simpler arrays in my applications, and prefer to define my own data structures.

Translated Dreadatour , the text read% username%.

Also popular now: