PyPy learned how to directly load CPython extensions

PyPy now able to download and run CPython'a expansion (ie, .pydand .sofiles) directly by means of a new subsystem called CPyExt. Unlike the solution presented in another blog post (in which numpy-type extension modules were launched on CPython and proxied via TCP), the new solution no longer requires running CPython. We have not yet reached full binary compatibility (like Ironclad), but in most cases, just recompiling the extension is enough.

The only requirement is that the necessary C API functions from CPython must be implemented in PyPy. If you are a user or author of a module, and you are missing some features in PyPy, we invite you to implement them. Already quite a few people (including many new contributors) have already joined the development and implemented some functions in order for their favorite modules to work. At the end of the post is a list of names.

Speaking of speed. Despite the fact that there is overhead for launching these modules, when we start the regular expression engine from CPython ( _sre.so) and go through the spambayes benchmark from the Unladen Swallow benchmark set (see speed.pypy.org ), we get significant acceleration: the test is twice as fast compared to the regular engine built into PyPy. OfAmdahl’s law implies that it _sre.soshould work several times faster than the built-in engine.

At the moment, work is underway on PIL and other modules. For example, distutils support is almost complete. And if you want to participate in this or get more information on how to use this new opportunity, go to our IRC channel #pypyon freenode .

CPyExt Developers:

• Alex gaynor
  
• Benjamin peterson
  
• Jean-paul calderone
  
• Maciej fijalkowski
  
• Jan de mooij
  
• Lucian Branescu Mihaila
  
• Andreas Stührk
  
• Zooko Wilcox-O Hearn