February 6, 2013 at 15:42Combining C ++ and Python. Subtleties of Boost.Python. Part two
- Tutorial
This article is a continuation of the first part .
Continuing to torment Boost.Python. This time it is the turn of the class, which can neither be created nor copied.
We wrap an almost ordinary sychny structure with an unusual constructor.
And we’ll work with returning the reference to the field of the C ++ object, so that the Python garbage collector does not delete it inadvertently. And vice versa, we’ll make an alternative for Python to clean up the garbage after deleting what has been stored.
Go…
For our purposes, it is quite enough to supplement the example project remaining from the previous part.
Let's add a couple more files to it to work with the singleton class:
single.h
single.cpp
And put the declarations of auxiliary functions for wrapping in Python into a separate file:
wrap.h
A file should remain from the previous project, which we will actively change:
wrap .cpp
And wonderful files with a miracle class that helped us so much in the first part, they will remain as they are:
some.h
some.cpp
To begin with, let's create a small C-style structure in single.h, just with a description of the fields.
For fun, let's say this will not be just a structure, but some mysterious type of configuration description:
It is not difficult to make a wrapper for such a structure, you only need to specifically describe the constructor with parameters using the constructor template boost :: python :: init <...> (...) the parameter of the boost :: python :: wrapper wrapper template:
As you can see, here you don’t even have to use return_value_policyfor a string field. Just because the field here is taken in essence by value, and therefore automatically converted to the standard string str of the Python language.
The make_setter functions do a very useful job of checking the type of the input value, for example, try assigning a string type value to the coef field in python or setting max_size to a float value, get an exception.
Fields of the Config structure are essentially transformed into properties of the object of a full-fledged Python class Config. Well, almost full-fledged ... Let's, by analogy with the Some class from the previous chapter, add the __str__ and __repr__ methods to the wrapper , and at the same time add the as_dict property to convert the structure fields to the standard python dict and vice versa.
The declaration of new functions, as well as old ones, is transferred to our new wrap.h file:
There will be nothing superfluous in the wrap.cpp file and immediately there will be an declaration of the example module, which will obviously add readability.
At the end of wrap.cpp, we will write an implementation of our new functions, similar to the way we wrote them in the first part:
This, of course, I am already mad with fat, but let's call it a repetition of the past.
Of course, we add new declarations to the structure wrapper:
There is nothing complicated with the structure, it made a wonderful Python class that mirrors the properties of the Config structure in C ++ and the class is completely pythonistic. The only problem with this class will be that when creating it, you will need to specify something in the constructor.
To fill in the configuration parameters and access them, let's get a singleton, at the same time supply it with a “useful” counter.
So singleton. Let it contain the aforementioned configuration parameters of the current application and some kind of magic counter to get the current identifier.
As you probably noticed, I don’t really like to pull out the useless Instance () method in the public section and prefer to work with the singleton functionality as a set of static methods. From this, the singleton does not cease to be a singleton, and the class user will thank you for hiding the call to Instance () in the implementation.
Here is the actual implementation in the single.cpp file :
There are only three static methods, the wrapper should not be complicated, if you do not take into account one thing ... but no, not exactly one:
1. You cannot create an instance of a class
2. You cannot copy an instance of a class
3. We have not wrapped static methods yet
As you can see, all the difficulties associated with the 1st and 2nd points come down to specifying the template parameter boost :: noncopyable and passing the parameter boost :: python :: no_init to the template class constructor boost :: python :: class_.
If you want the class to support copying or contain the default constructor, you can erase the corresponding silencer of the wrapper class property generator.
Generally speaking, the default constructor can be declared below in .def (init <> ()) , some do so, for consistency with other constructors with parameters described separately, also passing no_initinto the constructor of the wrapper template. There is also the option of replacing the default constructor with a constructor with parameters right when declaring a wrapper class, as we already did for the Config structure.
With the third paragraph, everything is simple in general, by declaring that the static method deals with .staticmethod () after declaring through .def () the wrapper of all overloads of this method.
In general, the rest no longer raises questions and is familiar to us in the first part, except for one funny little thing - the policy of return_value_policy return value by reference, about it further.
The biggest difficulty in wrapping our singleton's methods was the return of the object reference from the method
Just so that the Garbage Collector (GC) of the Python interpreter does not delete the contents of the field of the class returned by reference from the method, we use return_value_policy
The magic of Boost.Python is so harsh that when you execute Python code, changing the fields of the result of AppConfig () will lead to changes in the singleton field as if it were happening in C ++! By running the following code on the Python command line:
We get the conclusion:
Probably everyone has already noticed that I like to overload a method or two in the example so that the ad will turn out to be as furious as possible. Now, for the convenience of using the Single class in python, we will add properties for reading the counter and for getting and setting configuration parameters, since all the methods for this already exist.
The Single :: CurrentID method is wrapped by the current_id property for one or two, but see what a “beautiful” wrapper for the two overloads of Single :: AppConfig, respectively, get- and set-methods of the app_config property . And note that for the get method we had to use the special make_function function in order to set the return value policy return_value_policy
Be very careful, you cannot use the make_getter function for methods, it is used only for C ++ class fields, for methods you need to use functions as is. If you need to set a return value policy in one of the methods for one of the property methods, you need to use make_function . Subsidiary additional argument for return_value_policy in .def you have not, so you have to pass one argument and the function immediately, and return value policy.
So, we have already figured out how to prevent the Python GC from deleting the object returned by reference. However, sometimes you need to transfer a new object to python for storage. GC will correctly delete the object as soonas the last variable, referring to your result, dies in torment . There is a return_value_policy policy for this .
Let's get a method that clones configuration parameters into a new object. Add an ad to wrap.h:
And in wrap.cpp we add its implementation:
In the wrapper of the Single class, a new method with the manage_new_object policy will accordingly appear:
To verify that Config is indeed removed when necessary, declare the destructor in the completely non-C-style Config structure. In the destructor, we simply output to STDOUT via std :: cout the fields of the deleted Config instance:
We try!
In the test script in Python 3.x, we clone the configs, modify them in any way and reset all the links to the object created through CloneAppConfig () :
Destructors are called exactly when it is expected, when the last link to the object disappears.
Here's what should be displayed:
As a homework, try adding the __del__ method, an analogue of the destructor in Python, to the Config wrapper, you will see how the wrappers and the objects to which they refer are unlike.
So, we got acquainted in practice with two new return value policies by reference: reference_existing_object and manage_new_object . That is, we learned how to use the wrapper object of the return value as a reference to an existing C ++ object, and also to transfer new objects created in C ++ to the care of GC Python.
We figured out briefly how to act if the default class constructors in C ++ are imposed. This is true not only in the case of a singleton or an abstract class, but also for many specific classes, examples of which you probably have before your eyes.
In the third part, we will find a simple enum wrapper, we will write our own converter for a byte array from C ++ to Python and vice versa, and also learn how to use the inheritance of C ++ classes at the level of their wrappers.
Next, we are waiting for the magical world of converting exceptions from C ++ to Python and vice versa.
What will happen until I guess, the topic is being developed like a ball: so small and compact, until you start unwinding it ... A
link to the project of the 2nd part can be found here . MSVS v11 project, configured to build with Python 3.3 x64.
Boost.Python Documentation Booster
constructor for boost :: python :: class_
Policies for link return values in Boost.Python
Getting Started with Boost for Windows
Getting Started with Boost for * nix
Subtleties of Boost.Python assembly
Continuing to torment Boost.Python. This time it is the turn of the class, which can neither be created nor copied.
We wrap an almost ordinary sychny structure with an unusual constructor.
And we’ll work with returning the reference to the field of the C ++ object, so that the Python garbage collector does not delete it inadvertently. And vice versa, we’ll make an alternative for Python to clean up the garbage after deleting what has been stored.
Go…
Table of contents
- Combining C ++ and Python. Subtleties of Boost.Python. Part one
- Combining C ++ and Python. Subtleties of Boost.Python. Part two
- Type conversion in Boost.Python. We do the conversion between the familiar types of C ++ and Python
- An Exception Journey Between C ++ and Python or Round Trip
We are preparing a project
For our purposes, it is quite enough to supplement the example project remaining from the previous part.
Let's add a couple more files to it to work with the singleton class:
single.h
single.cpp
And put the declarations of auxiliary functions for wrapping in Python into a separate file:
wrap.h
A file should remain from the previous project, which we will actively change:
wrap .cpp
And wonderful files with a miracle class that helped us so much in the first part, they will remain as they are:
some.h
some.cpp
Wrap a simple structure
To begin with, let's create a small C-style structure in single.h, just with a description of the fields.
For fun, let's say this will not be just a structure, but some mysterious type of configuration description:
struct Config
{
double coef;
string path;
int max_size;
Config( double def_coef, string const& def_path, int def_max_size );
};
It is not difficult to make a wrapper for such a structure, you only need to specifically describe the constructor with parameters using the constructor template boost :: python :: init <...> (...) the parameter of the boost :: python :: wrapper wrapper template:
class_( "Config", init( args( "coef", "path", "max_size" ) )
.add_property( "coef", make_getter( &Config::coef ), make_setter( &Config::coef ) )
.add_property( "path", make_getter( &Config::path ), make_setter( &Config::path ) )
.add_property( "max_size", make_getter( &Config::max_size ), make_setter( &Config::max_size ) )
;
As you can see, here you don’t even have to use return_value_policy
The make_setter functions do a very useful job of checking the type of the input value, for example, try assigning a string type value to the coef field in python or setting max_size to a float value, get an exception.
Fields of the Config structure are essentially transformed into properties of the object of a full-fledged Python class Config. Well, almost full-fledged ... Let's, by analogy with the Some class from the previous chapter, add the __str__ and __repr__ methods to the wrapper , and at the same time add the as_dict property to convert the structure fields to the standard python dict and vice versa.
The declaration of new functions, as well as old ones, is transferred to our new wrap.h file:
#pragma once
#include
#include "some.h"
#include "single.h"
using namespace boost::python;
string Some_Str( Some const& );
string Some_Repr( Some const& );
dict Some_ToDict( Some const& );
void Some_FromDict( Some&, dict const& );
string Config_Str( Config const& );
string Config_Repr( Config const& );
dict Config_ToDict( Config const& );
void Config_FromDict( Config&, dict const& );
There will be nothing superfluous in the wrap.cpp file and immediately there will be an declaration of the example module, which will obviously add readability.
At the end of wrap.cpp, we will write an implementation of our new functions, similar to the way we wrote them in the first part:
string Config_Str( Config const& config )
{
stringstream output;
output << "{ coef: " << config.coef << ", path: '" << config.path << "', max_size: " << config.max_size << " }";
return output.str();
}
string Config_Repr( Config const& config )
{
return "Config: " + Config_Str( config );
}
dict Config_ToDict( Config const& config )
{
dict res;
res["coef"] = config.coef;
res["path"] = config.path;
res["max_size"] = config.max_size;
return res;
}
void Config_FromDict( Config& config, dict const& src )
{
if( src.has_key( "coef" ) ) config.coef = extract( src["coef"] );
if( src.has_key( "path" ) ) config.path = extract( src["path"] );
if( src.has_key( "max_size" ) ) config.max_size = extract( src["max_size"] );
}
This, of course, I am already mad with fat, but let's call it a repetition of the past.
Of course, we add new declarations to the structure wrapper:
class_( "Config", init( args( "coef", "path", "max_size" ) ) )
.add_property( "coef", make_getter( &Config::coef ), make_setter( &Config::coef ) )
.add_property( "path", make_getter( &Config::path ), make_setter( &Config::path ) )
.add_property( "max_size", make_getter( &Config::max_size ), make_setter( &Config::max_size ) )
.def( "__str__", Config_Str )
.def( "__repr__", Config_Repr )
.add_property( "as_dict", Config_ToDict, Config_FromDict )
;
There is nothing complicated with the structure, it made a wonderful Python class that mirrors the properties of the Config structure in C ++ and the class is completely pythonistic. The only problem with this class will be that when creating it, you will need to specify something in the constructor.
To fill in the configuration parameters and access them, let's get a singleton, at the same time supply it with a “useful” counter.
Class wrapper without the ability to create and copy
So singleton. Let it contain the aforementioned configuration parameters of the current application and some kind of magic counter to get the current identifier.
class Single
{
public:
static int CurrentID();
static Config& AppConfig();
static void AppConfig( Config const& );
private:
int mCurrentID;
Config mAppConfig;
Single();
Single( Single const& );
static Single& Instance();
int ThisCurrentID();
Config& ThisAppConfig();
void ThisAppConfig( Config const& );
};
As you probably noticed, I don’t really like to pull out the useless Instance () method in the public section and prefer to work with the singleton functionality as a set of static methods. From this, the singleton does not cease to be a singleton, and the class user will thank you for hiding the call to Instance () in the implementation.
Here is the actual implementation in the single.cpp file :
#include "single.h"
#include
using boost::mutex;
using boost::unique_lock;
const double CONFIG_DEFAULT_COEF = 2.5;
const int CONFIG_DEFAULT_MAX_SIZE = 0x1000;
const string CONFIG_DEFAULT_PATH = ".";
int Single::CurrentID()
{
return Instance().ThisCurrentID();
}
Config& Single::AppConfig()
{
return Instance().ThisAppConfig();
}
void Single::AppConfig( Config const& config )
{
Instance().ThisAppConfig( config );
}
Single::Single()
: mCurrentID( 0 )
{
mAppConfig.coef = CONFIG_DEFAULT_COEF;
mAppConfig.max_size = CONFIG_DEFAULT_MAX_SIZE;
mAppConfig.path = CONFIG_DEFAULT_PATH;
}
Single& Single::Instance()
{
static mutex single_mutex;
unique_lock single_lock( single_mutex );
static Single instance;
return instance;
}
int Single::ThisCurrentID()
{
static mutex id_mutex;
unique_lock id_lock( id_mutex );
return ++mCurrentID;
}
Config& Single::ThisAppConfig()
{
return mAppConfig;
}
void Single::ThisAppConfig( Config const& config )
{
mAppConfig = config;
}
There are only three static methods, the wrapper should not be complicated, if you do not take into account one thing ... but no, not exactly one:
1. You cannot create an instance of a class
2. You cannot copy an instance of a class
3. We have not wrapped static methods yet
class_( "Single", no_init )
.def( "CurrentID", &Single::CurrentID )
.staticmethod( "CurrentID" )
.def( "AppConfig", static_cast< Config& (*)() >( &Single::AppConfig ), return_value_policy() )
.def( "AppConfig", static_cast< void (*)( Config const& ) >( &Single::AppConfig ) )
.staticmethod( "AppConfig" )
;
As you can see, all the difficulties associated with the 1st and 2nd points come down to specifying the template parameter boost :: noncopyable and passing the parameter boost :: python :: no_init to the template class constructor boost :: python :: class_.
If you want the class to support copying or contain the default constructor, you can erase the corresponding silencer of the wrapper class property generator.
Generally speaking, the default constructor can be declared below in .def (init <> ()) , some do so, for consistency with other constructors with parameters described separately, also passing no_initinto the constructor of the wrapper template. There is also the option of replacing the default constructor with a constructor with parameters right when declaring a wrapper class, as we already did for the Config structure.
With the third paragraph, everything is simple in general, by declaring that the static method deals with .staticmethod () after declaring through .def () the wrapper of all overloads of this method.
In general, the rest no longer raises questions and is familiar to us in the first part, except for one funny little thing - the policy of return_value_policy return value by reference
The policy "do not beat me, I am a translator"
The biggest difficulty in wrapping our singleton's methods was the return of the object reference from the method
static Config& AppConfig();
Just so that the Garbage Collector (GC) of the Python interpreter does not delete the contents of the field of the class returned by reference from the method, we use return_value_policy
The magic of Boost.Python is so harsh that when you execute Python code, changing the fields of the result of AppConfig () will lead to changes in the singleton field as if it were happening in C ++! By running the following code on the Python command line:
from example import *
Single.AppConfig().coef = 123.45
Single.AppConfig()
We get the conclusion:
Config: { coef: 123.45, path: '.', max_size: 4096 }
Add a property with a policy for the get method
Probably everyone has already noticed that I like to overload a method or two in the example so that the ad will turn out to be as furious as possible. Now, for the convenience of using the Single class in python, we will add properties for reading the counter and for getting and setting configuration parameters, since all the methods for this already exist.
.add_static_property( "current_id", &Single::CurrentID )
.add_static_property( "app_config", make_function( static_cast< Config& (*)() >( &Single::AppConfig ), return_value_policy() ), static_cast< void (*)( Config const& ) >( &Single::AppConfig ) )
The Single :: CurrentID method is wrapped by the current_id property for one or two, but see what a “beautiful” wrapper for the two overloads of Single :: AppConfig, respectively, get- and set-methods of the app_config property . And note that for the get method we had to use the special make_function function in order to set the return value policy return_value_policy
Be very careful, you cannot use the make_getter function for methods, it is used only for C ++ class fields, for methods you need to use functions as is. If you need to set a return value policy in one of the methods for one of the property methods, you need to use make_function . Subsidiary additional argument for return_value_policy in .def you have not, so you have to pass one argument and the function immediately, and return value policy.
The policy "here is a new object - delete it"
So, we have already figured out how to prevent the Python GC from deleting the object returned by reference. However, sometimes you need to transfer a new object to python for storage. GC will correctly delete the object as soon
Let's get a method that clones configuration parameters into a new object. Add an ad to wrap.h:
Config* Single_CloneAppConfig();
And in wrap.cpp we add its implementation:
Config* Single_CloneAppConfig()
{
return new Config( Single::AppConfig() );
}
In the wrapper of the Single class, a new method with the manage_new_object policy will accordingly appear:
.def( "CloneAppConfig", Single_CloneAppConfig, return_value_policy() )
To verify that Config is indeed removed when necessary, declare the destructor in the completely non-C-style Config structure. In the destructor, we simply output to STDOUT via std :: cout the fields of the deleted Config instance:
Config::~Config()
{
cout << "Config destructor of Config: { coef: " << coef
<< ", path: '" << path << "', max_size: " << max_size << " }" << endl;
}
We try!
In the test script in Python 3.x, we clone the configs, modify them in any way and reset all the links to the object created through CloneAppConfig () :
from example import *
c = Single.CloneAppConfig()
c.coef = 11.11; c.path = 'cloned'; c.max_size = 111111
print( "c.coef = 12.34; c.path = 'cloned'; c.max_size = 100500" )
print( "c:", c ); print( "Single.AppConfig():", Single.AppConfig() )
print( "c = Single.CloneAppConfig()" ); c = Single.CloneAppConfig()
c.coef = 22.22; c.path = 'another'; c.max_size = 222222
print( "c.coef = 22.22; c.path = 'another'; c.max_size = 222222" )
print( "c:", c ); print( "Single.app_config:", Single.app_config )
print( "c = None" ); c = None
print( "Single.app_config:", Single.app_config )
Destructors are called exactly when it is expected, when the last link to the object disappears.
Here's what should be displayed:
c.coef = 12.34; c.path = 'cloned'; c.max_size = 100500
c: { coef: 11.11, path: 'cloned', max_size: 111111 }
Single.AppConfig(): { coef: 2.5, path: '.', max_size: 4096 }
c = Single.CloneAppConfig()
Config::~Config() destructor of object: { coef: 11.11, path: 'cloned', max_size: 111111 }
c.coef = 22.22; c.path = 'another'; c.max_size = 222222
c: { coef: 22.22, path: 'another', max_size: 222222 }
Single.app_config: { coef: 2.5, path: '.', max_size: 4096 }
c = None
Config::~Config() destructor of object: { coef: 22.22, path: 'another', max_size: 222222 }
Single.app_config: { coef: 2.5, path: '.', max_size: 4096 }
Config::~Config() destructor of object: { coef: 2.5, path: '.', max_size: 4096 }
As a homework, try adding the __del__ method, an analogue of the destructor in Python, to the Config wrapper, you will see how the wrappers and the objects to which they refer are unlike.
In the conclusion of the second part
So, we got acquainted in practice with two new return value policies by reference: reference_existing_object and manage_new_object . That is, we learned how to use the wrapper object of the return value as a reference to an existing C ++ object, and also to transfer new objects created in C ++ to the care of GC Python.
We figured out briefly how to act if the default class constructors in C ++ are imposed. This is true not only in the case of a singleton or an abstract class, but also for many specific classes, examples of which you probably have before your eyes.
In the third part, we will find a simple enum wrapper, we will write our own converter for a byte array from C ++ to Python and vice versa, and also learn how to use the inheritance of C ++ classes at the level of their wrappers.
Next, we are waiting for the magical world of converting exceptions from C ++ to Python and vice versa.
What will happen until I guess, the topic is being developed like a ball: so small and compact, until you start unwinding it ... A
link to the project of the 2nd part can be found here . MSVS v11 project, configured to build with Python 3.3 x64.
useful links
Boost.Python Documentation Booster
constructor for boost :: python :: class_
Policies for link return values in Boost.Python
Getting Started with Boost for Windows
Getting Started with Boost for * nix
Subtleties of Boost.Python assembly