Switch to Boost-1.65.1 and bugs that surfaced
Last year (almost a year has already passed), we still switched to the new version of Boost-1.65.1, and under the hood you will find the three boost bugs that we encountered. It is also important to mention that before we used boost -1.62.1 in the software, since some bugs appeared in boost earlier than version 1.65.1
. Our project has a special integration team whose main task is to migrate all the software to the new version of libraries , Visual Studio, newer versions of low-level components (base ones, on which most other components depend), etc. Also, the integration team is responsible for eliminating all the problems that arise, of course, with the assistance of the component maintainers, if necessary. So, the bugs that I especially remember.
This bug has surfaced fairly quickly. Tests began to fall with “Access violation” when searching for the full path to the specified file name. The function calls boost :: filesystem :: exist, and the program crashes. Running a few more tests, several more similar cases were noticed, while in all cases the call to boost :: filesystem :: exist was made for global variables. Apparently, something has changed in the lifetime of the boost-variable variables. Ticket for the detected bug is very easy google bug bug in boost :: filesystem :: exist
It turned out that this bug was in boost, starting with version 1.64. In fact, the problem was in the make_permissions call (used in filesystem :: exist). In 1.64, the implementation of make_permissions was changed and now used global variables, which means that when an attempt is made to call filesystem :: exist when initializing a global variable or object, the global variables used in make_permissions may not yet be initialized. Therefore, an attempt to access an uncreated variable throws an exception.
In tests using boost :: python, a strange thing was discovered. When making a trivial call to eval () for a literal (for example, "40 + 2") all the rules. And if the variables are determined and then used in expressions, we get the message that undefined variables are used in the calculations (ERROR: [name] not defined). To solve this problem, I have spent more time. I could not find the problem ticket in the boost tracker bug, so I had to ask for help from the command of this component. Information about the bug was promptly found on github .
It so happened that the global and local objects were not used in the implementation of eval. Wishing Good luck to find a fix without recompiling the source code, the team declined :)
This is my least favorite part. boost :: python :: numeric has been removed and now boost :: python :: numpy appears as an alternative. But the code that used numeric had to be pretty much redone, since it’s not just a matter of renaming namespaces, but also of implementing objects.
In addition, there was misinformation in the boost header, which misled me.
According to the source comment, the import_array () call is already done in numpy :: initialize ():
But in fact, as it turned out, import_array () is required.
In addition, there were problems with testing the changes, since pieces of code with numpy (previously with boost :: python :: numeric) were not covered at all by tests, and the code itself was also used in another component. Therefore, problems were detected only when testing the corresponding component. The integration team is not required to write tests for the components, and this situation was the omission of the command itself. Wow, I heard a lot from them that broke their code. But after the team grumbled, they finally covered their code with tests. However, the offense remained (during the next migration, the team did not want to give access to its component to my colleague, mentioning that last time, we broke the code for them. Sasha, weeds! But after three days of negotiations, they surrendered).
After this work, I can point out some positives for myself, since I had not used boost very often before (mostly std), so I can emphasize a lot of new things from migration. It's funny, but it’s a fact, for some reason after this you become a “boost expert” for many colleagues, and, accept, you will be asked questions about it for some time.
By the way, in recent years, many companies have begun to actively get rid of boost and, if possible, replace std with a library or something else if there are no possibilities in the standard library. And we also did not stand aside. The process started, but not completed, still a lot of work.
. Our project has a special integration team whose main task is to migrate all the software to the new version of libraries , Visual Studio, newer versions of low-level components (base ones, on which most other components depend), etc. Also, the integration team is responsible for eliminating all the problems that arise, of course, with the assistance of the component maintainers, if necessary. So, the bugs that I especially remember.
Bug in boost :: filesystem
This bug has surfaced fairly quickly. Tests began to fall with “Access violation” when searching for the full path to the specified file name. The function calls boost :: filesystem :: exist, and the program crashes. Running a few more tests, several more similar cases were noticed, while in all cases the call to boost :: filesystem :: exist was made for global variables. Apparently, something has changed in the lifetime of the boost-variable variables. Ticket for the detected bug is very easy google bug bug in boost :: filesystem :: exist
It turned out that this bug was in boost, starting with version 1.64. In fact, the problem was in the make_permissions call (used in filesystem :: exist). In 1.64, the implementation of make_permissions was changed and now used global variables, which means that when an attempt is made to call filesystem :: exist when initializing a global variable or object, the global variables used in make_permissions may not yet be initialized. Therefore, an attempt to access an uncreated variable throws an exception.
Workaround
For tests where global variables were used only once, they were transferred to the corresponding tests and became local variables. Do not even ask why this was not done before, I am not the maintainer of this code.
In other cases, singletons were used .
In other cases, singletons were used .
Bug in boost :: python
In tests using boost :: python, a strange thing was discovered. When making a trivial call to eval () for a literal (for example, "40 + 2") all the rules. And if the variables are determined and then used in expressions, we get the message that undefined variables are used in the calculations (ERROR: [name] not defined). To solve this problem, I have spent more time. I could not find the problem ticket in the boost tracker bug, so I had to ask for help from the command of this component. Information about the bug was promptly found on github .
It so happened that the global and local objects were not used in the implementation of eval. Wishing Good luck to find a fix without recompiling the source code, the team declined :)
Workaround
Но тут я вспомнила про release notes для boost-1.65.1 и там точно что-то было для boost::python.
Ура, выход есть! Баг был допущен при добавлении новой имплементации eval c поддержкой char const * аргумента, которая теперь вызывается в старой имплементации eval со string аргументом(Особо внимательные могли заметить вызов этой функции в коде по github-овской ссылке). И новая функция, как ожидалась, работает.
Ура, выход есть! Баг был допущен при добавлении новой имплементации eval c поддержкой char const * аргумента, которая теперь вызывается в старой имплементации eval со string аргументом(Особо внимательные могли заметить вызов этой функции в коде по github-овской ссылке). И новая функция, как ожидалась, работает.
boost :: numpy
This is my least favorite part. boost :: python :: numeric has been removed and now boost :: python :: numpy appears as an alternative. But the code that used numeric had to be pretty much redone, since it’s not just a matter of renaming namespaces, but also of implementing objects.
In addition, there was misinformation in the boost header, which misled me.
According to the source comment, the import_array () call is already done in numpy :: initialize ():
namespace boost { namespace python { namespace numpy {
/**
* @brief Initialize the Numpy C-API
*
* This must be called before using anything in boost.numpy;
* It should probably be the first line inside BOOST_PYTHON_MODULE.
*
* @internal This just calls the Numpy C-API functions "import_array()"
* and "import_ufunc()", and then calls
* dtype::register_scalar_converters().
*/BOOST_NUMPY_DECL voidinitialize(bool register_scalar_converters=true);
}}} // namespace boost::python::numpyBut in fact, as it turned out, import_array () is required.
In addition, there were problems with testing the changes, since pieces of code with numpy (previously with boost :: python :: numeric) were not covered at all by tests, and the code itself was also used in another component. Therefore, problems were detected only when testing the corresponding component. The integration team is not required to write tests for the components, and this situation was the omission of the command itself. Wow, I heard a lot from them that broke their code. But after the team grumbled, they finally covered their code with tests. However, the offense remained (during the next migration, the team did not want to give access to its component to my colleague, mentioning that last time, we broke the code for them. Sasha, weeds! But after three days of negotiations, they surrendered).
Conclusion
After this work, I can point out some positives for myself, since I had not used boost very often before (mostly std), so I can emphasize a lot of new things from migration. It's funny, but it’s a fact, for some reason after this you become a “boost expert” for many colleagues, and, accept, you will be asked questions about it for some time.
By the way, in recent years, many companies have begun to actively get rid of boost and, if possible, replace std with a library or something else if there are no possibilities in the standard library. And we also did not stand aside. The process started, but not completed, still a lot of work.