March 21, 2018 at 15:46How we saddled OPNsense
Hello, Khabrovites!
Until holivars subside on whether it is immoral or not to use free platforms to create commercial products, we quietly did it. And we’re not shy to take money from customers, because we’ve got a really cool thing, based on free code, that’s a universal hardware security gateway. We used to have a commercially successful firewall, but for Microsoft Windows. A stormy stream of ideas at some point emerged from the shores of "Windows", and the question arose - what next? And then Linux or Net / Open / Free BSD. Our gurus gathered, smoked and decided to use OPNsense instead of inventing their own bike. This article will help those who want to do something similar.
First, OPNsense is a fork of the open pfSense UTM system, which is considered one of the best solutions of its class based on FreeBSD. We studied the OPNsense code and we liked its quality, as well as the approach to the implementation of functionality. Here, modern development principles are actively used, and first of all, the MVC model, when data is stored in one place, and display is done using a special template subsystem. There are controllers for each module, normal WebAPI is implemented in all its functionality, the code is well-structured, and project leaders pay a lot of attention to the programming style. This makes it easier for us to develop and support, dividing functionality into separate modules, highlighting plugins and other programming routines. Among other things BSD license is ideal for the creators of a commercial product - it does not contain related to the so-called “Copyleft” of restrictions. Last but not least, the reason is that our developers used FreeBSD in their projects before. In no case should the existing experience and skills be discounted.
Collaboration with the OPNsense team began with the creation of Russian localization for the main branch of the project. It was expensive, professional translators worked for several months to Russify a huge number of interface elements, system messages and help information with a large number of special terms - all of this we still keep up to date. By the way, when we encounter bugs in OPNsense, we offer corrections, improvements, and improvements to the product that are made available to everyone absolutely free, so that the "money-makers who profit from free software" contributed to the development of free code.
The guys from OPNsense were ready to cooperate. When we make any changes, the key project developers respond promptly and immediately give feedback. If we submit new functionality to the main branch of the base project, we have to support it ourselves, but the search for errors and testing are greatly simplified. There are moments with which it is difficult for us to live, but we have to put up with them - the advantages of free software outweigh the disadvantages.
If we do refactoring that involves a lot of code, OPNsense developers are not always able to quickly digest changes.They simply do not have enough resources for this, therefore, Traffic Inspector Next Generation already has functions that are still not available in the basic version. OPNsense uses standard syslog, but we switched to syslog-ng a long time ago - it allows, for example, writing logs to the database. The logging system affects a lot of things, the volume of changes turned out to be huge. To transfer the code to OPNsense, we tried to break it into several iterations, but the process has not yet been completed.
The second point is related to changes in the web interface. In OPNsense, the MVC model is far from being used everywhere, the legacy code extends from pfSense there and we were asked to transfer it to new “rails”. We started the alteration from the page for working with certificates, but when OPNsense figured out the amount of work necessary to accept the new code in the main branch, their hair stood on end.
Although we are the official contributors of OPNsense, this does not give increased privileges to change the code of a free product. The community controls its economy on its own, the developers of the project must test and accept our changes themselves. We can only offer them to redo some solutions to facilitate the passage of changes.
The accumulation of differences with the code of the main product is a serious problem. We try to structure our work in such a way as to facilitate the update process as much as possible. When the OPNsense update is transferred to the Traffic Inspector Next Generation, our functionality should not break and it, in turn, should not break the new OPNsense functionality. It is important to understand that the Traffic Inspector Next Generation is not an independently developing fork - the interaction of products is ongoing and it is difficult to keep them in a non-conflicting state. The amount of work is frantic, but you have to live with it. It will not work just to stick your logo and take for a product for money, although some clever people try to do it.
We deal with problems as follows: if there is an opportunity to add functionality with a plugin, we do so. OPNsense has a well-designed API for embedding plug-ins, it rarely changes and developers pay enough attention to backward compatibility issues. When the new version of OPNsense is released, we also raise the new version of Traffic Inspector Next Generation and update the code base without any headache, although we can’t do without a headache.
Another interesting point, we do not give all our best practices to OPNsense. Some of them are needed only in the Russian market, while proprietary modules have licensing restrictions. Our country has a law that requires, for example, a cafe with a Wi-Fi access point to identify the client. In the Traffic Inspector Next Generation, this is done via SMS, but abroad, local Russian cuisine is of no interest to anyone.
You can mention Kaspersky Anti-Virus - for it we have compiled a package using the libraries of Kaspersky Lab. You can’t integrate such a package into the main branch. There is also a Russian resource filter by NetPolice categories, which implies a subscription to the service, and, of course, things related to FSTEC certification. Such as, for example, checking the integrity of the system and notifying administrators of its violations.
Dividing the functionality into modules and the active use of plugins allowed us to effectively separate flies from cutlets, but a lot of differences have accumulated between the two products. In addition to the already mentioned, one can call the traffic reporting module and a single Control Center for working with several devices of the Traffic Inspector Next Generation. There is also a plug-in for integration with Microsoft Active Directory, a system for restricting access to network resources by IP and MAC addresses, and NTLM authentication. Some of these goodies will sooner or later appear in OPNsense, but many will never be there.
The most important difference: Traffic Inspector Next Generation is a hardware-software complex, i.e. boxed solution. We take hardware from leading manufacturers of server hardware, create software for it and sell everything that is called in one piece. The advantages are obvious: the customer receives a product that does not require complex configuration without problems with software compatibility. Moreover, he has a limited warranty and, of course, free technical support by phone and email.
Free software creators cannot test all hardware platforms and offer to use software at their own risk. For support, they only have documentation and forums that we also have (moreover, Russian-speaking). Add here the additional functionality that we already talked about, and you will get Russified OPNsense on steroids along with ready-to-use iron - you can pay for it. Moreover, there are configurations with the FSTEC certificate on sale and OPNsense will definitely not offer you this for objective reasons.
Using free products as the basis for commercial solutions requires an integrated approach and a trained team. A large community of developers quickly notices and corrects errors, and also tracks the changes made. In our case, we get double control: we check the OPNsense code, and other project participants inspect our open code. As a result, quality increases and the reliability of both products increases. It’s hard not to notice a reduction in costs: we don’t need to pay for licenses for the operating system, development tools and modules that implement UTM-functionality. This does not mean that we got everything completely free. The developers of the Smart-Soft team have devoted many years to creating solutions for the FreeBSD platform. Our experience is a significant part of the cost of creating the Traffic Inspector Next Generation, and there are also many direct financial costs. Development of solutions based on open source software is far from free and if you miss this moment, you won’t be able to do something worthy.
Until holivars subside on whether it is immoral or not to use free platforms to create commercial products, we quietly did it. And we’re not shy to take money from customers, because we’ve got a really cool thing, based on free code, that’s a universal hardware security gateway. We used to have a commercially successful firewall, but for Microsoft Windows. A stormy stream of ideas at some point emerged from the shores of "Windows", and the question arose - what next? And then Linux or Net / Open / Free BSD. Our gurus gathered, smoked and decided to use OPNsense instead of inventing their own bike. This article will help those who want to do something similar.
Why OPNsense?
First, OPNsense is a fork of the open pfSense UTM system, which is considered one of the best solutions of its class based on FreeBSD. We studied the OPNsense code and we liked its quality, as well as the approach to the implementation of functionality. Here, modern development principles are actively used, and first of all, the MVC model, when data is stored in one place, and display is done using a special template subsystem. There are controllers for each module, normal WebAPI is implemented in all its functionality, the code is well-structured, and project leaders pay a lot of attention to the programming style. This makes it easier for us to develop and support, dividing functionality into separate modules, highlighting plugins and other programming routines. Among other things BSD license is ideal for the creators of a commercial product - it does not contain related to the so-called “Copyleft” of restrictions. Last but not least, the reason is that our developers used FreeBSD in their projects before. In no case should the existing experience and skills be discounted.
Beginning of work
Collaboration with the OPNsense team began with the creation of Russian localization for the main branch of the project. It was expensive, professional translators worked for several months to Russify a huge number of interface elements, system messages and help information with a large number of special terms - all of this we still keep up to date. By the way, when we encounter bugs in OPNsense, we offer corrections, improvements, and improvements to the product that are made available to everyone absolutely free, so that the "money-makers who profit from free software" contributed to the development of free code.
Pouring into the mainstream
The guys from OPNsense were ready to cooperate. When we make any changes, the key project developers respond promptly and immediately give feedback. If we submit new functionality to the main branch of the base project, we have to support it ourselves, but the search for errors and testing are greatly simplified. There are moments with which it is difficult for us to live, but we have to put up with them - the advantages of free software outweigh the disadvantages.
If we do refactoring that involves a lot of code, OPNsense developers are not always able to quickly digest changes.They simply do not have enough resources for this, therefore, Traffic Inspector Next Generation already has functions that are still not available in the basic version. OPNsense uses standard syslog, but we switched to syslog-ng a long time ago - it allows, for example, writing logs to the database. The logging system affects a lot of things, the volume of changes turned out to be huge. To transfer the code to OPNsense, we tried to break it into several iterations, but the process has not yet been completed.
The second point is related to changes in the web interface. In OPNsense, the MVC model is far from being used everywhere, the legacy code extends from pfSense there and we were asked to transfer it to new “rails”. We started the alteration from the page for working with certificates, but when OPNsense figured out the amount of work necessary to accept the new code in the main branch, their hair stood on end.
Problems…
Although we are the official contributors of OPNsense, this does not give increased privileges to change the code of a free product. The community controls its economy on its own, the developers of the project must test and accept our changes themselves. We can only offer them to redo some solutions to facilitate the passage of changes.
The accumulation of differences with the code of the main product is a serious problem. We try to structure our work in such a way as to facilitate the update process as much as possible. When the OPNsense update is transferred to the Traffic Inspector Next Generation, our functionality should not break and it, in turn, should not break the new OPNsense functionality. It is important to understand that the Traffic Inspector Next Generation is not an independently developing fork - the interaction of products is ongoing and it is difficult to keep them in a non-conflicting state. The amount of work is frantic, but you have to live with it. It will not work just to stick your logo and take for a product for money, although some clever people try to do it.
... and decisions
We deal with problems as follows: if there is an opportunity to add functionality with a plugin, we do so. OPNsense has a well-designed API for embedding plug-ins, it rarely changes and developers pay enough attention to backward compatibility issues. When the new version of OPNsense is released, we also raise the new version of Traffic Inspector Next Generation and update the code base without any headache, although we can’t do without a headache.
Another interesting point, we do not give all our best practices to OPNsense. Some of them are needed only in the Russian market, while proprietary modules have licensing restrictions. Our country has a law that requires, for example, a cafe with a Wi-Fi access point to identify the client. In the Traffic Inspector Next Generation, this is done via SMS, but abroad, local Russian cuisine is of no interest to anyone.
You can mention Kaspersky Anti-Virus - for it we have compiled a package using the libraries of Kaspersky Lab. You can’t integrate such a package into the main branch. There is also a Russian resource filter by NetPolice categories, which implies a subscription to the service, and, of course, things related to FSTEC certification. Such as, for example, checking the integrity of the system and notifying administrators of its violations.
Dividing the functionality into modules and the active use of plugins allowed us to effectively separate flies from cutlets, but a lot of differences have accumulated between the two products. In addition to the already mentioned, one can call the traffic reporting module and a single Control Center for working with several devices of the Traffic Inspector Next Generation. There is also a plug-in for integration with Microsoft Active Directory, a system for restricting access to network resources by IP and MAC addresses, and NTLM authentication. Some of these goodies will sooner or later appear in OPNsense, but many will never be there.
Machine of differences
The most important difference: Traffic Inspector Next Generation is a hardware-software complex, i.e. boxed solution. We take hardware from leading manufacturers of server hardware, create software for it and sell everything that is called in one piece. The advantages are obvious: the customer receives a product that does not require complex configuration without problems with software compatibility. Moreover, he has a limited warranty and, of course, free technical support by phone and email.
Free software creators cannot test all hardware platforms and offer to use software at their own risk. For support, they only have documentation and forums that we also have (moreover, Russian-speaking). Add here the additional functionality that we already talked about, and you will get Russified OPNsense on steroids along with ready-to-use iron - you can pay for it. Moreover, there are configurations with the FSTEC certificate on sale and OPNsense will definitely not offer you this for objective reasons.
Conclusion: free ≠ free
Using free products as the basis for commercial solutions requires an integrated approach and a trained team. A large community of developers quickly notices and corrects errors, and also tracks the changes made. In our case, we get double control: we check the OPNsense code, and other project participants inspect our open code. As a result, quality increases and the reliability of both products increases. It’s hard not to notice a reduction in costs: we don’t need to pay for licenses for the operating system, development tools and modules that implement UTM-functionality. This does not mean that we got everything completely free. The developers of the Smart-Soft team have devoted many years to creating solutions for the FreeBSD platform. Our experience is a significant part of the cost of creating the Traffic Inspector Next Generation, and there are also many direct financial costs. Development of solutions based on open source software is far from free and if you miss this moment, you won’t be able to do something worthy.