Experience developing a free application for collectors OpenNumismat
I want to share my experience developing an open source desktop application for collectors.
By the nature of my core business, I develop software for embedded systems and rarely intersect with end users. Therefore, we will discuss the features that I encountered while developing a non-commercial application for a wide range of users, which have become a discovery for me.
OpenNumismat, as the name implies, is intended primarily for registering a collection of coins. But it is also suitable for other types of collectibles - stamps, postcards, badges and more exotic things.
The application is written in Python, data is stored using a SQLite database, PyQt is used for the interface, data access and much more, Jinja2 is used for generating reports, and Matplotlib is used to build statistics graphs. Despite my initial skepticism, this was enough to ensure acceptable performance — several thousand image records are processed without any noticeable brakes.
Since all components are cross-platform, then OpenNumismat has builds for Windows, Linux (Debian / Ubuntu), macOS.
There is also a version for Android, currently abandoned. It was replaced by PWA (Progressive Web App), which allows you to view your collection in any browser - using SQLite.js. Developing PWA turned out to be much easier to create a native Android application similar in capabilities. In addition, it is immediately available on all platforms.
Now there are a lot of applications for collectors and, in particular, coin collectors. Most of them "look as if someone got their first version of Firefox and used it."
In the US, many use commercial software, the undoubted advantage of which is the availability of ready-made databases of existing coins. The user needs only to mark the available coins and enter the purchase information - price, date, etc.
Similar opportunities are provided by mobile applications (most for Android) and web services. OpenNumismat is more focused on creating a catalog of its own unique collection, rather than on “closing holes in a finished album,” as some coin collectors put it.
But the main competitor is Microsoft. The vast majority of collectors use Excel despite the fact that it does not work well with images. Also, some use Access or 1C and create their own collection database with everything they need. But it still requires some preparation.
To attract users, import from the most popular alternatives is implemented.
As for Linux and macOS, there are practically no analogues for these platforms.
The main advantage of the openness of the code was the possibility of free use of third-party services for distribution, translation, bug tracking. Pull requests, to my chagrin, there were only three.
Unexpectedly for me, another potential turned out to be a potential opportunity to convince skeptics of the security of the program - on the forums they claim that they checked the code with grep and no suspicions.
Initially, Google Code was chosen for distribution of OpenNumismat, which provided convenient bug tracking, a multilingual wiki for documentation, binary downloads, and of course a git repository. But starting in 2013, Google Code began to die. For relocation, the choice was between SourceForge and GitHub. Both services provide the ability to create project sites (the documentation from the wiki had to be converted to static pages of the site), but downloading SourceForge binaries looks scary for an unprepared user. Therefore, the choice fell on GitHub.
GitHub bug tracker requires registration, which is not so convenient for users far from programming. Therefore, it was decided to use a third-party service. Idea.Informer was found(Reformal for the Russian-speaking audience) is a free and convenient service, but is not currently being developed and this causes a number of difficulties (Idea.Informer stopped sending letters and does not support SSL).
Initially, OpenNumismat supported English and Russian. But a year after the first release from Venezuela, it was proposed to translate into Spanish. Then came the translation proposals in other languages. Initially, translation was carried out using the Qt Linguist tool. For the convenience of volunteers, an online service is currently used, providing a convenient infrastructure for open source projects. Currently, OpenNumismat has already been translated into 15 languages.
To promote OpenNumismat, I had to learn some of the tricks of the CEOs. So using semantic markup was able to promote and improve the display of the application site in search engines.
Currently, Windows version uses PyQt version 5.5.1. Switching to a more recent version is made difficult by replacing Qt WebKit with Qt WebEngine, which still does not know how to preview the printed pages used for working with reports.
A transition is needed to use Qt Charts for graphing statistics, but it is not available in the version used. Of course one could have collected it separately, but the hands did not reach. Therefore, the statistics use the Python package Matplotlib.
Also, fresh versions of Python do not support Windows XP, which is still used by a small number of users.
A Linux version can generally use any combination of PyQt, Python, Matplotlib versions. Therefore, statistics may not work on some configurations, report viewing is limited on others.
There were two options for building under macOS: use the official Python build with the required packages or MacPorts. In the first version, the described problems arise with the reports, the build process could not be fully configured, in the second one there were problems with Matplotlib, but MacPorts has an assembly of a fresh version of PyQt with Qt WebKit. Currently using a version with MacPorts.
I also want to point out that SQLite doesn’t support non-ASCII characters out of the box. This caused problems when sorting users using alphabets using characters other than Latin and Russian (for example, Ukrainian and Catalan - users of these countries especially paid attention to the problem). Transferring sorting from SQL queries to Qt not only solved the problem, but also increased performance.
Feedback and feedback is the most enjoyable part of the job. Best of all, OpenNumismat was adopted by the Spanish community of coin collectors. The Russians surprised me with reviews like: “The program is waiting for the collection to be filled to the maximum, and then informs where it should be.” Apparently fear "Russian hackers." Often, you have to justify yourself to skeptics who are not even going to use OpenNumismat - this is sometimes frustrating.
Many users write to simply thank, offer new ideas, ask for help to understand the application. Most describe the errors and problems in the operation of the application, which undoubtedly often arise, because the development and testing is conducted by one person.
The most common question is “I have a demo version of how to get the full one”. This is due to the fact that when you first start a demo database opens called demo. Apparently this greatly confuses new users.
Quite often they offer to add some very specific functionality that may be useful to individual users, but it may seem completely superfluous and complicating to the majority. Or it changes the basic logic of data organization. Such proposals must be rejected or, more often, ignored.
Sometimes there are error messages that I can not reproduce. If contact with the user is established, then the problem is usually solved after some correspondence and joint experiments. Sometimes contact fails, and the error remains uncorrected and, perhaps, no one else noticed.
Also come letters from the creators of numismatic sites and electronic catalogs that are trying to make a revolution, but no significant changes yet.
Development has been underway for more than 6 years. Much has been achieved, but the application is still far from perfect. There are still a lot of ideas that I would like to embody.
Getting positive feedback is definitely nice and I want to make a really useful tool for users. And since I myself am an active user of OpenNumismat, I’m not going to stop there.
By the nature of my core business, I develop software for embedded systems and rarely intersect with end users. Therefore, we will discuss the features that I encountered while developing a non-commercial application for a wide range of users, which have become a discovery for me.
Description
OpenNumismat, as the name implies, is intended primarily for registering a collection of coins. But it is also suitable for other types of collectibles - stamps, postcards, badges and more exotic things.
The application is written in Python, data is stored using a SQLite database, PyQt is used for the interface, data access and much more, Jinja2 is used for generating reports, and Matplotlib is used to build statistics graphs. Despite my initial skepticism, this was enough to ensure acceptable performance — several thousand image records are processed without any noticeable brakes.
Since all components are cross-platform, then OpenNumismat has builds for Windows, Linux (Debian / Ubuntu), macOS.
There is also a version for Android, currently abandoned. It was replaced by PWA (Progressive Web App), which allows you to view your collection in any browser - using SQLite.js. Developing PWA turned out to be much easier to create a native Android application similar in capabilities. In addition, it is immediately available on all platforms.
Competitors
Now there are a lot of applications for collectors and, in particular, coin collectors. Most of them "look as if someone got their first version of Firefox and used it."
In the US, many use commercial software, the undoubted advantage of which is the availability of ready-made databases of existing coins. The user needs only to mark the available coins and enter the purchase information - price, date, etc.
Similar opportunities are provided by mobile applications (most for Android) and web services. OpenNumismat is more focused on creating a catalog of its own unique collection, rather than on “closing holes in a finished album,” as some coin collectors put it.
But the main competitor is Microsoft. The vast majority of collectors use Excel despite the fact that it does not work well with images. Also, some use Access or 1C and create their own collection database with everything they need. But it still requires some preparation.
To attract users, import from the most popular alternatives is implemented.
As for Linux and macOS, there are practically no analogues for these platforms.
Benefits of openness
The main advantage of the openness of the code was the possibility of free use of third-party services for distribution, translation, bug tracking. Pull requests, to my chagrin, there were only three.
Unexpectedly for me, another potential turned out to be a potential opportunity to convince skeptics of the security of the program - on the forums they claim that they checked the code with grep and no suspicions.
Environment Used
Initially, Google Code was chosen for distribution of OpenNumismat, which provided convenient bug tracking, a multilingual wiki for documentation, binary downloads, and of course a git repository. But starting in 2013, Google Code began to die. For relocation, the choice was between SourceForge and GitHub. Both services provide the ability to create project sites (the documentation from the wiki had to be converted to static pages of the site), but downloading SourceForge binaries looks scary for an unprepared user. Therefore, the choice fell on GitHub.
GitHub bug tracker requires registration, which is not so convenient for users far from programming. Therefore, it was decided to use a third-party service. Idea.Informer was found(Reformal for the Russian-speaking audience) is a free and convenient service, but is not currently being developed and this causes a number of difficulties (Idea.Informer stopped sending letters and does not support SSL).
Initially, OpenNumismat supported English and Russian. But a year after the first release from Venezuela, it was proposed to translate into Spanish. Then came the translation proposals in other languages. Initially, translation was carried out using the Qt Linguist tool. For the convenience of volunteers, an online service is currently used, providing a convenient infrastructure for open source projects. Currently, OpenNumismat has already been translated into 15 languages.
To promote OpenNumismat, I had to learn some of the tricks of the CEOs. So using semantic markup was able to promote and improve the display of the application site in search engines.
Library Version Issues
Currently, Windows version uses PyQt version 5.5.1. Switching to a more recent version is made difficult by replacing Qt WebKit with Qt WebEngine, which still does not know how to preview the printed pages used for working with reports.
A transition is needed to use Qt Charts for graphing statistics, but it is not available in the version used. Of course one could have collected it separately, but the hands did not reach. Therefore, the statistics use the Python package Matplotlib.
Also, fresh versions of Python do not support Windows XP, which is still used by a small number of users.
A Linux version can generally use any combination of PyQt, Python, Matplotlib versions. Therefore, statistics may not work on some configurations, report viewing is limited on others.
There were two options for building under macOS: use the official Python build with the required packages or MacPorts. In the first version, the described problems arise with the reports, the build process could not be fully configured, in the second one there were problems with Matplotlib, but MacPorts has an assembly of a fresh version of PyQt with Qt WebKit. Currently using a version with MacPorts.
I also want to point out that SQLite doesn’t support non-ASCII characters out of the box. This caused problems when sorting users using alphabets using characters other than Latin and Russian (for example, Ukrainian and Catalan - users of these countries especially paid attention to the problem). Transferring sorting from SQL queries to Qt not only solved the problem, but also increased performance.
Feedback
Feedback and feedback is the most enjoyable part of the job. Best of all, OpenNumismat was adopted by the Spanish community of coin collectors. The Russians surprised me with reviews like: “The program is waiting for the collection to be filled to the maximum, and then informs where it should be.” Apparently fear "Russian hackers." Often, you have to justify yourself to skeptics who are not even going to use OpenNumismat - this is sometimes frustrating.
Many users write to simply thank, offer new ideas, ask for help to understand the application. Most describe the errors and problems in the operation of the application, which undoubtedly often arise, because the development and testing is conducted by one person.
The most common question is “I have a demo version of how to get the full one”. This is due to the fact that when you first start a demo database opens called demo. Apparently this greatly confuses new users.
Quite often they offer to add some very specific functionality that may be useful to individual users, but it may seem completely superfluous and complicating to the majority. Or it changes the basic logic of data organization. Such proposals must be rejected or, more often, ignored.
Sometimes there are error messages that I can not reproduce. If contact with the user is established, then the problem is usually solved after some correspondence and joint experiments. Sometimes contact fails, and the error remains uncorrected and, perhaps, no one else noticed.
Also come letters from the creators of numismatic sites and electronic catalogs that are trying to make a revolution, but no significant changes yet.
Conclusion
Development has been underway for more than 6 years. Much has been achieved, but the application is still far from perfect. There are still a lot of ideas that I would like to embody.
Getting positive feedback is definitely nice and I want to make a really useful tool for users. And since I myself am an active user of OpenNumismat, I’m not going to stop there.