ISPsystem, forgive and goodbye! Why and how we wrote our server control panel

    image

    Hello! We are “Hosting Technologies” and 5 years ago launched VDSina - the first vds hosting created specifically for developers. We strive to make it convenient, like DigitalOcean, but with Russian support, payment methods and servers in Russia. But DigitalOcean is not only reliability and price, it is also a service.

    Software from ISPsystem turned out to be a rope that bound our hands on the way to a cool service. Three years ago, we used Billmanager billing and VMmanager server control panel and quickly realized that it is almost impossible to provide good service without our own panel.


    How ISPsystem killed usability


    Bugs

    We could not fix a bug ourselves - each time we had to write in someone else's support and wait. The solution to any problem required the response of a third-party company.

    ISPsystem support responded normally, but fixes only came through a few releases, and not always and not all. Sometimes critical bugs fixed for several weeks. We had to reassure customers, apologize and wait for ISPsystem to fix the bug.

    The threat of downtime

    Updates could give rise to unpredictable downtimes that provoked new errors.

    Each update was a lottery: it was necessary to cover billing and make sacrifices to the gods of updates - a couple of times the update caused a downtime for 10-15 minutes. At that time, our admins were turning gray before our eyes - we never knew how long downtime would last and could not predict when ISPsystem decided to release a new update.

    In the fifth generation, Billmanager got better, but in order to get access to the necessary features, I had to install a beta, which was already updated every week. If something broke, you had to give access to foreign developers so that they corrected something.

    Inconvenient panel interface

    Everything was divided into different panels and controlled from different places. For example, clients paid through Billmanager, and they had to reboot or reinstall VDS in VMManager. Our employees also had to switch between windows to help the client, check the load on his server or see which OS he uses.

    Such an interface takes time - both ours and customers. About any convenience, like DigitalOcean, there is no question in such a situation.

    Short lifecycles with frequent API updates

    We wrote our own plugins - for example, a plugin with additional payment methods that are not available in VMManager.

    In recent years, VMManager had a relatively short life cycle, and in new versions the names of variables or functions in the API could arbitrarily change - this broke our plugins. Support for older versions was quickly minimized and had to be updated.

    You can’t modify it

    More precisely, you can, but it’s extremely inefficient. Licensing restrictions do not allow you to make changes to the source, you can only write plugins. The maximum of plugins is some menu items, a step-by-step wizard. ISPsystem is sharpened for versatility, and we needed specialized solutions.

    So the decision to write your panel matured. We set goals:

    • Quickly respond to errors, bugs and be able to fix them yourself, without making the client wait.
    • Freely modify the interface to work processes and customer needs.
    • Enhance usability with a clean and understandable design.

    And they started development.

    New Panel Architecture


    We have a self-sufficient development team, so we wrote the panel ourselves.
    The main work was done by three engineers - the technical director Sergey came up with the architecture and wrote the server agent, Alexey made the billing, and our frontend Artysh gathered the frontend.

    Step 1. Server Agent



    A server agent is a python web server that manages the libvirt library , which in turn manages the Qemu-kvm hypervisor .

    The agent manages all the services on the server: creating, stopping, uninstalling vds, installing operating systems, changing settings, and so on through the libvirt library. At the time of publication, this is more than forty different functions, which we complement depending on the task and needs of the client.

    In principle, libvirt could be managed directly from billing, but it required too much additional code and we decided to distribute these functions between the agent and billing - billing simply makes requests to the agent through the JSON API.

    The agent is the first thing we did, because it did not require any interface and it could be tested directly from the server console.

    What the server agent gave us: a layer appeared that simplifies life for everyone - billing does not need to send a whole bunch of commands, but only make a request. And the agent will do everything you need: for example, allocate disk space and RAM.

    Step 2. Billing


    For our developer, Alex, this was not the first control panel - Alex was in hosting for a long time, so he generally understood what the client needed and what the hoster needed.

    We call billing between ourselves a “control panel”: it contains not only money and services, but also their management, customer support and much more.

    To switch from ISPSystem software, it was necessary to completely retain the previous functionality for customers, transfer all financial user actions from the old billing to the new one, as well as all services and communications between them. We studied what is in the current product, then the decisions of competitors, mainly DO and Vultr. We looked at the disadvantages and advantages, collected feedback from people who worked with old products from ISPsystem.

    In the new billing, two stacks were used: classic PHP, MySQL (and in the future it is planned to switch to PostgreSQL), Yii2 as a framework on the backend and VueJS on the front. Stacks work independently of each other, developed by different people, and communicate using the JSON API. For development, then and now we use PHPStorm and WebStorm from JetBrains and love them dearly (guys, hello!).

    The panel is designed according to the modular principle: payment system modules, domain registrar module or, for example, SSL certificate module. You can easily add a new function or remove the old one. The groundwork for expansion was laid architecturally, including in the opposite direction, "to the iron."
    image
    What we got: a control panel over which we have full control. Now bugs are fixed in hours, not weeks, and new functions are implemented at the request of customers, and not at the request of ISPSystem.

    Step 3. Interface


    image
    The interface is our team brainchild.

    First, we looked at what would happen if we made an add-on over the ISPsystem API without changing anything dramatically in the interface. It turned out so-so and we decided to do everything from scratch.

    We believed that the main thing was to make the interface logical, with a clean and minimalistic design, and then we would get a beautiful panel. The arrangement of elements was discussed in Megaplan, and the interface that users see in the control panel now will gradually be born.

    The first appeared billing page design, because we already made payment plugins for ISPsystem.

    Frontend

    They decided to make the panel a SPA application - undemanding to resources and with fast data loading. Our front-end Artysh decided to write it on Vue - at that time Vue had just appeared. We assumed that the framework will develop dynamically, like React, after some time the Vue community will grow and a sea of ​​libraries will appear. We installed it on Vue and did not regret it - now adding to the front new features that have already been programmed on the backend takes little time. We will tell you more about the frontend panels in a separate article.

    Communication of the frontend with the backend

    The frontend was connected with the backend through the push. I had to sweat and write my own handler, but now the information on the page is updated almost instantly.

    What happened:The panel interface has become easier. We made it adaptive, and fast loading allows it to be used even from mobile phones in the last minutes before take-off, without installing a separate application for working with the panel.

    Step 4. Testing and migration scheme


    When everything started and the first tests passed, the question of migration arose. First of all, we set up billing and began to test its work with the server agent.

    Then they wrote a simple script that transfers the database from the old billing to the new one.

    I had to test and double-check literally everything, since the data was poured into one new database from the three old ones: Billmanager, VMmanager and IPmanager manager. Perhaps the test migration is the most difficult thing that we encountered in the process of developing a new panel.

    After rechecking, we covered up the old billing. The final data migration was a very worrying moment, but, thank God, it was completed in a few minutes and without noticeable problems. There were minor bugs that we fixed in a week. The main time was taken by testing what happened.

    Then we sent out letters to customers with the address of the new panel and billing and made a redirect.

    Bottom line: IT'S ALIVE!

    Happy end


    From the first hours of our software, we felt all the delights of the transition. The code was completely ours and with a convenient architecture, and the interface was clean and logical.
    image
    The first review after the launch of the new panel

    We launched the transition process in December, on the eve of the New Year 2017, when there was the least load to make the transition easier for customers - almost no one works on the eve of the holidays.

    The main thing that we got during the transition to our system (in addition to general reliability and convenience) is the ability to quickly add functionality for key customers - to be their face and not their ass.

    What's next?


    We are growing, the amount of data, customers, customer data is growing. A memcached server and two queue managers with different tasks had to be added to the backend. There are caching and queues on the front end.

    Of course, we still had adventures as we developed and complicated the product, for example, when we added HighLoad.

    In the next article we will tell you how the Hi-CPU tariff was launched: about hardware, software, what tasks we solved and what we did.



    Subscribe to our Instagram developer



    Also popular now: