5 stages of the API: what we realized by writing two versions

    Today we want to talk about the innermost - we have an API.

    We wrote and then rewrote it again for four years. And during this time almost all the classical stages of “accepting the inevitable” have passed. Except one - the fourth. And we want to share the conclusions acquired by overwork , what to do and not do if you decide to make your own “powerful epi”.



    The process of creating the uCoz API sometimes resembled the plot of the series The Knick ("Knickerbocker Hospital") - with unsuccessful operations, guts and experiments on living people.

    Stage One - Denial


    According to the concept of five stages:
    At the first stage, a person refuses to accept what happened to him.

    In general, APIs for site designers are rare even now. And in 2010, no one else had such an instrument on the market.

    On the one hand, there was a definite need for an API: we saw that people actively google “scripts for uCoz”, received requests directly - some users wanted to create their own add-ons and modifications. On the other hand - and this was a problem - in those days all resources were thrown to other projects.

    The question "to be or not to be" was decided simply. We needed the API ourselves - to run the PHP support function in the constructor. We selected one developer, and for six months he made our “initial API” - it was a get-only interface to which it was possible to receive page data from 11 modules. Data was given only in XML. By the time of the planned announcement of PHP, we did not have time to establish at least the addition of content, but the API had pluses: with it, you could run PHP scripts in the free version of yukoz.

    In general, we went to the users. It turned out straight in the classics:



    They didn’t really accept us ... More precisely, they mostly praised us. But for PHP. And in the “epiai” people did not see what they imagined with this magic word. Questions came to tech support: “How to add materials? How to edit? How to get in json? ”But this was not all.

    Stage Two - Anger


    According to the concept of five stages:
    At this stage, aggression towards the entire surrounding world is manifested.

    To the question that the API needs to be developed, the company returned about a year later. The priorities were mobile clients - and we decided to re-write everything taking into account the requirements of the guys who made clients for iOS and Android. The initial version was left to live by itself (and even is still alive, because some still use it), and we began to select an artist for a new project.

    “Your own manager.” A sensible guy Ilya just came to the Rostov office: he knew Perl, on which part of the old uCoz was written, and when, by tradition, they were offered several tasks, he chose from them work on the API. The problem was that for a while the guy had to become his own manager.

    Then anger came: “As it turned out in the process, I understood Perl syntax, but the spirit didn’t.



    Therefore, I tried for a long time not work, but the world is better to do: I thought to rewrite everything, to introduce objects, classes, and patterns. It took a lot of time to get comfortable with the ideology of the language. And then what at first seemed scary to me and wanted to rewrite, began to seem the norm. ”

    By the time of enlightenment, a manager appeared and began to demand a report. And it was only ready to deliver content on several modules ... In general, once again anger, now on the other hand, and Ilya was transferred to a new project. Its version of the API was never destined to see the light.


    ... between the stages ...


    “The perfect learning challenge.” By that time, the company was trying to open an R&D office in Kazan. In place, we needed a carrier of knowledge about the whole system - and the idea came up to “grow” it through work on an API that would affect the main modules of the system.

    So Rinat appeared in this story:



    On the one hand, he was rather excited to take on the project (he’s a bit extreme in our country). On the other hand, he is calm and prudent in his work: nevertheless, not only 700+ parachute jumps are behind him, but also 20 years of experience in IT.

    He was also familiar with Perl and had fresh experience working with foreign APIs - integrated Metric and GA into the webmaster panel.

    The first thing that came in handy was his discretion. Rinat asked for a pause, in order to understand in more detail the tangled "guts" of the system, and only then announce the dates and implementation plan. A month later, he came up with the following sentence:

    * Rewrite and duplicate some functions purely under the API - a lot of old code has accumulated in it during the creation of the system. And some functions, if you had tried to put something else in there, you would have turned out to be “kilometer”. So, we need cleaned doubles purely under the API.

    * We use REST - to simplify the architecture of the request, which will help increase productivity.

    * We use Oauth 1.0a - authentication, which seemed the most secure at that time.

    *We give in different formats - JSON, XML, Text Plain.

    * Well: get, post, put, delete, peace, labor, May ...


    Third Stage - Bargaining


    According to the concept of five stages:
    Here you have thoughts about negotiating with someone about a better fate

    To a common understanding of how to make it convenient, we kind of came. But the devil was in the details and debate.

    The principal issue was the scope of tokens. Our server system, as Rinat says, “is a complete encapsulated figovina”: N users spin on each server, and when the space runs out on it, new hardware is taken.

    Rinat wanted to use one token to gain access to all sites. “Ok, let's talk about it,” - and we invited colleagues to skype chat. As it happens in collective chats, they argued, they argued, but they did not come to a collective decision. And in the heat of development, the topic was forgotten and surfaced again when integration was ready for half the modules.



    We had no other option how to replicate the token: this is how the architecture with one main authorization server appeared - after receiving one token there, it can be used everywhere.

    To issue as an array or object? For us, this was not a problem of the category of “what is right and what is not”. It was a problem of detail.

    For example, we send data to JSON - and there is a problem with typed languages. Not all structures received from the API are convenient to parse - because the amount of code on the client side is increasing.

    But the API is focused on web-stray, so we listened to the opinion of developers in Java and C ++ and came to the standard: we give the fields in any case, we supplement the named parameters with the code.

    Fields and parameters.Here, bidding and discussion went on all the time of implementation. For example, is it worth issuing an empty parameter? Since we also focused on mobile development using the API, Rinat proved in the process that it is not necessary: ​​traffic is important in mobile applications.

    And what fields are needed? Here already, to me, the project manager, it was necessary to search and give examples from real practice. Often I took the help of the audience - interviewed future API users. If the sent case of a third-party developer seemed valuable, we implemented the necessary fields in the API itself, so that people would not do it after crutches. And later they came up with a more elegant solution - we’ll talk about it below.



    The fourth stage - ... (and the timing)


    According to the concept of five stages:
    In the fourth stage, you have depression. [We somehow passed this stage]

    The work planned for the year took almost two. In the process there were completely unforeseen difficulties. We quickly realized that:



    Problems with the second developer. According to the plan, after Rinat makes authorization, the second developer was supposed to connect to the integration of the API with the modules.

    We painted Google Dock by putting the modules in alphabetical order. The work on them was divided between the performers in half. We determined a schedule - one module per month from each. When it was time to start, the second man left. And we already did a new project - the uKit constructor, on which we threw the main resources. With the loss of the second programmer, almost 7 months were added to the implementation period.

    Testing problems. In theory, the process was conceived as follows: after the developer's server, everything goes to the alpha server, and if the tester says “come on,” then it passes beta testing and the update is distributed across hundreds of servers.

    But it turned out that our regular testers were not very suitable for working with the API - after all, they are “tailored” for testing web pages. API is a subtle thing. For example, when there was a change inside the module with which we integrated, something could fall off - but testers did not notice this at first (then we taught them).

    Then we opened 4 test servers and invited advanced users at the earliest stages. Such a crowd, and therefore not very controlled (you are not sure when a person will do something and not quit), testing also affected the increase in terms.


    Fifth Stage - Humility (and Conclusions)


    According to the concept of five stages:
    According to the canons, there is agreement with the inevitable

    In the end, we have come to terms with the inevitable - the API, like the repair, can be started, but not so easy to finish. Here are some recipes for organizing the process to make it easier for you.

    1. Get feedback. More connection.

    In addition to the forum thread and blog posts, i.e. public means of exchanging comments, I started the “Laboratory” section on the site with new documentation. In fact, it was a feedback form where you could discuss your case in personal correspondence.

    We informed the early users about the new API in February 2015, and the process of rolling out all the modules to the servers was completed only this year. All this time we received reports, offers and interesting cases from users through the “Laboratory” (which I also used in the “bidding” with Rinat).


    The flow of appeals has decreased only in the last two months.

    Each time, having received the appeal “how to do this or what to do here”, we took the time and sat down to watch, but what is better? Sometimes interesting stories were born from this.

    2. Personally help those who have not figured it out or [Script as a gift].

    It used to be that the message from the Lab was so interesting that we started to write the script ourselves. And then they presented a ready-made solution to the user.

    What is the profit? As the saying goes, "if you want to find a bug, be a bug." And if you want to immerse yourself in the project - do as much as possible on it to check the performance and convenience. At the same time, the first solutions using uAPI appeared on the uScript exchange - authorization through social networks and the inaccurate output of pictures in the block of recommended materials.

    3. Run an internal hackathon.

    As you recall, we lacked testers. So the idea of ​​an internal hackathon to test the product came up. For some of our developers, working with the API was a completely new and unusual task, so we had the opportunity to closely monitor how people with different skills act.


    Rinat usually didn’t communicate directly with users, but he came from Kazan to the hackathon to the Rostov developers. He says, “with the exception of trolling, I was pleased with the data collected and the process of criticism.”


    4. Try to automate something.

    My favorite feature for working with the API was the semi-automatic creation of an application and a token.



    Initially, this system emerged from my personal needs - at one point I was tired of wasting time signing requests when creating an application. Then I decided to write a semiautomatic mode - so as not to think about anything, but copied the data - and the application is ready. As it turned out, now people mostly use a semi-automatic solution.

    5. Consult and change documentation.

    Initially, we wrote a sample request in PHP and CURL.


    It was. As it turned out, nobody used CURL.

    But people asked, for example, in what format and in what way to send a request. It became clear that the first implementation was not obvious to everyone.

    We decided that there should be an additional module (written in a modern, OOP), where the signature for oauth will be automatically generated. Once called this module - and then just write the path and request method.

    In parallel, I went through our programmers and asked - the documentation for which API they consider a worthy role model? I looked at the recommended examples and, based on them, compiled a new version of the documentation :



    And as a test before publication, I asked a colleague from our second office to check that the API works in accordance with the descriptions in the documentation for all methods.

    6. Implement mobility.

    First of all, it will help to get good reviews and will expand your audience:

    It has long been a dream to create a mobile application for your site. I will study uAPI in order to finally bring it to life

    Secondly, it can be a valuable and new experience for an internal team. For example, at a hackathon, several of our guys sawed such an application:


    Here you can see the application code

    7. Do not limit the wishes of users.

    Let's get back to the issue of fields. Since you do not foresee all cases, in the end we made a query constructor with which you can implement any bold fantasies on this topic:



    Finally, we have come to the conclusion that the API is a two-way movement. Today, using uAPI, you can authorize for any site, transfer materials from uCoz and vice versa, use only our database, but display data on a third-party site. The more user cases you select into the tool, the longer the implementation. But at the same time, the scope of its application is growing.


    PS

    The whole process of rolling out and combat testing the new version took us 14 months and 20 updates. Here is the visualization .

    It happens that after the next update they write to us: “When will you finish it?” But the process is really very difficult to stop (we did not joke about the repair). Sometimes - for technical reasons: let's say when the system update requires changes to the API. And sometimes it’s creative. For example, now we are thinking: when all the integrations with the modules are completed, why not work out the topics of changing the design and site settings by API?

    Also popular now: