August 12, 2014 at 15:31

Building your own JS SDK - why and how?


    If you write an SDK for the internal needs of a single project, then many things are simplified: backward compatibility issues are not so acute, instead of writing detailed documentation, you can personally answer questions from a colleague, and detected errors are relatively easy to include in the project. If the SDK is done for all projects of a large company, then there are much more problems. But if your SDK is designed for third-party developers around the world, then you can’t do without things like good documentation and autotests. Our APS JavaScript SDK can be attributed to the latter , and in this article I want to talk about how it works and how we try to make life as easy as possible for third-party developers who use or will use our SDK.

    Slidesfor those who do not like to read multi-letters.

    And for the most impatient: what we got and our sandbox .

    If you are only interested in technical details and a code, I advise you to start reading right away from the second part.


    What for


    We did not need our own SDK in order to come up with a new type of vehicle, shown in the picture to attract attention.

    We needed it to support the plug-in system within the OSS system of Parallels Automation and the Parallels Plesk hosting control panel . These systems are based on various code bases and they are very large. At Parallels Automation millions of lines of code, and the decision itself is not the final product. A contract is signed with a potential client and the product is branded. The product needed to be expanded, so we needed to enable unified creation of UI by third-party developers.

    To get a clear idea ofParallels Automation needs to understand what OSS is . This is a class of operational support systems that manage the network infrastructure, accounting and allocation of resources. The main consumers are telecoms and large hosters. It is important for them not only to manage the infrastructure as such, but also to sell services to their customers. From the sale of the service, telecoms receive good income. Imagine a telecom with 10 million users. Even if 1% of users buy a service worth $ 1, its revenue will be $ 100,000.

    APS




    A standard developed by Parallels with which it is possible to integrate services into the Parallels Automation ecosystem . The latest version of the standard is APS 2 , which corrects errors of version 1.x.

    APS 1.x

    APS 1.x allowed integrating applications by describing the business model of integrable objects and declaratively describing the UI in a meta file. Thus, the UI was automatically generated from the business model, which negatively affected the UX. A script-oriented user interface was simply impossible to do. Therefore, APS 2.x was created .

    APS 2.x

    In APS 2, the business model was completely separate from presentation logic. Business objects are now available on REST.
    After heated discussions and a design review, we settled on a thin server architecture, in which the UI began to be described in HTML with very active use of JavaScript. This allowed us to create a script-oriented interface without any significant restrictions.

    Limitations

    Although there were still limitations.
    Parallels Automation , unlike, for example, Parallels Desktop for Mac , cannot just be out of the box. As a rule, when it is sold, serious contracts are concluded and the Parallels brand is hiding in favor of the hoster or telecom that bought it. Therefore, the UI in APS was supposed to be branded. As stated earlier, the level of developers can vary greatly. The commercial product and copyleft restrictive licenses did not suit us.

    APS JS SDK

    As a result, we came up with the idea of ​​creating our own JavaScript SDK, which allows us to abstract the developer from markup. Abstraction is an additional bonus for us and end developers, as legacy markup, strictly speaking, is not semantic.

    We also made the initial emphasis on documentation, as JS SDK is not only used internally.

    how



    How did we create our framework and what rules did we follow?

    • used existing frameworks;
    • Do not forget about the thoughtful API;
    • adhered to Test Driven Development - initially created automated tests;
    • Worked on documentation
    • and above the sandbox where you can try the framework in action.


    Using existing frameworks


    Creating and maintaining your own JavaScript framework is a very time-consuming task. Therefore, we decided to take one of the existing frameworks as the basis for building our JS SDK. He had to satisfy several requirements:
    • because we already had an existing code base for typesetting that could change in the future, we would like to work with UI elements as integral objects - widgets that would be based on HTML templates that are as isolated from JS logic as possible ;
    • the tasks of our users are very different, so the framework had to be flexible enough to support a wide variety of UX options ;
    • we did not want to force developers to learn some complex proprietary technologies, so the basic framework should be as widespread as possible ;
    • as already mentioned, users can customize the design of their applications, so the framework should make it easy to change the appearance of widgets ;
    • in addition, I wanted to find a holistic framework on the basis of which we could immediately start creating our own widgets;
    • and, of course, everyone loves beautiful and dynamic widgets , so the framework should contain them, or at least allow them to be created.

    As options, we considered:
    • Extjs
    • JQuery UI
    • loader (RequireJS / CommonJS / ...) + MV * Framework (KnockoutJS / AngularJS / ...)
    • Dojo toolkit

    Extjs


    A powerful, popular, actively developing framework with a large number of cool widgets and a convenient data binding model. But still, this is a thing that was created to be taken and used. This is not the foundation for creating your own framework. Its widgets are difficult to customize, and design customization is added only in the latest versions. But its main disadvantage for us, of course, was the very high royalties.

    JQueryUI


    An even more popular framework, with a low entry threshold and a lack of royalties. Widgets included in the official supply are not enough, but this is not a problem, because there are a huge number of widgets from third-party developers. Unfortunately, at the time of the research, the framework API was not quite settled yet and often changed. In addition, the widget code is not separated from the layout.

    loader + MV * Framework

    At the time of research, the future of angular was vague, knockout solved very limited tasks. But there are customers, contracts and tight deadlines, therefore, in order to start the final development as soon as possible, we needed a holistic basic framework.

    Dojo toolkit


    Powerful AMD- based modular framework . Rich, numerous easily customizable widgets with template support. Many years of support for Deferred and Promise . Lack of royalties. But Dojo is strong and weak. It has a relatively large entry threshold due to which it is poorly distributed. In addition, he recently began to actively move towards mobile devices.

    But we still decided to use Dojo :
    • the specifics of the system does not imply active use on mobile devices (quite a lot has changed now, but, on the other hand, we have solved the corresponding problems of the framework);
    • we tried to lower the entry threshold, which we will discuss below.


    For more information on the merits of Dojo described in the translation of articles by David Walsh (by David Walsh is) . You can also read Dojo and ExtJS comparisons in the comments .

    Thoughtful API


    The basis of any framework is a well-thought-out API.

    AMD

    The foundation of our JS SDK are AMD modules which are visual components - widgets, components for working with data and various auxiliary utilities. In addition, the AMD format allows you to include any third-party libraries supporting this packaging format in the project. Thus, we do not limit developers to their preferences.

    Widgets

    The visual components — widgets — are logically separate from the HTML representation. They can dynamically change the values ​​of their properties and be inherited from each other. For the convenience of developers, there are 3 ways to describe widgets. Moreover, on one screen it is absolutely not necessary to use only one, the methods can be freely combined.

    Incorporate widgets into each other

    Widgets can include each other at the template level. For example, the aps / Slider widget consists of dijit / form / HorizontalSlider horizontal scroll and aps / TextBox:

    In addition to the described inclusion of widgets in the template, you can add child widgets dynamically or when describing the screen. For example, add inputs to the form.


    Declarative announcement

    Our framework supports 3 ways to describe screens. The first is declarative. In it, the location of widgets and their properties are set in the form of HTML layout with the specified special attributes. The widget hierarchy is defined by the hierarchy of HTML Elements. After loading the page, the parser is called, which creates widgets. The parser can be called on the whole page or on its separate part.

    Example code
    require(["dojo/parser", "aps/ready!"], function(parser){
	parser.parse();
});




    Widget Software Announcement

    The second way to declare widgets is software. Using require , the necessary modules are connected and by calling the constructor with parameters, the necessary widgets are created. The widget hierarchy is defined by adding child widgets using the addChild method .

    Example code
    require([
	"aps/FieldSet", 
	"aps/CheckBox", 
	"aps/ready!"
], function (FieldSet, CheckBox) {
	var fs = new FieldSet({
		title : “I am aps / FieldSet”
	}, "idDiv");
	fs.addChild(new CheckBox({
		label : “CheckBox”,
		description : “I am aps / CheckBox” 
	});
	fs.startup();
});



    Using bootloader

    The third way to describe the screen is through the bootloader. It was developed by us, and we recommend it. The location of widgets and their properties are set in the form of a JSON-like structure. Each widget is described by a triple: module name, constructor parameters and an array of child elements. The second and third element are optional. In addition to widgets, the loader can also create HTML tags.

    • No time wasted parsing HTML (first method);
    • in the program announcement (second method), the logical order is violated - first a child widget is created, and only then the parent (container) is created, which is inconvenient and poorly readable.


    Example code
    require([
	"aps/load", 
	"aps/ready!"
], function (load) {
	load(["aps/FieldSet", {
			title : "I am aps / FieldSet"
		}, [
			["aps / CheckBox", {
				label : "CheckBox",
				description : "I am aps / CheckBox"
			}
	]]]);
});



    Data sources

    Two modules can be used as a data source for a widget: Store for a remote source and Memory for a local one.
    For server requests, aps / Store uses an extended version of the RQL query language , which is also supported by aps / Memory.

    Example code
    require([
	"aps/Store", 
	"aps/Grid", 
	"aps/ready!"
], function (load) {
	var store = new Store({
		target : "http://localhost/resources"
	});
	var grid = new Grid({
		columns: layoutSimpleGrid,
		store: store
	}, "gridDiv");
});



    Data binding

    For two-way communication of data and widgets, modules of the Model family and the at method are used . If you want to connect data and widgets, then you specify the at method as the constructor parameter with the model name passed to it and the name of the property of the model object that you are mapping. In addition to mapping, the model supports the ability to track the value of its properties using the watch method .

    Example code
    require([
	"aps/TextBox", 
	"dojox/mvc/getStateful", 
	"dojox/mvc/at", 
	"aps/ready!"
], function (TextBox, getStateful, at) {
	model = getStateful({val : "Hello, world!"}); 
	new TextBox({value : at("model", "val")}, "divTB").startup(); 
});



    Uniform rules for naming properties and methods for all modules

    The naming rules are simple and standard:
    • names of private / protected properties and methods begin with an underscore “_”;
    • имена методов, функций и свойств начинаются с маленькой буквы, имена классов — с большой; если класс является миксином, то перед большой буквой ставится знак подчеркивания.


    Единые для всех модулей способы взаимодействия между собой и внешним миром

    • данные и виджеты связываются через модули Model и Store;
    • слежение за состоянием виджета осуществляется с помощью метода watch, виджеты не генерируют никаких специальных событий, вся работа со свойствами выполняется только через методы set и get.


    Пример взаимодействия с виджетом

    As an example of interaction with widgets, consider the selection of rows in a Grid widget .

    In the simplest case, to create a table with the ability to select rows, it is enough to specify the column structure, the row selection mode (whether it is possible to select one or several rows at the same time) and the data source.

    If a string is selected, its id is added to a special array stored in the selectionArray property . When the selection is removed, it is removed from there. Thus, to monitor the selection of lines, it is enough to add the callback method watchElements and it will automatically be called when the set of selected lines is changed.

    Two-way communication. Therefore, to visually highlight a row in the table, we just need to add its id toselectionArray .

    Example code
    var grid = new Grid({ //Грид с возможностью выбора строк
	columns : layoutSimpleGrid,
	selectionMode : "single",
	store : store
}); 
grid.get("selectionArray") //Отследим события выбора строк
	.watchElements(function (index, removals, adds) {
		alert(adds);
});
//выделим строку программно
grid.get("selectionArray").push("ea7865aa");



    Automated Tests


    Tests are important, and automated tests are essential. They allow you to keep the code in a stable state. And it’s better to develop them from the very beginning. But before developing tests, you need to think about a build system.

    Build system


    The build runs Jenkins , the standard build-scheduler. Maven , node.js and a headless phantomjs browser are deployed on the assembly machine . Our build is built into the maven build life cycle and includes checking JSHint code, compiling the clojure compiler , packaging, including creating layers (or merging files), testing and deploying the generated archive to the nexus artifact management system .

    Automated Testing

    Tests can determine the whole future fate of the framework, as stability is the key to development.

    What we followed when creating autotests:
    • can be run without infrastructure , as tests can serve as an example for developers;
    • because the framework is modular, there must be a strictly separate test for each component ;
    • tests should be located next to the framework itself , right in the distribution, so that you can quickly run them away and see how to use the framework;
    • and finally, to understand the tests, you must strictly adhere to naming conventions ; in our case, the test directories begin with test, everything else is for manual testing.


    Unit tests should always be run during assembly. It often happens that new functionality is required immediately. At the same time, during the development process, it turns out that some of the tests are optional "at first glance" and disabling them "will not hurt anyone." The test is disabled. Then another. And over time, the number of discrepancies turns out to be such that running unit tests becomes pointless.

    The assembly is canceled if at least one test fails. You cannot disable tests.

    Tool selection


    We chose QUnit and our own system that runs phantomjs , IE, Firefox and Safari in virtual machines. Previously, we examined:
    • TestSwarm
    • Buster.js;
    • DOH Robot;
    • External Farms (browsershots, browserling, etc);
    • Selenium


    Why QUnit

    The main thing for us is the ease of embedding. One JavaScript programming language for writing both code and tests, which is very convenient. QUnit is simple and widespread - if necessary, this makes it easy to outsource testing.

    Why not TestSwarm


    TestSwarm is heavily tied to infrastructure. But the main thing is that the build is marked after assembly, and not at the time, which contradicts our methodology.

    Why not Buster.js


    The framework is designed to run modules on the server and on the client - we do not have such a task. In addition, he does not know how to launch browsers on his own.


    Why not Dojox Robot

    The framework included in the Dojo Toolkit . Unfortunately, despite this, it is abandoned and significantly outdated.

    Why not external farms


    I really didn’t want to be tied to external systems, besides, this would require complicated configuration of VPN access to our assembly machines. Plus, their use is quite expensive.

    Why not Selenium

    Bulky, requires infrastructure tuning.


    Total

    We have a hybrid Selenium Server , which, on a command from the outside, launches a browser in a virtual machine. All tests run in one page, and the results can be delivered to the collector or displayed in the browser if the tests were run manually.

    Test Infrastructure Components

    The diagram shows the components of the test infrastructure. On the build server, phantom.js is launched in parallel (to get a quick result) and tests on different browsers running in different OSs on virtual machines.

    Documentation



    Even the most sophisticated API requires convenient documentation. The APS JS SDK documentation consists of two parts: API and Reference Guide .

    API

    The API section contains a brief description of all available modules and their interfaces. This section is primarily aimed at developers who have already figured out the SDK and want to quickly clarify the presence and correct spelling of a method or property. This information is generated based on comments in the code during code building.

    Reference Guide

    In the description of each module there is a link to the corresponding page of the second part of the documentation - Reference Guide . This section contains an extended (compared to the API) description of APS SDK modules and their main properties and methods. It is regularly updated and updated in accordance with the problems faced by users of our SDK. Each Reference Guide article also contains examples of the use of modules.

    Examples

    Examples of using and creating a module are given for all three ways of declaring widgets: declarative, program, and using the loader. But examples that cannot be run and tried are of little interest. The page of each module in the Reference Guide contains a link to a page with autotests. The user can see how the tests are carried out, and having looked at the source code, he can see examples demonstrating all the functionality of the widget.

    Sandbox



    But you always want to try it yourself. And, preferably, quickly and simply. To do this, most examples can be run in a special sandbox.

    When creating our sandbox, we were inspired by the notorious jsfiddle project , so it is called apsfiddle . As in it, the user has 4 areas available: HTML editor, CSS editor, JS editor, and the result of code execution (you can see it both in the designated area of ​​the screen and in a new tab). To create code editors, we used the CodeMirror project .

    In addition to manually entering text in the sandbox, you can open your files simply by dragging them into the browser window. Editors will fill up automatically. You can also open files in the traditional way.

    For collaboration, TogetherJS is used .

    Editors support
    • syntax highlighting;
    • verification of the entered code on the fly;
    • automatic substitution of closing brackets and tags;
    • folding code blocks;
    • beautiful formatting;
    • smart code completion.


    Auto completion

    Usually, developers, creating a sandbox for HTML + JS, are limited to simple editors and the frame in which the entered code is run. We believe that this is not correct. The person who came to the sandbox does not know the framework API well and without a maximum of hints, he most likely will not be able to do anything. Therefore, we implemented smart code completion with tooltips.



    For autocompletion the project Tern already mentioned on Habr is used . In his work, he uses several dictionaries. A dictionary of ECMA5 and JQuery methods and properties was provided by the project author; for the APS JS SDK objects, a dictionary is generated based on the documentation. This allows the editor to prompt methods and properties of the object based on which constructor it was created.

    "MessageList":{
	"!type":"fn(options: object)->!custom:MessageList_ctor",
	"prototype":{
	"add":{
		"!type": "fn(description: string, error: string)",
		"!doc":  "added new message"
	},
	...


    It often happens that within a quarter of the screen it gets crowded, so all editors support full-screen mode, in which you can switch between screens with hot keys, without going into normal mode.

    Hotkeys

    Hotkeys are an important element of working with apsfiddle . They duplicated all the functionality, and some operations can be done only by them. This is done so as not to clutter up the screen with a bunch of buttons. In order for the user to immediately recognize which key combinations can be used, when you open apsfiddle, he is first shown a complete list of keyboard combinations:

    System
    Ctrl + H - help
    Ctrl + O - download file
    Ctrl + R - clear fields
    Ctrl + Enter - launch
    Ctrl + 1 -focus on HTML editor
    Ctrl + 2 - focus on CSS editor
    Ctrl + 3 - focus on JavaScript editor
    F11- switching full-screen
    Esc mode - exiting full-screen mode

    General
    Ctrl + B - make beautiful formatting
    Ctrl + F - search
    Ctrl + K - minimize code block
    Ctrl + / - commenting
    Ctrl + Space - auto-complete

    HTML code
    Ctrl + J - switch by

    JavaScript tag
    Ctrl + I - show type
    Alt +. - jump to ad
    Alt +, - jump back

    Autosave

    To avoid accidental data loss, the entered code is periodically stored in localStorage. The last saved code is substituted into the editors when apsfiddle is opened again .

    Note

    When we created our sandbox, we did not set the task to develop a full-fledged Cloud IDE. First of all, we needed a platform where our users could easily try our SDK and quickly learn how to work with it based on the examples we provided. Therefore, we did not take the code of such large projects as, for example, Cloud9.

    In creating the most intelligent and convenient code completion for our framework, we are only at the very beginning of the journey. Now it requires compliance with the naming conventions for plug-ins, and we also do not know how to prompt property names during their implicit use. Those. when declaring an object that is an argument to the constructor, and when specifying the name of the property as a string, for example, to call encapsulating methods.

    Brief Summary


    Now APS JS SDK
    • has a modular structure;
    • includes "thick widgets", data binding and template;
    • supplied with "handwritten" and auto-generated documentation;
    • checked by automatic tests and manually;
    • has a sandbox with advanced code completion.

    Everything described can be tried here . Our main clients are located in the West, and the servers are hosted there, so the first download of apsfiddle can be a long one.

    In general, the article turned out to be a review, if any implementation details are interesting, then ask in the comments. If there are many questions about something, we will try to cover it in a separate article.

    The authors of the article are Timur Nizametdinov and Evgeny Uspensky, Parallels
    Tags:

    Also popular now: