SDK to you, SDK to me, SDK to everyone! How to do the SDK and why it is needed

Hello!

Our company makes a service for storing and processing data from industrial devices (pumps, drills and other industrial equipment). We store the data of our clients and provide the functionality for their analysis: the construction of reports, graphs and much more.

And in the course of the work, we noticed that the integration of each new client is greatly delayed, and the number of different errors is constantly increasing. Then it became clear that it was time to figure it out. As the situation analysis showed, the IT department of each of our clients developed their own solution for local data collection from devices and sending them to our service. Everything is complicated by the fact that, taking into account the specifics of the industry, there is not always access to the Internet and it is necessary to store data locally and send as soon as possible. And there are quite a few such nuances, which leads to an increase in the number of errors.

And then we realized that the best solution in this situation would be to develop an SDK and provide it to the client. Immediately I began to look for best practices and reasoning on the development of the SDK and I was very surprised - there is practically nothing about it in runet, and there is very little information on the Basurmanian Internet and it is fragmented. Well, the task is clear, thought out and implemented.

But it was the lack of information on this topic that gave rise to the desire to tell the community about the thoughts, decisions taken and conclusions on the development of the SDK. The article discusses the solution for .NET, but this is a concept, so it will be interesting to many. Details under the cut!

Time to be determined

To begin with, we define what the SDK is and why it may be needed.

The SDK (from the English software development kit) is a set of development tools that allows software specialists to create applications for a specific software package, basic software development tools, a hardware platform, a computer system, game consoles, operating systems, and other platforms. The SDK takes advantage of each platform and reduces integration time.
...
The software engineer usually receives the SDK from the target system developer.

Well, it is logical. In simple terms, the SDK is a library package, so that the client can quickly and easily start working with your system (this article will discuss our service, but everything stated in the article applies to other types of SDKs) or perform the same type of actions.

But, like any approach, the SDK Path has both advantages and disadvantages.

Benefits

High speed integration of a new client - your customers need to write less code.

Reuse code - the same code is used in several places at once. It can be said that this is a duplication of the previous paragraph, but it’s about the fact that the logic of work is lonely everywhere, which means

Predictable behavior - the use of the same libraries leads the behavior of systems to a certain standard, which greatly facilitates the search and elimination of errors and vulnerabilities.

The quality of the code - many places like to save on testing (it’s a pity for the budget, deadlines and other reasons are burning). It is clear that in the real world, to cover all project areas with tests is a laborious task. But to qualitatively test all the SDK modules and then use them is a way to increase the percentage of test coverage, which will lead you to reduce the number of errors.

Documentation - the same script as with tests. Cover documentation of the entire project is quite problematic. Re-use of SDK modules increases the percentage of documentation coverage, which reduces the threshold for new employees to enter the project and generally helps with life.

All the advantages, in fact, are the consequences of the most important thing - we write the code very qualitatively once, and then reuse it .

disadvantages

The high quality requirements of the SDK code are a consequence of the main advantage. An error in the SDK will generate errors in all systems that use it.

Setting Restrictions - The SDK is a set of libraries for implementing standard scripts. Sometimes SDK developers believe that in addition to the implementation of one of the provided scenarios, the client will not need anything, that it is easier for the client to do everything from scratch on their own than to build a pedestal of crutches for the SDK.

Dependency hell and updates - when expanding the functionality (for example, customizing the solution for a specific client), you will release a new version of the library. But there are dependencies, different sets of versions of libraries for different clients, and you need to very carefully monitor backward compatibility or strict versioning.

When the SDK is really needed

You have several standard scenarios that are re-implemented from one time to another - in fact, our case.

Internal development - in different projects you use logging systems, system configuration, work with HttpRequest, DB, files? Develop an internal SDK - a set of libraries for internal use. You can expand the functionality of the SDK at any time, but the speed of developing new projects, the percentage of coverage with tests and documentation will increase, and the threshold for new developers to fall.

When the SDK is likely to be superfluous

Usage scenarios are not defined or are constantly changing - leave the implementation of custom solutions to customers and help them. No need to fence vundervaflyu, which will only interfere. Very important for young companies and startups.

You do not know how to do qualitatively - I have bad news for you: it's time to learn. But to give a crooked solution to the client is very, very wrong. Customers need to be respected, after all.

So, we have decided what the SDK is, with its advantages and disadvantages and when we need it. If after this you understand that the SDK is really needed - I invite you to take the "SDK path" and figure out what it should be and how the hell can it be done?

"Do you like Lego?" - Modularity

Imagine all the possible scenarios for using the SDK (you have already decided why you need it, right?) And make a script for the library. What is not an option? But this is a bad approach, and we will not do that. And we will be like this:

• we break all the scenarios into steps
• identify common steps
• build a list of modules that implement all possible steps (one module is responsible for the implementation of something specific, for example, work with configurations)

For example, taking into account the specifics of the problem, we need to set all the logic from the configs. We implement the module for working with configs (reading, writing, updating, validating and processing configurations) and will use it in all other modules.

And for the implementation of standard scenarios, we will actually make modules — sort of “control” modules, each of which implements one specific script using other modules of the same SDK. Thus, for the implementation of standard scenarios, the client only needs to connect the script control module (and he himself will pull up all dependencies), and for the implementation of non-standard ones, we use the basic modules, also reusing the code.

This is the reason that the SDK should not be one library (although I really want to, I understand. After all, when the entire SDK is in one library, you can forget about the dependencies and everything connected with them), and be a set of libraries. An additional advantage of this approach will be to reduce the "weight" of the client's program - it will pull the heavy SDK, and only the necessary modules will be pulled.

But it’s not worthwhile to produce modules as horrible, because the more modules, the more headache from their dependencies! Those. It is important to correctly break the logic into modules, observing a balance between the decision "all in one" and "for each function its own module".

"And so it was possible?" - Versatility

Provide the client with various interfaces for working with your library. I will give an example:

publicvoidLoadConfiguration(string filename);
publicasync Task LoadConfigurationAsync(string filename);

If you provide only a synchronous version, then when implementing an asynchronous application, the client will be forced to make asynchronous wrappers for your synchronous method. If you provide only an asynchronous version - the situation is similar. Give the customer both and he will thank you.

A nice plus will be generics. For example, we have a class for working with configurations that implements methods for packing a config into a string, loading a config from a file, etc. The configuration of a specific module will be inherited from our base class, but to work with the new class, we also need to provide unpacking methods.

classBaseConfiguration{
    public BaseConfiguration FromString(string source){...}
    public BaseConfiguration FromString(string source,Type configurationType){...}
    public T FromString<T>(string source) where T:BaseConfiguration
}
classCustomConfiguration : BaseConfiguration{}

Thus, we have provided the client with as many as three implementations that he can use. Generics are very convenient, but when working with dynamic types, they can only be called through reflection, which is expensive. The general principle of universality, I hope, is clear.

"Parent 1, Parent 2, Children []" - Naming

What is the most difficult thing about a programmer? To invent names for variables.

And nevertheless ... Proper naming of modules, classes, properties and methods will greatly help those who will work with your SDK. Example not requiring comments:

Kinect 2.0 SDK example

var skeletons = new Skeleton[0];
using (var skeletonFrame = e.OpenSkeletonFrame())
{
    if (skeletonFrame != null)
    {
        skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
        skeletonFrame.CopySkeletonDataTo(skeletons);
    }
}
if (skeletons.Length == 0) { return; }
var skel = skeletons.FirstOrDefault(x => x.TrackingState == SkeletonTrackingState.Tracked);
if (skel == null) { return; }
var rightHand = skel.Joints[JointType.WristRight];
XValueRight.Text = rightHand.Position.X.ToString(CultureInfo.InvariantCulture);
YValueRight.Text = rightHand.Position.Y.ToString(CultureInfo.InvariantCulture);
ZValueRight.Text = rightHand.Position.Z.ToString(CultureInfo.InvariantCulture);

Everything is clear from the names of classes and methods. And if there is auto-completion of the code in your IDE, then it is often possible not to look in the documentation, if everything is clear.

"I’m sure if Death knew what bureaucracy is, people would never die, always standing in line ..." - Documentation

But even if all the modules, classes, methods and properties are named very beautifully and actually, the documentation still needs to be written. Firstly, it will greatly save your nerves (the number of customer questions decreases by an order of magnitude. Everything is in the documentation), and secondly, it is always clear why you did this and not otherwise.

Documentation in the SDK is usually simple and concise. It is usually divided into two parts: Tutorial is a step-by-step course in the style of “Build a city in 10 minutes” and the Reference section is a guide to everything that can be done using this SDK.

We chose the easiest way - summary + articles. We add Xml attributes for methods and classes that are lit in intellisense as hints. Using Docfx we build documentation on these attributes and get detailed and convenient documentation, which complements articles describing usage scenarios and examples.

"- To be clean! - How will I clean something with a fork?" - Testing

What can you say about testing in the discussion of the SDK ... Must have! The best solution would be TDD (despite the fact that I have a negative attitude to this approach, in this case I decided to use it). Yes, for a long time. Yes, tedious. But in the future, you will not hang from the constant falls of the SDK on the side and the consequences of this fall.

The main juice of the situation is that by giving the SDK to the client you lose control: you cannot quickly fix the error, it’s difficult to find this very error, and you will be quite foolish to look in such a situation. Therefore - test. Test better. Once again. And just in case, test your tests. And tests tests. So, something I got carried away, but the importance of testing the SDK, I hope, is clear.

"The victim, who could not resist his past, was absorbed by him" - Logi

Since you give away the SDK to a third-party company, as a result of which you lose control of the situation, in the event of an error (at the testing stage, you all decided so and it will do, yes?) You will have a rather long and painful process of searching for this error itself. This is where logs come to your rescue.

Log everything , absolutely everything, and in case of an error, request logs from your client. Thus, you will save a lot of time and you will not have a chance to face the client.

"Alarm! Achtung! Attention!" - Errors

Long thinking about errors I came to an interesting conclusion - no method in your SDK should give an error not described in the documentation . You have to admit that it’s very unpleasant when you connect a third-party library to work with HttpRequest, and it dumps some NullPointerException and StackTrace on you that takes you to the library. And you have to dive into these very "bowels", trying to understand how deep the rabbit hole is, and what the problem is.

Therefore, I propose the following solution - declare a closed list of possible exceptions and document them. But, since you cannot be sure that you have foreseen everything, wrap the method in try-catch, and the caught error - in the declared one. For example, a ConfigurationException that will contain an InnerException — a caught error. This will allow a third-party developer to catch all the possible errors, but in case something happens quickly to figure out what's wrong.

Versions or "how not to bite your own tail"

To avoid problems in the future, I highly recommend using strict versioning. Choose the versioning system that suits you and use it. But if the new version of the library does not have backward compatibility, this must be indicated. How to solve it - to think you. But it’s definitely worth thinking about.

"The engine that could" - Deploy

The need for updated documentation and versions generates a requirement for correct deployment. In our decision we use the following solution (crutches, but they work).
When you need to release a new release, the developer jerks bat'nik with the release number, and then the batch file:

• build release
• puts all the libraries in the archive
• build the latest documentation (docfx)
• indicates release version in documentation and archive name
• puts all the tiniest thing in the git repository
• WebApp on MS Azure pulls up a fresh commit for hikuk and publishes changes

At the exit we get an updated version of the site with documentation, where you can download the archive with the latest version of the SDK.
Future plans include packaging everything in Nuget packages and publishing to a local Nuget repository.

I recommend to pay attention to this point, because you can significantly reduce the number of headaches caused by the lack of current information about the new version of the library.

"-And so can you? - Garbage. Look how it should be!" - & toolkit examples

An important item of documentation are examples of use. But besides this, it is often required to provide not a library, but an application that implements the most standard scripts. I recommend doing these applications with an open and well-commented source code, which will allow you to kill two birds with one stone at a time - to provide a working application and provide an example of using the SDK.

Conclusion

The development of the SDK has become an interesting new task for me, which has raised many important architectural issues. A lot of things described in the article are obvious things (for me), but I consider it important to read out even the obvious things in order to get a clear overall picture.

PS

Thanks for reading, I will be glad to your comments. I hope this article will be useful for you.