May 2, 2019 at 13:21

Self-documenting REST server (Node.JS, TypeScript, Koa, Joi, Swagger)

  • Tutorial

Quite a lot of articles have already been written about the advantages and disadvantages of REST (and even more in the comments to them)). And if it so happened that you have to develop a service in which this architecture should be applied, then you will certainly come across its documentation. After all, creating each method, we certainly understand that other programmers will refer to these methods. Therefore, the documentation should be comprehensive, and most importantly - relevant.

Welcome to the cat, where I will describe how we solved this problem in our team.

A bit of context.

Our team was tasked to issue a backend product on Node.js of medium complexity in a short time . Frontend programmers and mobilizers were supposed to interact with this product.

After some thought, we decided to try using TypeScript as a YaP . Well-tuned TSLint and Prettier helped us achieve the same code style and tight check at the coding / assembly stage (and husky even at the commit stage). Strong typing led everyone to clearly describe the interfaces and types of all objects. It has become easy to read and understand what exactly this function takes as an input parameter, what it will eventually return, and which of the properties of the object are mandatory and which are not. The code began to resemble Java pretty much). And of course, TypeDoc added readability to every function.

This is how the code began to look:

/**
 * Interface of all responses
 */
export interface IResponseData {
  nonce: number;
  code: number;
  message?: string;
  data?: T;
}
/**
 * Utils helper
 */
export class TransferObjectUtils {
  /**
   * Compose all data to result response package
   *
   * @param responseCode - 200 | 400 | 500
   * @param message - any info text message
   * @param data - response data object
   *
   * @return ready object for REST response
   */
  public static createResponseObject(responseCode: number, message: string, data: T): IResponseData {
    const result: IResponseData = {
      code: responseCode || 200,
      nonce: Date.now()
    };
    if (message) {
      result.message = message;
    }
    if (data) {
      result.data = data;
    }
    return result;
  }
}

We thought about the descendants, it will not be difficult to maintain our code, it is time to think about the users of our REST server.

Since everything was done quite rapidly, we understood that writing code separately and documentation separately for it would be very difficult. Especially add additional parameters to answers or requests according to the requirements of front-end or mobilchiki and do not forget to warn others about it. This is where a clear requirement appeared: the code with the documentation should always be synchronized . This meant that the human factor should be excluded and the documentation should influence the code, and the code should affect the documentation.

Here I delved into the search for suitable tools for this. Fortunately, the NPM repository is just a storehouse of all kinds of ideas and solutions.

The requirements for the tool were as follows:

  • Documentation synchronization with code;
  • TypeScript support;
  • Validation of incoming / outgoing packets;
  • Live and supported package.

I had to write on a REST service using many different packages, the most popular of which are: tsoa, ​​swagger-node-express, express-openapi, swagger-codegen.



But in some there was no TypeScript support, in some package validation, and some were able to generate code based on documentation, but they did not provide further synchronization.

This is where I came across joi-to-swagger. A great package that can turn the described in Joi scheme into swagger documentation and even with TypeScript support. All items are executed except for synchronization. Rushing for some time, I found an abandoned repository of a Chinese who used joi-to-swaggerin conjunction with the Koa framework. Since there were no prejudices against Koa in our team, and there were no reasons to blindly follow the Express trend, we decided to try to take off on this stack.

I forked this repository, fixed bugs, completed some things, and now my first contribution to OpenSource Koa-Joi-Swagger-TS was released. We successfully passed that project and after it there were already several others. It has become very convenient to write and maintain REST services, and users of these services need nothing but a link to Swagger online documentation. After them, it became clear where this package can be developed and it has undergone several more improvements.

Now let's see how using Koa-Joi-Swagger-TS you can write a self-documenting REST server. I posted the finished code here .

Since this project is a demo, I simplified and merged several files into one. In general, it is good if the index initializes the application and calls the app.ts file, which in turn will read resources, calls to connect to the database, etc. The server should start with the latest command (just what will now be described below).

So, for starters, create index.ts with this content:

index.ts
import * as Koa from "koa";
import { BaseContext } from "koa";
import * as bodyParser from "koa-bodyparser";
import * as Router from "koa-router";
const SERVER_PORT = 3002;
(async () => {
  const app = new Koa();
  const router = new Router();
  app.use(bodyParser());
  router.get("/", (ctx: BaseContext, next: Function) => {
    console.log("Root loaded!")
  });
  app
    .use(router.routes())
    .use(router.allowedMethods());
  app.listen(SERVER_PORT);
  console.log(`Server listening on http://localhost:${SERVER_PORT} ...`);
})();



When you start this service, a REST server will be raised, which so far does not know how. Now a little about the architecture of the project. Since I switched to Node.JS from Java, I tried to build a service with the same layers here.

  • Controllers
  • Services
  • Repositories

Let's start connecting Koa-Joi-Swagger-TS . Naturally install it.

npm install koa-joi-swagger-ts --save

Create the “controllers” folder and the “schemas” folder in it . In the controllers folder, create our first controller base.controller.ts :

base.controller.ts
import { BaseContext } from "koa";
import { controller, description, get, response, summary, tag } from "koa-joi-swagger-ts";
import { ApiInfoResponseSchema } from "./schemas/apiInfo.response.schema";
@controller("/api/v1")
export abstract class BaseController {
  @get("/")
  @response(200, { $ref: ApiInfoResponseSchema })
  @tag("GET")
  @description("Returns text info about version of API")
  @summary("Show API index page")
  public async index(ctx: BaseContext, next: Function): Promise {
    console.log("GET /api/v1/");
    ctx.status = 200;
    ctx.body = {
      code: 200,
      data: {
        appVersion: "1.0.0",
        build: "1001",
        apiVersion: 1,
        reqHeaders: ctx.request.headers,
        apiDoc: "/api/v1/swagger.json"
      }
    }
  };
}


As you can see from decorators (annotations in Java), this class will be associated with the path “/ api / v1”, all methods inside will be relative to this path.

This method has a description of the response format, which is described in the file "./schemas/apiInfo.response.schema":

apiInfo.response.schema
import * as Joi from "joi";
import { definition } from "koa-joi-swagger-ts";
import { BaseAPIResponseSchema } from "./baseAPI.response.schema";
@definition("ApiInfo", "Information data about current application and API version")
export class ApiInfoResponseSchema extends BaseAPIResponseSchema {
  public data = Joi.object({
    appVersion: Joi.string()
      .description("Current version of application")
      .required(),
    build: Joi.string().description("Current build version of application"),
    apiVersion: Joi.number()
      .positive()
      .description("Version of current REST api")
      .required(),
    reqHeaders: Joi.object().description("Request headers"),
    apiDoc: Joi.string()
      .description("URL path to swagger document")
      .required()
  }).required();
}


The possibilities of such a description of the scheme in Joi are very extensively and described in more detail here: www.npmjs.com/package/joi-to-swagger

And here is the ancestor of the described class (actually this is the base class for all the answers of our service):

baseAPI.response.schema
import * as Joi from "joi";
import { definition } from "koa-joi-swagger-ts";
@definition("BaseAPIResponse", "Base response entity with base fields")
export class BaseAPIResponseSchema {
  public code = Joi.number()
    .required()
    .strict()
    .only(200, 400, 500)
    .example(200)
    .description("Code of operation result");
  public message = Joi.string().description("message will be filled in some causes");
}


Now register these circuits and controllers in the Koa-Joi-Swagger-TS system.
Next to index.ts, create another routing.ts file :

routing.ts
import { KJSRouter } from "koa-joi-swagger-ts";
import { BaseController } from "./controllers/base.controller";
import { BaseAPIResponseSchema } from "./controllers/schemas/baseAPI.response.schema";
import { ApiInfoResponseSchema } from "./controllers/schemas/apiInfo.response.schema";
const SERVER_PORT = 3002;
export const loadRoutes = () => {
  const router = new KJSRouter({
    swagger: "2.0",
    info: {
      version: "1.0.0",
      title: "simple-rest"
    },
    host: `localhost:${SERVER_PORT}`,
    basePath: "/api/v1",
    schemes: ["http"],
    paths: {},
    definitions: {}
  });
  router.loadDefinition(ApiInfoResponseSchema);
  router.loadDefinition(BaseAPIResponseSchema);
  router.loadController(BaseController);
  router.setSwaggerFile("swagger.json");
  router.loadSwaggerUI("/api/docs");
  return router.getRouter();
};


Here we create an instance of the KJSRouter class, which essentially is a Koa-router, but with middlewares and handlers added.

Therefore, in the index.ts file, we simply change

const router = new Router();

on the 

const router = loadRoutes();

Well, delete the unnecessary handler:

index.ts
import * as Koa from "koa";
import * as bodyParser from "koa-bodyparser";
import { loadRoutes } from "./routing";
const SERVER_PORT = 3002;
(async () => {
  const app = new Koa();
  const router = loadRoutes();
  app.use(bodyParser());
  app
    .use(router.routes())
    .use(router.allowedMethods());
  app.listen(SERVER_PORT);
  console.log(`Server listening on http://localhost:${SERVER_PORT} ...`);
})();


When starting this service, we have 3 routes available to us:
1. / api / v1 - the documented route
Which in my case is shown:

http: // localhost: 3002 / api / v1
{
  code: 200,
  data: {
    appVersion: "1.0.0",
    build: "1001",
    apiVersion: 1,
    reqHeaders: {
      host: "localhost:3002",
      connection: "keep-alive",
      cache-control: "max-age=0",
      upgrade-insecure-requests: "1",
      user-agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36",
      accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
      accept-encoding: "gzip, deflate, br",
      accept-language: "uk-UA,uk;q=0.9,ru;q=0.8,en-US;q=0.7,en;q=0.6"
    },
    apiDoc: "/api/v1/swagger.json"
  }
}


And two service routes:

2. /api/v1/swagger.json

swagger.json
{
  swagger: "2.0",
  info: {
    version: "1.0.0",
    title: "simple-rest"
  },
  host: "localhost:3002",
  basePath: "/api/v1",
  schemes: [
    "http"
  ],
  paths: {
    /: {
      get: {
        tags: [
          "GET"
        ],
        summary: "Show API index page",
        description: "Returns text info about version of API",
        consumes: [
          "application/json"
        ],
        produces: [
          "application/json"
        ],
        responses: {
          200: {
            description: "Information data about current application and API version",
            schema: {
              type: "object",
              $ref: "#/definitions/ApiInfo"
            }
          }
        },
        security: [ ]
      }
    }
  },
  definitions: {
    BaseAPIResponse: {
      type: "object",
      required: [
        "code"
      ],
      properties: {
        code: {
          type: "number",
          format: "float",
          enum: [
            200,
            400,
            500
          ],
          description: "Code of operation result",
          example: {
            value: 200
          }
        },
        message: {
          type: "string",
          description: "message will be filled in some causes"
        }
      }
    },
    ApiInfo: {
      type: "object",
      required: [
        "code",
        "data"
      ],
      properties: {
        code: {
          type: "number",
          format: "float",
          enum: [
            200,
            400,
            500
          ],
          description: "Code of operation result",
          example: {
            value: 200
          }
        },
        message: {
          type: "string",
          description: "message will be filled in some causes"
        },
        data: {
          type: "object",
          required: [
            "appVersion",
            "apiVersion",
            "apiDoc"
          ],
          properties: {
            appVersion: {
              type: "string",
              description: "Current version of application"
            },
            build: {
              type: "string",
              description: "Current build version of application"
            },
            apiVersion: {
              type: "number",
              format: "float",
              minimum: 1,
              description: "Version of current REST api"
            },
            reqHeaders: {
              type: "object",
              properties: { },
              description: "Request headers"
            },
            apiDoc: {
              type: "string",
              description: "URL path to swagger document"
            }
          }
        }
      }
    }
  }
}


3. / api / docs

This page with the Swagger UI is a very convenient visual representation of the Swagger scheme, in which, besides being easy to see, you can even generate requests and get real answers from the server.



This UI requires access to the swagger.json file, which is why the previous route was included.

Well, everything seems to be there and everything works, but! ..

After a while, we circled that in such an implementation, we get quite a lot of code duplication. In the case when the controllers need to do the same thing. It was because of this that I later finalized the package and added the ability to describe the “wrapper” for the controllers.

Consider an example of such a service.

Suppose we have a “Users” controller with several methods.

Get all users
  @get("/")
  @response(200, { $ref: UsersResponseSchema })
  @response(400, { $ref: BaseAPIResponseSchema })
  @response(500, { $ref: BaseAPIResponseSchema })
  @tag("User")
  @description("Returns list of all users")
  @summary("Get all users")
  public async getAllUsers(ctx: BaseContext): Promise {
    console.log("GET /api/v1/users");
    let message = "Get all users error";
    let code = 400;
    let data = null;
    try {
      let serviceResult = await getAllUsers();
      if (serviceResult) {
        data = serviceResult;
        code = 200;
        message = null;
      }
    } catch (e) {
      console.log("Error while getting users list");
      code = 500;
    }
    ctx.status = code;
    ctx.body = TransferObjectUtils.createResponseObject(code, message, data);
  };


Update user
  @post("/")
  @parameter("body", { $ref: UsersRequestSchema }, ENUM_PARAM_IN.body)
  @response(200, { $ref: BaseAPIResponseSchema })
  @response(400, { $ref: BaseAPIResponseSchema })
  @response(500, { $ref: BaseAPIResponseSchema })
  @tag("User")
  @description("Update user data")
  @summary("Update user data")
  public async updateUser(ctx: BaseContext): Promise {
    console.log("POST /api/v1/users");
    let message = "Update user data error";
    let code = 400;
    let data = null;
    try {
      let serviceResult = await updateUser(ctx.request.body.data);
      if (serviceResult) {
        code = 200;
        message = null;
      }
    } catch (e) {
      console.log("Error while updating user");
      code = 500;
    }
    ctx.status = code;
    ctx.body = TransferObjectUtils.createResponseObject(code, message, data);
  };


Insert user
  @put("/")
  @parameter("body", { $ref: UsersRequestSchema }, ENUM_PARAM_IN.body)
  @response(200, { $ref: BaseAPIResponseSchema })
  @response(400, { $ref: BaseAPIResponseSchema })
  @response(500, { $ref: BaseAPIResponseSchema })
  @tag("User")
  @description("Insert new user")
  @summary("Insert new user")
  public async insertUser(ctx: BaseContext): Promise {
    console.log("PUT /api/v1/users");
    let message = "Insert new user error";
    let code = 400;
    let data = null;
    try {
      let serviceResult = await insertUser(ctx.request.body.data);
      if (serviceResult) {
        code = 200;
        message = null;
      }
    } catch (e) {
      console.log("Error while inserting user");
      code = 500;
    }
    ctx.status = code;
    ctx.body = TransferObjectUtils.createResponseObject(code, message, data);
  };


As you can see, the three controller methods contain duplicate code. It is for such cases that we are now using this opportunity.

First, create a wrapper function, for example, directly in the routing.ts file .

const controllerDecorator = async (controller: Function, ctx: BaseContext, next: Function, summary: string): Promise => {
  console.log(`${ctx.request.method} ${ctx.request.url}`);
  ctx.body = null;
  ctx.status = 400;
  ctx.statusMessage = `Error while executing '${summary}'`;
  try {
    await controller(ctx);
  } catch (e) {
    console.log(e, `Error while executing '${summary}'`);
    ctx.status = 500;
  }
  ctx.body = TransferObjectUtils.createResponseObject(ctx.status, ctx.statusMessage, ctx.body);
};

Then connect it to our controller.

Replace

router.loadController(UserController);

on the

router.loadController(UserController, controllerDecorator);

Well, let's simplify our controller methods

User controller
  @get("/")
  @response(200, { $ref: UsersResponseSchema })
  @response(400, { $ref: BaseAPIResponseSchema })
  @response(500, { $ref: BaseAPIResponseSchema })
  @tag("User")
  @description("Returns list of all users")
  @summary("Get all users")
  public async getAllUsers(ctx: BaseContext): Promise {
    let serviceResult = await getAllUsers();
    if (serviceResult) {
      ctx.body = serviceResult;
      ctx.status = 200;
      ctx.statusMessage = null;
    }
  };
  @post("/")
  @parameter("body", { $ref: UsersRequestSchema }, ENUM_PARAM_IN.body)
  @response(200, { $ref: BaseAPIResponseSchema })
  @response(400, { $ref: BaseAPIResponseSchema })
  @response(500, { $ref: BaseAPIResponseSchema })
  @tag("User")
  @description("Update user data")
  @summary("Update user data")
  public async updateUser(ctx: BaseContext): Promise {
    let serviceResult = await updateUser(ctx.request.body.data);
    if (serviceResult) {
      ctx.status = 200;
      ctx.statusMessage = null;
    }
  };
  @put("/")
  @parameter("body", { $ref: UsersRequestSchema }, ENUM_PARAM_IN.body)
  @response(200, { $ref: BaseAPIResponseSchema })
  @response(400, { $ref: BaseAPIResponseSchema })
  @response(500, { $ref: BaseAPIResponseSchema })
  @tag("User")
  @description("Insert new user")
  @summary("Insert new user")
  public async insertUser(ctx: BaseContext): Promise {
    let serviceResult = await insertUser(ctx.request.body.data);
    if (serviceResult) {
      ctx.status = 200;
      ctx.statusMessage = null;
    }
  };


In this controllerDecorator, you can add any logic of checks or detailed logs of inputs / outputs.

I posted the finished code here .

Now we have almost CRUD ready. Delete can be written by analogy. In fact, now to write a new controller, we must:

  1. Create controller file
  2. Add it to routing.ts
  3. Describe methods
  4. In each method, use input / output circuits
  5. Describe these schemes
  6. Connect these schemes in routing.ts

If the incoming packet does not match the scheme, the user of our REST service will receive a 400 error with a description of what is exactly wrong. If the outgoing packet is invalid, then a 500 error will be generated.

Well, also as a pleasant trifle. In Swagger UI, you can use the “ Try it out ” functionality on any method. A request will be generated via curl to your running service, and of course you can see the result right away. And just for this it is very convenient to describe the parameter “ example ” in the circuit . Because the request will be generated right away with a ready-made package based on the described examples.



conclusions


Very convenient and useful in the end, the thing turned out. At first, they did not want to validate outgoing packets, but then with the help of this validation they caught several significant bugs on their side. Of course, you cannot fully use all the features of Joi (since we are limited by joi-to-swagger), but those that are quite enough.

Now the documentation is always online and always strictly corresponds to the code - and this is the main thing.
What other ideas are there? ..

Is it possible to add express support?
I just read it .

It would really be cool to describe entities once in one place. Because now it is necessary to edit both circuits and interfaces.

Maybe you will have some interesting ideas. Better yet Pull Requests :)
Welcome to contributors.
Tags:

Also popular now: