Documenting and testing the REST API using SpringRestDocs
Good afternoon, I want to touch on the topic of documenting the REST API. Immediately make a reservation, this material will be focused on engineers working in the Spring ecosystem.
I used the SpringRestDocs framework on several recent projects, it was successfully fixed in the portfolio, it was shown to acquaintances who also started to use it successfully and now I want to share with you in an article about its capabilities and advantages. This article will help you understand how to use SpringRestDocs and start using it.
In order to start working with SpringRestDocs, you need to understand the principle of operation of its pipeline, it is quite simple and linear:
All actions come from tests, except for the logic of resource verification, snippets are also generated. Snippets are the serialized value of a specific HTTP attribute that our controller interacted with. We are preparing a special template file in which we indicate in which sections the generated snippets should be included. The output is a compiled documentation file, please note that we can set the documentation format - it can be in the format html, pdf, epub, abook.
Further along the text of the article, we will collect this pipeline, write tests, configure SpringRestDocs and compile the documentation.
Below are the dependencies from my project working with spring rest docs, on the example of which we will analyze the work
I will show the part of the controller for which we will write a test, connect SpringRestDocs and generate documentation.
Let's analyze its logic. Using SpringDataRepository, I go to the database for the record with the ID that was passed to the controller. SpringDataRepository returns Optional - if there is a value in it, we are transforming the JPA Entity into a resource (at the same time we can encapsulate some of the fields that we do not want to show in the response), if Optional.isEmpty () then we return 404 NOT_FOUND code .
In the test, I connect the auto-configuration mockMVC, RestDocs. For restdocs, you must specify the directory where snippets will be generated (outputDir = "buid / generated-snippets") , this is a regular test using mockMvc, which we write almost every day when we test rest services. I use the proprietary library from the spring.tests mockMvc Dependence, however if you prefer to use RestAssured, then everything you read will also be relevant - there are only minor modifications. My test makes a call to the controller’s HTTP method, verifies the status, fields and prints request / response flow to the console.
After running the test in its output, we see the following:
This is the output to the console of the HTTP request and response content. Thus, we can trace what values were transferred to our controller and what was the response from it. The output to the console is performed by the connected handler:
As you can see, ResultHandler has access and can interpret the values of MvcResult - an object containing the results of the mockMvc test, and through it to the attributes of two key players - MockHttpServletRequest, MockHttpServletResponse. Here is a partial list of these attributes:
Here is an example of MyResultHandler, which logs the type of the HTTP method called and the status of the response code:
It is this idea with processing and registration that Pivotal used to generate documentation. We need to connect a handler from the MockMvcRestDocumentation class in our test:
Run the test again and pay attention to the fact that after its execution , the folders with files were created in the build / generated-snippets directory:
These are the generated snippets. By default, rest docs generates 6 types of snippets, I will show some of them.
curl-request.adoc
http-request.adoc
http-response.adoc
Now you need to prepare the template file and mark out its sections, where the generated snippet blocks will be included. The template is maintained in a flexible asciidoc format, by default the template should be in the src / docs / asciidoc directory :
Using the asciidoc syntax, we can include static files (for example, in the rest_conv.adoc file, I made a description of what methods the service supports, in which cases what status codes should be returned), as well as auto-generated snippets files.
In order for the documentation to compile, you need to make a basic setup - connect the necessary dependencies, asciidoctor-gradle-plugin must be added to gradle.build buildscript.dependencies
Apply the plugin
Now we need to make the basic asciidoctor configuration:
Let's check the documentation assembly, run it in the console
since we indicated that the asciidoctor task depends on running the tests, first the tests will break through, generate snippets and these snippets will be included in the generated documentation.
All the described configuration steps must be performed once when raising the project. Now every time you run tests, we will additionally generate snippets and documentation. Here are some screenshots:
Agreement section on HTTP methods and status codes
Example documentation for the Get All Speakers method
Identical documentation is also available in pdf format. It is convenient as an offline version, it can be sent along with the specifications of your service to customers.
Well, since we work with spring boot, now we can use one of its interesting properties - all resources that are in the src / static directory or src / public will be available as static content when accessed from the browser
This is exactly what we will do - after assembling the documentation, we will copy it to the / static / docs directory. Thus, each collected jar artifact will contain a static endpoint with documentation. Regardless of where it will be deployed, what environment it will be on - the current version of the documentation will always be available in front of us.
I used the SpringRestDocs framework on several recent projects, it was successfully fixed in the portfolio, it was shown to acquaintances who also started to use it successfully and now I want to share with you in an article about its capabilities and advantages. This article will help you understand how to use SpringRestDocs and start using it.
From the moment I got acquainted with this tool, I realized that there was a solution that I was waiting for and it was not enough in development. Judge for yourself - you could only dream about it:Let's understand the work of SpringRestDocs, I will combine the material with theoretical insets and guide the practical line of the tutorial, after reading which you can configure and use the framework.
- Documentation is automatically generated when tests are run.
- You can control the source format of the documentation file — for example, compile html. Since we use Spring boot, we can modify the steps and tasks in gradle, copy the documentation file and include it in a jar, make the documentation documentation on the remote server, copy the documentation to the archive. Thus, you will always have a static endpoint with documentation wherever your service is deployed. For offline version, you can connect pdf, epub, abook permissions.
- The documentation of our REST service is guaranteed to correspond to the logic of work. The documentation is synchronized with the application logic. Made changes, forgot to reflect them in the documentation - instantly you see falling tests with a detailed description of the difference of non-compliance.
- Documentation is generated from tests. Now, in order to add new sections to the documentation or to start conducting it, you need to start by writing a test, yes a test. Indeed, very often developers, in conditions of lack of time, undelivered processes on the project, or other reasons, write tons of code, but do not pay attention to the importance of tests. Oddly enough, the documentation framework encourages you to work on TDD
- As a result, you keep coverege high. More precisely, it is not coverage percentages that will be drawn in code analysis systems or reports that are important. The important thing is that you will cover different scenarios with separate tests and include their results in the documentation. Green tests are always a pleasure.
SpringRestDocs Pipeline
In order to start working with SpringRestDocs, you need to understand the principle of operation of its pipeline, it is quite simple and linear:
All actions come from tests, except for the logic of resource verification, snippets are also generated. Snippets are the serialized value of a specific HTTP attribute that our controller interacted with. We are preparing a special template file in which we indicate in which sections the generated snippets should be included. The output is a compiled documentation file, please note that we can set the documentation format - it can be in the format html, pdf, epub, abook.
Further along the text of the article, we will collect this pipeline, write tests, configure SpringRestDocs and compile the documentation.
Dependencies
Below are the dependencies from my project working with spring rest docs, on the example of which we will analyze the work
dependencies {
compile "org.springframework.boot:spring-boot-starter-data-jpa"
compile "org.springframework.boot:spring-boot-starter-hateoas"
compile "org.springframework.boot:spring-boot-starter-web"
compile "org.springframework.restdocs:spring-restdocs-core:$restdocsVersion"
compile "com.h2database:h2:$h2Version"
compile "org.projectlombok:lombok"
testCompile "org.springframework.boot:spring-boot-starter-test"
asciidoctor "org.springframework.restdocs:spring-restdocs-asciidoctor:$restdocsVersion"
testCompile "org.springframework.restdocs:spring-restdocs-mockmvc:$restdocsVersion"
testCompile "com.jayway.jsonpath:json-path"
}
Testable Controller
I will show the part of the controller for which we will write a test, connect SpringRestDocs and generate documentation.
@RestController
@RequestMapping("/speakers")
public class SpeakerController {
@Autowired
private SpeakerRepository speakerRepository;
@GetMapping(path = "/{id}")
public ResponseEntity getSpeaker(@PathVariable long id) {
return speakerRepository.findOne(id)
.map(speaker -> ResponseEntity.ok(new SpeakerResource(speaker)))
.orElse(new ResponseEntity(HttpStatus.NOT_FOUND));
} Let's analyze its logic. Using SpringDataRepository, I go to the database for the record with the ID that was passed to the controller. SpringDataRepository returns Optional - if there is a value in it, we are transforming the JPA Entity into a resource (at the same time we can encapsulate some of the fields that we do not want to show in the response), if Optional.isEmpty () then we return 404 NOT_FOUND code .
SpeakerResource Resource Code
@NoArgsConstructor
@AllArgsConstructor
@Getter
@Relation(value = "speaker", collectionRelation = "speakers")
public class SpeakerResource extends ResourceSupport {
private String name;
private String company;
public SpeakerResource(Speaker speaker) {
this.name = speaker.getName();
this.company = speaker.getCompany();
add(linkTo(methodOn(SpeakerController.class).getSpeaker(speaker.getId())).withSelfRel());
add(linkTo(methodOn(SpeakerController.class).getSpeakerTopics(speaker.getId())).withRel("topics"));
}
}
Let's write a basic test for this endpoint.
@RunWith(SpringRunner.class)
@SpringBootTest
@AutoConfigureMockMvc
@AutoConfigureRestDocs(outputDir = "build/generated-snippets")
public class SpControllerTest {
@Autowired
private MockMvc mockMvc;
@Autowired
private SpeakerRepository speakerRepository;
@After
public void tearDown() {
speakerRepository.deleteAll();
}
@Test
public void testGetSpeaker() throws Exception {
// Given
Speaker speaker = Speaker.builder().name("Roman").company("Lohika").build();
speakerRepository.save(speaker);
// When
ResultActions resultActions = mockMvc.perform(get("/speakers/{id}", speaker.getId()))
.andDo(print());
// Then
resultActions.andExpect(status().isOk())
.andExpect(jsonPath("name", is("Roman")))
.andExpect(jsonPath("company", is("Lohika")));
}
}
In the test, I connect the auto-configuration mockMVC, RestDocs. For restdocs, you must specify the directory where snippets will be generated (outputDir = "buid / generated-snippets") , this is a regular test using mockMvc, which we write almost every day when we test rest services. I use the proprietary library from the spring.tests mockMvc Dependence, however if you prefer to use RestAssured, then everything you read will also be relevant - there are only minor modifications. My test makes a call to the controller’s HTTP method, verifies the status, fields and prints request / response flow to the console.
ResultsHandler
After running the test in its output, we see the following:
MockHttpServletRequest:
HTTP Method = GET
Request URI = /speakers/1
Parameters = {}
Headers = {}
Handler:
Type = smartjava.domain.speaker.SpeakerController
Method = public org.springframework.http.ResponseEntity smartjava.domain.speaker.SpeakerController.getSpeaker(long)
Async:
Async started = false
Async result = null
Resolved Exception:
Type = null
ModelAndView:
View name = null
View = null
Model = null
FlashMap:
Attributes = null
MockHttpServletResponse:
Status = 200
Error message = null
Headers = {Content-Type=[application/hal+json;charset=UTF-8]}
Content type = application/hal+json;charset=UTF-8
Body = {
"name" : "Roman",
"company" : "Lohika",
"_links" : {
"self" : {
"href" : "http://localhost:8080/speakers/1"
},
"topics" : {
"href" : "http://localhost:8080/speakers/1/topics"
}
}
} This is the output to the console of the HTTP request and response content. Thus, we can trace what values were transferred to our controller and what was the response from it. The output to the console is performed by the connected handler:
resultActions.andDo(print());ResultHandler is a functional interface. Having created our own implementation and connecting it in the test, we can have access to the HttpRequest / HttpResponse , which was executed in the test and interpret the results of the execution, if we want to record these values in the console, on the file system, in our own documentation file, and so on.
public interface ResultHandler {
/**
* Perform an action on the given result.
*
* @param result the result of the executed request
* @throws Exception if a failure occurs
*/
void handle(MvcResult result) throws Exception;
}Mvcresult
As you can see, ResultHandler has access and can interpret the values of MvcResult - an object containing the results of the mockMvc test, and through it to the attributes of two key players - MockHttpServletRequest, MockHttpServletResponse. Here is a partial list of these attributes:
Here is an example of MyResultHandler, which logs the type of the HTTP method called and the status of the response code:
public class MyResultHandler implements ResultHandler {
private Logger logger = LoggerFactory.getLogger(MyResultHandler.class);
static public ResultHandler myHandler() {
return new MyResultHandler();
}
@Override
public void handle(MvcResult result) throws Exception {
MockHttpServletRequest request = result.getRequest();
MockHttpServletResponse response = result.getResponse();
logger.error("HTTP method: {}, status code: {}", request.getMethod(), response.getStatus());
}
} resultActions.andDo(new MyResultHandler())It is this idea with processing and registration that Pivotal used to generate documentation. We need to connect a handler from the MockMvcRestDocumentation class in our test:
// Document
resultActions.andDo(MockMvcRestDocumentation.document("{class-name}/{method-name}"));Let's generate snippets
Run the test again and pay attention to the fact that after its execution , the folders with files were created in the build / generated-snippets directory:
./sp-controller-test/test-get-speaker:
total 48
-rw-r--r-- 1 rtsypuk staff 68B Oct 31 14:17 curl-request.adoc
-rw-r--r-- 1 rtsypuk staff 87B Oct 31 14:17 http-request.adoc
-rw-r--r-- 1 rtsypuk staff 345B Oct 31 14:17 http-response.adoc
-rw-r--r-- 1 rtsypuk staff 69B Oct 31 14:17 httpie-request.adoc
-rw-r--r-- 1 rtsypuk staff 36B Oct 31 14:17 request-body.adoc
-rw-r--r-- 1 rtsypuk staff 254B Oct 31 14:17 response-body.adoc
These are the generated snippets. By default, rest docs generates 6 types of snippets, I will show some of them.
Snippet is a part of the HTTP request / response component serialized in a file in a textual representation. The most commonly used snippets are curl-request, http-request, http-response, request-body, response-body, links (for HATEOAS services), path-parameters, response-fields, headers.
curl-request.adoc
[source,bash]
----
$ curl 'http://localhost:8080/speakers/1' -i
----http-request.adoc
[source,bash]
[source,http,options="nowrap"]
----
GET /speakers/1 HTTP/1.1
Host: localhost:8080
----http-response.adoc
[source,bash]
[source,http,options="nowrap"]
----
HTTP/1.1 200 OK
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 218
{
"name" : "Roman",
"company" : "Lohika",
"_links" : {
"self" : {
"href" : "http://localhost:8080/speakers/1"
},
"topics" : {
"href" : "http://localhost:8080/speakers/1/topics"
}
}
}
----Preparing the template file
Now you need to prepare the template file and mark out its sections, where the generated snippet blocks will be included. The template is maintained in a flexible asciidoc format, by default the template should be in the src / docs / asciidoc directory :
== Rest convention
include::etc/rest_conv.adoc[]
== Endpoints
=== Speaker
==== Get speaker by ID
===== Curl example
include::{snippets}/sp-controller-test/test-get-speaker/curl-request.adoc[]
===== HTTP Request
include::{snippets}/sp-controller-test/test-get-speaker/http-request.adoc[]
===== HTTP Response
====== Success HTTP responses
include::{snippets}/sp-controller-test/test-get-speaker/http-response.adoc[]
====== Response fields
include::{snippets}/sp-controller-test/test-get-speaker/response-fields.adoc[]
====== HATEOAS links
include::{snippets}/sp-controller-test/test-get-speaker/links.adoc[]Using the asciidoc syntax, we can include static files (for example, in the rest_conv.adoc file, I made a description of what methods the service supports, in which cases what status codes should be returned), as well as auto-generated snippets files.
Static rest_conv.adoc
=== HTTP verbs
Speakers Service tries to adhere as closely as possible to standard HTTP and REST conventions in its
use of HTTP verbs.
|===
| Verb | Usage
| `GET`
| Used to retrieve a resource
| `POST`
| Used to create a new resource
| `PATCH`
| Used to update an existing resource, including partial updates
| `PUT`
| Used to update an existing resource, full updates only
| `DELETE`
| Used to delete an existing resource
|===
=== HTTP status codes
Speakers Service tries to adhere as closely as possible to standard HTTP and REST conventions in its
use of HTTP status codes.
|===
| Status code | Usage
| `200 OK`
| Standard response for successful HTTP requests.
The actual response will depend on the request method used.
In a GET request, the response will contain an entity corresponding to the requested resource.
In a POST request, the response will contain an entity describing or containing the result of the action.
| `201 Created`
| The request has been fulfilled and resulted in a new resource being created.
| `204 No Content`
| The server successfully processed the request, but is not returning any content.
| `400 Bad Request`
| The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).
| `404 Not Found`
| The requested resource could not be found but may be available again in the future. Subsequent requests by the client are permissible.
| `409 Conflict`
| The request could not be completed due to a conflict with the current state of the target resource.
| `422 Unprocessable Entity`
| Validation error has happened due to processing the posted entity.
|===Configure build.gradle
In order for the documentation to compile, you need to make a basic setup - connect the necessary dependencies, asciidoctor-gradle-plugin must be added to gradle.build buildscript.dependencies
buildscript {
repositories {
jcenter()
mavenCentral()
mavenLocal()
maven { url "https://plugins.gradle.org/m2/" }
}
dependencies {
classpath "org.asciidoctor:asciidoctor-gradle-plugin:$asciiDoctorPluginVersion"
classpath "org.springframework.boot:spring-boot-gradle-plugin:$springBootVersion"
}
}Apply the plugin
apply plugin: 'org.asciidoctor.convert'
Now we need to make the basic asciidoctor configuration:
asciidoctor {
dependsOn test
backends = ['html5']
options doctype: 'book'
attributes = [
'source-highlighter': 'highlightjs',
'imagesdir' : './images',
'toc' : 'left',
'toclevels' : 3,
'numbered' : '',
'icons' : 'font',
'setanchors' : '',
'idprefix' : '',
'idseparator' : '-',
'docinfo1' : '',
'safe-mode-unsafe' : '',
'allow-uri-read' : '',
'snippets' : snippetsDir,
linkattrs : true,
encoding : 'utf-8'
]
inputs.dir snippetsDir
outputDir "build/asciidoc"
sourceDir 'src/docs/asciidoc'
sources {
include 'index.adoc'
}
}
Let's check the documentation assembly, run it in the console
gradle asciidoctorsince we indicated that the asciidoctor task depends on running the tests, first the tests will break through, generate snippets and these snippets will be included in the generated documentation.
Documentation
All the described configuration steps must be performed once when raising the project. Now every time you run tests, we will additionally generate snippets and documentation. Here are some screenshots:
Agreement section on HTTP methods and status codes
Example documentation for the Get All Speakers method
Identical documentation is also available in pdf format. It is convenient as an offline version, it can be sent along with the specifications of your service to customers.
Modification of the jar task
Well, since we work with spring boot, now we can use one of its interesting properties - all resources that are in the src / static directory or src / public will be available as static content when accessed from the browser
jar {
dependsOn asciidoctor
from ("${asciidoctor.outputDir}/html5") {
include '**/index.html'
include '**/images/*'
into 'static/docs'
}
}
This is exactly what we will do - after assembling the documentation, we will copy it to the / static / docs directory. Thus, each collected jar artifact will contain a static endpoint with documentation. Regardless of where it will be deployed, what environment it will be on - the current version of the documentation will always be available in front of us.
Conclusion
This is only a small part of the capabilities of this wonderful tool, it is impossible to cover everything in one article. For anyone interested in SpringRestDocs, I offer links to resources:
- this is how the compiled documentation looks like, in this example you can look at the asciidoc format, how powerful this tool is (by the way, you can automatically download the dock on githubpages) tsypuk.github.io/springrestdoc
- my github with a customized demo project with SpringRestDocs github.com/tsypuk/springrestdoc (everything is configured, use the code in your projects for a quick start, there are demo syntax asciidoctor, examples of extensions, diagrams that can be easily generated and included in the documentation)
- And of course the official documentation