The history of testing the project "K": Kotlin & Spek

Hi, Habr!

In this article we will talk about automated testing on one of the many QIWI projects, which received the code name "K". When we organized the testing of this project, we decided to choose a practical and hypo Kotlin , as well as Spek , which says “You call them tests, we call them specifications” (You call them tests, we call them specifications). Perhaps this approach will suit you if you run into similar tasks. Why Kotlin, and not something else? Kotlin was chosen by development, to experiment, as this particular product was not critical, and it was possible to practice it live without fear that there would be problems. Official documentation







tells us that “Spek is written in Kotlin, and the specifications you write will be written in Kotlin” - this very clearly answers the question: “Why is it necessary?”.

So…

What is it and why is it needed?

The project provides its partner with software, which is an Android application. The lion's share of tests falls on the back-end, so we will focus on testing the REST API.

For a bundle that allows you to write tests and get results, everything is clear: you need a programming language, a test framework, an HTTP client and reports. And what about the entry point to our test universe?

Requirements, they are specifications, the project developers decided to write in the form of tests. The result was an interesting picture - BDD. Thus, Kotlin, Spek and khttp appeared on the scene.
The attentive reader will ask - OK, but where are the testers?

Testers

Having finished two birds with one stone, the development gave the product tester both the requirements and the autotests. Since then, the tester extends the test coverage according to the requirements, and also supports and creates new tests with the developers.

“This cannot go on forever and should not end tragically for the testing process!” - when colleagues had such an idea, the service team of the Testing Department entered the game. The service department was faced with the task: to study Kotlin in a short time in order to immediately take over the support of the tests if necessary.

Getting started

In service, the service department has IntelliJ IDEA , and since Kotlin works on top of the JVM and is developed by JetBrains , I didn’t have to install anything extra for writing code.

The process of learning the language for obvious reasons, skip.

The first thing to start with was to clone the repository:
git clone https://gerrit.project.com/k/autotests

Then the project was opened and the gradle settings were imported :



For complete satisfaction and comfort (* Actually, this is mandatory), the Spek plugin was installed:



It ensured the launch of tests in the development environment:



First the stage was completed, and it was time to start writing the tests themselves.

Tests

Brave guys from the service department do not belong to one or another product. These are the employees who are in a hurry to assist in setting up automation on the project, including all the stages of the process, and then handing over the tests and support to the operation of the product testers.

And since the interaction of the internal teams of the testing department is organized in a similar way, then at the entrance the service department “asks” at least the requirements for the feature.

It may seem like a deadlock in the case of “K”. But it was not there:

• Required read access to the repository where the source of the project is stored;
• Cloned the repository;
• Began to dive into the functionality of the product through reading the source code written in Java.

What did you read?

Development "K" asked to write tests for the feature, which allowed you to add, update and delete products for sale. The implementation consisted of two parts: “web” and “mobile”.

In the case of web:

• To add products using a POST request, the body of which contains JSON with data.
• To update or edit products use a PUT request, the body of which contains JSON with modified data.
• To delete products use a DELETE request, the body of which is empty.

In the case of mobile:

To add, update and delete products use a POST request, the body of which contains JSON with data for the specified operations.

Those. JSON has three nodes:

• "Added": list of added goods,
• "Removed": list of products to be removed,
• Updated: list of updated products.

What did you write?

The test class containing the test-specifications was already created and contained test methods (* a little out of Spek), so all that was needed was to expand it.

For web

Test for successful addition of goods:

• Add a product
• Check that the item is added.
• Delete created product (postcondition)

Code:

on("get changed since when goods added") {
            val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8)
            val date = Date.from(Instant.now()).time - 1
            val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader,
                    json = dataToMap(goods.copy(name = goodsUpdateName)))
            it("should return the status code OK") {
                goodsAdded.statusCode.should.be.equal(OK)
            }
            val goodId = goodsAdded.jsonObject?.optLong("id")
            val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" touser.AppUID))
            it("should return the status code OK") {
                goodsUpdates.statusCode.should.be.equal(OK)
            }
            val goodsInsert = goodsUpdates.jsonObject.getJSONArray("updated").toList()
                    .map { it as JSONObject }
                    .find {
                        it.optLong("goodId") == goodId
                    }
            it("should contain goods insert") {
                goodsInsert.should.be.not.`null`
                goodsInsert?.optString("name").should.be.equal(goodsUpdateName)
            }
            delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader)
        }

Test for successful removal of goods:

• Add a product (precondition)
• Remove the goods
• Check that the product has been deleted.

Code:

on("get changed since when goods deleted") {
            val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8)
            val date = Date.from(Instant.now()).time - 1
            val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader,
                    json = dataToMap(goods.copy(name = goodsUpdateName)))
            it("should return the status code OK") {
                goodsAdded.statusCode.should.be.equal(OK)
            }
            val goodId = goodsAdded.jsonObject?.optLong("id")
            val responseDelete = delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader)
            it("should return the status code NO_CONTENT") {
                responseDelete.statusCode.should.be.equal(NO_CONTENT)
            }
            val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" touser.AppUID))
            it("should contain goods deletes") {
                goodsUpdates.statusCode.should.be.equal(OK)
                goodsUpdates.jsonObject.getJSONArray("removed").toList()
                        .map { it asInt }
                        .find {
                            it == goodId.toInt()
                        }
                        .should.be.not.`null`
            }
        }

Negative test for unauthorized user request execution

• Add a product
• Check the status of the response
• A request to add a product is sent without an authorization header. The answer comes with a status of 401 Unauthorized.

Code:

on("get changed since when goods added without authorization") {
            val response = post(baseUrl + "goods/${user.storeId}",
                    json = dataToMap(goods))
            it("should contain an Unauthorized response status and an empty body") {
                response.statusCode.should.be.equal(UNAUTHORIZED)
                response.text.should.be.equal("")
            }
        }

For mobile

Auxiliary functions were written to get the nodes from the response body and form the request body.

Code:

package com.qiwi.k.tests
import com.fasterxml.jackson.databind.ObjectMapper
import khttp.responses.Response
import org.json.JSONObject
val mapper = ObjectMapper()
funarrayAdded(n: Int): Array<GoodsUpdate> {
    return Array(n) { i -> GoodsUpdate() }
}
fungetGoodsIds(list: List<GoodsUpdate>): List<Long> {
    return Array(list.size) { i -> list[i].goodId }.toList()
}
fungetResult(response: Response): List<GoodsUpdate> {
    return mapper.readValue(
            response.jsonObject.getJSONArray("result").toString(),
            Array<GoodsUpdate>::class.java
    ).toList()
}
fungetCountryIdFromTheResult(response: Response): List<Int> {
    val listGoods = mapper.readValue(
            response.jsonObject.getJSONArray("result").toString(),
            Array<GoodsUpdate>::class.java
    ).toList()
    return Array(listGoods.size) { i -> listGoods[i].countryId }.toList()
}
fungetBody(added: Array<GoodsUpdate> = emptyArray(),
            removed: List<Long> = emptyList(),
            updated: List<GoodsUpdate> = emptyList()): JSONObject {
    return JSONObject(
            mapOf(
                    "added" to added,
                    "removed" to removed,
                    "updated" to updated
            )
    )
}

Test for the successful addition of goods

• Add a product
• Check that the item is added.
• Remove the item (postcondition)

Code:

on("adding goods") {
            valrespAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count)))valresultOfAdding = getResult(respAdd)it("should return the status code OK") {
                respAdd.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count") {
                resultOfAdding.should.be.size.equal(count)
            }
            post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding)))
        }

Test for successful product update

• Add a product (precondition)
• We update the product
• Check that the added product has been updated.
• Remove the item (postcondition)

Code:

on("updating goods") {
            valrespAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count)))valresultOfAdding = getResult(respAdd)it("should return the status code respAdd OK") {
                respAdd.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count (resultOfAdding)") {
                resultOfAdding.should.be.size.equal(count)
            }
            valrespUpdate = post(urlGoodsUpdate,
                    authHeaderWithAppUID,
                    json = getBody(updated = resultOfAdding.map { it.copy(countryId = REGION_77) })
            )it("should return the status code respUpdate OK") {
                respUpdate.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count (respUpdate)") {
                getResult(respUpdate).should.be.size.equal(count)
            }
            it("should be all elements are 77") {
                getCountryIdFromTheResult(respUpdate).should.be.all.elements(REGION_77)
            }
            post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding)))
        }

Test for successful removal of goods:

• Add a product (precondition)
• Remove the goods
• Check that the added item has been deleted.

Code:

on("deleting goods") {
            valrespAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count)))valresultOfAdding = getResult(respAdd)it("should return the status code respAdd OK") {
                respAdd.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count") {
                resultOfAdding.should.be.size.equal(count)
            }
            valrespRemoved = post(urlGoodsUpdate,
                    authHeaderWithAppUID,
                    json = getBody(removed = getGoodsIds(resultOfAdding))
            )it("should return the status code respRemoved OK") {
                respRemoved.statusCode.should.be.equal(OK)
            }
            it("should be empty") {
                getResult(respRemoved).should.be.empty
            }
        }

After writing the tests, it was necessary to go through the review code.

Review

More than a dozen commits, a lot of correspondence with dev, visiting forums, chatting with Google. And that's what the result.

Code:

package com.qiwi.k.tests.catalog
import …
class GoodsUpdatesControllerSpec : WebSpek({
    given("GoodsUpdatesController") {
        val OK = HttpResponseStatus.OK.code()
        val NO_CONTENT = HttpResponseStatus.NO_CONTENT.code()
        val UNAUTHORIZED = HttpResponseStatus.UNAUTHORIZED.code()
        val REGION_77 = 77
        val auth = login(user)
        val accessToken = auth.tokenHead + auth.tokenTail
        val authHeader = mapOf("Authorization" to "Bearer $accessToken")
        val baseUrl = "http://test.qiwi.com/catalog/"
        val count = 2
        val authHeaderWithAppUID = mapOf("Authorization" to "Bearer $accessToken", "AppUID" touser.AppUID)
        val urlGoodsUpdate = "http://test.qiwi.com/catalog/updates/goods/"
        on("get changes since") {
            val goodsName: String = goodsForUpdate.name + Random().nextInt(1000)
            val date = Date.from(Instant.now()).time - 1
            put(baseUrl + "goods/${user.storeId}", authHeader,
                    json = dataToMap(goodsForUpdate.copy(name = goodsName)))
            val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" touser.AppUID))
            it("should contain goods updates") {
                val updates = goodsUpdates.jsonObject.getJSONArray("updated").toList()
                        .map { it as JSONObject }
                        .find {
                            it.optLong("goodId") == goodsForUpdate.id
                        }
                updates.should.be.not.`null`
                updates?.optString("name").should.be.equal(goodsName)
            }
        }
        on("get changed since when goods added") {
            val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8)
            val date = Date.from(Instant.now()).time - 1
            val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader,
                    json = dataToMap(goods.copy(name = goodsUpdateName)))
            it("should return the status code OK") {
                goodsAdded.statusCode.should.be.equal(OK)
            }
            val goodId = goodsAdded.jsonObject?.optLong("id")
            val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" touser.AppUID))
            it("should return the status code OK") {
                goodsUpdates.statusCode.should.be.equal(OK)
            }
            val goodsInsert = goodsUpdates.jsonObject.getJSONArray("updated").toList()
                    .map { it as JSONObject }
                    .find {
                        it.optLong("goodId") == goodId
                    }
            it("should contain goods insert") {
                goodsInsert.should.be.not.`null`
                goodsInsert?.optString("name").should.be.equal(goodsUpdateName)
            }
            delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader)
        }
        on("get changed since when goods deleted") {
            val goodsUpdateName: String = RandomStringUtils.randomAlphabetic(8)
            val date = Date.from(Instant.now()).time - 1
            val goodsAdded = post(baseUrl + "goods/${user.storeId}", authHeader,
                    json = dataToMap(goods.copy(name = goodsUpdateName)))
            it("should return the status code OK") {
                goodsAdded.statusCode.should.be.equal(OK)
            }
            val goodId = goodsAdded.jsonObject?.optLong("id")
            val responseDelete = delete(baseUrl + "goods/${user.storeId}/$goodId", authHeader)
            it("should return the status code NO_CONTENT") {
                responseDelete.statusCode.should.be.equal(NO_CONTENT)
            }
            val goodsUpdates = get(baseUrl + "updates/goods/since/" + date, authHeader.plus("AppUID" touser.AppUID))
            it("should contain goods deletes") {
                goodsUpdates.statusCode.should.be.equal(OK)
                goodsUpdates.jsonObject.getJSONArray("removed").toList()
                        .map { it asInt }
                        .find {
                            it == goodId.toInt()
                        }
                        .should.be.not.`null`
            }
        }
        on("get changed since when goods added without authorization") {
            val response = post(baseUrl + "goods/${user.storeId}",
                    json = dataToMap(goods))
            it("should contain an Unauthorized response status and an empty body") {
                response.statusCode.should.be.equal(UNAUTHORIZED)
                response.text.should.be.equal("")
            }
        }
        on("adding goods") {
            val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count)))
            val resultOfAdding = getResult(respAdd)
            it("should return the status code OK") {
                respAdd.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count") {
                resultOfAdding.should.be.size.equal(count)
            }
            post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding)))
        }
        on("updating goods") {
            val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count)))
            val resultOfAdding = getResult(respAdd)
            it("should return the status code respAdd OK") {
                respAdd.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count (resultOfAdding)") {
                resultOfAdding.should.be.size.equal(count)
            }
            val respUpdate = post(urlGoodsUpdate,
                    authHeaderWithAppUID,
                    json = getBody(updated = resultOfAdding.map { it.copy(countryId = REGION_77) })
            )
            it("should return the status code respUpdate OK") {
                respUpdate.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count (respUpdate)") {
                getResult(respUpdate).should.be.size.equal(count)
            }
            it("should be all elements are 77") {
                getCountryIdFromTheResult(respUpdate).should.be.all.elements(REGION_77)
            }
            post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(removed = getGoodsIds(resultOfAdding)))
        }
        on("deleting goods") {
            val respAdd = post(urlGoodsUpdate, authHeaderWithAppUID, json = getBody(added = arrayAdded(count)))
            val resultOfAdding = getResult(respAdd)
            it("should return the status code respAdd OK") {
                respAdd.statusCode.should.be.equal(OK)
            }
            it("should be equal to the size of the variable count") {
                resultOfAdding.should.be.size.equal(count)
            }
            val respRemoved = post(urlGoodsUpdate,
                    authHeaderWithAppUID,
                    json = getBody(removed = getGoodsIds(resultOfAdding))
            )
            it("should return the status code respRemoved OK") {
                respRemoved.statusCode.should.be.equal(OK)
            }
            it("should be empty") {
                getResult(respRemoved).should.be.empty
            }
        }
    }
})

Total

The code itself, language proficiency and knowledge of the framework are far from perfect, but the beginning is generally good.

When meeting Kotlin, there was a feeling that he was syntactic sugar in Java. And while writing the code with all the fibers of the soul, I managed to feel the words: “fully compatible with Java”.

Spek, where simple language constructs are used to describe specifications, provides a complete pool of test methods. Those. gives what they want from it as from a test framework.

Total - all tests in master. Everything turned out, and the service department now knows for sure that it will be able to support colleagues from “K” in difficult times.