Consumer Driven Contracts or Gitlab CI-eyed QA test automation
The objectives of this publication are :
- A Brief Introduction to Consumer Driven Contracts (CDC)
- Configure CDI-based CI pipeline
Consumer Driven Contracts
In this part, we will go over the main points of the CDC. This article is not exhaustive on the subject of contract testing. There is a sufficient amount of materials on this subject on the same Habré .
To continue, we need to get acquainted with the main provisions of the CDC:
- Contact testing is at the Service / Integration Tests level above Unit Tests according to the Mike Cohn pyramid.
- Contract testing can be applied when there are 2 (or more) services that interact with each other.
- Consumer driven approach means that the first step in implementation is to write a test on the consumer side. The test result is a pact (contract) in json format that describes the interaction between the consumer (for example, web interface / mobile interface: a service that wants to receive some data) and the provider (for example, server API: a service that provides data)
- The next step is to check the contract with the provider. This is fully implemented by the Pact framework .
So, let's start with the consumer side test . I used Pactman . This is what the test looks like:
import pytest
from pactman import Like
from model.client import Client
@pytest.fixture()
def consumer(pact):
return Client(pact.uri)
def test_app(pact, consumer):
expected = '123456789'
(pact
.given('provider in some state')
.upon_receiving("request to get user's phone number")
.with_request(
method='GET',
path=f'/phone/john',
)
.will_respond_with(200, body=Like(expected))
.given('provider in some state')
.upon_receiving("request to get non-existent user's phone number")
.with_request(
method='GET',
path=f'/phone/micky'
)
.will_respond_with(404)
)
with pact:
consumer.get_users_phone(user='john', host=pact.uri)
consumer.get_users_phone(user='micky', host=pact.uri)Using Pact DSL, we describe request / response interactions. After starting the test, we get a new file ({consumer} - {provider} -pact.json):
{
"consumer": {
"name": 'basic_client'
},
"provider": {
"name": 'basic_flask_app'
},
"interactions": [
{
"providerStates": [
{
"name": "provider in some state",
"params": {}
}
],
"description": "request to get user's phone number",
"request": {
"method": "GET",
"path": "/phone/john"
},
"response": {
"status": 200,
"body": "123456789",
"matchingRules": {
"body": {
"$": {
"matchers": [
{
"match": "type"
}
]
}
}
}
}
},
{
"providerStates": [
{
"name": "provider in some state",
"params": {}
}
],
"description": "request to get non-existent user's phone number",
"request": {
"method": "GET",
"path": "/phone/micky"
},
"response": {
"status": 404
}
}
],
"metadata": {
"pactSpecification": {
"version": "3.0.0"
}
}
}Next, we need to pass the pact to the provider for verification. This is done using Pact Broker.
Pact Broker is a contract repository with some additional features that allow us to track compatibility of service versions, as well as generate network diagrams (service interaction).
Pact broker
Pact
Version Matrix
Provider Verification
This part of the test is completely performed by the framework. After verification, the results are sent back to Pact Broker.
provider-verifier_1 | Verifying a pact between basic_client and basic_flask_app
provider-verifier_1 | Given provider in some state
provider-verifier_1 | request to get user's phone number
provider-verifier_1 | with GET /phone/john
provider-verifier_1 | returns a response which
provider-verifier_1 | WARN: Skipping set up for provider state 'provider in some state' for consumer 'basic_client' as there is no --provider-states-setup-url specified.
provider-verifier_1 | has status code 200
provider-verifier_1 | has a matching body
provider-verifier_1 | Given provider in some state
provider-verifier_1 | request to get non-existent user's phone number
provider-verifier_1 | with GET /phone/micky
provider-verifier_1 | returns a response which
provider-verifier_1 | WARN: Skipping set up for provider state 'provider in some state' for consumer 'basic_client' as there is no --provider-states-setup-url specified.
provider-verifier_1 | has status code 404
provider-verifier_1 |
provider-verifier_1 | 2 interactions, 0 failuresRunning both parts of the test in the pipeline
Now that both parts of the contract testing have been disassembled, it would be nice to run them on every commit. This is where Gitlab CI comes to the rescue. Pipeline jobs are described in .gitlab-ci.yml. Before we move on to the pipeline, we need to say a few words about the GitLab Runner, which is an open-source project, and is used to run jobs and send the results back to GitLab. Jobs can be run locally or using docker containers. In our project we use Docker. The test infrastructure is implemented in containers and is described in docker-compose.yml, located at the root of the project.
version: '2'
services:
basic-flask-app:
image: registry.gitlab.com/tknino69/basic_flask_app:latest
ports:
- 5005:5005
postgres:
image: postgres
ports:
- 5432:5432
env_file:
- test-setup.env
volumes:
- db-data:/var/lib/postgresql/data/pgdata
pactbroker:
image: dius/pact-broker
links:
- postgres
ports:
- 80:80
env_file:
- test-setup.env
provider-states:
image: registry.gitlab.com/tknino69/cdc/provider-states:latest
build: provider-states
ports:
- 5000:5000
consumer-test:
image: registry.gitlab.com/tknino69/cdc/consumer-test:latest
command: ["sh", "-c", "find -name '*.pyc' -delete && pytest $${TEST}"]
links:
- pactbroker
environment:
- CONSUMER_VERSION=$CI_COMMIT_SHA
provider-verifier:
image: registry.gitlab.com/tknino69/cdc/provider-verifier:latest
build: provider-verifier
ports:
- 5001:5000
links:
- pactbroker
depends_on:
- consumer-test
- provider-states
command: ['sh', '-c', 'find -name "*.pyc" -delete
&& CONSUMER_VERSION=`curl --header "PRIVATE-TOKEN:$${API_TOKEN}"
https://gitlab.com/api/v4/projects/$${BASIC_CLIENT}/repository/commits | jq ".[0] .id" | sed -e "s/\x22//g"`
&& echo $${CONSUMER_VERSION}
&& pact-provider-verifier $${PACT_BROKER}/pacts/provider/$${PROVIDER}/consumer/$${CONSUMER}/version/$${CONSUMER_VERSION}
--provider-base-url=$${BASE_URL}
--pact-broker-base-url=$${PACT_BROKER}
--provider=$${PROVIDER}
--consumer-version-tag=$${CONSUMER_VERSION}
--provider-app-version=$${PROVIDER_VERSION} -v
--publish-verification-results=PUBLISH_VERIFICATION_RESULTS']
environment:
- PROVIDER_VERSION=$CI_COMMIT_SHA
- API_TOKEN=$API_TOKEN
env_file:
- test-setup.env
volumes:
db-data:So, we have services that run in containers as needed.
Service Provider:
basic-flask-app:
image: registry.gitlab.com/tknino69/basic_flask_app:latest
ports:
- 5005:5005Pact Broker and its database. Volumes allow us to have a permanent repository for pacts and results of provider verification:
postgres:
image: postgres
ports:
- 5432:5432
env_file:
- test-setup.env
volumes:
- db-data:/var/lib/postgresql/data/pgdata
pactbroker:
image: dius/pact-broker
links:
- postgres
ports:
- 80:80
env_file:
- test-setup.envService Provider States. In practice, it should bring the provider into a certain state (for example, get the user in the database). However, in our example, it simply performs a dummy function.
provider-states:
image: registry.gitlab.com/tknino69/cdc/provider-states:latest
build: provider-states
ports:
- 5000:5000A service that runs the Consumer Test. Pay attention to the command that runs in the containerfind -name '* .pyc' -delete && pytest $$ {TEST}
consumer-test:
image: registry.gitlab.com/tknino69/cdc/consumer-test:latest
command: ["sh", "-c", "find -name '*.pyc' -delete && pytest $${TEST}"]
links:
- pactbroker
environment:
- CONSUMER_VERSION=$CI_COMMIT_SHAService Provider Verifier:
provider-verifier:
image: registry.gitlab.com/tknino69/cdc/provider-verifier:latest
build: provider-verifier
ports:
- 5001:5000
links:
- pactbroker
depends_on:
- consumer-test
- provider-states
command: ['sh', '-c', 'find -name "*.pyc" -delete
&& CONSUMER_VERSION=`curl --header "PRIVATE-TOKEN:$${API_TOKEN}"
https://gitlab.com/api/v4/projects/$${BASIC_CLIENT}/repository/commits | jq ".[0] .id" | sed -e "s/\x22//g"`
&& echo $${CONSUMER_VERSION}
&& pact-provider-verifier $${PACT_BROKER}/pacts/provider/$${PROVIDER}/consumer/$${CONSUMER}/version/$${CONSUMER_VERSION}
--provider-base-url=$${BASE_URL}
--pact-broker-base-url=$${PACT_BROKER}
--provider=$${PROVIDER}
--consumer-version-tag=$${CONSUMER_VERSION}
--provider-app-version=$${PROVIDER_VERSION} -v
--publish-verification-results=PUBLISH_VERIFICATION_RESULTS']
environment:
- PROVIDER_VERSION=$CI_COMMIT_SHA
- API_TOKEN=$API_TOKEN
env_file:
- test-setup.envThe Consumer Pipeline.gitlab-ci.yml at the root of the consumer project describes the processes that are performed on the consumer side:
image: gitlab/dind:latest
variables:
TEST: 'tests/docker-compose.app.yml'
CONSUMER_VERSION: $CI_COMMIT_SHA
BASIC_APP: '11993024'
services:
- gitlab/gitlab-runner:latest
before_script:
- docker login -u $GIT_USER -p $GIT_PASS registry.gitlab.com
stages:
- clone_test
- get_broker_up
- test
- verify_provider
- clean_up
clone test:
tags:
- cdc
stage: clone_test
script:
- git clone https://$GIT_USER:$GIT_PASS@gitlab.com/tknino69/cdc.git && ls -ali
artifacts:
paths:
- cdc/
broker:
tags:
- cdc
stage: get_broker_up
script:
- cd cdc && docker-compose -f docker-compose.yml up -d pactbroker
dependencies:
- clone test
test:
tags:
- cdc
stage: test
script:
- cd cdc && CONSUMER_VERSION=$CONSUMER_VERSION docker-compose -f docker-compose.yml -f $TEST up consumer-test
dependencies:
- clone test
provider verification:
tags:
- cdc
stage: verify_provider
script:
- curl -X POST -F token=$CI_JOB_TOKEN -F ref=master https://gitlab.com/api/v4/projects/$BASIC_APP/trigger/pipeline
when: on_success
clean up:
tags:
- cdc
stage: clean_up
script:
- cd cdc && docker-compose stop consumer-test
dependencies:
- clone testHere the following happens:
As before_scriptwe login into our roster gitlab, using variables $ GIT_USER and $ GIT_PASS, we have set in "Settings»> «CI / CD»
- Next, we clone a test project
- In the next step, we raise Pact Broker
- Then the Consumer Test starts.
- After that, use the Gitlab API to start the provider verification.
- And finally, we clean ourselves
Provider Pipeline
The provider pipeline configuration is stored in .gitlab-ci.ymlthe root of the provider project.
image: gitlab/dind:latest
variables:
TEST: 'tests/docker-compose.app.yml'
PROVIDER_VERSION: $CI_COMMIT_SHA
services:
- gitlab/gitlab-runner:latest
stages:
- clone_test
- provider_verification
- clean_up
clone test:
tags:
- cdc
stage: clone_test
script:
- git clone https://$GIT_USER:$GIT_PASS@gitlab.com/tknino69/cdc.git
artifacts:
paths:
- cdc/
verify provider:
tags:
- cdc
stage: provider_verification
before_script:
- cd cdc
- docker login -u $GIT_USER -p $GIT_PASS registry.gitlab.com && docker-compose -f docker-compose.yml up -d basic-flask-app
script:
- PROVIDER_VERSION=$PROVIDER_VERSION docker-compose -f docker-compose.yml -f $TEST up provider-verifier
dependencies:
- clone test
.clean up:
tags:
- cdc
stage: clean_up
script:
- cd cdc && docker-compose down --rmi localAs in the Consumer Pipeline, we have several jobs:
- Clone a test project
- Verify provider
- We clean ourselves
Summarize :
- Wrote a contract test in Python
- Set up a test environment in Docker containers
- Configured CI based on contract tests, i.e. commit to the consumer project will launch the CI pipeline ( on the consumer side : cloning the test environment -> starting Pact Broker -> testing the consumer -> starting provider verification -> clean up; on the provider side : cloning the test environment -> provider verification -> clean up )
Commit to the provider project initiates provider verification to ensure that the provider complies with the pact
Thanks for attention.