Microservices development using Scala, Spray, MongoDB, Docker and Ansible

Original author: Viktor Farcic
  • Transfer
  • Tutorial
The purpose of this article is to show a possible approach for building microservices using Scala , RESTful JSON, Spray, and Akka . We will use MongoDB as the database . As a result of our work, we will pack our project in a Docker container, and Vagrant and Ansible will allow us to control the configuration of the application.

In this article you will not find details about the Scala language and other technologies that will be used in the project. In it you will not find a guide that will answer all your questions. The purpose of the article is to show a technique that can be used in the development of microservices. In fact, most of this article is not technology specific. Docker has a wider scope than just microservices. Ansible allows you to quickly deploy any required environment, and Vagrant is a great tool for creating virtual machines.

So, let's start creating the “Book Service” with the following methods:

  • Get all books
  • Get book info
  • Update existing book
  • Delete existing book

Environment


We will use Ubuntu as a server. The easiest way to create one is to use Vagrant. If you haven’t installed it yet, please download it and install it. You will also need Git to clone the source repository. The rest of the article will not require you to manually install additional packages.

Proceed to clone the repository

git clone https://github.com/vfarcic/books-service.git
cd books-service

Next, you need to create an Ubuntu server using Vagrant with the following settings:

# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "ubuntu/trusty64"
  config.vm.synced_folder ".", "/vagrant"
  config.vm.provision "shell", path: "bootstrap.sh"
  config.vm.provider "virtualbox" do |v|
    v.memory = 2048
  end
  config.vm.define :dev do |dev|
    dev.vm.provision :shell, inline: 'ansible-playbook /vagrant/ansible/dev.yml -c local'
  end
  config.vm.define :prod do |prod|
    prod.vm.provision :shell, inline: 'ansible-playbook /vagrant/ansible/prod.yml -c local'
  end
end


We defined box (OS) as Ubuntu, synced_folder means that everything inside the ./vagrant directory on the host machine will be accessible inside the virtual machine. The rest of the work on installing applications and preparing the environment will be assigned to Ansible, which will be installed using bootstrap.sh. There are two virtual machines in Vagrantfile: dev and prod. Each of them will use Ansible, so make sure it is installed correctly.

The classic way to work with Ansible is to split the configuration into roles. In our case, we have 4 roles located in ansible / roles. The first role includes the installation of Scala and SBT. Another installs Docker. The third installs a MongoDB container. The last role (books) will be used to deploy the application to the combat virtual machine.
As an example, declare a mongodb role as follows:

- name: Directory is present
  file:
    path=/data/db
    state=directory
  tags: [mongodb]
- name: Container is running
  docker:
    name=mongodb
    image=dockerfile/mongodb
    ports=27017:27017
    volumes=/data/db:/data/db
  tags: [mongodb]

This role ensures that the folder exists and the mongodb container is running. The ansible / dev.yml playbook binds these roles together:

- hosts: localhost
  remote_user: vagrant
  sudo: yes
  roles:
    - scala
    - docker
    - mongodb

Each time we launch a playbook, all tasks from the roles scala, docker and mongodb are performed.

One of Ansible's charm is that it only performs tasks when it is needed. If you run it a second time, it will verify that everything is in place and will do nothing. However, if you delete the / data / db folder, Ansible will notice the loss and create it again.

Time to run a virtual machine! The first launch will be a little long, since Vagrant needs to download the Ubuntu distribution package, install the packages and download the MongoDB Docker image. Subsequent launches will be noticeably faster.

vagrant up dev
vagrant ssh dev
ll /vagrant


The vagrant up command creates a new virtual machine or starts one of the existing ones. With vagrant ssh we go to a newly created machine. Finally, ll / vagrant shows a list of files and directories as proof that our local files are accessible inside the virtual machine.
It's all. Our development server with Scala, SBT and MongoDB is ready to go. Let's start the development of our service.

Book Service


I like Scala, it is a very powerful language, and akka is my favorite framework for building message-driven JVM applications. Despite the fact that Akka came from Scala, nothing prevents us from using it in Java.
Spray is a simple but powerful tool for building REST / HTTP services. It is asynchronous due to the use of Akka actors and has a wonderful DSL for describing HTTP routes.
In the TDD usage mode, we write tests before implementation. Here is an example of tests that checks the route by which a list of books is given.
"GET /api/v1/books" should {
  "return OK" in {
    Get("/api/v1/books") ~> route ~> check {
      response.status must equalTo(OK)
    }
  }
  "return all books" in {
    val expected = insertBooks(3).map { book =>
      BookReduced(book._id, book.title, book.author)
    }
    Get("/api/v1/books") ~> route ~> check {
      response.entity must not equalTo None
      val books = responseAs[List[BookReduced]]
      books must haveSize(expected.size)
      books must equalTo(expected)
    }
  }
}

This is a simple test, which I hope will show the direction in which you need to move to develop tests for APIs built on Spray. The first thing we check is the server, upon this request should return the code 200 (OK). The second thing we pay attention to is that after adding the book to the database, it returns correctly. You can look at the complete source code with tests in ServiceSpec.scala

How are these checks implemented? The code that will allow this is presented below:

val route = pathPrefix("api" / "v1" / "books") {
  get {
    complete(
      collection.find().toList.map(grater[BookReduced].asObject(_))
    )
  }
}


We defined the route / api / v1 / books, the GET method and the response inside the complete () expression. In our case, we got a list of all the books from the database and converted it to the BookReduced case class. You can find all source code, including all methods (GET, PUT, DELETE) in ServiceActor.scala.
Both tests and implementation are given as an example, in practice they are usually more complicated. But Spray does it really cool.

During development, you can run tests in quick mode.

#[Внутри виртуальной машины]
cd /vagrant
sbt ~test-quick

Each time you change the source code, all affected tests will be run again. Usually I have a terminal window open with test results, this gives me the opportunity to receive instant feedback on the quality of the code I'm working on.

Testing, Assembly, and Deployment


Our application, like any other, needs to be tested, built and deployed.
Let's create a docker container with a service. You can specify the necessary settings in the Dockerfile.

#[Внутри виртуальной машины]
cd /vagrant
sbt assembly
sudo docker build -t vfarcic/books-service .
sudo docker push vfarcic/books-service

We compiled the JAR (passing the tests is part of the build phase), assembled the Docker container, and sent it to the Docker Hub. If you plan to repeat these steps again, please create an account at hub.docker.com and change “vfarcic” to your username.
The container that we created contains everything you need to launch our service. It is based on Ubuntu, contains JDK7 with MongoDB, and a compiled JAR file. This container can be run on any machine with Docker installed. It does not require the installation of additional dependencies on the server, the container is self-sufficient and can be run anywhere.
Let's deploy (run) the container that we created on another virtual machine. This is very similar to deploying an application in a production environment.
To create a production virtual machine, with our service, you must run the following commands:

#[из директории с исходным кодом]
vagrant halt dev
vagrant up prod

The first command stops the dev virtual machine. Each machine requires 2GB of RAM, and if you have enough free RAM, you can skip this step. The second command starts a production machine with a deployed service.
After a while, the virtual machine will be created, Ansible will be installed and run playbook prod.yml. It will install Docker and run vfarcic / books-service, compiled in the previous step and sent to the Docker Hub. During operation, it will use port 8080 and share the / data / db folder with the host system.
Let's try what we got. First, let's try to send a PUT request to add test data:

curl -H 'Content-Type: application/json' -X PUT -d '{"_id": 1, "title": "My First Book", "author": "John Doe", "description": "Not a very good book"}' http://localhost:8080/api/v1/books
curl -H 'Content-Type: application/json' -X PUT -d '{"_id": 2, "title": "My Second Book", "author": "John Doe", "description": "Not a bad as the first book"}' http://localhost:8080/api/v1/books
curl -H 'Content-Type: application/json' -X PUT -d '{"_id": 3, "title": "My Third Book", "author": "John Doe", "description": "Failed writers club"}' http://localhost:8080/api/v1/books

Let's check that the service returned the correct data to us:
curl -H 'Content-Type: application/json' http://localhost:8080/api/v1/books

We can delete the book:
curl -H 'Content-Type: application/json' -X DELETE http://localhost:8080/api/v1/books/_id/3

Verify that the deleted book no longer exists:
curl -H 'Content-Type: application/json' http://localhost:8080/api/v1/books

In conclusion, let's try to extract a specific copy of the book:
curl -H 'Content-Type: application/json' http://localhost:8080/api/v1/books/_id/1

We tried a quick way to develop, build and deploy a microservice. Docker simplifies deployment and does not require additional dependencies. Every service that requires JDK and MongoDB does not require installed applications on the destination machine. This is all part of a container that runs as a docker process.

Summary


The idea of ​​microservices has existed for a long time, and until recently did not receive due attention, due to application compatibility problems necessary for the simultaneous operation of hundreds and thousands of different service instances. The advantages that arose due to the use of microservices (separation, reduced development time, scalability, etc.) were not so significant compared to the problems that they brought while providing the necessary environment and deployment. Docker and tools such as Ansible help significantly reduce effort. With the departure from this problem, the most heterogeneous microservices become fashionable due to the advantages they bring.

Spray is a great choice for microservice. Docker containers contain everything you need for the application to work, and nothing more. Using large web servers such as JBoss and WebSphere may not be justifiable for a small service. Even small servers like Tomcat are usually not needed. Play! - This is a great framework for building a RESTFul API, but it contains many things that we don’t use. Spray, on the other hand, does just one thing - it provides asynchronous routing for the RESTFul API, and it does it great.

We can continue to expand the functionality of the service. For example, we can add a registration and authentication module.
However, this brings us one step closer to a monolithic application. In the world of microservices, new services should be new applications, and in the case of Docker, separate containers, each of which listens to its port and responds to requests addressed to it.

When building microservices, you need to try to create them in such a way that they do only one, or a small amount of work. Complexity is solved by combining them together, and not by building a large monolithic application.

Also popular now: