Setting up a home environment for development (docker + gitlab + DNS)

Published on July 14, 2018

Setting up a home environment for development (docker + gitlab + DNS)

  • Tutorial

Intro


I could not think of a suitable name for the post, so I will briefly describe what will be discussed.


Most of us have some small personal crafts that do not go beyond our homes. Someone is hosting them on a work computer, someone is on Heroku, someone is on a VPS, and someone has a home server. Reddite even has a community r / homelab , in which people discuss different hardware and software for the so-called. home lab .


I'm not so keen on this issue, but at my home is the Intel NUC, which plays music from the NAS using MPD . In addition to MPD, my small crafts that help me work with it are spinning: now the dead telegraph bot, HTTP API on the synatra and a clumsy frontend for it.


In the post I will describe the process of installing a DNS server for working with domain names for services, the scheme of simultaneous operation of several services using Docker and installing Gitlab with CI without any particular details (many of which I don’t understand myself). You will not learn anything new, but suddenly this “guide” will be useful to someone. In addition, I would like to hear suggestions on how to make it simpler / more elegant / more correct.


Initially, the code of my services was on the bitback / githaba and after creating the docker images I had to log in under SSH and run a couple of scripts that created / updated containers with services. I caught myself thinking that I see a minor annoying bug in the application, which I do not fix just because I'm too lazy to perform the whole procedure. Obviously, it was time to automate everything. This is where the idea of ​​installing Gitlab + CI came.


Local domains using DNS


All containers were created with a flag --network=hostfor simplicity - it was enough to use different ports in the applications. However, as the number of services grows, remember which application uses which port. Yes, and entering each time the IP address with the port in the browser is not very nice, so before installing the glitb I decided to deal with hosting several applications on the same server.


The idea is simple: we configure DNS, feed it to the router, install Nginx and with its configuration we redirect requests to different ports depending on the domain. This will allow not to bother with the ports during development, as containers will start using --publishinstead --network=host.


When installing this guide was used . It is configured for Ubuntu 16.04, I have Debian.


Further actions are made from the user root.


First of all, install bind9and utilities:


apt-get install -y bind9 bind9utils bind9-doc dnsutils

Next, we need to configure the domain zone. To do this, add the /etc/bind/named.conf.localfollowing to the file :


zone "nondv.home" IN { // Желаемое доменное имя
     type master;
     file "/etc/bind/fwd.nondv.home.db"; // Forward lookup file
     allow-update { none; }; // Since this is the primary DNS, it should be none.
};

Also, a reverse lookup configuration is added to the guide, but, to be honest, I don’t really understand why this is needed, so I didn’t do that.


Now create the file /etc/bind/fwd.nondv.home.db:


$TTL    604800
@       IN      SOA     ns1.mydomain.home. root.mydomain.home. (
                             20         ; Serial
                         604800         ; Refresh
                          86400         ; Retry
                        2419200         ; Expire
                         604800 )       ; Negative Cache TTL
;Name Server Information
        IN       NS     ns1.nondv.home.
;IP address of Name Server
ns1     IN       A      192.168.0.3
;A - Record HostName To Ip Address
nuc     IN       A      192.168.0.3
gitlab  IN       A      192.168.0.3
mpd     IN       A      192.168.0.3
@       IN       A      192.168.0.3

Next, restart bind9 and set autorun:


systemctl restart bind9
systemctl enable bind9

Please note that I used .homeinstead .local. This was done because the domain was nondv.localnot resolved without subdomains. Well, more accurately digrecognized it normally, but browsers and curl- no. As a colleague explained to me, this is most likely due to a different software like Bonjour (my working laptop with an apple on the lid). In general, .homethere should be no such problems with the domain .


Actually, that's all. I then added the DNS as primary to the router and reconnected to it (so that the file was automatically updated /etc/resolve.conf).


Nginx


As I said, in order to be able to access all services at the same time via HTTP on port 80, we need to configure Nginx so that it proxies requests to different ports depending on the domain.


Documentation on the nginx image is available on the Docker Hub website .


Prepare the main configuration file /srv/nginx/nginx.conf:


user  nginx;
worker_processes  1;
error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;
events {
    worker_connections  1024;
}
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    access_log  /var/log/nginx/access.log  main;
    sendfile        on;
    keepalive_timeout  65;
    server {
      listen 80;
      server_name nondv.home;
      rewrite ^/$ http://mpd.nondv.home redirect; # основной домен, на который будут пеправляться ненастроенные запросы
    }
    include /etc/nginx/conf.d/*.conf;
}

Next, configure the domains. I will show only one:


# /srv/nginx/conf.d/gitlab.conf
server {
  listen       80;
  server_name  gitlab.nondv.home;
  location / {
    proxy_pass http://127.0.0.1:3080;
  }
}

The container is started by the command:


docker run --detach \
           --network host \
           --name nginx \
           --restart always \
           --volume /srv/nginx/nginx.conf:/etc/nginx/nginx.conf:ro \
           --volume /srv/nginx/conf.d:/etc/nginx/conf.d:ro \
           nginx:alpine

That's all, now HTTP requests for port 80 will be caught by nginx and redirected to the correct port.


Gitlab


Here, everything is simple according to the official manual :


docker run --detach \
           --hostname gitlab.nondv.home \
           --publish 3080:80 --publish 3022:22 \
           --name gitlab \
           --restart always \
           --volume /srv/gitlab/config:/etc/gitlab:Z \
           --volume /srv/gitlab/logs:/var/log/gitlab:Z \
           --volume /srv/gitlab/data:/var/opt/gitlab:Z \
           gitlab/gitlab-ce:latest

We are waiting for when everything is set up (we glance in docker logs -f gitlab) and then we enter the container ( docker exec -it gitlab bash) for add. settings:


nano /etc/gitlab/gitlab.rb # or vim
# /etc/gitlab/gitlab.rb
external_url 'http://gitlab.nondv.home'
gitlab_rails['gitlab_shell_ssh_port'] = 3022
# /etc/gitlab/gitlab.rb
gitlab-ctl reconfigure

For reliability, you can restart the container ( docker container restart gitlab).


CI


Gitlab CI is already integrated, but it needs Gitlab Runner ( documentation ).


For this, I wrote a small script:


NAME="gitlab-runner$1"
echo $NAME
docker run -d --name $NAME --restart always \
           --network=host \
           -v /srv/gitlab-runner/config:/etc/gitlab-runner \
           -v /var/run/docker.sock:/var/run/docker.sock \
           gitlab/gitlab-runner:alpine

After creating the runner, we need to register it. To do this, go to the guitar tab (through the browser), go to the Admin area → Overview → Runners. There is described the installation runners. In short, you just do:


docker exec -it gitlab-runner register

and answer the questions.


Your own HTTP services


They run like a gitlab. Publish them on any port and add the config to nginx.


Conclusion


Now you can host your projects on a home server and use the power of Gitlab CI to automate the assembly and publishing of your projects. It's convenient to do git pushand not worry about running, right?


I would also recommend setting up mail for gitlab. Personally, I used a mailbox on Yandex. Documentation .