Another home media server based on Docker

Good afternoon, Habr. To write this article, I was prompted by the closure of the rather well-known ex.ua and fs.to resources in UA-IX. Since the valiant law enforcement officers did not bother to provide any worthy alternative, it was decided to take the process into their own hands, besides, I had long wanted to organize a home storage / file storage independent of external providers. In the process of implementation, quite a lot of different systems were tried, but in the end everything was built just like that, and not otherwise. The opinion is subjective, the implementation is affordable. The article itself is designed for beginners who just need a working solution, or for those who are thinking about their media server, but have not yet decided on the implementation.

So the decision is made, let's begin.

Iron

The selection criteria were quite simple: compact, normal operation with WiFi, silence. After studying the iron market, the choice of which now gives the widest opportunities, several candidates remained. Namely, the HP MicroServer G8, ASUS VivoMini VC65R, and Rapsberry Pi 3 itself.

After studying the available software for creating a media server, it was decided to stop at the ASUS VivoMini VC65R, since Rapsberry, which is based on ARM, does not support transcoding, but the available options MicroServer was equipped with a fairly old Celeron G1610T, which might not provide enough performance. So, VC65R and 4x Samsung ST1500LM006 1.5TB were acquired.

operating system

On the one hand, there are a rather large number of specialized distributions on the market, such as FreeNAS, NAS4Free, OpenMediaVault, Rockstor. On the other hand, I would like to fully control the system, and, if possible, not use unnecessary services. So the choice was limited to Ubuntu Server 16.04 LTS. In general, the resulting configuration thanks to Docker does not depend on the operating system, but some services are somehow tied to the distribution, so if you want to repeat this configuration, keep this in mind.

To organize the files, the following directory structure was selected:

/ data - a separate logical volume for storing media data and backups
/ data / Media - media data
/ data / TimeMachine - backups TimeMachine
/ opt / mediacenter - a separate logical volume for storing service data

The installation process itself is not worth describing, since it depends on the distribution you choose, the only caveat is that since it would be nice to work with files without worrying about access permissions, we will give our user access using setfacl

apt-get install acl
setfacl -Rm u::rwx /data
setfacl -Rm d:u::rwx /data
setfacl -Rm u::rwx /opt/mediacenter
setfacl -Rm d:u::rwx /opt/mediacenter

Software

So, what requirements can we set for such a media server, given the fact that there is a wide range of equipment for Mac OS X, iOS, Windows, and Android:

- Storage for TimeMachine + share for Mac OS X
- Share for Windows
- Torrent client
- Monitoring of updated torrents
- Personal cloud storage
- Media server

Let's start in order.

Storage for TimeMachine + share for Mac OS X

Installation is very trivial, as well as everything described below.

apt-get install netatalk avahi-daemon

With this command, we install open source implementations of the AppleTalk protocol and Bonjour. Now all that remains is to export the / data / Media directory as an available shared resource, and the / data / TimeMachine directory as available for TimeMachine backup.

To do this, edit the file /etc/netatalk/AppleVolumes.default adding at the end

/data/TimeMachine "TimeMachine" cnidscheme:dbd options:tm,upriv,usedots volsizelimit:300000 allow:
/data/Media "Media" cnidscheme:dbd options:upriv,usedots dperm:0776 fperm:0666 allow:

An option has been added to limit the space available for TimeMachine.

volsizelimit:300000

An option has been added to restrict access for a specific user.

allow:

Share for Windows

Since there is Windows on my network, I also need Samba. Everything is super simple here, there are a lot of guides for using Samba on the network. However, to complete the article:

apt-get install samba smbclient

And the contents of smb.conf

Torrent Client and Monitoring Updating Torrents and Personal Cloud Storage and Media Server

It was decided to combine these requirements into one part, since all this is implemented on the basis of docker compose, and in fact they are one file thanks to which everything can be started with one command. I’ll probably omit the installation of docker and docker-compose, since they are not complicated and are described quite well on the docker website.

Let's start with the selected software:

As a torrent client, we will use a relatively lightweight Transmission.

To monitor updated torrents, two solutions specializing in the Russian-language segment of the network were selected for testing, namely TorrentMonitor and Monitorrent. Now they both work on my media server, and as a result, the one that will show itself in the most optimal way will remain. By the way, the authors of both solutions are present on Habré.

We will use our ownCloud as a personal dropbox. I'm not sure that this is exactly the solution that will suit me one hundred percent, but at this stage it suits me. Alternatively, you can use NextCloud, Pydio, or Seafile. In fact, the choice is much wider and there are quite a few worthy alternatives on the market.

We will use Plex as a media server. An alternative called Emby was also tested, but I didn’t like it.

For all of the above, ready-made docker images are already present, the only thing I did not like was the image of TorrentMonitor, so it was decided to write my own.


The repository itself is here

UPD: My pull request to nawa / torrentmonitor was stink, so the article was updated.

So, the images are ready, let's write docker-compose.yml

UPD2: For those who do not have DLNA, you can comment out the ports section and uncomment network_mode: host





Now you need to create a directory structure for storing metadata and media data:

mkdir -p /opt/mediacenter/{mariadb-data,monitorrent-db,owncloud-apps,owncloud-config,plex-config,plex-transcode,torrentmonitor-db,torrentmonitor-torrents,transmission-config}
mkdir -p /data/Media/{Anime,Books,Documents,Downloads,Games,Home\ Videos,Movies,Music,ownCloud,Photos,Torrents,TV\ Shows}

And for complete convenience, create a systemd service


Now do:

systemctl daemon-reload
systemctl enable mediacenter
systemctl start mediacenter

Docker compose will download images from the docker hub and launch our services. Now they will be available at:

Plex : http: //: 32400
Transmission : http: //: 9091
TorrentMonitor : http: //: 8080
Monitorrent : http: //: 6687
ownCloud : http: //: 8081

Well, it seems everything has started and works. Now you can start testing, for example, save the .torrent file to the Torrents directory. Transmission should start the download, and soon the downloaded files will appear in the Downloads / complete directory.

Now the last step remains, to facilitate the work of the media server in recognizing downloaded movies / TV shows / music. To do this, we use an excellent product called FileBot which can rename files based on information from open sources like TheTVDB.

Installation is simple:

apt-get install openjdk-8-jdk mediainfo openjfx
wget -O filebot_4.7.5_amd64.deb http://downloads.sourceforge.net/project/filebot/filebot/FileBot_4.7.5/filebot_4.7.5_amd64.deb?r=http%3A%2F%2Fwww.filebot.net%2F&ts=1482609251&use_mirror=heanet
dpkg -i filebot_4.7.5_amd64.deb

Well, for the convenience of renaming, we write a couple of aliases:

~ / .bash_aliases

alias anime-test = 'filebot --action test --conflict skip --format "{plex}" --db AniDB -r -non-strict --output / data / Media -rename'
alias series-test = 'filebot --action test --conflict skip --format "{plex}" --db TheTVDB -r -non-strict --output / data / Media -rename '
alias movie-test =' filebot --action test --conflict skip --format "{plex}" --db TheMovieDB -r -non-strict --output / data / Media -rename '
alias music-test =' filebot --action test --conflict skip --format "{plex } "--db AcoustID -r -non-strict --output / data / Media -rename '
alias anime-hardlink =' filebot --action hardlink --conflict skip --format" {plex} "--db AniDB - r -non-strict --output / data / Media -rename '
alias series-hardlink = 'filebot --action hardlink --conflict skip --format "{plex}" --db TheTVDB -r -non-strict --output / data / Media -rename'
alias movie-hardlink = 'filebot --action hardlink --conflict skip --format "{plex}" --db TheMovieDB -r -non-strict --output / data / Media -rename '
alias anime-move =' filebot --action move --conflict skip --format "{plex}" --db AniDB -r -non-strict --output / data / Media -rename '
alias series-move =' filebot --action move --conflict skip --format "{plex } "--db TheTVDB -r -non-strict --output / data / Media -rename '
alias movie-move =' filebot --action move --conflict skip --format" {plex} "--db TheMovieDB - r -non-strict --output / data / Media -rename '
alias music-move = 'filebot --action move --conflict skip --format "{plex}" --db AcoustID -r -non-strict --output / data / Media -rename'

Now, for example, to work with the series, you can use the command:

series-test 

If the result of the work does not suit us, we can modify the search conditions using the --q and --filter options. If the result of work suits us, we execute the command:

series-move 

As a result, the files of our series will be in the directory / data / Media / TV Shows / Series Name. Well, if we work with ongaming or want to continue distribution, you can run the command:

series-hardlink 

As a result, the hardlinks appear in the / data / Media / TV Shows / Series Name directory.

In general, such a trivial way to quickly and painlessly make your own media server.