EazyPhoto: a cozy photo hosting for your server
In the recent times, when flickr did not yet offer terabytes for storing photos, and BitTorrent Sync just went to the Internet in its unsafe alpha version, I had a need: to share photos with friends and more. But as usually happens with programmers, the word “share” costs much more than putting photos in VKontaktik. Namely:
After such thoughts, the idea begins to arise in the head: “We need to create ...” - and you can no longer stop.
Google has found a lot of cloud services for uploading photos, intended mainly for photographers (and not only). The essence of all services comes down to the fact that starting from a certain amount you have to pay a fixed amount per month. If for six months the amount does not look large, then two years later it will be an unpleasant reminder. An analogy can be made with the payment of ru-domains at 600 rubles per year in nic.ru and with a constant reminder to yourself that these domains should be transferred to other registrars :).
On the other hand, if you already have a server, you should use it to the maximum. I found two projects that, in principle, could fit my tasks. These are Piwigo (former phpWebGallery) and Koken. The last project was generally very pleased. But getting to know them better, I realized that this also did not suit me.
Dropbox, Skydrive, Google+ (Picasa), Yandex.Fotki - these guys performed their functions perfectly as long as there was enough space.
I want to talk more about what the term “share” means to me. After some event (whether it's a trip to nature or walking around the city), in which several people take part, many photos appear. After processing, photos (300-700 MB) are sent to the local NAS (Synology DS210j). And this folder with the NAS should be posted on the Internet so that it can be downloaded by the very friends who are on the pictures.
Uploading / downloading through a browser such a volume is not for me.
But in the end, I want to have a link to the album, which you can throw off to someone else. Well, a list of albums would also be nice to get. Simple ordinary process.
Most importantly, there should be original photos and a normal resize without filters (not what VK and others are now offering). Then, when everyone has a 4K TV (as FullHD is now), it will be very cool to look through your photo archive and be glad again.
After I read the news about the release of the first version of BTSync for all platforms - I realized everything. The algorithm of work is clearly lined up in my head:
BTSync on Linux still does not have a console API. But there is a web interface that can act as an API server. Armed with Go, I wrote a console utility for working with BTSync. Well, then I made a class for PHP. And then it’s small: write everything else;)
By my main specialty I am a php programmer. Layout is the worst thing for me that could be :). Therefore, taking Foundation, I tried to portray something on it.
On the main page we see a list of albums with names and dates. And on the album page - square photos with a preview, as in dropbox. If you add a description to each photo and set its serial number, you get something like a blog. The size of large photos for the web-based interface I chose directly as a marketer - FullHD :)
Management of albums and photos takes place inside the admin panel. This is the enemy’s way of adding a new album: just fill in the name, start date and ReadOnly Secret.
First I wrote an indexer in PHP: it ran through new albums, added them to the queue for uploading to BTSync, and then ran through existing ones and added new found files. It is enough to hang the script in cron once every 5 minutes: that would be enough. But not to me.
It would be cool if the pictures were added to the database as they were downloaded. And then Go comes to the rescue.
A small daemon that hangs and checks directories for new files. Creates a preview for photos and saves to the database. Communicates with BTSync. Here are his main tasks.
To implement these tasks, I used the following packages:
Having checked all the necessary functions in the demo project, I began to collect all this into a working version. It took about two days to get the finished result .
The service copes with its tasks: the storage is all the time on the network, BTSync is installed on it. After the photos have been sent to the storage, the folder has been added to BTSync and an album has been created in the web interface - you can go to bed. The server with eazyphoto will download the photos and add them to the current album. In the morning you can go in, check, and send the link to friends. And friends, if necessary, download the originals to themselves through BitTorrent Sync.
For those who decided to try to raise it all at home - here are the links:
All instructions(or their similarity) are inside projects (open source such open source :).
I would like not to scare potential users with the admin panel, but make them a normal web interface for management and a simple script / package for installation. But all this is relevant only with eazyphotod, because the cron and php process for geeks is clearly not an option;)
If someone has thoughts on functionality or how it should all look (or suddenly someone wants to help;) - Wellcome.
Finally: lyrical digression. Take pictures, share with friends and parents. Life passes quickly, but the photo archive remains (preferably in RAID1 and an additional copy in case of the apocalypse:) ...
- Put photos somewhere on your server with a beautiful and simple web-interface.
- Simply and massively upload and download original images.
- Have the ability to restrict access to certain albums.
- Try to associate this with local network storage so as not to duplicate photos.
- If possible, delete the original photos from the server after some time and leave only the resized photos.
- To the maximum for free ! :)
After such thoughts, the idea begins to arise in the head: “We need to create ...” - and you can no longer stop.
What do photo hosting companies offer now?
Google has found a lot of cloud services for uploading photos, intended mainly for photographers (and not only). The essence of all services comes down to the fact that starting from a certain amount you have to pay a fixed amount per month. If for six months the amount does not look large, then two years later it will be an unpleasant reminder. An analogy can be made with the payment of ru-domains at 600 rubles per year in nic.ru and with a constant reminder to yourself that these domains should be transferred to other registrars :).
On the other hand, if you already have a server, you should use it to the maximum. I found two projects that, in principle, could fit my tasks. These are Piwigo (former phpWebGallery) and Koken. The last project was generally very pleased. But getting to know them better, I realized that this also did not suit me.
Dropbox, Skydrive, Google+ (Picasa), Yandex.Fotki - these guys performed their functions perfectly as long as there was enough space.
Share with friends
I want to talk more about what the term “share” means to me. After some event (whether it's a trip to nature or walking around the city), in which several people take part, many photos appear. After processing, photos (300-700 MB) are sent to the local NAS (Synology DS210j). And this folder with the NAS should be posted on the Internet so that it can be downloaded by the very friends who are on the pictures.
Uploading / downloading through a browser such a volume is not for me.
But in the end, I want to have a link to the album, which you can throw off to someone else. Well, a list of albums would also be nice to get. Simple ordinary process.
Most importantly, there should be original photos and a normal resize without filters (not what VK and others are now offering). Then, when everyone has a 4K TV (as FullHD is now), it will be very cool to look through your photo archive and be glad again.
Bittorrent sync
After I read the news about the release of the first version of BTSync for all platforms - I realized everything. The algorithm of work is clearly lined up in my head:
- The initial upload of photos occurs through BitTorrent Sync.
- The user does not matter where he shares the folder and receives a readonly secret key, while he can do this on the NAS or at his place.
- Next, the user goes to the web-based management interface and adds the album via readonly secret key.
- The web interface sends the BTSync command to download this folder on the server. Gradually, downloading to the server begins.
- At this time, in the kroon or where else there is a check whether everything was downloaded or not (a la file indexing). All files, for example, are added to the database, and a small copy is created for them (i.e. all meta-information gets from the original files to the database).
- After some time, we delete the originals from the server (but! They are saved by users who managed to download).
BTSync on Linux still does not have a console API. But there is a web interface that can act as an API server. Armed with Go, I wrote a console utility for working with BTSync. Well, then I made a class for PHP. And then it’s small: write everything else;)
Eazyphoto
By my main specialty I am a php programmer. Layout is the worst thing for me that could be :). Therefore, taking Foundation, I tried to portray something on it.
On the main page we see a list of albums with names and dates. And on the album page - square photos with a preview, as in dropbox. If you add a description to each photo and set its serial number, you get something like a blog. The size of large photos for the web-based interface I chose directly as a marketer - FullHD :)
Management of albums and photos takes place inside the admin panel. This is the enemy’s way of adding a new album: just fill in the name, start date and ReadOnly Secret.
What is it all written on?
The project is written on the internal engine Eaze . It is far from perfect, but it allows you to quickly solve certain problems in a known short period of time. It does not have normal documentation, but it is also written in PHP, so to understand something, just look inside (just like in the ideology of go-lang :).
Database: chose mysql because everyone has it. These two labels are mostly interesting here: albums and photos
Database: chose mysql because everyone has it. These two labels are mostly interesting here: albums and photos
File indexing
First I wrote an indexer in PHP: it ran through new albums, added them to the queue for uploading to BTSync, and then ran through existing ones and added new found files. It is enough to hang the script in cron once every 5 minutes: that would be enough. But not to me.
It would be cool if the pictures were added to the database as they were downloaded. And then Go comes to the rescue.
eazyphotod
A small daemon that hangs and checks directories for new files. Creates a preview for photos and saves to the database. Communicates with BTSync. Here are his main tasks.
To implement these tasks, I used the following packages:
- code.google.com/p/gcfg - work with configuration ini-files in the style of gitconfig. The output is a filled structure.
- github.com/disintegration/imaging - work with images (resize, crop, fit, etc ..). Without using gd, pure go.
- github.com/go-sql-driver/mysql - driver for mysql.
- github.com/howeyc/fsnotify is a cross-platform package for tracking changes in the file system.
- github.com/rwcarlsen/goexif/exif - a package for reading EXIF data.
- github.com/sergeyfast/btsync-cli is my package for working with the BTSync API.
Having checked all the necessary functions in the demo project, I began to collect all this into a working version. It took about two days to get the finished result .
How it works?
eazyphotod is launched through upstart (in the future I plan to redo it under supervisord). After starting, it loads all the albums into memory and starts scanning directories for new photos. In parallel with this, gourutin is launched to track changes in the file system and to process HTTP requests (update album meta-information and add a new album to the download queue).
All requests (resize, meta-information update, event of a new file) are put in one queue :
Therefore, there is no need to track simultaneous access to maps and other important variables. Everything goes in one common queue.
Working with the database is also quite simple .
Generally, if you thought to write on! Go, but did not dare to - my advice: take and write has something on it :)
And another thing: my go-projects on github ideology placed wrong, because only one src folder should be root in the repository. And I have the whole project. I’m somehow not comfortable when next to the source code in one folder are examples of configs, README.md and other supporting materials.
All requests (resize, meta-information update, event of a new file) are put in one queue :
var SyncQueue = make(chan *SyncItem)
type SyncItem struct {
Album *model.Album
FsPhotos model.PhotoList
FullSync bool
Filename string
}
Therefore, there is no need to track simultaneous access to maps and other important variables. Everything goes in one common queue.
Working with the database is also quite simple .
Generally, if you thought to write on! Go, but did not dare to - my advice: take and write has something on it :)
And another thing: my go-projects on github ideology placed wrong, because only one src folder should be root in the repository. And I have the whole project. I’m somehow not comfortable when next to the source code in one folder are examples of configs, README.md and other supporting materials.
Total
The service copes with its tasks: the storage is all the time on the network, BTSync is installed on it. After the photos have been sent to the storage, the folder has been added to BTSync and an album has been created in the web interface - you can go to bed. The server with eazyphoto will download the photos and add them to the current album. In the morning you can go in, check, and send the link to friends. And friends, if necessary, download the originals to themselves through BitTorrent Sync.
Try to collect
For those who decided to try to raise it all at home - here are the links:
- github.com/sergeyfast/eazyphoto-web - php part
- github.com/sergeyfast/eazyphotod - demon itself
For those who want to poke and see an example
I don’t knowingly publish a link to the demo. But Yandex will help you find according to the words on the black die.
All instructions
Installing auxiliary software using Ubuntu as an example
1. Install btsync:
Data port = 0. Port for the web interface: 8888. IP: 127.0.0.1
2. Install go. I
recommend via godeb blog.labix.org/2013/06/15/in-flight-deb-packages-of-go
Minimum go version: 1.1
3. Install eazyphotod
After that, we got a binary.
I recommend creating / usr / local / eazyphotod / (user www-data), copy eazyphotod + config.gcfg there.
Drop eazyphotod.conf into / etc / init / and run (after configuring config.gcfg):
(deb-way: put the config in /etc/eazyphotod.conf, the binary in / usr / bin / eazyphotod, and specify the link to the config via the -config parameter).
sudo add-apt-repository ppa:tuxpoldo/btsync
sudo apt-get update
sudo apt-get install btsync
Data port = 0. Port for the web interface: 8888. IP: 127.0.0.1
2. Install go. I
recommend via godeb blog.labix.org/2013/06/15/in-flight-deb-packages-of-go
Minimum go version: 1.1
3. Install eazyphotod
git clone git://github.com/sergeyfast/eazyphotod.git
cd eazyphotod
export GOPATH=`pwd`
go get code.google.com/p/gcfg github.com/disintegration/imaging github.com/go-sql-driver/mysql github.com/howeyc/fsnotify github.com/rwcarlsen/goexif/exif github.com/sergeyfast/btsync-cli/src/btsync
go build -o eazyphotod src/*.go
After that, we got a binary.
I recommend creating / usr / local / eazyphotod / (user www-data), copy eazyphotod + config.gcfg there.
Drop eazyphotod.conf into / etc / init / and run (after configuring config.gcfg):
service eazyphotod start(deb-way: put the config in /etc/eazyphotod.conf, the binary in / usr / bin / eazyphotod, and specify the link to the config via the -config parameter).
Further development
I would like not to scare potential users with the admin panel, but make them a normal web interface for management and a simple script / package for installation. But all this is relevant only with eazyphotod, because the cron and php process for geeks is clearly not an option;)
If someone has thoughts on functionality or how it should all look (or suddenly someone wants to help;) - Wellcome.
Finally: lyrical digression. Take pictures, share with friends and parents. Life passes quickly, but the photo archive remains (preferably in RAID1 and an additional copy in case of the apocalypse:) ...