Another implementation of Data Processing

Imagine a tool with which you can transform any data into ... any data.

The most accessible example is Images. What if I want to get a picture of a certain size and put a watermark somewhere (also of a certain size) somewhere.

And here is one example of how it might look :



PS: This is a review article, if it is interesting, I will describe the approaches in more detail later.

Motivation

Recently, in November 2018, I had to change the size of the photo ...

Online services because of their "features" (advertising, strange interface, etc.) made this process as painful as possible.

Standard Windows 10 tools seemed to me not obvious. I had to download GIMP and from there I had to do everything I needed - to change the size of the photo.

In my defense:

1. Problem time ~ 1AM (one in the morning)
2. There was no working laptop at hand
3. Windows at home for game only. Now sometimes I write code with an apple keyboard.

And then another programmer decided to create another cost ... Tool.

Requirements

This Tool should be all crutches crutch:

1. Simple UI
2. Simple API
3. Convenient organization of project entities
4. Minimum buttons and settings
5. Ability to implement your ... "standards"
6. Open Source: Backend and Frontend

And most importantly - all who understand what they want should be able to use it.

Several possible sequences:

1. Picture, Picture smaller, Filter
2. Download the contents of the URL, leave only useful content, leave only text
3. Download the picture, determine that there is any object on it (template matching)

Implementation

Recently I read on Habré. Before microservices you need to grow ... and decided to follow this. There have already been a couple of attempts to make such a thing through “beautiful” (in my imagination) structures: each function (like template_match , md5 ) as a microservice optimized to solve its problem. As a result, everything will come to this, but not today :) In

total, there are:

1. Frontend without beautiful wallpaper
2. Backend API
3. Celery (in local development I use Redis, and in test and prod RabbitMQ)
4. Backend workers

I have to say: I am very mediocre familiar with Frontend, however, thanks to Stackoverflow and Google search.

More examples

All examples can be found here , they are combined into "projects".

One of them: extracting text from the HTML page



Attention : the site is in read-only mode, but the result will be returned to you. He will not be able to save your changes to Project and Pipeline.

Small explanation

A list of what data can transform: here

What this Project is not:

1. Data storage
2. Bitcoin miner

The project is very similar to:

1. Zapier
2. Yandex Nirvana
3. Parse.com (a couple of years bought by Facebook)