Uploadcare - file storage for sites and applications

Hello! I want to talk about a project that is likely to be useful to many developers. In a nutshell, explaining why it is needed is quite difficult, but I'll try. Uploadcare is a service for applications and sites that simplifies the receipt of files from users, their storage and transmission over the network.

Anyone who has ever done a form with  knows that there is nothing complicated about it, but there are unpleasant moments that arise along the way. Here are just a few of them:

- you cannot save a form with ajax file;
- You can not show the form with the selected file;
- if you expect a picture, you need to make sure that the picture is loaded;
- the server should be ready to accept a large request body;
- in some frameworks, the downloaded file is a source of increased danger;
- Convenient download of several files is quite difficult;
- indication of the loading process is even more complicated;
- the server disk may run out of space;

Uploadcare aims to solve these problems. The principle of operation is very simple: you embed a widget in the form with which users select files. The widget passes you the identifier of the file selected by the user. In the future, you work only with this identifier: store it in the database, use it to build links to files.



This removes many of the problems listed above. The identifier can be saved by ajax. You can transfer it to the form again, and the widget will recognize and show the user the file. When a file is selected, the widget can tell the user that only pictures can be downloaded. Using the widget, the user can upload multiple files. Uploadcare does not suddenly run out of space, and its increase is not related to problems such as disk re-partitioning and server shutdown.

Files uploaded by users are not immediately tied to the account. They are visible through api, but there is no other way to access them. Only after the server confirms with a secret key and a special api method that the file is really needed, it will become public. This usually happens after the form is “validated” with the rest of the user data. Accordingly, you don’t have to worry that some user or even an attacker will be able to download what you do not need. To call api methods, you can use ready-made libraries that are available for most popular languages ​​and frameworks. However, if the application does not assume a server, there is a way to change this behavior: you can enable the setting in your account and widget, which allows you to save files without a secret key.

But accepting a file is not so bad. After that, they also need to be distributed. This requires a wide channel. Pictures need to be pinched in a different resolution, or even in several. It’s also good that a separate server does this, because the distribution of large files clogs the operating system caches.

That's why Uploadcare has a smart CDN . Like any normal CDN, it can simply give away files by their identifier. And he can apply different operations to them. For pictures, these are: resizing, cropping, converting to a different format, rotation, and others. You describe in url what picture you want to receive, and CDN gives it.

Of course, before Uploadcare there were ready-made solutions, for example, for resizing images. Or various libraries that allow you to save files on the "limitless" S3 . But setting them up is still tiring enough, and they did not solve all the problems. And few people came to the realization of such amenities as downloading files from social networks and cloud services, or to the ability for the user to cut out a suitable part of the image during file upload. With Uploadcare you get it automatically.



The project has as many as two api (except for the CDN), which you can use directly if you have specific requirements: file upload api , accessible by public key, and  rest apiavailable in secret. This allows you to use Uploadcare not only on the web, where downloading files is a common thing, but also on mobile devices and desktop applications. By the way, there are already ready-made libraries for iOS.

You can write a separate article about Uploadcare architecture (by the way, can this be done?). But in short, everything is in order with reliability. Servers run in the Amazon cloud and scale easily horizontally. Different nodes are responsible for different functions. Even if the base and the main site fall, CDN will continue to distribute files and resize pictures. CloudFlare is responsible for delivering content. User files are stored on S3, and it is possible to connect your own S3 storage for your account, which can be beneficial if you intend to store large files.



If the project interests you, but something does not suit you, do not hesitate to ask and suggest in the comments.