Support for cloud file storage in 1C-Bitrix: Site Management 11

    With this article, we open a whole series of posts devoted to new products implemented in the new, 11th version of “1C-Bitrix: Site Management” .

    One of the most interesting features that has appeared in the new release is cloud storage support.



    In this post we will tell you what it is, what kind of storage there are, how they work, and how they can be used with great benefit on your site.

    To get started, let's look at any new or existing web project?

    What is its owner striving for?With the exception of highly specialized resources - to the growth of the site, the expansion of its audience, the increase in the profit (money - in an explicit form, or “profit by attention”) that the project brings. A good and understandable goal for any online store, information resource, social network, etc.

    Such a goal by itself implies several smaller tasks that need to be learned to most effectively solve:
    • Minimization of operating costs and reduction of financial risks at the start of the project
    • Load Scaling and Back Scaling
    • Reliability - providing SLA, with which, perhaps, a different level of SLA for different categories of customers
    • Fast return of dynamic and static content

    You can invent several bicycles to solve all these problems: put several servers, instead of one, to ensure reliability; constantly upgrade iron as it grows; move the statics to a separate file server ...

    The solution, as it seems to us, is not very flexible.

    It is much more convenient, and most importantly - more effective (in our opinion) - to learn how to work in the cloud infrastructure, use its services, scale and be ready to develop not just a site, but a real cloud service. ( We already wrote about the advantages of the “cloud” compared to traditional hosting on Habré , we are all interested in the topic - we invite you to a discussion).

    That is why in the 11th version of “1C-Bitrix: Site Management” appearedsupport module for the most common cloud storage .

    What is cloud storage?

    If you try to describe in a nutshell: for the user - this is a large, large server for storing static content, which can be very quickly distributed via HTTP. :) From the inside (from the point of view of the provider), everything is a bit more complicated: all the data in the storage is replicated to several points, which ensures its reliability; There is an API for working with files in the repository, etc. (We will write about some of the features in more detail below.)

    The idea of ​​the cloud storage module in our product is very simple - any files that the system user can work with can be stored absolutely transparently not on the local server of the website, but in the cloud storage .

    What tasks do we solve by transferring files to the "cloud"?

    1. Reduce the cost of operation

    In general, the cost of placing files in the cloud will be lower than using a similar file server. If you will conduct independent calculations, do not forget to calculate the cost of the server (you cannot use the “bare” disk;)), organization of backups, traffic, etc.

    2. We can use it together with CDN to accelerate the delivery of content

    Almost all cloud storage providers offer customers CDN (Content Delivery Network or Content Distribution Network - a geographically distributed network infrastructure that allows optimizing the delivery and distribution of content to end users on the Internet, Wikipedia ).

    3. Reduce the load on web sites.

    Your application servers can only do scripts and deliver dynamic content to users. Web server processes are not busy returning statics (especially true for video and distributions). Plus reduced disk load.

    4. Using centralized storage, we solve the problem of synchronizing content between multiple web sites.

    If you use a load balancer and several web servers, you need to solve the problem of synchronizing content between them. Options - either periodically start synchronization on local storage (rsync, csync2), which is quite problematic when working with large amounts of data; either use one or another centralized storage. For example, cloud.

    5. Accelerate the rendering of pages in the browser

    If all the pictures of the site are given from the main domain (for example, www.site.ru ), then almost all browsers by default open a limited number of connections to one domain (most often - 5).

    Therefore, even if the speed of your Internet channel allows you to receive content from the site very quickly, you will be limited by the number of connections. Visually, the page in the browser will not render too quickly.

    If the pictures are placed on a separate domain relative to the site itself - this speeds up the loading of all necessary content.

    Cloud Storage Providers

    We support Amazon S3, Google Storage, Windows Azure Storage cloud storage from Microsoft, RackSpace, OpenStack.

    Almost all of them offer roughly similar functionality (from the point of view of the user):
    • Any number of objects (most often up to several TB each)
    • Possibility of placement in different data centers (regions)
    • Grouping objects
    • Authorization Mechanisms and ACLs
    • REST and SOAP interfaces for working with objects
    • Direct file upload via HTTP
    • High availability
    • Low price
    • Access through external tools (FUSE, clients)

    However, of course, they have their own characteristics. Amazon Simple Storage Service (Amazon S3) A service implemented as part of a whole range of cloud services from Amazon - Amazon Web Services (AWS). It can be used together with another Amazon service - CloudFront - Amazon’s own CDN. Files from S3 can be distributed not only via HTTP, but also via the BitTorrent protocol. An interesting feature - you can use a special type of storage - Reduced Redundancy Storage (RRS). In this case, less reliability of data storage is guaranteed (compared to standard storage), however, users are offered a lower cost of data storage. Can be used for renewable content.











    Prices (for standard storage; vary by region):
    • Storage - 1 GB (up to 1 TB) $ 0.14 / month.
    • $ 0.01 per 1000 PUT, COPY, POST or LIST requests
    • $ 0.01 per 10,000 GET requests
    • Traffic - 1 GB (up to 10 TB) $ 0.12 (first 1 GB - free)


    Google Cloud Storage

    Storage, implemented by Google, mainly for storing data when working with the Google App Engine (however, of course, it can be used in other applications).

    Content may be served through the Google CDN.

    Until the end of 2011, it is possible to take advantage of the free trial offer (storage size - up to 5 GB, there are limits on traffic and the number of requests). A good offer for those who want to just try and evaluate the service.

    Prices (vary by region):
    • Storage - 1 GB (up to 1 TB) $ 0.13 / month.
    • $ 0.01 per 1000 PUT, COPY, POST or LIST requests
    • $ 0.01 per 10,000 GET requests
    • Traffic - 1 GB (up to 1 TB) $ 0.12


    Windows Azure Storage

    Storage created by Microsoft as part of the development of the Windows Azure cloud platform.

    There is own CDN. Additional services are interesting (for example, Table Service, Queue Service).

    Users can take advantage of the free trial for the entire Windows Azure platform as a whole, including by gaining access to the storage (trial limits - 20 GB of storage, 50,000 transactions).

    Prices (vary by region):
    • Storage - 1 GB $ 0.15 / month.
    • $ 0.01 per 10,000 requests
    • Traffic - 1 GB $ 0.15


    Rackspace Cloud Files

    Rackspace is one of the world's largest cloud providers. Which, among other services, offers its customers cloud storage.

    Files can be distributed via CDN, which is organized in partnership with Akamai .

    Rates:
    • Storage - 1 GB $ 0.15 / month.
    • Requests for files less than 250 Kb, as well as HEAD, GET, DELETE - for free
    • Traffic - 1 GB $ 0.18


    The previously mentioned Rackspace, together with NASA, developed the Nebula cloud platform.

    In mid-2010, it was decided to open this platform and create a new project on its basis - OpenStack.

    Currently, OpenStack is a whole range of software that anyone can use to create their own cloud. One of the components of OpenStack - OpenStack Swift - a complex for organizing file storage.

    The project is supported by many companies (Citrix, Dell, AMD, Intel, etc.)

    Universal OpenStack Swift API support at the 1C-Bitrix platform level enables any hoster to implement its own file storage based on OpenStack - and it can be used right away any owner of a site created on Bitrix(without additional costs for integration and learning a new specific API).

    The first (and, so far as we know, so far the only) in Russia's own cloud file storage based on OpenStack implemented Clodo.ru .

    Some technical features of the storage implementation are described in the blog Clodo.ru on Habré .

    As a result, you can buy a hosting (virtual machine) from Clodo under 1C-Bitrix, immediately with it - a product license, and then connect cloud storage to the project. Great batch offer for customers !

    We urge Russian hosting providers to develop their own cloud solutions!

    We hope that the emergence of such solutions will give impetus to the development of cloud infrastructure and CDN networks in Russia.

    How does it all work in the 1C-Bitrix platform?

    You can connect one or several storages to any site running on 1C-Bitrix. This is done in the administrative interface - it is simple, just specify several parameters:



    In the simplest version, you can simply transfer all the project files to the clouds by selecting the desired item in the context menu:



    In the future, all new files (photos in the photo gallery, pictures to product descriptions in the online store, user avatars in social networks, etc.) will be automatically uploaded to the cloud, and links to them will be automatically generated in the correct way.

    If you want more flexibility in managing files, you can use the system of "filter rules" implemented in the platform.

    Rules are configured:
    • by system modules
    • by file extensions
    • by file size



    For example, you can connect two different storages and configure filter rules for each of them. Suppose all files larger than 100 MB are moved to the Google Storage cloud, and all videos are transferred to Amazon S3. Depending on the profitability of providers' offers, you can change your rules by transferring data to other cloud folders. Or completely disable storage that has become expensive or uncomfortable.

    A bit of “insides” for developers

    The most important thing is that we consider the possibility of “transparent” work with the “cloud” very important for customers. That is why the cloud storage support module is included in all editions of the product, starting with Start.

    At first glance, the implementation of support for "clouds" in the platform may seem quite simple. However, our developers had to modify about 30 modules in the system! All of them somehow work with files. And for the user who decides to "move to the cloud," everything should be transparent.

    The API for file developers remained just as transparent (you can read in detail on Maxim Smirnov's blog ).

    And this means that any developer working with the platform API can fully use all the functionality of cloud storage.

    * * *

    The 1C-Bitrix platform has become even more flexible for both end users and developers. New tools have appeared for scaling sites and effectively solving other problems.

    We strive to make 1C-Bitrix a platform not only for creating sites, but also for developing cloud web services, large scalable web projects hosted in the cloud.

    Also popular now: