January 14, 2014 at 15:26

Import data to online stores

    image


    What do we mean by importing data into online stores? To explain, we describe the scenario that is most typical for owners of online stores.
    Typically, online store owners want to import the actual data from suppliers. They need this in order to update the state of the database of their online store, so that they quickly get new items, so that those products that are no longer sold by the supplier are not available for order in their online store. And they want to repeat this process as often as possible.


    Supplier data is:

    - categories (with or without hierarchy),
    - products with descriptions, parameters and attributes, images, their combinations, if any,
    - status of availability, quantity in stock, purchase price, sale price, discounts, minimum order.

    It is about this data that will be discussed. There are two options for working with this data:
    1) the online store is empty, and we do not need to consider what is in it;
    2) the online store is already full.

    We will consider the second option, because it most fully reflects the specifics of the data import process for online stores.

    Requirements



    What does an online store owner usually want? The requirements on his part for the automation of data import are as follows:

    - if a new category has appeared at the supplier, it is necessary that it also be created in his store;
    - that the goods that are not in his store and are present at the supplier appear (meaning with descriptions, specifications, pictures, color options, etc.) in the desired category and (or) the category “News”;
    - if the product is present in the customer’s store, but is not in the supplier’s warehouse, it is necessary to disconnect (deactivate) this product or change the status to “out of stock” (this is necessary to make the product unavailable for ordering);
    - if the goods are present both in the store and at the supplier, then you need to update the prices and quantity (if such data is available).

    Hidden requirements



    In fact, the online store manager wants more from imports. These are the so-called hidden requirements, about which no one speaks out loud, but as if all implies already when accepting the project or operation in the first month.
    Such hidden requirements for importing data to an online store are the following points:
    - so that the goods that the user brings to the online store on their own, the automatic import process does not “change” (do not delete, do not deactivate);
    - so that editing product descriptions that are updated from the supplier’s warehouse, which he makes using copywriters, does not disappear (this is important for SEO);
    - for new categories and products to form correct and beautiful CNCs - “URL rewrite”;
    - change the price based on your margin, which depends on the category and popularity of the product;
    - Receive reports on which new products have appeared, and which are no longer on sale;
    - so that the process of updating data in the online store is simple and takes little time (several or 1 click);
    - if the product is present in the supplier’s system in more than one category, then you need to link this product to the appropriate categories on the side of the online store;
    - if the goods that we import into the online store are combination products (for example, a boot “red, 42 size, 2300 rubles” and “black, 43 size, 3100 rubles, etc.), it is necessary that they were processed correctly.

    Solutions



    All the requirements described above are caused by the specifics of e-commerce, and the algorithm that will import the data must take them into account in full.

    How to fulfill these requirements, what tools can be used?
    - use the standard CMS import (usually from files from csv / xls / xml / yml, etc.);
    - import through the 1C-Bitrix exchange mechanism (1C-Bitrix, UMI, HostCMS, etc., where there is an exchange with 1C);
    - program your own import mechanism.

    An analysis of each of the proposed options will be presented below.

    Import data via csv- The thing itself is very simple and should work reliably. True, as practice shows, this is not always the case. Questions arise if you need to import something more complex than the “name, description, picture, article, price” set. What to do with features, options, etc.? Today, this option to import data into the CMS is not always programmed well in most implementations and always requires improvements. Plus, these are intermediate files (they also need to be generated), hosting requirements (restrictions). And the most important minus is manual work, or rather, a lot of boring and homogeneous handmade work, which few people like to do.

    Import data through the 1C-Bitrix exchange mechanism.The idea itself is very good. Everyone has 1C, and if it is not, then the protocol itself can be emulated. But the implementation on the side of online stores makes me ask a lot of questions. If you look towards the reference for all Bitrix, you can simply say that on volumes of more than 10 thousand goods, this mechanism is already failing. Failures are different. For example, ZIP support may not be available. There may be problems with the amount of available memory for the php process. Speed ​​is the most painful topic to import into Bitrix through 1C. Why so slowly is difficult to say, but today it is. For example, Bitrix does not import by default into the standard information block of goods, but into the information block, which is specially created for 1C. Plus, when working through 1C, we have a time loss, that is, the data is downloaded twice - the first time in 1C, the second time in Bitrix. If you look at other CMS, then there implementations of this protocol are in a state where it is impossible to use it by ordinary people without the help of programmers. Read more about importing goods into Bitrix in this article:habrahabr.ru/post/133993 .

    Programming your own mechanism. Importing this way is difficult and expensive, but the implementation provides a lot of advantages that are not available today out of the box in the proposed solutions. In our opinion, if the task is to fulfill the points indicated in the requirements for importing data in full, then only this option is able to satisfy the requirements.

    The following factors are key when choosing an import tool:
    - frequency of data updates,
    - amount of data transferred,
    - speed of data updates,
    - details of information,
    - budget,
    - competencies and workload of the team.

    Those. it turns out here is a schedule for selection.



    Regarding budgets, the situation is ambiguous and depends primarily on the cost of the integrator team. The graph shows the arrangement of budgets and proposals in the Russian market. The overestimated cost of 1C-Bitrix integration is due to the shortage of specialists of the required 1C and Bitrix level in the market, which causes a corresponding increase in the cost of developing and supporting the solution. CSV integration is possible and acceptable with small amounts of data or in the absence of a budget, or with little detail in importing data.

    How to implement support for “direct import into the database” of more than 10 popular CMS online stores will be described in the following articles.

    Nikolay Kekish,
    Director of CatalogLoader.com
    Tags:

    Also popular now: