Magento 2: import products from external sources

    Magento is an e-commerce solution, i.e. It is more focused on the sale of products than on the accompanying sales of warehouse, logistics or financial accounting. For a companion, other applications are better suited (for example, ERP systems). Therefore, quite often in the practice of using Magento, the task of integrating the store with these other systems (for example, with 1C) arises.


    By and large, integration can be reduced to data replication by:


    • catalog (products, categories);
    • inventory data (stock balances and prices);
    • customers;
    • orders;

    Magento for data manipulation in the database offers a separate class of objects - repositories . Due to the specifics of Magento, adding data to the database via repositories is easily encoded, but it happens, let's say, not fast. In this publication, I consider the main stages of programmatically adding a product to Magento 2 in a “classical” way - using repo classes.


    Customers and orders are usually replicated in the other direction - from Magento to external ERP systems. Therefore, with them easier, on the side of Magento, you just need to select the relevant data, and then - " from our side the bullets flew out ."


    Principles of data recording in the database


    At the moment, the creation of objects stored in the database in a programmatic way in Magento is done through the Factory :


    function __construct (\Magento\Cms\Model\BlockFactory $blockFactory) {
        $this->blockFactory = $blockFactory;
    }
    /** @var \Magento\Cms\Model\Block $block */
    $block = $this->blockFactory->create();

    and writing to the database via the Repository :


    function __construct (\Magento\Cms\Api\BlockRepositoryInterface $blockRepo) {
        $this->blockRepo = $blockRepo;
    }
    $this->blockRepo->save($block);

    The approach using "Factory" and "Repository" can be used for all major models in the Magento 2 subject area.


    Basic Product Information


    I am considering a data structure that corresponds to the version of Magento 2.3. The most basic information about the product is in the table catalog_product_entity(product registry):


    entity_id
    attribute_set_id
    type_id
    sku
    has_options
    required_options
    created_at
    updated_at

    I confine myself to one type of product ( type_id='simple'), a set of attributes by default ( attribute_set_id=4) and ignore attributes has_optionsand required_options. Since attributes entity_id, created_atand updated_atautomatically generated, then, in fact, we have to add a new product is enough to set sku. I do this:


    /** @var \Magento\Catalog\Api\Data\ProductInterfaceFactory $factProd */
    /** @var \Magento\Catalog\Api\ProductRepositoryInterface $repoProd */
    /** @var \Magento\Catalog\Api\Data\ProductInterface $prod */
    $prod = $factProd->create();
    $prod->setAttributeSetId(4);
    $prod->setTypeId('simple');
    $prod->setSku($sku);
    $repoProd->save($prod);

    and get an exception:


    The "Product Name" attribute value is empty. Set the attribute and try again.

    I add the product name to the request and I get a message that the attribute is missing Price. After adding the price, the product falls into the base:


    $prod = $factProd->create();
    $prod->setAttributeSetId(4);
    $prod->setTypeId('simple');
    $prod->setSku($sku);
    $prod->setName($name);
    $prod->setPrice($price);
    $repoProd->save($prod);

    The product name is stored in the varchar-product attribute table ( catalog_product_entity_varchar), the price is in the table catalog_product_entity_decimal. Before adding a product, it is desirable to explicitly indicate that we are using an administrative storefront to import data:


    /** @var \Magento\Store\Model\StoreManagerInterface $manStore */
    $manStore->setCurrentStore(0);

    Additional attributes


    Processing additional product attributes with Magento is a pleasure. The EAV data model for basic entities (see table eav_entity_type) is one of the key features of this platform. Simply add the appropriate attributes to the product model:


    $prodEntity->setData('description', $desc);
    $prodEntity->setData('short_description', $desc_short);
    // или
    $prodEntity->setDescription($desc);
    $prodEntity->setShortDescription($desc_short);

    and while saving the model through the repo object:


    $repoProd->save($prod);

    additional attributes will also be saved in the corresponding database tables.


    Inventory data


    In a simple way - the amount of product in stock. In Magento 2.3, the structures in the database that describe the format for storing inventory data differ significantly from what was previously. However, adding a quantity of a product in stock through a product model is not much more difficult than adding other attributes:


    /** @var \Magento\Catalog\Model\Product $prodEntity */
    /** @var \Magento\Catalog\Api\ProductRepositoryInterface $repoProd */
    $inventory = [
        'is_in_stock' => true,
        'qty' => 1234
    ];
    $prodEntity->setData('quantity_and_stock_status', $inventory);
    $repoProd->save($prodEntity);

    Media


    As a rule, media support for a product for a customer in a store (e-commerce) is different from media support for the same product for an employee in an internal accounting system (ERP). In the first case, it is desirable to show the "product face", in the second - it is enough to give a general idea of ​​the product. However, transferring at least the primary product image is quite common casewhen importing data.


    When adding an image through the admin panel, the image is first stored in the temporary directory ( ./pub/media/tmp/catalog/product) and only when the product is saved is moved to the media directory ( ./pub/media/catalog/product). Also, when adding an image through the admin panel exhibited tags image, small_image, thumbnail, swatch_image.


    /** @var \Magento\Catalog\Api\ProductRepositoryInterface $repoProd */
    /** @var \Magento\Catalog\Model\Product\Gallery\CreateHandler $hndlGalleryCreate */
    /* $imagePath = '/path/to/file.png';  $imagePathRelative = '/f/i/file.png' */
    $imagePathRelative = $this->imagePlaceToTmpMedia($imagePath);
    /* reload product with gallery data */
    $product = $repoProd->get($sku);
    /* add image to product's gallery */
    $gallery['images'][] = [
        'file' => $imagePathRelative,
        'media_type' => 'image'
        'label' => ''
    ];
    $product->setData('media_gallery', $gallery);
    /* set usage areas */
    $product->setData('image', $imagePathRelative);
    $product->setData('small_image', $imagePathRelative);
    $product->setData('thumbnail', $imagePathRelative);
    $product->setData('swatch_image', $imagePathRelative);
    /* create product's gallery */
    $hndlGalleryCreate->execute($product);

    For some reason, the media is tied up only after first saving the product and retrieving it from the repository again. And you need to specify an attribute labelwhen adding an entry to the media gallery of the product (otherwise we get an exception Undefined index: label in .../module-catalog/Model/Product/Gallery/CreateHandler.php on line 516).


    Categories


    Often, the structure of store categories and backend applications or the placement of products in them can vary considerably. Strategies for transferring data about categories and products in them depend on many factors. In this example, I adhere to the following:


    • categories of backend and store are matched by name;
    • if a category is imported that is not in the store, then it is created under the root category ( Default Category) and its further positioning in the store catalog is assumed to be manually;
    • the binding of a product to a category occurs only when it is created in a store (first import);

    Basic information about the category is in the table catalog_category_entity(catalog of categories). Creating a category in Magento:


    /** @var \Magento\Catalog\Api\Data\CategoryInterfaceFactory $factCat */
    /** @var \Magento\Catalog\Api\CategoryRepositoryInterface $repoCat */
    $cat = $factCat->create();
    $cat->setName($name);
    $cat->setIsActive(true);
    $repoCat->save($cat);

    Binding of a product to a category is carried out by category ID and product SKU:


    /** @var \Magento\Catalog\Model\CategoryProductLinkFactory $factCatProdLink */
    /** @var \Magento\Catalog\Api\CategoryLinkRepositoryInterface $repoCatLink */
    $link = $factCatProdLink->create();
    $link->setCategoryId($catMageId);
    $link->setSku($prodSku);
    $repoCatLink->save($link);

    Total


    Writing code to add products to Magento 2 programmatically is quite easy. All of the above, I brought in the demo module " flancer32 / mage2_ext_demo_import ". There is only one console command in the module fl32:import:prodthat imports the products described in the " ./etc/data/products.json " JSON file :


    [
      {
        "sku": "...",
        "name": "...",
        "desc": "...",
        "desc_short": "...",
        "price": ...,
        "qty": ...,
        "categories": ["..."],
        "image_path": "..."
      }
    ]

    Pictures for import are in the catalog ./etc/data/img.


    Import time 10 products in a similar way is about 10 seconds on my laptop. If we develop this idea further, it is easy to come to the conclusion that about 3,600 products can be imported per hour, and it can take about 30 hours to import 100K products. Replacing the laptop on the server allows you to smooth out the situation somewhat. Maybe even at times. But not by orders of magnitude. Maybe thisspeedslowness is to some extent one of the reasons for the emergence of the magento / async-import project .


    A direct solution to the database can be a cardinal decision to increase the speed of import, but in this case all the “bugs” regarding Magento extensibility are lost - you have to do everything “extended” yourself. However, it is worth it. If it works, then I will consider the approach with direct writing to the database in the next article.


    Also popular now: