An example of using the Product API from Fetchee to parse products of an online store / Fetchee's blog

In this manual, we will talk about how to use the Fetchee Product API to get product data by URL using the example of the lamoda online store.

For those who have not read our previous article , the Product API will be useful for developers who need to receive product data from any store, but who do not want to spend time creating their own parsing system or have already realized that open-source libraries have significant limitations and require a lot of time for support. Our automatic and configuration-free API for parsing eCommerce data makes it possible to concentrate on developing the main functions of your application. In addition, trying it is very simple. Details under the cut.

First you need an API access key. Leave a request forhttps://fetch.ee/ru/developers/ . We provide access to all comers and usually respond to applications within 24 hours.

1. Sending a request for parsing a product by URL

Please note that the API accepts only POST requests in application / json format at the https://fetch.ee/api/v1/product URL :

{
   "url":"",
   "api_key":"",
   "callback_url":""
}

An example of a request for a product at http://www.lamoda.ru/p/ug174awohj47/shoes-uggaustralia-uggi/ will look like this (we added a demo API key that will last a couple of days):

curl -X POST -H "Content-type: application/json" -d "{\"url\": \"http://www.lamoda.ru/p/ug174awohj47/shoes-uggaustralia-uggi/\", \"api_key\": \"0e2bc30838d8bbbbbec787667c4ff34b\", \"callback_url\": \"http://requestb.in/onpdf2on\"}" https://fetch.ee/api/v1/product

All parameters are required, if any of them is not present, the server will return a 403 error, as in the case of an incorrect API key.

IMPORTANT: The example specifies a temporary callback_url. It will stop working after a couple of hours, replace it with your URL, which can be obtained at http://requestb.in .

The Product API processes requests asynchronously, the response time can vary from a couple of seconds to several tens of minutes. This is due to the load on the system in a particular geographical region. The API will send data to the URL you specify at the end of the parsing (callback_url).

The maximum waiting time is 30 minutes, after which in any case a response will be sent with the result or a message about the impossibility to complete the parsing of the goods at the specified URL.

For testing the API it is convenient to use the http://requestb.in service . Get a new container on it (Create a RequestBin) and specify the received URL as callback_url. All answers from the API will come to him, just refresh the browser page.

2. Fields and API response options

The response uses the following fields:

• success - the result of sending parsing URLs;
• id - a unique request ID;
• already_processed - the request has already been recently processed;
• not_shop - the URL does not belong to the store;
• not_product - there is no product data for this URL.

Any of the fields, except success, may be absent.

Immediately after the request, you will receive a response in JSON format. The ID is needed to match the requests you send and the responses received on callback_url. In this case, everything went fine and the product was accepted for parsing.

{
  "success": true,
  "id": "583ea83a09e9497a0eb1b82a"
}

Such an answer will follow if you try to send to parsing a URL that was already processed less than 6 hours ago. In this case, the call to callback_url will not follow.

{
   "success":true,
   "id":"583ea83a09e9497a0eb1b82a",
   "already_processed":true
}

You will receive this answer if the API already knows that the specified URL does not apply to the site of the online store. The item is not added for parsing and there will be no callback_url call. We maintain a black list of sites, which include, among other things, dubious and affiliate stores (acting as display cases with goods from other stores).

{
   "success":false,
   "not_shop":true
}

When the system knows that there is no product at this URL (category page, store main page, etc.). The item is not added for parsing and there will be no callback_url call.

{
   "success":false,
   "not_product":true
}

3. Getting the result of parsing

The API sends a POST request with a response in JSON to the address specified in callback_url. After 20 seconds, we got the following parsing result:

{
   "id":"583ea83a09e9497a0eb1b82a",
   "url":"http://www.lamoda.ru/p/ug174awohj47/shoes-uggaustralia-uggi/",
   "created_at":"2016-11-30T10:21:46.865Z",
   "title":"Обувь угги UGG Australia",
   "price":22800,
   "currency":"RUB",
   "img_url":"https://fetch.ee/assets/item-images/583e/a843a9436d700e54ef37.jpg",
   "brand":"UGG Australia"
}

Consider the resulting JSON in more detail. There are always 3 required parameters in the answer:

• id - a unique request ID;
• url - the final URL of the product (if there were redirects, or we cut out various unnecessary query_params from the URL (UTM tags, for example);
• created_at - time to create a request for parsing the goods;

Any of the following options may be missing:

• title - product name;
• price - price;
• currency - currency code (for example, RUB, USD, EUR);
• img_url - link to the product image;
• brand - producer of goods;
• out_of_stock: true - when the product is not on sale;
• removed: true - when the product is completely gone from the store and redirected to the category page, the main page of the store, or we get a 404 response from the server;
• not_shop: true - when during parsing the API determined that the URL does not belong to the store, although it previously accepted a request for processing;
• not_product: true - the goat during the parsing API determined that the URL does not contain information about the product, although the request for processing was accepted;
• unprocessed: true - when the Product API could not get the key parameters for the product title or price.

If the Product API may be useful to you, leave a request for fetch.ee/ru/developers ! Write in the comments what other product parameters you would like to receive in addition to those already available?