How to gash a feature and not shoot yourself in the foot

    The main goal of projects is to make money. The project I worked on was no exception.


    I’m a developer of Wheels Roof Market, and today's post will be devoted to how we differentiated prices for paid services on our “classified”.


    Our company develops 3 products, each for 3 platforms - web, android and ios. Users can apply various paid services to ads, for example, paid extension of the life of an advertisement or placement in a block of hot offers.


    When I was attracted to this project, in my head, even before the start of the discussion, the idea was held: what are the differentiated prices?


    Differentiated price - the price of which depends on the characteristics of the advertisement (region, brand, model, year, etc.).


    The team was faced with the task of increasing the average check. It was decided to “gash” a feature that contains a functionality that will be discussed later. The meaning of the feature was that through the admin panel we can change the price of any paid service, based on various parameters.


    At the beginning of development, we already had a microservice written in Go. Sites and applications communicated with him through the client, sending an ad object with a POST request, and then, in response to receiving prices for any paid services, rendered them to the user.


    In the process of studying the microservice, it turned out that the prices that were given to the user were hardcoded, that is, the price for placing an ad in hot offers was described by the “hot” variable: 300, and the result of the answer looked something like this:


    {
        status: "ok",
        data: {
            color-red: 45,
            hot: 300,
            paid-auto-re: 5,
            re: 0,
            s: 90,
            unset-auto-re: 0,
            up: 200
        }
    }

    It was decided to do important refactoring and get rid of the hardcode in favor of a method that would give a differentiated price for the service.


    We divided the development process into several stages:


    1. The choice of data storage format.
    2. Development of an admin panel for price regulation.
    3. Differentiated price return method.

    Choosing a data storage format


    Initially, according to the ToR, the product manager wanted to be able to set the price for a paid service from the administrative panel and this was one of the most important tasks. I had a question in what format to store data.


    I decided to fill the database with “rules”, that is, if the category of the announcement is “Auto” and the region is “Almaty”, then we use the following price for the announcement. It remains to deal with the database.


    The first thing that came to mind is the MySQL database, where a table with rules for prices will be stored. For instance:


    IdcatIdregionIdcoeffserviceName
    1thirteen141.4hot

    According to the statement of work, it was necessary to set the price based on the region and category. But, knowing how all these “Wishlist” work, I thought that the price would need to be changed not only for the category and region, but also for a specific car model or for some other reason.


    In general, MySQL was dropped as an option, and the choice fell on MongoDB, which would provide us with the widest possibilities of dynamic scaling and be able to work with data arrays, without which the rules would be useless.




    Basically, at the time of development, all of our ads were already stored in MongoDB. Feeding rules for regulating prices there was easier. On this we stopped

    Development of admin panels for price regulation


    The admin panel is not a complicated thing, it should have standard CRUD functionality, that is, adding, editing, deleting and displaying these rules in an easy-to-read form.


    We wrote this whole thing on phalcon, since this admin panel was only part of the main admin panel for a site running on phalcon. We also wrote functionality in the API, which validated and saved our rules in the MongoDB collection.


    The json ad object looked like this:




    As we see here, there are some levels of nesting, that is, you need not only to save data with nesting, but also to search for this nesting in MongoDB. For data validation, we introduced a small collection where we stored possible fields for use, so as not to store everything that the user sent to the API.

    Differential price return functionality


    The last stage of our development was the stage of writing code that would be able to work with all these rules and would return a response with a differentiated price to the request.


    Like before:



    Like now:



    And where is the refactoring and where are the differentiated prices? But.


    These hardcoded variables in the code eventually remained as default values.


    The price return algorithm used to work like this:


    1. Formation of an array with prices for services.
    2. Type exception handling for this section. If the service is free, then give 0.
    3. The return of the result.

    Now everything works as well, except that before returning the answer to the user, there is now a trip to the method of price differentiation.


    The first approach to solving the problem was as follows.


    At the time of the formation of the array with prices for each of its elements, that is, services, there was an additional trip to MongoDB.



    The getPriceForService method returned the price and accepted the following arguments:


    1. Ad Object
    2. Service name
    3. Price before processing

    Remember, I wrote above that there is a collection with possible rules for regulating prices, they are needed here.


    In order not to go through each of the fields of the object, only a selection of possible rules was made. In the next screenshot, we see the process of generating a request in MongoDB.


    // Возвращает стоимость услуги для объявления
    func getPriceForService(advert *Advert, serviceName string, basePrice int) (result int) {
        var mongoResult map[string]interface{}
        query := bson.M{}
        query["serviceName"] = serviceName
        query["$and"] = []bson.M{}
        //Генерируем поисковый запрос в монго
        for _, data := range getUsableValuesForPrice() {
            var value, err = advert.GetValueString(data.Rule)
            stringVal, err := getStringFromMixed(value)
            if err == nil {
                list := []string{"rules.", data.Rule}
                var str bytes.Buffer
                for _, l := range list {
                    str.WriteString(l)
                }
                if err == nil {
                    query["$and"] = append(query["$and"].([]bson.M), bson.M{"$or": []bson.M{bson.M{str.String(): stringVal}, bson.M{str.String(): nil}}})
                } else {
                    checkErr(err)
                }
            }
        }
        //Получаем правила по нашему запросу
        err := mongo.GetSession().
            DB(mongo.GetDbName()).
            C("COLLECTION_NAME").
            Find(query).
            Sort("-priority").
            One(&mongoResult)
        if err == nil {
            stringResult, err := getStringFromMixed(mongoResult["coeff"])
            checkErr(err)
            coeff, err := strconv.ParseFloat(stringResult, 64)
            if err == nil {
                //Умножаем цену на полученный коэффицент
                return int(math.Ceil(coeff * float64(basePrice)))
            } else {
                checkErr(err)
            }
        }
        return basePrice
    }

    In the end, we received a request that could only be fulfilled and a new price for the service was calculated using the received coefficient.


    However, after the release of this code in production, our MongoDB died due to the fact that with one request to the microservice, it gives the results for all services, and I call the method to form an array for each element. That is, I increased the load on MongoDB by 7-8 times and, in a sweat of my face, started rewriting my code.


    Information for note: To work with MongoDB, we used mgo , which makes it easy to build queries into the database .


    That evening, having studied the code again, I decided to call this functionality at the very last moment, that is, before giving the results to the client. I will knock on the same method, only slightly rewritten. The rewritten method began to take no longer the name of the service, but a list of services with prices ready for delivery.


    // Возвращает стоимости услуг массивом
    func getPriceForServices(advert *Advert, serviceList Services) (result Services) {
        var mongoResult []map[string]interface{}
        var services []string
        for key, _ := range serviceList {
            services = append(services, key)
        }
        query := bson.M{}
        query["serviceName"] = bson.M{"$in": services}
        query["$and"] = []bson.M{}
        //Генерируем поисковый запрос в монго
        for _, data := range getUsableValuesForPrice() {
            var value, err = advert.GetValueString(data.Rule)
            stringVal, err := getStringFromMixed(value)
            if err == nil {
                list := []string{"rules.", data.Rule}
                var str bytes.Buffer
                for _, l := range list {
                    str.WriteString(l)
                }
                if err == nil {
                    query["$and"] = append(query["$and"].([]bson.M), bson.M{"$or": []bson.M{bson.M{str.String(): stringVal}, bson.M{str.String(): nil}}})
                } else {
                    checkErr(err)
                }
            }
        }
        //Получаем коэффиценты по нашему запросу
        err := mongo.GetSession().
            DB(mongo.GetDbName()).
            C("Collection_name").
            Find(query).
            Select(bson.M{"serviceName": 1, "coeff": 1}).
            Sort("priority").
            All(&mongoResult)
        checkErr(err)
        //Собираем массив с ключ, значением
        for _, element := range mongoResult {
            coeff, err := getStringFromMixed(element["coeff"])
            checkErr(err)
            intCoeff, error := strconv.ParseFloat(coeff, 64)
            checkErr(error)
            serviceName, err := getStringFromMixed(element["serviceName"])
            if val, ok := serviceList[serviceName]; ok {
                price := int(math.Ceil(intCoeff * float64(val)))
                serviceList[serviceName] = price
            }
        }
        return serviceList
    }

    As before, having received the request and fulfilling it, we get data with the rules for each service and the coefficient by which you need to multiply the old price.


    This approach eliminated all unnecessary trips to MongoDB, thereby we stopped loading our database and got differentiated prices :) (profit).


    Also popular now: