Simple things with a complicated AppEngine

I thought of making a simple thing in the toy that I wrote about earlier - to calculate what place a person occupies in the overall rating: As I wrote, the game uses AppEngine for various statistics. I’ll tell you about the optimizations that I had to apply for this simple feature. Obviously, calculating a place in the rating “on the fly” is a rather demanding task - you need to sort all users each time and with an impressive number of them, the page will start to slow down. Therefore, I decided to denormalize this value and added the appropriate field to the user class:

class Player(db.Model):
    name = db.StringProperty()
    scores = db.IntegerProperty() # очки
    #...
    rank = db.IntegerProperty() # <-- занимаемое место

Each time a player gains or loses points, his place is recalculated as follows. Let's say the table of players looks like this:

Name	Glasses	A place
...
User 1	123	fifty
User 2	121	51
User 3	111	52
User 4	105	53
User 5	100	54
User 6	99	55
...

Now, if User 5 scores 21 points and has a total of 121, I recount the places of all players with points between 100 and 121. Before recounting:

Name	Glasses	A place
...
User 1	123	fifty
User 5	121	54
User 2	121	51
User 3	111	52
User 4	105	53
User 6	99	55
...

After recounting:

Name	Glasses	A place
...
User 1	123	fifty
User 5	121	51
User 2	121	52
User 3	111	53
User 4	105	54
User 6	99	55
...

To my surprise, such a simple algorithm gave a high load on AppEngine (CPU goes

overboard ): I was surprised and decided that the problem is in more than 30 fields in the Player class, and the update without selecting all of these fields in AppEngine does not exist. Then I decided to unhook the rank field from the Player class and make a separate “label” for counting places.

class Player(db.Model):
    #... более 30-ти полей ...
    rank = db.ReferenceProperty(reference_class=PlayerRank)
class PlayerRank(db.Model):
    score = db.IntegerProperty()
    rank = db.IntegerProperty()

Recounting now occurred only for the PlayerRank class with two fields. Well, of course, this helped a lot, but you can’t call the result satisfactory:

AppEngine, which spends all its resources on recounting places in the ranking, is obviously a bad AppEngine. The problem was in too many operations. For example, if a player has 1 rating point, and another 1000 players have 2 points, then having collected this player only a couple of points, you have to count all 1000 other players who now need to lower their place. I had to optimize further as follows.
Keep the occupied place not for the player, but for the number of points. Those. if 1000 players have 2 points, then all of them will have one place (say 3000th).

Glasses	A place
...
2	3000
1	3001
...

Thus, it is necessary to recount not a thousand records of players, but only two.

class Player(db.Model):
    #... более 30-ти полей ...
    def rank(self):
        return ScoreRank.all().filter('score =', self.score).get().rank
class ScoreRank(db.Model):
    score = db.IntegerProperty()
    rank = db.IntegerProperty()
    count = db.IntegerProperty()

The ScoreRank.count property has been added to control how many players have a given number of points. If this count becomes 0, then the ScoreRank record for a given number of points is deleted.
AppEngine responded:

Conclusion

On the one hand, AppEngine forces the developer to write complicated algorithms that would be no longer needed, use the developer a relational database and the traditional one say LAMP. On the other hand, the algorithms thus turn out to be fast, the pages fly, in other approaches these pages would probably be a bottleneck and brakes (remember braking online stores on php + mysql). On the third side, AppEngne's quotas are surprising. 10 thousand requests to select an object with 30 fields and its update threw the application for a free quota. It comes to understanding that AppEngine is expensive , contrary to popular belief.

Link to the toy: www.vkubiki.ru