LUA in nginx: hot cache in memory

I decided to replenish the piggy bank of articles on Habré about such a wonderful YP as lua, with a couple of examples of its use under the hood of nginx. Broke into two independent posts, the second here .

In this post, nginx is used as a "hot cache" of some constantly updated data requested by clients at an interval with optional grouping (a certain analogue of BETWEEN and GROUP BY / AGGREGATE from SQL). Data is loaded into the cache by lua + nginx from Redis. Redis source data is added every second, and customers still want it (interval in seconds, minutes, hours ...) with aggregation by N (1 <= N <= 3600) seconds, sorted by date and in json format.
With a good hitrate on an existing machine, it turns out to provide 110-130k Wishlist per second, but with a bad one - only 20-30k. Which, in general, is also acceptable for us on the same nginx instance.


From a certain source every second comes the data that is added to Redis ZSET. An important point is the binding of data precisely to time - the selection will go through time intervals. One client came - “give me one by one every now and then”, another came - “but I have this interval, but come with hourly aggregation”, the third one needed one last second, the fourth for a day with aggregation of 27 seconds, well, etc. d ... Knocking on data directly in Redis is unrealistic. It is very problematic to cache prepared data in advance, as the required intervals and aggregation step in the general case, each client / request has its own and can arbitrarily vary. The server should be ready to quickly respond to any reasonable request.

Initially, the idea was to perform aggregation on the Redis side, calling through the EVAL redis-lua code from nginx-lua code. This “We need to go deeper technology” did not fit because of the single-threaded nature of Redis itself: quickly delivering “raw data” comes out much faster than grouping and stuffing the finished result.

Data in Redis is stored element-wise in the json format of the form:

ZADD ns:zs:key 1386701764 "{\"data100500\":\"hello habr\",\"dt\":\"10.12.2013 10:05:00\",\"smth\":\"else\"}"

The key is timestamp, in dt the string equivalent according to the “filler” version.
Accordingly, the range selection:

ZREVRANGEBYSCORE ns:zs:data:sec 1386701764 1386700653 WITHSCORES

And on lua via resty Redis:

local redis = require 'redis'
local R, err = redis:new()
R:connect('12.34.56.78', 6379)
R:zrevrangebyscore('ns:zs:data:sec', to, from, 'WITHSCORES')
-- и т.п.

We collect nginx ( lua-nginx-module + lua-resty-redis ), quickly configure:

http {
    lua_package_path '/path/to/lua/?.lua;;';
    init_by_lua_file '/path/to/lua/init.lua';
    lua_shared_dict ourmegacache 1024m;
    server {
        location = /data.js {
            content_by_lua_file '/path/to/lua/get_data.lua';
        }
    }
}

To diversify the boundaries of the requested interval and the aggregation step, we will obtain from the GET request parameters from , to and step . With this agreement, an example format for a request to a service would be:

/data.js?step=300&from=1386700653&to=1386701764

local args = ngx.req.get_uri_args()
local from = tonumber(args.from) or 0
...

So, we have element-wise json entries stored in Redis that we can get from there. What is the best way to cache and give them to customers?

• You can store per second records individually in a table. However, as practice has shown, the execution of several dozen queries to the table has an extremely negative impact on performance. And if you receive a request for a day, then a response with a short timeout can not wait;
• Records can be stored in blocks, combining through some common separator or serializing them even in the same json. And when prompted, you need to sort through the delimiter or deserialize. So-so option;
• Store data hierarchically, with partial repetitions at different levels of aggregation. The cache blocks of different sizes are used: 1 second (single record), 10 seconds, 1 minute, 10 minutes, hour. Each block contains data of all its seconds. Most importantly, the content of the block does not change in any way and is not given in pieces: either as a whole or not.

The last option was selected, consuming more memory, but significantly reducing the number of table accesses. The cache blocks of different sizes are used: 1 second (single record), 10 seconds, 1 minute, 10 minutes, hour. Each block contains data of all its seconds. Each block is aligned to the border of its interval, for example, the first element of the 10 second interval always has a timestamp with a decimal remainder of 9 (sorting in descending order, as customers want), and the time block contains elements 59:59, 59: 58, ... 00:00. When combining elements, they are immediately glued with a separator - a comma, which allows you to give these blocks to the client with one action: '[', block, ']', and also quickly combine them into larger pieces.

To cover the requested interval, it is divided into the maximum possible blocks with completion on the edges with smaller blocks. Because Since we have single blocks, it is always possible to fully cover the required interval. To request the interval 02: 29: 58 ... 03:11:02 we get the cache layout:

1sec - 03:11:02
1sec - 03:11:01
1sec - 03:11:00
1min - 03:10:59 .. 03:10:00
10min - 03:09:59 .. 03:00:00
30min - 02:59:59 .. 02:30:00
1sec - 02:29:59
1sec - 02:29:58

This is just an example. Real calculations are performed on timestamps.
It turns out that we need 8 requests to the local cache. Or to Redis, if locally they are already / not yet. And in order not to burst over the same data from different workers / connects, you can use the atomicity of operations with shared dict to implement locks (where key is a cache cache key containing information about the interval and aggregation step):

local chunk
local lock_ttl = 0.5 -- пытаемся получить блокировку не дольше, чем полсекунды
local key_lock = key .. ':lock'
local try_until = ngx.now() + lock_ttl
local locked
while true do
    locked = cache:add(key_lock, 1, lock_ttl)
    chunk = cache:get(key)
    if locked or chunk or (try_until < ngx.now()) then
        break
    end
    ngx.sleep(0.01) -- ожидание, не блокирующее nginx evloop
end
if locked then
    -- удалось получить блокировку. делаем, что собирались
elseif chunk then
    -- лок получить не удалось, но в кеш положили нужные нам данные
end
if locked then
    cache:delete(key_lock)
end

Having the necessary cache layout, the ability to select the desired range from Redis, and the aggregation logic (it’s very specific here, I don’t give an example), we get an excellent caching server, which, after warming up, knocks on Redis only once per second for a new element + for an old one, if they have not yet been selected or have been thrown by LRU. And do not forget about the limited connection pool in Redis.
In our case, the warm-up looks like a short-term jump in incoming traffic of the order of 100-110Mb / s for several seconds. According to cpu on a machine with nginx, heating is almost not noticeable at all.

The image in the header is taken from here .