April 29, 2013 at 16:05Proxying and auto-sharing in Tarantool / Box
After numerous discussions, a draft specification for proxying and auto-sharing in Tarantool / Box was created.
The general idea is that in order to hide the presence of sharding from the user, a proxy is raised on the local application host (PHP / Perl / etc), which already routes requests for shards and can find the necessary key in case of resharding.
Resharding itself is already done using Lua on the shards themselves, while the proxy transfers the hottest keys to new shards when accessing them, which should reduce the load on the system during resharding.
The full specification is on the github wiki .
It would be extremely useful to discuss the specification for potential problems, unaccounted situations, usability, before implementing it.
The general idea is that in order to hide the presence of sharding from the user, a proxy is raised on the local application host (PHP / Perl / etc), which already routes requests for shards and can find the necessary key in case of resharding.
Resharding itself is already done using Lua on the shards themselves, while the proxy transfers the hottest keys to new shards when accessing them, which should reduce the load on the system during resharding.
The full specification is on the github wiki .
It would be extremely useful to discuss the specification for potential problems, unaccounted situations, usability, before implementing it.