Purely unscientific: Tarantool 1.6 vs Golang (speed)

    I read recently about Tarantool, it became interesting. The idea is good - the code next to the database is stored in such a fast Redis-like environment.


    And something was thinking - we are now actively using Golang at work, in fact, the thought came that Go wrote a lot of things, including and embedded databases. But what if, for example, you compare Go + LevelDB (actually, you could have any other) against Tarantool. I also tested Go + RocksDB, but everything turned out to be a little more complicated there , and the result is about the same on small data.


    I tested a simple task - an HTTP server, upon request - write the key to the database, get it by name (without any race checks), send back a simple JSON from this value.


    Compare: go+leveldb, tarantool, go+go-tarantool,nginx upstream tnt_pass


    Looking ahead - in my unscientific test I won Go + LevelDB due to the use of all processor cores. Most likely, if you launch several Tarantulas and a balancer, there may be some gain, but not to say that it will be significant ... But, the truth is, here it will already be necessary to do replication or something like that.


    But overall, Tarantool is a very impressive thing.


    Please note: I am comparing a very specific case , this does not mean that in all other cases Go / LevelDB wins or loses.


    Well and still: instead of LevelDB - probably, it is better to use RocksDB.


    So the result (briefly)


    4-10= 4 threads, 10 simultaneous connections
    10-100= 10 threads, 100 connections


    image


    Please note Tarantool takes only 1 CPU thread (or rather, in view of 2), and it was tested on a 4-thread CPU. Go uses all cores and threads by default.


    nginx lua tnt_pass taken from dedokOne comment ( result )


    wrk -t 4 -c 10 (4 threads, 10 connections):


    Golang:


      Latency Distribution
         50%  269.00us
         99%    1.64ms
    Requests/sec:  25637.26

    Tarantool:


      Latency Distribution
         50%  694.00us
         99%    1.43ms
    Requests/sec:  10377.78

    But, Tarantula took up only about half of the cores, so their speed is probably about the same.


    Under heavy load ( wrk -t 10 -c 100), the Tarantula remained in place by RPS (but latency dipped much more noticeably than Golang, especially the upper part), and Golang even perked up (but latency also dipped, of course).


    Go:


      Latency Distribution
         50%    2.85ms
         99%    8.12ms
    Requests/sec:  33226.52

    Tarantool:


      Latency Distribution
         50%    8.69ms
         99%   73.09ms
    Requests/sec:  10763.55

    Tarantool has its advantages: secondary index, replication ...


    We Go also has a huge library ecosystem ( about 100 thousand by my count, among them implementations built (and not) database - the sea ), and, as an example, the same bleve provides full-text search (which, as I understand it, for example , not in Tarantool).


    Feels like the Tarantula ecosystem is poorer. At least everything that is offered is msgpack, http server, client, json, LRU cache, ... in Go it is implemented in countless versions ..


    That is, in general, there is no crazy speed gain.


    So far, my personal choice remains in the direction of Go, because there is no feeling that the Tarantool ecosystem will shoot so hard in the near future, and Go - has been actively developing for a long time.


    The Tarantool code is, of course, shorter, but mainly because errors are handled by the language. In Go, you can also cut everything errand remain about the same.


    Can someone have other opinions?


    We also noticed in the comments about atomic code updates in Tarantool, but since we are talking about HTTP requests, we (at the current place of work) use endless for go and according to our tests (and we have thousands of requests per second) - we update Go code without losing HTTP requests. An example at the end of the article.


    And if more about the test:


     ➜  ~ go version
     go version go1.6 darwin/amd64
     ➜  ~ tarantool --version
     Tarantool 1.6.8-525-ga571ac0
     Target: Darwin-x86_64-Release

    Golang:


    ➜  ~ wrk -t 4 -c 10 -d 5 --latency http://127.0.0.1:8081/
    Running 5s test @ http://127.0.0.1:8081/
      4 threads and 10 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency   346.71us  600.80us  26.94ms   97.89%
        Req/Sec     6.54k     0.88k   13.87k    73.13%
      Latency Distribution
         50%  269.00us
         75%  368.00us
         90%  493.00us
         99%    1.64ms
      130717 requests in 5.10s, 15.08MB read
    Requests/sec:  25637.26
    Transfer/sec:      2.96MB

    Tarantool:


    ➜  ~ wrk -t 4 -c 10 -d 5 --latency http://127.0.0.1:8080/
    Running 5s test @ http://127.0.0.1:8080/
      4 threads and 10 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency   767.53us  209.64us   4.04ms   87.26%
        Req/Sec     2.61k   437.12     3.15k    45.59%
      Latency Distribution
         50%  694.00us
         75%    0.90ms
         90%    1.02ms
         99%    1.43ms
      52927 requests in 5.10s, 8.58MB read
    Requests/sec:  10377.78
    Transfer/sec:      1.68MB

    Under greater load:


    Go:


    ➜  ~ wrk -t 10 -c 100 -d 5 --latency http://127.0.0.1:8081/
    Running 5s test @ http://127.0.0.1:8081/
      10 threads and 100 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     3.04ms    1.48ms  25.53ms   80.21%
        Req/Sec     3.34k   621.43    12.52k    86.20%
      Latency Distribution
         50%    2.85ms
         75%    3.58ms
         90%    4.57ms
         99%    8.12ms
      166514 requests in 5.01s, 19.21MB read
    Requests/sec:  33226.52
    Transfer/sec:      3.83MB

    Tarantool:


    ➜  ~ wrk -t 10 -c 100 -d 5 --latency http://127.0.0.1:8080/
    Running 5s test @ http://127.0.0.1:8080/
      10 threads and 100 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency    10.65ms   14.24ms 269.85ms   98.43%
        Req/Sec     1.09k   128.17     1.73k    94.56%
      Latency Distribution
         50%    8.69ms
         75%   10.50ms
         90%   11.36ms
         99%   73.09ms
      53943 requests in 5.01s, 8.75MB read
    Requests/sec:  10763.55
    Transfer/sec:      1.75MB

    Test sources:


    Go:


    package main
    import (
        "encoding/json"
        "fmt"
        "io"
        "net/http"
        "github.com/syndtr/goleveldb/leveldb"
    )
    var db *leveldb.DB
    func hello(w http.ResponseWriter, r *http.Request) {
        err := db.Put([]byte("foo"), []byte("bar"), nil)
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        res, err := db.Get([]byte("foo"), nil)
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        result, err := json.Marshal(string(res))
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        w.Write(result)
    }
    func main() {
        var err error
        db, err = leveldb.OpenFile("level.db", nil)
        if err != nil {
            panic(err)
        }
        http.HandleFunc("/", hello)
        fmt.Println("http://127.0.0.1:8081/")
        http.ListenAndServe("127.0.0.1:8081", nil)
    }

    Tarantool:


    #!/usr/bin/env tarantool
    box.cfg{logger = 'tarantool.log'}
    space = box.space.data
    if not space then
        space = box.schema.create_space('data')
        space:create_index('primary', { parts = {1, 'STR'} })
    end
    local function handler(req)
      space:put({'foo','bar'})
      local val = space:get('foo')
      return req:render({ json = val[2] })
    end
    print "http://127.0.0.1:8080/"
    require('http.server').new('127.0.0.1', 8080)
        :route({ path = '/' }, handler)
        :start()

    Golang (atomic code substitution, no connection loss):


    package main
    import (
        "encoding/json"
        "fmt"
        "io"
        "net/http"
        "syscall"
        "io/ioutil"
        "time"
        "github.com/fvbock/endless"
        "github.com/gorilla/mux"
        "github.com/syndtr/goleveldb/leveldb"
    )
    var db *leveldb.DB
    func hello(w http.ResponseWriter, r *http.Request) {
        if db == nil {
            // (необязательная) гарантия себе, что тест и правда отработал
            panic("DB is not yet initialized")
        }
        err := db.Put([]byte("foo"), []byte("bar"), nil)
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        res, err := db.Get([]byte("foo"), nil)
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        result, err := json.Marshal(string(res))
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        w.Write(result)
    }
    func main() {
        var err error
        mux1 := mux.NewRouter()
        mux1.HandleFunc("/", hello).Methods("GET")
        fmt.Println("http://127.0.0.1:8081/")
        server := endless.NewServer("127.0.0.1:8081", mux1)
        server.BeforeBegin = func(add string) {
            ioutil.WriteFile("server.pid", []byte(fmt.Sprintf("%d", syscall.Getpid())), 0755)
            db, err = leveldb.OpenFile("level.db", nil)
            for err != nil {
                time.Sleep(10 * time.Millisecond)
                db, err = leveldb.OpenFile("level.db", nil)
            }
        }
        server.ListenAndServe()
        if db != nil {
            db.Close()
        }
    }

    After that, you can do a go buildrun and try to do during the load go build; kill -1 $(cat server.pid)- in my tests, there was no data loss.


    In the comments, we recommended trying go + go-tarantool


    I tried:


    Less load


    ➜  ~ wrk -t 4 -c 10 -d 5 --latency http://127.0.0.1:8081/
    Running 5s test @ http://127.0.0.1:8081/
      4 threads and 10 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency   799.14us  502.56us  25.22ms   95.74%
        Req/Sec     2.55k   248.65     2.95k    85.22%
      Latency Distribution
         50%  727.00us
         75%  843.00us
         90%    1.02ms
         99%    2.03ms
      51591 requests in 5.10s, 5.95MB read
    Requests/sec:  10115.52
    Transfer/sec:      1.17MB

    Huge pressure:


    ➜  ~ wrk -t 10 -c 100 -d 5 --latency http://127.0.0.1:8081/
    Running 5s test @ http://127.0.0.1:8081/
      10 threads and 100 connections
      Thread Stats   Avg      Stdev     Max   +/- Stdev
        Latency     7.49ms    4.00ms  65.06ms   81.21%
        Req/Sec     1.38k   357.31     8.40k    94.61%
      Latency Distribution
         50%    6.78ms
         75%    8.86ms
         90%   11.77ms
         99%   22.74ms
      69091 requests in 5.10s, 7.97MB read
    Requests/sec:  13545.12
    Transfer/sec:      1.56MB

    Source:


    tarantool.lua:


    #!/usr/bin/env tarantool
    box.cfg{ listen = '127.0.0.1:3013', logger = 'tarantool.log' }
    space = box.space.data
    if not space then
        box.schema.user.grant('guest', 'read,write,execute', 'universe')
        space = box.schema.create_space('data')
        space:create_index('primary', { parts = {1, 'STR'} })
    end
    print(space.id)
    print('Starting on 3013')

    main.go:


    package main
    import (
        "encoding/json"
        "fmt"
        "io"
        "log"
        "net/http"
        "time"
        "github.com/tarantool/go-tarantool"
    )
    var client *tarantool.Connection
    func hello(w http.ResponseWriter, r *http.Request) {
        spaceNo := uint32(512)
        _, err := client.Replace(spaceNo, []interface{}{"foo", "bar"})
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        indexNo := uint32(0)
        resp, err := client.Select(spaceNo, indexNo, 0, 1, tarantool.IterEq, []interface{}{"foo"})
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        first := resp.Data[0].([]interface{})
        result, err := json.Marshal(first[1])
        if err != nil {
            w.WriteHeader(500)
            io.WriteString(w, err.Error())
            return
        }
        w.Write(result)
    }
    func main() {
        var err error
        server := "127.0.0.1:3013"
        opts := tarantool.Opts{
            Timeout: 500 * time.Millisecond,
        }
        client, err = tarantool.Connect(server, opts)
        if err != nil {
            log.Fatalf("Failed to connect: %s", err.Error())
        }
        http.HandleFunc("/", hello)
        fmt.Println("http://127.0.0.1:8081/")
        http.ListenAndServe("127.0.0.1:8081", nil)
    }

    Also popular now: