Load test with Go

    Good afternoon, Habrahabr.
    You are probably familiar with JMeter . If in short - a very convenient tool for conducting load testing, it has huge functionality and many, many useful features. But the article is not about him.

    Where did it start

    There is a rather loaded node in our project, JMeter helped for a long time. Profiling and optimization gave their profit, but everything ran into a small problem. JMeter could not create very large traffic, and more precisely, after 10 seconds of the required mode, OutOfMemory took place and testing stopped, in some cases there was no problem, but the speed of sending requests decreased noticeably, while the CPU load was 400%, it was solved restarting the program. It was extremely uncomfortable to use.
    So, we have a problem, and it needs to be solved, the first thing that came to mind was to make your own mini-test that meets the minimum requirements. It has long been interesting to try Go's taste. So the go-meter application was born. When writing, there were a lot of questions, the answers to which either were not, or they did not explain the problem, so I decided to share my experience and an example of working code, if you are interested, I ask a tackle.


    I think writing about what kind of language does not make sense, you can always see a tour of the language that reveals the basic elements. How to install and configure the environment is also not worth it; the documentation is written in a completely understandable language.
    Why did you choose Go? There are several criteria that are very important for me: it works fast, cross-platform, there are flows that are easy to manage, unusual. Of course, you say that you can write this in any other language. I agree with you, but the task was not only to write, but also to learn something new.

    Let's get started

    Without hesitation, it was decided to store the test profile in JSON format, after launching the application, the profile is read and testing is started. During testing, a summary table is displayed in the console (response time, number of requests per second and percentage of errors, warnings and successful requests). With JSON, everything is simple, for this you need to make structures for each element, open and read the file:
    func (this *Settings) Load(fileName string) error {
    	file, e := ioutil.ReadFile(fileName); if e != nil {
    		return e
    	e = json.Unmarshal(file, this); if e != nil {
    		return e
    	return nil

    Let's go further. After starting, we need to start N-threads, and after working out each of them, aggregate the data, then output it beautifully to the console. For this, in this interesting language there are Channels. A kind of "pipe" between different streams. No synchronization, locks are needed, everything is done for us. The idea is this: a thread sends a request, determines the result, and reports this to the main thread, which in turn waits until all the threads have completed and output all the data received. Streams will communicate with us by means of transmission of the structure:
    type Status struct {
    	IsError bool
    	IsWarning bool
    	IsSuccess bool
    	Duration *time.Duration
    	Size int64
    	IsFinished bool
    	Error *error
    	FinishedAt *time.Time
    	StartedAt *time.Time

    Each thread will execute an M-time HTTP request to the specified resource. If we have a POST request, then still sending certain data that the user wants:
    func StartThread(setts *settings.Settings, source *Source, c chan *Status){
    	iteration := setts.Threads.Iteration
    	//Формируем объект key, value для заголовков запроса
    	header := map[string]string{}
    	for _, s := range setts.Request.Headers {
    		keyValue := regexp.MustCompile("=").Split(s, -1)
    		header[keyValue[0]] = keyValue[1]
    	sourceLen := len(*source)
    	//необходимый URL
    	url := setts.Remote.Protocol + "://" + setts.Remote.Host + ":" + strconv.Itoa(setts.Remote.Port) + setts.Request.Uri
    	if iteration < 0 {
    		iteration = sourceLen
    	index := -1
    	for ;iteration > 0; iteration-- {
    		status := &Status{false, false, false, nil, 0, false, nil, nil, nil}
    		if index >= sourceLen {
    			if setts.Request.Source.RestartOnEOF {
    				index = 0
    			} else {
    		//Получаем данные для отправки запроса
    		var s *bytes.Buffer
    		if strings.ToLower(setts.Request.Method) != "get" {
    			s = bytes.NewBuffer((*source)[index])
    		//Создаем HTTP запрос
    		req, err := http.NewRequest(setts.Request.Method, url, s); if err != nil {
    			status.Error = &err
    			status.IsError = true
    			c <- status
    		//Выставляем заголовки
    		for k,v := range header {
    		//Засекаем время
    		startTime := time.Now()
    		//Отправляем запрос
    		res, err := http.DefaultClient.Do(req); if err != nil {
    			status.Error = &err
    			status.IsError = true
    			c <- status
    		endTime := time.Now()
    		//Записываем служебную информацию
    		status.FinishedAt = &endTime
    		status.StartedAt = &startTime
    		diff := endTime.Sub(startTime)
    		//Проверяем статус ответа и причисляем в одной из 3 групп (Error, Warning, Success)
    		checkStatus(setts.Levels, res, diff, status)
    		//Закрываем соединение
    		//Оповещаем главный поток
    		c <- status
    		//Если установлена в настройках задержка, выполняем ее
    		if setts.Threads.Delay > 0 {
    			sleep := time.Duration(setts.Threads.Delay)
    			time.Sleep(time.Millisecond * sleep)
    	//Оповещаем главный поток о завершении работы
    	status := &Status{false, false, false, nil, 0, true, nil, nil, nil}
    	c <- status

    It remains only to start our threads at program startup and listen to data from them
    c := make(chan *Status, iteration * setts.Threads.Count)
    for i := 0; i < setts.Threads.Count; i++{
    	go StartThread(&setts, source, c)
    for i := iteration * setts.Threads.Count; i>0 ; i-- {

    Instead of a conclusion

    These are the most interesting moments, in my opinion. All sources are available on GitHub , there you can see the entire cycle of work with an example of use. In fact, this miracle language coped with this task with interest, when generating traffic 3 times more than it was in the case of JMeter, the processor load rarely exceeds 15%.
    If it’s interesting, I’ll talk about the process of writing an HTTP Restfull Web service with storage in MongoDB and Redis.

    Thanks for attention!

    Also popular now: