Browser strategy "Paths of History". Architecture and evolution of the project

    In this article I will talk about the development and evolution of the technical part of the browser-based game " History Paths ".
    I will pay attention to the choice of a programming language, database, technology and architecture. I'll tell you about hosting.

    Ways of History is a massive browser-based strategy game. The project began with the enthusiasm of one person and grew into a serious project with a considerable audience.

    To develop the engine, C ++ was chosen for three reasons:
    1. he is fast, which is important for this project;
    2. it is flexible that allows to realize some opportunities optimally;
    3. I know him better than other suitable ones.

    The essence of the engine is receiving a request, generating and submitting a page.
    MySQL chose the database only because it is quite popular and similar projects are often done using MySQL. At that moment I did not have experience with databases.
    The question of architecture immediately arose. The following model was chosen:
    The engine is divided into two parts (let's call them D1, D2).
    D1 receives a request, sends it to one of 8 free threads. The stream analyzes the request, requests the necessary data from the database, forms a page and returns it. D1 does not know how to make changes to the database. To reduce the number of queries to the database, many data are cached on D1.

    In some cases, D1 receives a request for a change in the state of the world (a structure was ordered in the player’s city, troops were sent, etc.). In this case, D1 sends a request to D2 (communication on sockets). More than one D1 can be connected to D2 (each of which has 8 threads, and therefore can simultaneously transmit 8 instructions to D2). D2 only executes one instruction at a time, the rest are waiting in line. The execution of one statement for the database is carried out as a single transaction. Upon successful execution of the instruction, changes are made to the database, if the instruction is invalid, the transaction is canceled and all its changes are rolled back. It is important to note that invalid instructions are cut off even on D1, but it happens that the instruction became invalid after transferring it to D2 (for example, in the previous instruction, the city spent resources, but in this one he is trying to order a building for non-existent resources, but both instructions came almost simultaneously). The whole system can work without D2, but only in read only mode - nothing can be changed, all event timers go on, but at the end they hang. If after that D2 is turned on, then the system will recover, as if there was no failure, all events will be processed in the correct order.

    Initially, the Apache web server was used. It was chosen because it is popular and has a build under Windows. D1 connected to Apache using ISAPI technology, i.e. as a dll library. Apache accepted requests and transferred them to the library connected to itself. Apache itself was quite slow. Therefore, at some point, the project was transferred to a bunch of nginx + FastCGI.

    The nginx web server is very convenient both in configuration and in use. Page rendering speed has increased. In addition, nginx very quickly distributes static content.
    How does FastCGI work? The engine itself from the dll library has been redone into a standalone application. The application accepts requests from the web server via sockets, processes them, generates pages and, through the same sockets, returns pages to the web server. At the same time, sockets remain open and new requests come in. Learn more about C ++ development using the FastCGI protocol here .

    Now about hosting.
    Prior to launching the project, everything worked on a regular home computer over a regular home cable Internet.
    At that time there was no financial opportunity to rent a server in a data center, so the first game world was launched all on the same home computer. This created a number of inconveniences: access to the network is unstable, sometimes the lights in the house turn off, the provider does not issue traffic at the declared tariff and often carries out technical work. The project began to fill up with players, the load grew. One more computer with a different Internet connection was delivered to the service of the 1st world. Now one world easily works on the 1st server, but then not everything was optimized, and the computers used were weak.

    The next world was soon discovered. Two more computers and two network connections were used on it. Already preparing to launch world 3. All of these computers were located in my home in the living room. As the number of servers grew, so did the problems. I could no longer leave the house, because something was constantly falling. In addition to force majeure problems, there were also regular bugs. If any abnormal situation occurs, the application immediately crashes on the assert, not trying to somehow get out of this situation. This solution was specially chosen. This made me always first of all fight in bugs, and not drag them through the entire development period.
    The project began to generate income, and a server was rented in the data center. The site of the game and both worlds were transferred to it. Administering the system has become much easier, but costs have increased. The third world was also launched on this server, but after a DDoS attack, I transferred it to a separate server so that the first two worlds were out of danger.

    Development was conducted and tested on Windows. But the code was written right away without being tied to this OS, and, in the future, it took only one day to fix the code and compile the project under FreeBSD.
    To work with streams, the POSIX library was selected. To create image files, I used the FreeImage library .

    System monitoring.
    Initially, system monitoring was using monitors! Any “crash” of the server could be detected as a window with an error or lack of outgoing traffic on the chart. Even at night I had to wake up several times and look around at all the monitors.
    This could not continue for so long and a special php script was written that constantly polls the servers, collects status data from them and, if necessary, sends an e-mail or SMS message to the phone. This script was launched on a free hosting, where it still works today. Thanks to him, it is always possible to quickly find out about problems and, if possible, eliminate them immediately.

    In the following articles I will talk about the development of the project from idea to release, about technical solutions in the engine and formats for storing data in the database, about backing up data and protecting against attacks, about the mechanism for creating pages.

    Basis D1:
    
    void* operateRequest(void* listen_socket)
    {
    	//инициализация
    	FCGX_Request request;
    	assert(!FCGX_InitRequest(&request, *(int*)listen_socket, 0));
    	Session* s = new Session;
    	//условно бесконечный цикл по приему запросов
    	while(FCGX_Accept_r(&request) == 0) {
    		stringstream out;
    		stringstream header;
    		header << "Content-type: text/html";
    		//чтение параметров запроса
    		string query;
    		string addr;
    		string referer;
    		string post;
    		string cookie;
    		string agent;
    		int content_lenght = 0;
    		for(char** envp = request.envp; *envp; ++envp) {
    			string v = *envp;
    			string::size_type e = v.find('=');
    			string p = v.substr(0, e);
    			if(p == "REQUEST_URI") query = v.substr(e + 2, v.length());
    			if(p == "REMOTE_ADDR") addr = v.substr(e + 1, v.length());
    			if(p == "HTTP_COOKIE") cookie = v.substr(e + 1, v.length());
    			if(p == "HTTP_REFERER") referer = v.substr(e + 1, v.length());
    			if(p == "CONTENT_LENGTH") content_lenght = toInt(v.substr(e + 1, v.length()));
    			if(p == "HTTP_USER_AGENT") agent = v.substr(e + 1, v.length());
    		}
    		//чтение тела запроса
    		maximize(content_lenght, 9999);
    		char p[10000];
    		FCGX_GetStr(p, content_lenght, request.in);
    		p[content_lenght] = 0;
    		post = p;
    		//основная функция. Генерирует header и страницу
    		s->work(header, out, addr, cookie, referer, query, post);
    		//сборка и возвращение страницы
    		header << "\r\n\r\n" << out.str();
    		FCGX_PutStr(header.str().c_str(), int(header.str().length()), request.out);
    		FCGX_Finish_r(&request);
    	}
    	return 0;
    }
    int main() {
    	assert(initSocketSystem());
    	assert(!FCGX_Init());
    	int listen_socket = FCGX_OpenSocket(":8000", 400);
    	assert(listen_socket >= 0);
    	//создать потоки
    	for(int i = 0; i < threads; ++i) {
    		pthread_t thread;
    		assert(pthread_create(&thread, 0, operateRequest, (void*)&listen_socket) == 0);
    	}
    	while(true) sleep(1000);
    	return 0;
    }
    


    Basis D2:
    
    void operateCommand(asComType com, Socket& sock)
    {
    	//критическая секция. В ней работает одновременно только один поток
    	pthread_mutex_lock(&ascs);
    	bool res;
    	//сообщить бд о начале транзакции
    	assert(sql.put("BEGIN"));
    	switch(com) {
    		case ASC_TOWNUPDATE:
    		{
    			//инструкция на простое обновление состояния города
    			int id = sock.readVal();
    			res = asUpdateTown(id);
    		}
    		break;
    		//другие инструкции
    		//…
    		//…
    		//…
    	}
    	//в зависимости от успешности выполнения инструкции заканчиваем или отменяем транзакцию
    	assert(sql.put(res ? "COMMIT" : "ROLLBACK"));
    	//возвращаем результат
    	sock.sendVal(res);
    	//критическая секция заканчивается
    	pthread_mutex_unlock(&ascs);
    }
    void* clientThread(void* client_socket)
    {
    	Socket& sock = *(Socket*)client_socket;
    	asComType com;
    	int bytes;
    	//висеть в ожидании запроса
    	while((bytes = sock.readVal(com)) && bytes >= 0) {
    		//обработать запрос
    		operateCommand(com, sock);
    	}
    	delete &sock;
    	return 0;
    }
    int main()
    {
    	while(Socket* client = sock.listen()) {
    		//создать поток для чтения и обработки данных сокета
    		pthread_t thread;
    		assert(pthread_create(&thread, 0, clientThread, (void*)client) == 0);
    	}
    	return 0;
    }
    

    All of the code presented in some places is specially simplified for clarity. Some classes and functions are omitted.

    Also popular now: