Cocos2d-x for Android: faster file reading
One of my recent projects is porting the game from iOS to Android. The game is written using Cocos2d-x , a fairly popular cross-platform game engine.
I want to tell you how you can easily increase the download speed of a big game many times, as well as improve the overall smoothness of the reaction to user actions.
The game on iOS uses version 0.13, so for Android they took the same one (including the latest fixes from the official repository). Everything went fine in general, without big disappointments and significant bugs found in the engine, the version of the toy for poor graphics (the old iPhone 480x320) worked fine. It's time to add the “HD” graphics mode (960x640).
On iOS, the first branch of cocos2d supports other modes by automatically using a file with the suffix of the current mode, if such a file exists, otherwise, a file without such a suffix. Suffixes are usually ' -hd ' for iPhone HD and iPad SD modes, as well as' -ipadhd'for iPad HD. So, when you try to download the file 'image.png', the file 'image-hd.png' may actually load. For the Android version, this was not - added independently. The graphics got better.
The toy began to load longer, and it was much worse to respond to user actions (transitions to the menu, for example).
At first I decided that since every time a file with a suffix is checked for existence, then a simple cache in the form of std :: unordered_map is enough- to check the file for existence only 1 time - partially helped, but the situation remained very difficult. It was surprising that there was no such problem for the port of the same game on Win8 (and the version for Win8 supported everything, including the iPad HD mode - with textures for the screen 2048x1536).
He began to understand further.
CCFileUtils :: getFileData - the method responsible for reading files in Cocos2d-x (on github - and hereinafter links there too) - to download from the APK, it calls CCFileUtils :: getFileDataFromZip :
That is, every time for any file from the APK - there is a separate opening / closing archive. The simplest action is to open it at least once, decide to check if it is fixed in 2.0.x versions of Cocos2d-x - I find that it is not only not fixed, but also worsened - due to the changed file logic for different graphics (already added for Android version) - a double attempt to read from the APK file is possible .
Okay, I’m looking further - calling unzLocateFile. What does this function do?
That is, every time - a linear search in all files in the archive. Everytime. It’s deeper in code - with saving memory, so every now and then - moving around the file with the archive and reading. In the APK with a toy - more than 1300 files with resources.
And for gles20, the version of cocos2d-x is generally the worst case when the file has not been read, and you need to search for the second one again.
Good - I decided to somehow cache at least a list of files, and it is desirable to have its position in the archive so that unzLocateFile works faster. I start digging further - and with surprise I find that everything is already there, it is enough to start using:
So, in general, it’s elementary to generate a list of files with positions 1 time, and even further - quietly directly read what is required.
Testing the idea showed a 5-fold acceleration of loading (moreover, versions with heavy graphics compared to the old version with simplified graphics) and an overall improvement in the game (since there are a lot of graphics, it is impossible to store everything in memory, and you have to constantly load something) .
Later it turned out that it is not required to separately call unzGetCurrentFileInfo after unzGoToFirstFile / unzGoToNextFile, they still read the same information.
So in the end - a helper class has been created that reads the entire list of files from ZIP in about the same time as searching for one existing original unzLocateFile file.
Pull requestfor the latest version of Cocos2d-x, for Android - it definitely works. The version for 0.13 is obtained by a minor edit.
Use on health.
Why is this possible? Everything in the source code is actually ready, the search for a solution took literally a couple of hours. Nevertheless, the problem existed and was solved by various methods, mainly - “fewer files in the APK”.
Then I already found an offer 3 months ago to use Asset Manager - it will be necessary to look at what is there and how, and whether there will be a performance increase (or already a drop).
I think the same thing has been repeatedly solved in various companies, just no one returned the corrections back.
The used parts of MiniZip in Cocos2d-x - in fact, simply demonstrate how to do it, and were never intended to read random files constantly.
So in spite of two years, in the public version there are still many places where it is elementary to improve something. Performance - yes, the same classes from CCProfiling.h / .cpp, designed to profile the engine, where something average is calculated by adding the old average and the new value, and further dividing in half . So the results for, for example, the same values of 10 4 1 and 1 4 10 are different (4 and 6.25, respectively). Exactly the same problem is in the source code for cocos2d-iphone, so I’ll do a general correction a bit later - if it doesn’t interest anyone else.
I hope there will be time, and other corrections will also be possible to add to the official version, with a transfer from 0.13 to the current 2.x.
Learn more about Cocos2d-x for Android
Cocos2d-x has been in existence for Android for almost two years now, a fairly respectable age. Open source, MIT license (no change required).
The latest stable release for OpenGL ES 1.x is 0.13.0, released in March this year.
The first release for OpenGL ES 2.0 - 2.0.2, appeared in late August.
The latest stable release for OpenGL ES 1.x is 0.13.0, released in March this year.
The first release for OpenGL ES 2.0 - 2.0.2, appeared in late August.
I want to tell you how you can easily increase the download speed of a big game many times, as well as improve the overall smoothness of the reaction to user actions.
Calm work
The game on iOS uses version 0.13, so for Android they took the same one (including the latest fixes from the official repository). Everything went fine in general, without big disappointments and significant bugs found in the engine, the version of the toy for poor graphics (the old iPhone 480x320) worked fine. It's time to add the “HD” graphics mode (960x640).
On iOS, the first branch of cocos2d supports other modes by automatically using a file with the suffix of the current mode, if such a file exists, otherwise, a file without such a suffix. Suffixes are usually ' -hd ' for iPhone HD and iPad SD modes, as well as' -ipadhd'for iPad HD. So, when you try to download the file 'image.png', the file 'image-hd.png' may actually load. For the Android version, this was not - added independently. The graphics got better.
Problem
The toy began to load longer, and it was much worse to respond to user actions (transitions to the menu, for example).
At first I decided that since every time a file with a suffix is checked for existence, then a simple cache in the form of std :: unordered_map is enough
He began to understand further.
CCFileUtils :: getFileData - the method responsible for reading files in Cocos2d-x (on github - and hereinafter links there too) - to download from the APK, it calls CCFileUtils :: getFileDataFromZip :
Source code
unsigned char* CCFileUtils::getFileDataFromZip(const char* pszZipFilePath, const char* pszFileName, unsigned long * pSize)
{
unsigned char * pBuffer = NULL;
unzFile pFile = NULL;
*pSize = 0;
do
{
CC_BREAK_IF(!pszZipFilePath || !pszFileName);
CC_BREAK_IF(strlen(pszZipFilePath) == 0);
pFile = unzOpen(pszZipFilePath);
CC_BREAK_IF(!pFile);
int nRet = unzLocateFile(pFile, pszFileName, 1);
CC_BREAK_IF(UNZ_OK != nRet);
char szFilePathA[260];
unz_file_info FileInfo;
nRet = unzGetCurrentFileInfo(pFile, &FileInfo, szFilePathA, sizeof(szFilePathA), NULL, 0, NULL, 0);
CC_BREAK_IF(UNZ_OK != nRet);
nRet = unzOpenCurrentFile(pFile);
CC_BREAK_IF(UNZ_OK != nRet);
pBuffer = new unsigned char[FileInfo.uncompressed_size];
int nSize = 0;
nSize = unzReadCurrentFile(pFile, pBuffer, FileInfo.uncompressed_size);
CCAssert(nSize == 0 || nSize == (int)FileInfo.uncompressed_size, "the file size is wrong");
*pSize = FileInfo.uncompressed_size;
unzCloseCurrentFile(pFile);
} while (0);
if (pFile)
{
unzClose(pFile);
}
return pBuffer;
}
That is, every time for any file from the APK - there is a separate opening / closing archive. The simplest action is to open it at least once, decide to check if it is fixed in 2.0.x versions of Cocos2d-x - I find that it is not only not fixed, but also worsened - due to the changed file logic for different graphics (already added for Android version) - a double attempt to read from the APK file is possible .
Okay, I’m looking further - calling unzLocateFile. What does this function do?
err = unzGoToFirstFile(file);
while (err == UNZ_OK)
{
char szCurrentFileName[UNZ_MAXFILENAMEINZIP+1];
err = unzGetCurrentFileInfo64(file,NULL, szCurrentFileName,sizeof(szCurrentFileName)-1,NULL,0,NULL,0);
if (err == UNZ_OK)
{
if (unzStringFileNameCompare(szCurrentFileName, szFileName,iCaseSensitivity)==0)
return UNZ_OK;
err = unzGoToNextFile(file);
}
}
That is, every time - a linear search in all files in the archive. Everytime. It’s deeper in code - with saving memory, so every now and then - moving around the file with the archive and reading. In the APK with a toy - more than 1300 files with resources.
And for gles20, the version of cocos2d-x is generally the worst case when the file has not been read, and you need to search for the second one again.
Decision
Good - I decided to somehow cache at least a list of files, and it is desirable to have its position in the archive so that unzLocateFile works faster. I start digging further - and with surprise I find that everything is already there, it is enough to start using:
/* unz_file_info contain information about a file in the zipfile */
typedef struct unz_file_pos_s
{
uLong pos_in_zip_directory; /* offset in zip file directory */
uLong num_of_file; /* # of file */
} unz_file_pos;
int ZEXPORT unzGetFilePos(unzFile file, unz_file_pos* file_pos);
int ZEXPORT unzGoToFilePos(unzFile file, unz_file_pos* file_pos);
So, in general, it’s elementary to generate a list of files with positions 1 time, and even further - quietly directly read what is required.
Testing the idea showed a 5-fold acceleration of loading (moreover, versions with heavy graphics compared to the old version with simplified graphics) and an overall improvement in the game (since there are a lot of graphics, it is impossible to store everything in memory, and you have to constantly load something) .
Later it turned out that it is not required to separately call unzGetCurrentFileInfo after unzGoToFirstFile / unzGoToNextFile, they still read the same information.
So in the end - a helper class has been created that reads the entire list of files from ZIP in about the same time as searching for one existing original unzLocateFile file.
Pull requestfor the latest version of Cocos2d-x, for Android - it definitely works. The version for 0.13 is obtained by a minor edit.
Use on health.
conclusions
Why is this possible? Everything in the source code is actually ready, the search for a solution took literally a couple of hours. Nevertheless, the problem existed and was solved by various methods, mainly - “fewer files in the APK”.
Then I already found an offer 3 months ago to use Asset Manager - it will be necessary to look at what is there and how, and whether there will be a performance increase (or already a drop).
I think the same thing has been repeatedly solved in various companies, just no one returned the corrections back.
The used parts of MiniZip in Cocos2d-x - in fact, simply demonstrate how to do it, and were never intended to read random files constantly.
So in spite of two years, in the public version there are still many places where it is elementary to improve something. Performance - yes, the same classes from CCProfiling.h / .cpp, designed to profile the engine, where something average is calculated by adding the old average and the new value, and further dividing in half . So the results for, for example, the same values of 10 4 1 and 1 4 10 are different (4 and 6.25, respectively). Exactly the same problem is in the source code for cocos2d-iphone, so I’ll do a general correction a bit later - if it doesn’t interest anyone else.
I hope there will be time, and other corrections will also be possible to add to the official version, with a transfer from 0.13 to the current 2.x.