Rewriting from java to C ++ on the Android platform

I want to share with you the experience of rewriting from java to C ++ on the Android platform and what happened as a result.

For your small home project, the Viola-Jones face search algorithm was used, the java source code for the model was taken from here code.google.com/p/jviolajones with a slight modification - two classes were added: Point and Rectangle. I’ll clarify why I didn’t use OpenCV for Android - I need to install a separate library application for it, which in my case will be very inconvenient, and the experiments showed it crashed without warning, I didn’t figure it out for a long time, as well as searching for other libraries, and decided to take the simplest finished implementation.

The speed of the algorithm showed disastrous results, in a 400 by 300 photo on my old broken GT-I9300I - 54 seconds, on avd (virtual device) and even longer - 250 seconds.

Often came under my gaze discussion of the performance of code in java and C ++, somewhere it was shown that java is behind, in some cases, and vice versa, small sections of code with one loop were cited. Here, the algorithm is a little more than more complicated, of the order of 6 nested loops, as you can see from the source. Therefore, the decision was made - to try on our own experience rewriting in C ++. For all the articles I read, I got the impression that the speed would increase by a maximum of 20 percent, but as it turned out it was wrong.

Naturally, the following tasks arose - how to transfer input and receive output data and how to rewrite the code. Detector decided to leave the filling of the model from xml in the constructor to java, it fills up, of course, not quickly, but since working with xml in C ++ sounds very scary for me, I left it as it is. My occupation is related to java, I only contacted C \ C ++ at the institute and a little at work on old projects. Therefore, I had to study a little documentation, read articles and fill a few cones.

Rewriting logic. There were no special problems, a method was adopted - without looking to copy the classes, where the eclipse highlighted with a red hatchet. I converted all ArrayLists into an array, fortunately, they did not change the size.

I will not describe the environment settings for calling native code, there are a lot of articles on this topic.

How to transfer data. With simple types - int, float, boolean, everything is simple and clear. With one-dimensional, it seems to be also simple:

JNIEXPORT jint JNICALL Java_com_example_Computations_intFromJni(JNIEnv* env, jobject thiz, jintArray arr) {
	jsize d = env->GetArrayLength(arr);
	jboolean j;
	int * p = env->GetIntArrayElements(arr, &j);
...
}

With two-dimensional a little more complicated:

JNIEXPORT jint JNICALL Java_com_example_Computations_findFaces(JNIEnv* env, jobject thiz, jobjectArray image) {
	int width = env -> GetArrayLength(image);
	jboolean j2;
	jintArray dim=  (jintArray)env->GetObjectArrayElement(image, 0);
	int height = env -> GetArrayLength(dim);
	int **imageLocal;
	imageLocal = new int*[width];
	for (int i = 0; i < width; i++) {
		jintArray oneDim= (jintArray)env->GetObjectArrayElement(image, i);
		int *element = env->GetIntArrayElements(oneDim, &j2);
		imageLocal[i] = new int[height];
		for(int j=0; j < height; ++j) {
			imageLocal[i][j]= element[j];
		}
	}
...
}

Let's go further, how to pass objects that have a bunch of fields, among which there are List types. To obtain an object field, the following construction is applied:

jclass clsDetector = env->GetObjectClass(objDetector);
jfieldID sizeFieldId = env->GetFieldID(clsDetector, "size", "Ldetection/Point;");
jobject pointObj = env->GetObjectField(objDetector, sizeFieldId);

For sheets, we need two get and size methods:

	jfieldID stagesFieldId = env->GetFieldID(clsDetector, "stages", "Ljava/util/List;");
	jobject stagesList = env->GetObjectField(detectorJObj, stagesFieldId);
	jclass listClass = env->FindClass( "java/util/List" );
	jmethodID getMethodIDList = env->GetMethodID( listClass, "get", "(I)Ljava/lang/Object;" );
	jmethodID sizeMethodIDList = env->GetMethodID( listClass, "size", "()I" );
	int listStagesCount = (int)env->CallIntMethod( stagesList, sizeMethodIDList );
	for( int i=0; i < listStagesCount; ++i )
	{
		jobject stage = env->CallObjectMethod( stagesList, getMethodIDList, i);
...

Learned how to get data. We start, falls on an error - Local reference table overflow 512 entries. It turns out that you need to clean all the local jclass and jobject links, this is done like this:

	env->DeleteLocalRef(jcls);
	env->DeleteLocalRef(jobj);

And for arrays too:

	env->ReleaseIntArrayElements(oneDim, element, JNI_ABORT);
	env->DeleteLocalRef(oneDim);

We return the result. To simplify my task, I returned the result as an array of Rectangle:

	jclass cls = env->FindClass("detection/Rectangle");
	jobjectArray jobAr =env->NewObjectArray(faces->currIndex, cls, NULL);
	jmethodID constructor = env->GetMethodID(cls, "", "(IIII)V");
	for (int i = 0; i < faces->currIndex; i++) {
		Rectangle* re = faces->rects[i];
		jobject object = env->NewObject(cls, constructor, re->x, re->y, re->width, re->height);
		env->SetObjectArrayElement(jobAr, i, object);
	}
	return jobAr;

So, the solemn moment - the search in the same photo - 14 seconds, i.e. 4 times faster, in other photos similar results. On a virtual android, 132 seconds vs 300 seconds. But as we know, it is impossible to use the results of one experiment, it is necessary to repeat several times, I will give for one photograph, the processing time in seconds.
Virtual deviceVirtual device using cppMy galaxy phoneMy galaxy phone with cpp
2381328414
3181375414
4721355414
2641505414
2661385414
2621295314

And in conclusion, I note. Despite the fact that rewriting has given great acceleration, there is still a lot of perfection limit, multithreading can be used, which I plan to study in the near future. And making any adjustments to the algorithm is probably the most difficult part.

update I posted the sources . Use as follows:
	Detector detector = Detector.create(inputHaas);
	List res = detector.getFaces(background_image, 1.2f, 1.1f, .05f, 2, true, useCpp);

inputHaas - stream of the model, i.e. haarcascade_frontalface_default.xml file from the original algorithm, useCpp - use C ++ or not. In C ++ sources I do not free memory, because wrote in haste.

Also popular now: