Configuring and Using Apache Ignite as MyBatis L2 Cache / AT Consulting Blog

In this article, I will talk about how to configure apache ignite as the 2nd level cache for MyBatis and see the cache entry in Apache Ignite. What is Apache Ignite? This is a distributed, high-performance platform for in-memory computing with the main characteristics:

• with distributed storage of objects In-memory data grid, implementation of JSR 107 (Jcache)
• with distributed computing in RAM
• with distributed messaging and events
• with an in-memory accelerator for Hadoop and Spark.

Why Apache Ignite? We used EhCache and Oracle Coherence for a long time, after which we switched to HazelCast because of its ease of use. In recent versions, HazelCast reduced performance in the open source version, and we were also interested in using it as a single platform for spark and Hadoop.

Why MyBatis? The choice between Hibernate and MyBatis is like choosing between the BMW and Mercedes brands. Native SQL support and the location of SQL scripts in one place (not scattered across all source codes) are very important for us, so that it is convenient to optimize SQL queries.

Apache Ignite recently announced support for MyBatis as a 2nd level cache, and we decided to test its functionality and performance. Any database operations are expensive, so one of the main tasks for increasing system performance is to reduce the number of database accesses: i.e. use cache.
The response time to a query can be calculated using a simple formula:

T = t _acq + t _req + t _exec + t _res

where:

t _acq - connection acquisition time
t _req - time to send a request to the database
t _exec - query time to the database
t _res- time to receive a response from the database

For a well-optimized query, the minimum response time is from 20 to 150 ms.

MyBatis tech supports 2 cache levels by default:

• caching in local session Local cache (enabled by default)
• second level cache 2 ^nd level

By default, MyBatis only uses first level caching (L1 Cache), that is, objects cached in one session are not available for another. However, the global 2nd level can also be used: in it, cached objects will be available for all sessions. This usually improves performance because each new session uses data from the L2 cache.

MyBatis 2 ^nd level cache stores data or information about objects (entitiy data), and not its own object as in hibernate. The data in the cache is stored in the format 'Serialized' - a hash table, where the key is the identifier of the entity, and the values ​​are a list of parameter values.

In the example below, you will see the cache of entries in apache ignite for MyBatis 2 ^nd level cache.



Where:

cache key:[idHash=1499858, hash=2019660929, checksum=800710994, count=6, multiplier=37,hashcode=2019660929, updateList=[com.blu.ignite.mapper.UserMapper.getUserObject, 0, 2147483647, SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name=?, USERS, SqlSessionFactoryBean]]
Value class: java.util.ArrayList
Cache value:[UserObject [idHash=243119413, hash=1658511469, owner=C##DONOTDELETE, object_type=TABLE, object_id=94087, created=Mon Feb 15 13:59:41 MSK 2016, object_name=USERS]]

In our case, the key is an SQL query."SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name=?"

As an example, I took the system table 'all_objects' from the Oracle DBMS and the following queries:

QUERY_1: SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name='EMP';
QUERY_2: SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE';
QUERY_3: SELECT count(*) FROM all_objects;

SandBox Features:

Apache ignite cluster	2 virtual machines (VM Ware)	CPU: 2 RAM : 4 GV Java HEAP : 2 GV OS : Red Hat Santigo JVM : Oracle JVM 1.7_45
Oracle 12c	virtual machine (VM Ware)	CPU : 4 RAM : 8 GV OS : Red Hat Santigo
Standalone java app + SoapUI	Macbook pro	CPU : 4 RAM : 16 GV JVM : Oracle JVM 1.7_45

If you execute the above SQL queries (QUERY1-3) through SQL Developer, we get the following response time:

No.	Name of request	Response Time (mc)
1	QUERY_1: SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name='EMP';	~ 660
2	QUERY_2: SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE';	~ 378
3	QUERY_3: SELECT count(*) FROM all_objects;	~ 700

Now add apache ignite as a level 2 cache and look at the result. You can find instructions on installing apache ignite on my blog , and all source codes on github .

Add mybatis-ignite library in the Maven project:

org.mybatis.cachesmybatis-ignite1.0.0-beta1

Add MyBatis sql mapper

      SELECT * FROM all_objects t where t.OBJECT_TYPE='TABLE' and t.object_name=#{objectName}

      SELECT t.object_type, count(*) as cnt FROM all_objects t group by t.OBJECT_TYPE
 

      SELECT count(*) FROM all_objects

Here we:

• specify the cache adapter on the IgniteCacheAdapter
• for each SQL query, specify useCache = "true", that is, enable caching mode.

Add ignite spring configurations

        
                 
                      
                
                      
               
               
            
        
    
    
        
            
                    
                        
                            IP_ADDRESS_IGNITE_NODE
                            IP_ADDRESS_IGNITE_NODE
                        
                    
                
            
        
    

Pay particular attention to setting 'clientMode' with false. It allows you to connect cacheMode = Partitioned, where we use the Partitioned cache to share data between cache nodes. Another option is to enable the Replicated mode, using which data is replicated between all caching nodes.

statisticsEnabled = true, allows you to get cache usage statistics: hit count, etc.

writeSynchronizationMode= FULL_SYNC, allows you to fully synchronize cached data with backup nodes.

Add the appropriate Java interface:

public interface UserMapper {
       User getUser( String id);
List getUniqueJob();
UserObject getUserObject(String objectName);
String allObjectCount();
List getAllObjectsTypeByGroup();
}

As well as a simple soap service

@WebService(name = "IgniteTestServices",
        serviceName=" IgniteTestServices ",
        targetNamespace = "http://com.blu.rules/services")
public class WebServices {
    private UserServices userServices;
    @WebMethod(operationName = "getUserName")
    public String getUserName(String userId){
        User user = userServices.getUser(userId);
        return user.getuName();
    }
    @WebMethod(operationName = "getUserObject")
    public UserObject getUserObject(String objectName){
        return userServices.getUserObject(objectName);
    }
    @WebMethod(operationName = "getUniqueJobs")
    public List getUniqueJobs(){
        return userServices.getUniqueJobs();
    }
    @WebMethod(exclude = true)
    public void setDao(UserServices userServices){
        this.userServices = userServices;
    }
    @WebMethod(operationName = "allObjectCount")
    public String allObjectCount(){
        return userServices.allObjectCount();
    }
    @WebMethod(operationName = "getAllObjectsTypeCntByGroup")
    public List getAllObjectsTypeCntByGroup(){
        return userServices.getAllObjectCntbyGroup();
    } 
}

After successfully compiling the project, if we call the web method 'getAllObjectsTypeCntByGroup', then through SoapUi the response time will increase to ~ 1600 ms in my case.



From the second time, the response time should decrease significantly, because the result is returned from the apache ignite cache, and the database request is not received.



Now, after the first time the web method is called, the response time will take from 5-6 ms.

In apache ignite, the cache entry will look like this:



cache key: [idHash=46158416, hash=1558187086, checksum=2921583030, count=5, multiplier=37, hashcode=1558187086, updateList=[com.blu.ignite.mapper.UserMapper.getAllObjectsTypeByGroup, 0, 2147483647, SELECT t.object_type, count(*) as cnt FROM all_objects t group by t.OBJECT_TYPE, SqlSessionFactoryBean]]
Value class: java.util.ArrayList
Cache value: [UobjectGroupBy [idHash=2103707742, hash=1378996400, cnt=1, object_type=EDITION], UobjectGroupBy [idHash=333378159, hash=872886462, cnt=444, object_type=INDEX PARTITION], UobjectGroupBy [idHash=756814918, hash=1462794064, cnt=32, object_type=TABLE SUBPARTITION], UobjectGroupBy [idHash=931078572, hash=953621437, cnt=2, object_type=CONSUMER GROUP], UobjectGroupBy [idHash=1778706917, hash=1681913927, cnt=256, object_type=SEQUENCE], UobjectGroupBy [idHash=246231872, hash=1764800190, cnt=519, object_type=TABLE PARTITION], UobjectGroupBy [idHash=1138665719, hash=1030673983, cnt=4, object_type=SCHEDULE], UobjectGroupBy [idHash=232948577, hash=1038362844, cnt=1, object_type=RULE], UobjectGroupBy [idHash=1080301817, hash=646054631, cnt=310, object_type=JAVA DATA], UobjectGroupBy [idHash=657724550, hash=1248576975, cnt=201, object_type=PROCEDURE], UobjectGroupBy [idHash=295410055, hash=33504659, cnt=54, object_type=OPERATOR], UobjectGroupBy [idHash=150727006, hash=499210168, cnt=2, object_type=DESTINATION], UobjectGroupBy [idHash=1865360077, hash=727903197, cnt=9, object_type=WINDOW], UobjectGroupBy [idHash=582342926, hash=1060308675, cnt=4, object_type=SCHEDULER GROUP], UobjectGroupBy [idHash=1968399647, hash=1205380883, cnt=1306, object_type=PACKAGE], UobjectGroupBy [idHash=1495061270, hash=1345537223, cnt=1245, object_type=PACKAGE BODY], UobjectGroupBy [idHash=1328790450, hash=1823695135, cnt=228, object_type=LIBRARY], UobjectGroupBy [idHash=1128429299, hash=1267824468, cnt=10, object_type=PROGRAM], UobjectGroupBy [idHash=760711193, hash=1240703242, cnt=17, object_type=RULE SET], UobjectGroupBy [idHash=317487814, hash=61657487, cnt=10, object_type=CONTEXT], UobjectGroupBy [idHash=1079028994, hash=1960895356, cnt=229, object_type=TYPE BODY], UobjectGroupBy [idHash=276147733, hash=873140579, cnt=44, object_type=XML SCHEMA], UobjectGroupBy [idHash=24378178, hash=1621363993, cnt=1014, object_type=JAVA RESOURCE], UobjectGroupBy [idHash=1891142624, hash=90282027, cnt=10, object_type=DIRECTORY], UobjectGroupBy [idHash=902107208, hash=1995006200, cnt=593, object_type=TRIGGER], UobjectGroupBy [idHash=142411235, hash=444983119, cnt=14, object_type=JOB CLASS], UobjectGroupBy [idHash=373966405, hash=1518992835, cnt=3494, object_type=INDEX], UobjectGroupBy [idHash=580466919, hash=1394644601, cnt=2422, object_type=TABLE], UobjectGroupBy [idHash=1061370796, hash=1861472837, cnt=37082, object_type=SYNONYM], UobjectGroupBy [idHash=1609659322, hash=1543110475, cnt=6487, object_type=VIEW], UobjectGroupBy [idHash=458063471, hash=1317758482, cnt=346, object_type=FUNCTION], UobjectGroupBy [idHash=1886921697, hash=424653540, cnt=7, object_type=INDEXTYPE], UobjectGroupBy [idHash=1455482905, hash=1776171634, cnt=30816, object_type=JAVA CLASS], UobjectGroupBy [idHash=49819096, hash=2110362533, cnt=2, object_type=JAVA SOURCE], UobjectGroupBy [idHash=1916179950, hash=1760023032, cnt=10, object_type=CLUSTER], UobjectGroupBy [idHash=1138808674, hash=215713426, cnt=2536, object_type=TYPE], UobjectGroupBy [idHash=305229607, hash=340664529, cnt=23, object_type=JOB], UobjectGroupBy [idHash=1365509716, hash=623631686, cnt=12, object_type=EVALUATION CONTEXT]]

Performance rating:


Although our tests are not an example of a correct performance calculation (we did not use the connection pool, nor did we optimize the SQL queries), they still help to calculate the performance gain using the usual formula:

Performance gain = Response time without caching / Response time with caching = 1589 / 6 which is approximately 265 times faster or performance gain = ((Response time without caching — Response time with caching) / Response time with caching * 100) is approximately 26,383% faster.

Thus, the cache of the 2nd level allows you to increase system performance by a hundred compared to the approach without using a cache.