Delphi: the fastest DataSet (TJvMemoryData, TMemTableEh, TdxMemData, TkbmMemTable)

In this article, I will conduct a comparative analysis of DataSets that hold data in RAM.

List of DataSets

1. TJvMemoryData
   Developer: JEDI Visual Component Library (JCL + JVCL) community
   JCL (version 2.8)
   JVCL (version 3.50)
   → Official site
2. TMemTableEh
   Developer: EhLib
   Version: 9.0.040
   → Official website
3. TdxMemData
   Developer: DevExpress
   Version: 15.2.2
   → Official site
4. TkbmMemTable
   Developers: Components4Developers
   Version: 7.74.00 Professional Edition
   → Official site

DataSet comparison options

1. Insert Records
2. Sort Records

Environment

Delphi	10.2 Tokyo Starter
operating system	Windows 7 SP1 Ultimate x64
CPU	Intel Core i5
RAM	8 GB

Test data

DataSets will be tested on data received from the Firebird database. For comparative analysis, I created 100,000 records in the database with various types of data:

• integers;
• real numbers;
• dates;
• lines;
• Images.

Comparison

1. Data loading
   

   Dataset	Loadfromdataset	Manual	Average
   TJvMemoryData	7.7846 sec	5.7500 sec	6.7673 sec
   TMemTableEh	4.5114 sec	7.2978 sec	5.9046 sec
   TdxMemData	6.3804 sec	6.5082 sec	6.4443 sec
   TkbmMemTable	5,4474 sec	6.0562 sec	5.7518 sec

   Downloads for each DataSet were performed 5 times, the arithmetic mean was calculated from the obtained values. The average is the arithmetic average of the load using the LoadFromDataSet method and the manual load value.
   
   Results:
   
   TMemTableEh - the fastest data loading using the method LoadFromDataSet
   TJvMemoryData - the fastest manual loading data
   TkbmMemTable - the fastest average data download
   
   
   Time source code

   var
     start_time, end_time, total_time: double;
   start_time := GetTickCount;
   ...
   end_time   := GetTickCount;
   total_time := (end_time - start_time) / 1000;

   
   
   Source code for loading data LoadFromDataSet

   //TJvMemoryData
   //function LoadFromDataSet(Source: TDataSet; RecordCount: Integer; Mode: TLoadMode; DisableAllControls: Boolean = True): Integer;
   JvMemoryData.LoadFromDataSet(FIBDataSet, -1, lmCopy);
   //TMemTableEh
   //function LoadFromDataSet(Source: TDataSet; RecordCount: Integer; Mode: TLoadMode; UseCachedUpdates: Boolean): Integer;
   MemTableEh.LoadFromDataSet(FIBDataSet, -1, lmCopy, true);
   //TdxMemData
   //procedure LoadFromDataSet(DataSet : TDataSet);
   dxMemData.LoadFromDataSet(FIBDataSet);
   //TkbmMemTable
   //procedure TkbmCustomMemTable.LoadFromDataSet(Source:TDataSet; CopyOptions:TkbmMemTableCopyTableOptions; Mapping:string='');
   kbmMemTable.LoadFromDataSet(FIBDataSet, [mtcpoAppend]);

   
   
   Manual Download Source

   while not FIBDataSet.Eof do
   begin
     table.Append;
     table.FieldByName('ID').AsInteger     := FIBDataSet.FieldByName('ID').AsInteger;
     table.FieldByName('SUMM').AsFloat     := FIBDataSet.FieldByName('SUMM').AsFloat;
     table.FieldByName('COMMENT').AsString := FIBDataSet.FieldByName('COMMENT').AsString;
     ...
     table.Post;
     FIBDataSet.Next;
   end;
   table.First;

   
   TJvMemoryData is the only DataSet in which manual data loading is faster than loading using the LoadFromDataSet method.
   
   TdxMemData is the only DataSet that, after loading data using the LoadFromDataSet method, does not return the position in the DataSet to the first record.
2. Sorting
   

   Dataset	Integer	Real number	Line	Average
   TJvMemoryData	0.3492 sec	0.8330 sec	1,9938 sec	1.0587 sec
   TMemTableEh	0.9014 sec	0.8642 sec	3.6876 sec	1.8177 sec
   TdxMemData	0.3616 sec	0.3650 sec	0.9134 sec	0.5467 sec
   TkbmMemTable	0.1996 sec	0.2186 sec	0.7550 sec	0.3897 sec

   Sorting for each DataSet was performed 5 times, from the obtained values ​​the arithmetic mean was calculated. The average value is the arithmetic mean of the sort values.
   
   Results:
   
   TkbmMemTable - the fastest sorting of integers
   TkbmMemTable - the fastest sorting of real numbers
   TkbmMemTable - the fastest sorting of strings
   TkbmMemTable - the fastest sorting
   
   
   Time source code

   var
     start_time, end_time, total_time: double;
   start_time := GetTickCount;
   ...
   end_time   := GetTickCount;
   total_time := (end_time - start_time) / 1000;

   
   
   Source code for sorting data

   //TJvMemoryData
   //procedure SortOnFields(const FieldNames: string = ''; CaseInsensitive: Boolean = True; Descending: Boolean = False);
   JvMemoryData.SortOnFields(fields, false, false);
   //TMemTableEh
   //procedure SortByFields(const SortByStr: string);
   MemTableEh.SortByFields(fields);
   //TdxMemData
   dxMemData.SortedField := fields;
   //TkbmMemTable
   //procedure SortOn(const FieldNames:string; Options:TkbmMemTableCompareOptions);
   kbmMemTable.SortOn(fields, []);

   
   

I note that only for the DataSet TMemTableEh when sorting by several fields, you can set a different sort direction for each field (ascending / descending).

Conclusion

The DataSet TkbmMemTable turned out to be the undisputed leader , but all other DataSets showed good results. But you can use TkbmMemTable only with Delphi XE2.

Of course, on real data, it is unlikely to need to load 100,000 records for display to the user. Also, when choosing a DataSet, you need to consider that on a different data set (for example, when there is no image loading or for an x64 application), the results of the speed of operation may turn out to be different. In addition, many DataSets have additional functionality for working with a grid from their library.

More on TMemTableEh

The TMemTableEh component appeared in version EhLib 4.0. All library components, including TMemTableEh, work starting with Delphi 7.

The advantages of using MemTableEh in comparison with other DataSets are as follows:

1. It supports a special interface that allows the DBGridEh component to view all data without moving the active record.
2. Allows you to upload data from a TDataDriverEh object (DataDriver property).
3. Allows you to upload changes back to the DataDriver, promptly or postponed (depending on the CachedUpdates properties).
4. Allows you to create a master / child relationship on the client (filtering records) or on an external source (updating [Params] parameters and re-querying data from DetailDataDriver).
5. Allows you to sort data, including Calculated and Lookup fields.
6. Allows you to create and fill data in design-time and save the data in the form dfm-file.
7. Allows you to store entries in the form of a tree. Each record can have records nodes / branches and itself can be a node of another parent record. The TDBGridEh component supports the display functionality of the tree structure of these records.
8. Allows you to connect to the internal array of another TMemTableEh component (ExternalMemData property) and work with its data: sort, filter, edit.
9. It has an interface for obtaining a list of all column values, ignoring the local DataSet filter. TDBGridEh uses this property to automatically generate a list in the DropDownBox drop-down filter.

About TRxMemoryData

Developers: RxLib
Component TRxMemoryData appeared in version RxLib 2.60.

DataSet TRxMemoryData is not involved in the comparison, because in 2002 the RxLib library was officially included in the JVCL. JVCL has a utility for quickly replacing all Rx components, functions, and units with the JVCL version.

Why you should upgrade to JVCL:

Unlike RxLib, JVCL is evolving. Errors are fixed. Regularly release versions with support for new versions of Delphi. JVCL components support new versions of Windows and Windows styles.

The procedure for using DataSets in my practice

1. TRxMemoryData
2. TMemTableEh
3. TRxMemoryData
4. TJvMemoryData
5. TkbmMemTable