Databases in MIDP Part 1: The Record Management System Concept

Original author: Eric Giguere
  • Transfer
One of the key components of MIDP is the Record Management System (RMS). This is an API that provides the ability to store data locally in the device’s memory. For most MIDP-compatible phones, this is the only way to store data - only a small number of devices support access to the regular file system. It is easy to guess that a complete understanding of the RMS mechanism is necessary for writing any application that requires local data storage.

This is the first article in a series in which the most common problems regarding the use of RMS in applications, for example, interaction with external data sources, such as relational databases, will be considered. To get started, we will find out what RMS can offer us and write some simple debuggers.

Key concepts


Posts

From the name it’s clear that RMS is a system for managing records. A record is a data item. RMS does not impose any restrictions on the contents of a record; it can contain a number, a string, an array, an image — anything that can be represented as a sequence of bytes. If you can encode the available data in a binary format (and decode it back), then you can save it in a record, unless, of course, they fit into the size limit imposed by the system.

Many newcomers to RMS are puzzled by the notion of recording. They ask: “where are the fields?”, Wondering how the system divides individual records into separate data sequences. The answer is simple: the RMS record does not contain any fields.More precisely, the record contains one binary field of arbitrary length. The function of interpreting the contents of the record rests entirely with the application. RMS provides storage and a unique identifier, nothing more. This creates difficulties for applications, but keeps RMS simple and flexible, which is quite important for the MIDP subsystem.

At the API level, records are simply arrays of bytes.

Record stores

A record store is an ordered collection of records. Each record belongs to the repository and is accessible only through it. The storage guarantees atomic reading and writing of data, preventing their damage.

When a record is created, the store assigns it a unique integer identifier (record ID). The first record gets id 1, the second - 2, etc. This is not an index: when a record is deleted, the remaining elements are not renumbered.

The name is used to identify the repository inside the midlet. The name can contain from 1 to 32 unicode characters and must be unique inside the MIDlet that created the repository. In MIDP 1.0, storage cannot be used by more than one application. MIDP 2.0 optionally allows this, in which case the repository is identified not only by the name, but also by the name and manufacturer of the application that created this repository.

In addition, the repository contains information on the date of the last change and version. Applications can also bind to the repository a data change event handler in it.

At the API level, the repository is represented by an instance of the javax.microedition.rms.RecordStore class. All RMS classes and interfaces are defined in the javax.microedition.rms package.

Aspects of RMS


Data size limits

The amount of memory available for storing records differs on different devices. The MIDP specification requires at least 8 kB of memory to be backed up for permanent data storage. Moreover, the size of one record is not limited. RMS provides methods for determining record size, total storage size, and free memory size. Remember that read-only memory is common to all applications; use it sparingly.

Any MIDlet using RMS should indicate the minimum storage size in bytes required for its operation. To do this, the MIDlet-Data-Size attribute must be set in the manifest of the jar file and in the jad file. Do not indicate too much value there - some devices may prohibit the installation of the application if there is not enough free space. In practice, most devices allow applications to go beyond the starting data size.

Note that some MIDP implementations require you to specify additional attributes related to the required memory size. This should be indicated in the documentation for the device.

Work speed

Operations with permanent memory are usually slower than with random access. In particular, on some platforms, data recording can take a long time. To increase performance, use caching of frequently used data in RAM. In order not to slow down the response of the user interface, do not perform operations with RMS in the MIDlet event processing flow.

Thread Safety

RMS operations are thread-safe. Nevertheless, flows need to be coordinated among themselves, as when working with any shared resource. This also applies to running midlets that use the same storage.

Exceptions

In general, RMS API methods throw a few exceptions (in addition to standard exceptions like java.lang.IllegalArgumentException). These exceptions are defined in the javax.microedition.rms package:
  • InvalidRecordIDException - The operation could not be completed because the wrong record id was passed.
  • RecordStoreFullException - the available memory has run out.
  • RecordStoreNotFoundException - The specified store does not exist.
  • RecordStoreNotOpenException - The application is trying to use storage that has been closed.
  • RecordStoreException - a superclass of previous exceptions, is also thrown for common errors that are not covered by them.

Note that for brevity we will neglect the exception handling in some examples (due to its simplicity).

Using RMS


We will devote the rest of the article to basic recording operations through the RMS API. Some of them are presented in the RMSAnalyzer class, designed for storage analysis. You can use it as a debugging tool in your projects.

Storage Search

A list of storages can be obtained using RecordStore.listRecordStores (). This static method returns an array of strings, each of which is the name of the repository belonging to the MIDlet. If no repositories are created, null is returned.

The RMSAnalyzer.analyzeAll () method uses listRecordStores () to call analyze () for each store:
public void analyzeAll() {
    String[] names = RecordStore.listRecordStores();
    for( int i = 0;
         names != null && i < names.length;
         ++i ) {
        analyze( names[i] );
    }
}

Please note that the array contains the names of the storages created by our midlet only. The MIDP specification does not contain any way to get a list of all the repositories of the other midlets. MIDP 1.0 does not allow access to other people's repositories. In MIDP 2.0, an application can mark a repository as shareable, but other MIDlets can only use it if they know its name.

Opening and closing storage

RecordStore.openRecordStore () is used to open (and sometimes create) storage. This static method returns an instance of the RecordStore object, as can be seen from this version of RMSAnalyzer.analyze ():
public void analyze( String rsName ) {
    RecordStore rs = null;
    try {
        rs = RecordStore.openRecordStore( rsName, false );
        analyze( rs ); // перегруженный метод
    } catch( RecordStoreException e ) {
        logger.exception( rsName, e );
    } finally {
        try {
            rs.closeRecordStore();
        } catch( RecordStoreException e ){
            // игнорируем это исключение
        }
    }
}

The second parameter of the openRecordStore () method indicates whether the repository will be created if it does not exist. In MIDP 2.0, the following form of openRecordStore () is used to open storage created by another application:
...
String name = "mySharedRS";
String vendor = "EricGiguere.com";
String suite = "TestSuite";
RecordStore rs = RecordStore.openRecordStore( name, vendor, suite );
...

The manufacturer and the name of the midlet must match the ones specified in the manifest.

When finished, close the repository by calling RecordStore.closeRecordStore (), as in the analyze () method above.

The RecordStore instance is unique in the midlet: after opening it, all subsequent calls to openRecordStore () with the same arguments will return a reference to the same object. This instance is common to all midlets in the collection.

Each instance of RecordStore counts how many times the store has been opened. It will not close until closeRecordStore () is called the same number of times. After the repository is closed, an attempt to use it will throw a RecordStoreNotOpenException.

Storage creation

To create a repository (inaccessible to other midlets), you need to call openRecordStore (), setting the second parameter to true:
...
// создать хранилище
RecordStore rs = null;
try {
    rs = RecordStore.openRecordStore( "myrs", true );
} catch( RecordStoreException e ){
    // не удалось создать или открыть хранилище
}
...

To complete the initialization of the storage, check the value of getNextRecordID () - if it is 1, there are no entries in the storage:
if( rs.getNextRecordID() == 1 ){
    // первоначальная инициализация
}

An alternative way is to check the value returned by getNumRecords ():
if( rs.getNumRecords() == 0 ){
    // хранилище пусто, реинициализация
}

To create a public repository (MIDP 2.0 only), use the following option to call openRecordStore () with four parameters:
boolean writable = true;
rs = RecordStore.openRecordStore( "myrs", true, 
       RecordStore.AUTHMODE_ANY, writable );

If the second parameter is true and the repository does not exist, the last two parameters control its access mode and write ability. The access mode determines whether other applications can use this storage. Two options are possible: RecordStore.AUTHMODE_PRIVATE (only our application has access) or RecordStore.AUTHMODE_ANY (any application has access). The writable flag determines whether another application will have write access - if writable = false, it can only read data.

The storage owner application can change these parameters at any time using RecordStore.setMode ():
rs.setMode( RecordStore.AUTHMODE_ANY, false );

In fact, it is best to create a private storage, and access only after its initialization.

Adding and editing entries

Recall that records are arrays of bytes. To add a new record to open storage, use the RecordStore.addRecord () method:
...
byte[] data = new byte[]{ 0, 1, 2, 3 };
int    recordID;
recordID = rs.addRecord( data, 0, data.length );
...

You can create an empty entry by passing null as the first parameter. The second and third parameters specify the starting position of the read and the number of bytes to be saved. If successful, the record id is returned, otherwise an exception is thrown (for example, RecordStoreFullException).

At any time, you can update the record using RecordStore.setRecord ():
...
int    recordID = ...; // ID некоторой записи
byte[] data = new byte[] { 0, 10, 20, 30 };
rs.setRecord( recordID, data, 1, 2 ); 
    // заменить все данные в записи на 10, 20
...

You can find out which id will be assigned to the next record to be added using the RecordStore.getNextRecordID () method. All existing entries have an id less than this.

In the second part, we will look at ways to convert objects and other data into an array of bytes.

Reading records

To read records, RecordStore.getRecord () is used in one of two forms. In the first version, this method creates an array of the required length and writes data to it:
...
int    recordID = .... // ID некоторой записи
byte[] data = rs.getRecord( recordID );
...

In the second embodiment, the data is copied to the already created array, starting from the given position, and the number of bytes written is returned:
...
int    recordID = ...; // ID записи
byte[] data = ...; // массив
int    offset = ...; // стартовая позиция
int numCopied = rs.getRecord( recordID, data, offset );
...

The array must be long enough to hold the data, otherwise a java.lang.ArrayIndexOutOfBoundsException will occur. To determine the required size of the array, RecordStore.getRecordSize () is used. In fact, the first form of getRecord () is equivalent to the following:
...
byte[] data = new byte[ rs.getRecordSize( recordID ) ];
rs.getRecord( recordID, data, 0 );
...

The second form is useful if you iterate over many records in a loop, as the number of memory requests is reduced. For example, you can use it with getNextRecordID () and getRecordSize () to search by enumerating all the records in the repository:
...
int    nextID = rs.getNextRecordID();
byte[] data = null;
for( int id = 0; id < nextID; ++id ) {
    try {
        int size = rs.getRecordSize( id );
        if( data == null || data.length < size ) {
            data = new byte[ size ];
        }
        rs.getRecord( id, data, 0 );
        processRecord( rs, id, data, size ); // что-то делаем с найденной записью
    } catch( InvalidRecordIDException e ){
        // игнорируем и переходим к следующей записи
    } catch( RecordStoreException e ){
        handleError( rs, id, e ); // обработка ошибок
    }
}
...

However, it is better to use RecordStore.enumerateRecords () for this. We will consider this method in the third part of the series of articles.

Deleting Records and Storage

To delete records, use the RecordStore.deleteRecord () function:
...
int recordID = ...; 
rs.deleteRecord( recordID );
...

After deleting a record, any attempt to use it results in an InvalidRecordIDException.

You can delete the entire storage using RecordStore.deleteRecordStore ():
...
try {
    RecordStore.deleteRecordStore( "myrs" );
} catch( RecordStoreNotFoundException e ){
    // нет такого хранилища
} catch( RecordStoreException e ){
    // хранилище открыто
}
...

Storage cannot be deleted while it is opened by any application. Only the MIDlet that created it can delete the repository.

Other operation

There are only a few operations with RMS, all of them are methods of the RecordStore class:
  • getLastModified () returns the time of the last storage modification in the same format as System.currentTimeMillis ().
  • getName () returns the name of the store.
  • getNumRecords () returns the number of records in the repository.
  • getSize () returns the total storage size in bytes, including the length of records and service fields necessary for the system to organize it.
  • getSizeAvailable () returns the size of free space in bytes. Actual available size may be smaller due to additional bytes used by the repository to store each record.
  • getVersion () returns the repository version number. This is a positive integer that increments by one every time the data changes.

The midlet can also monitor changes to the repository by registering the handler with addRecordListener (); removeRecordListener () is designed to remove it. In the third part, these methods will be discussed in more detail.

RMSAnalyzer Class


We finish this article with the source code for the RMSAnalyzer class, our warehouse analyzer. For analysis, you need to run the following code:
...
RecordStore rs = ...; // открываем хранилище
RMSAnalyzer analyzer = new RMSAnalyzer();
analyzer.analyze( rs );
...

By default, the output is redirected to System.out and looks something like this:
=========================================
Record store: recordstore2
Number of records = 4
Total size = 304
Version = 4
Last modified = 1070745507485
Size available = 975950

Record #1 of length 56 bytes
5f 62 06 75 2e 6b 1c 42 58 3f _b.u.k.BX?
1e 2e 6a 24 74 29 7c 56 30 32 ..j$t)|V02
5f 67 5a 13 47 7a 77 68 7d 49 _gZ.Gzwh}I
50 74 50 20 6b 14 78 60 58 4b PtP k.x`XK
1a 61 67 20 53 65 0a 2f 23 2b .ag Se./#+
16 42 10 4e 37 6f .B.N7o
Record #2 of length 35 bytes
22 4b 19 22 15 7d 74 1f 65 26 "K.".}t.e&
4e 1e 50 62 50 6e 4f 47 6a 26 N.PbPnOGj&
31 11 74 36 7a 0a 33 51 61 0e 1.t6z.3Qa.
04 75 6a 2a 2a .uj**
Record #3 of length 5 bytes
47 04 43 22 1f G.C".
Record #4 of length 57 bytes
6b 6f 42 1d 5b 65 2f 72 0f 7a koB.[e/r.z
2a 6e 07 57 51 71 5f 68 4c 5c *n.WQq_hL\
1a 2a 44 7b 02 7d 19 73 4f 0b .*D{.}.sO.
75 03 34 58 17 19 5e 6a 5e 80 u.4X..^j^?
2a 39 28 5c 4a 4e 21 57 4d 75 *9(\JN!WMu
80 68 06 26 3b 77 33 ?h.&;w3

Actual size of records = 153
-----------------------------------------

This format is convenient to use when testing with the J2ME Wireless Toolkit. When testing on a real device, you can send the analyzer output to a serial port or even over a network. To do this, create a new class with the RMSAnalyzer.Logger interface and pass it an instance to the RMSAnalyzer constructor.

The article ends with a J2ME Wireless Toolkit project called RMSAnalyzerTest, which demonstrates the use of the analyzer: pastebin.com/n36QLuAs

The rest of the article in English can be seen here . Does it make sense to continue translating them?

Also popular now: