Dive into Berkeley DB JE. Introduction to Collections API

  • Tutorial

Introduction


A little about the subject. BerkleyDB is a high-performance embedded DBMS, delivered as a library for various programming languages. This solution involves storing key-value pairs; it also supports the ability to assign multiple values ​​to one key. BerkeleyDB supports multi-threaded, replication, and more. The attention of this article will be drawn primarily to the use of the library provided by Sleepycat Software in the bearded 90s.

In the previous articleWe examined the main aspects of working with the Direct Persistence Layer API, thanks to which you can work with Berkeley as a relational database. Today, however, attention will be paid to the Collections API, which provides the ability to work through all the familiar Java Collections interface adapters.

Note : all examples in this article will be given in the Kotlin language.

Some general information


We all know that working with annotations, delayed initialization, and nullable types in Kotlin is a big, big pain. Due to its specificity, DPL does not allow these problems to be eliminated, and the only loophole is the creation of its implementation EntityModel- a mechanism that determines how to work with beans. The main incentive, personally for me, to use the Collections API is the ability to work with completely clean, familiar data-class beans. Let's look at how to transfer the code from the previous article to this framework.

Description of Entities


To describe any object in the database that can be represented as a separate entity, we need to create three classes: a solid entity, which we will operate outside the context of the database, its key and data.

No requirements are imposed on the essence, and the key and value (in standard work with the database, which this article covers) are required to implement the interface Serializable. Everything is standard here, we want in-memory fields - add annotation to them @Transient. Everything that is not marked as @Transientwill be serialized.

As we recall, in order to organize records in a selection, it is required to use an interface implementation as a key Comparable. Here the principle is the same: the selection will be sorted by key.

Bean Description Example
data class CustomerDBO(
        val id: String,
        val email: String,
        val country: String,
        val city: String,
        var balance: Long
)
data class CustomerKey(
        val id: String
): Serializable
data class CustomerData(
        val email: String,
        val country: String,
        val city: String,
        val balance: Long
): Serializable


Entity Operations


In order to create a connection with the Collections API, you have to sweat a little. To begin with, you should consider the principle of work in the most common case - N: 1.

We EnvironmentConfigomit the standard steps for creating , since it does not fundamentally differ from the configuration for DPL. The differences begin immediately after them.

For each of the entities, we must create a separate database, giving it a unique name, plus, we need to create a separate database that stores information about the entities in this Environmentand wrap it in ClassCatalog. We can say that in Berkeley, databases have roughly the same essence as tables in SQL. An example under a cat.

Creating a database for an entity and directory
    private val databaseConfig by lazy {
        DatabaseConfig().apply {
            transactional = true
            allowCreate = true
        }
    }
    private val catalog by lazy {
        StoredClassCatalog(environment.openDatabase(null, STORAGE_CLASS_CATALOG, databaseConfig))
    }
    val customersDatabase by lazy {
        environment.openDatabase(null, STORAGE_CUSTOMERS, databaseConfig)
    }


Further, it is logical that we need some kind of convenient point of contact with the framework, because it itself Databasehas a very nasty low-level API. These adapters are StoredSortedMapand classes StoredValueSet. It is most convenient to use the first as an immutable point of contact with the database, and the second mutable.

Collection adapters

    private val view = StoredSortedMap(
           customersDatabase,
           customerKeyBinding,
           customerDataBinding,
           false
    )
    private val accessor = StoredValueSet(
           customersDatabase,
           customerBinding,
           true
    )


You may notice that at the moment, Berkeley does not know how mapping is performed (key, data) -> (dbo)and dbo -> (key, data). In order for the mapping to work, it is required to implement one more mechanism for each of the bins - binding. The interface is extremely simple - two methods for mapping to data and to key, and one to essence.

Binding example
class CustomerBinding(
        catalog: ClassCatalog
): SerialSerialBinding(catalog, CustomerKey::class.java, CustomerData::class.java) {
    override fun entryToObject(key: CustomerKey, data: CustomerData): CustomerDBO = CustomerDBO(
            id = key.id,
            email = data.email,
            country = data.country,
            city = data.city,
            balance = data.balance
    )
    override fun objectToData(dbo: CustomerDBO): CustomerData = CustomerData(
            email = dbo.email,
            country = dbo.country,
            city = dbo.city,
            balance = dbo.balance
    )
    override fun objectToKey(dbo: CustomerDBO): CustomerKey = CustomerKey(
            id = dbo.id
    )
}


Now we can safely use working collections, which will be automatically synchronized as data changes in the database. At the same time, the “pulp” is the ability to calmly use parallel writing and reading from different streams. This possibility is primarily due to the fact that iteratorthese collections will be a copy of the current state, and will not change when the collections change, while the collections themselves are mutable. Thus, the only thing a programmer should think about is the control of data relevance.

Well, then, with the usual CRUD, we figured out, go to the connections!

Relationships between Entities


To work with relationships, we will need to additionally create SecondaryDatabaseone that will provide access to one entity using the key of another. An important point is the need to indicate in the DatabaseConfigvalue sortedDuplicateshow trueif the connection is not 1: 1 or 1: M. This action is quite logical, based on the fact that indexing will occur on a foreign key, and several entities will correspond to one key.

Secondary DB example with configuration
    val ordersByCustomerIdDatabase by lazy {
        environment.openSecondaryDatabase(null, STORAGE_ORDERS_BY_CUSTOMER_ID, ordersDatabase, SecondaryConfig().apply {
            transactional = true
            allowCreate = true
            sortedDuplicates = true
            keyCreator = OrderByCustomerKeyCreator(catalog = catalog)
            foreignKeyDatabase = customersDatabase
            foreignKeyDeleteAction = ForeignKeyDeleteAction.CASCADE
        })
    }


It is noteworthy that as a foreign key, you can choose not only the field by which the connection will be set, but also any arbitrary data type. The implementation of the interface takes on the role of creating keys SecondaryKeyCreator, or SecondaryMultiKeyCreator(more specific options are present, but it is enough to implement one of these two).

SecondaryKeyCreator Example
class OrderByCustomerKeyCreator(
        catalog: ClassCatalog
): SerialSerialKeyCreator(catalog, OrderKey::class.java, OrderData::class.java, CustomerKey::class.java) {
    override fun createSecondaryKey(key: OrderKey,
                                    data: OrderData): CustomerKey = CustomerKey(
            id = data.customerId
    )
}


It remains a little bit - to create a collection for obtaining samples by our foreign keys, the code under the cut is no different in principle from creating a collection for non-secondary databases.

Creating a collection for sampling N: 1
private val byCustomerKeyView = StoredSortedMap(
            database.ordersByCustomerIdDatabase,
            database.customerKeyBinding,
            database.orderDataBinding,
            false
    )


Instead of a conclusion


This article was the last part of the BerkeleyDB basics series. After reading this and the previous article, the reader is able to use the DBMS in their projects as a local storage, for example, in a client application. In the following articles more interesting aspects will be considered - migration, replication, some interesting configuration parameters.

As usual - if someone has additions or corrections to my stocks - you are welcome to comment. I am always happy for constructive criticism!

Also popular now: