March 4, 2014 at 19:29PHP and various kinds of NoSQL
Recently, various NoSQL databases are gaining popularity. This article began as a study of the features of the
In the course of this small study, DBMSs that have been successfully applied in the field of Web have been selected for detailed consideration. And, since “PHP” is present in the tags, I chose DBMSs that can already be used with this language.
The article turned out to be voluminous, for the convenience of navigation I suggest using the table of contents:
- NoSQL Views
- Key-value stores
- Memcachedb
- Redis
- Tarantool
- Riak
- Aerospike
- Foundationdb
- Bigtable stores
- Hbase
- Hypertable
- Cassandra
- Graph stores
- Neo4j
- Why only one
- Document Stores
- Mongodb
- Couchdb
- Oriententb
- Some conclusions
NoSQL Views
All NoSQL DBMSs are divided into several categories:
- Key-value stores
- Column Family (Bigtable) stores / Scalable Distributed Storage
- Graph Stores / Graph DBMS
- Document Stores / Document Oriented Database Management Systems
The figure below schematically indicates the amount of data used and the complexity of this data in these NoSQL views.
In each section, I tried to arrange the DBMS in order of increasing functionality. Perhaps it turned out to be somewhat subjective.
There are databases combining several categories, for example, OrientDB . According to the official description from the link above, it is both graphical and document-oriented. Sometimes it is even attributed to Key-value stores and Column Family stores. More about it later in the graph DBMS section.
Let’s take a closer look at each category:
Key-value stores
Key-value stores is the direction in which NoSQL solutions show their superiority over SQL.
And many consider this direction to be the most popular in the short and long term.
For example, Michael Widenius, the author of the original version of the open MySQL DBMS, thinks so.
Key-value NoSQL are very popular and they are developing quickly and well, apparently due to their large number and strong competition. The largest number of NoSQL databases that were studied during the writing of the article related specifically to key-value stores.
On a habr there is an article about key-value storages for PHPwith which I do not agree in everything. The general selection of the repositories presented in it (Voldemort, Scalaris, MemcacheDB, ThruDB, CouchDB) seemed to me not so relevant after almost five years that have passed since the publication of the article. And CouchDB described there is not a key-value storage at all, but a document-oriented DBMS (see the section on document-oriented DBMS ).
Memcachedb
Description : the same memcached, only with a background in the form of BerkeleyDB.
Performance : the developers presented test results , according to which the average performance in one stream is 18868 w / s (write operations per second) and 44444 r / s (read operations per second). We tested on the Dell 2950III server, which even in the weakest configuration is a decent device .
Installation : everything is collected from source. In PHP, we use the usual Memcached from PECL.
License : BSD-like License - use for free for commercial and non-commercial projects.
Redis
Description : There is an introductory article on the
Productivity : ~ 110.000 w / s, ~ 81.000 r / s on average iron.
Installation : Redis itself and the client for PHP recommend building from source. There are many clients ( list ), I would recommend phpredis for a good description and support of all (or almost all) of the existing Redis functionality.
License : BSD license - all for free, but if something broke, then no complaints to the developers.
Tarantool
Description : In-memory storage. Redis is opposed, from which it differs, according to the developers, by increased speed, due to the fact that all the data is in memory. There is a built-in queuing mechanism. There is a good article describing the main features.
Installation : on Ubuntu it is installed using apt-get and a bit of magic ( official page ), the client for PHP is assembled from source ( github )
Performance : at the level with Redis, test results are inconsistent: Tarantool is faster than Redis at its developer , Tarantool at the level with Redis in an ordinary person
Licensed : Simplified BSD - everything is free.
Riak
Description : A database with a strong focus on fault tolerance and distribution. This emphasis is so strong that the development company recommends allocating at least five servers for Riak in order to be able to evaluate its capabilities. At first glance, this is a key-value storage, but it contains a search in all fields, secondary keys, MapReduce. No transactions. Detailed and detailed habrostatya .
Installation : there are many ways up to installation from packages for Debian / Ubuntu. For PHP there is a PECL package, as well as the official PHP-client .
Productivity : it is not given the most important place, but there are mentions of 2,500 operations per second.
License: Apache 2 License is free for ordinary people, but for commercial use, prices for a single copy of Riak Enterprise start at $ 2,800 / year.
Aerospike
Description : scalable storage for huge amounts of data with minimal latency. By default, ACID support has a separate page . Version 3 introduced secondary indexes. An impressive amount of proprietary technologies for scaling, replication and clustering ( link ). For myself, I remembered this system as a powerful industrial Memcached.
Installation : Aerospike is installed from the distribution kit, the official client for PHP exists only for Aerospike2, it is assembled from source.
Performance : claimed speed from 180,000 to 400,000 operations per second with a delay in microseconds ( source ).
License :
- Community Edition - a free version with restrictions: a maximum of two servers with 200GB of data each;
- Enterprise Edition - trial 30 days, no restrictions. According to rumors , the cost is from $ 50,000 per data center.
Foundationdb
Description : It is positioned as a comprehensive and most simple to install and configure solution. Easy scalability, easy management are the keywords that catch. Users are offered "uncompromising ACID transactions." The ability to use various data models - key / value, document and even SQL. This DBMS seemed especially interesting to me when I read about its performance.
Productivity : 3,750,000 r / s * . * Reading random entries from RAM (cache). On the official website in the performance section there are many interesting tests, the "slowest" of which shows a result of ~ 235,000operations per second (50/50 read and write operations). Read latency less than 2ms, commit delay less than 15ms. The results were obtained on a cluster of 24 machines, each with 16Gb RAM, 2x200Gb SSD, the test base consisted of 2 million key-value records, all operations were transactional with maximum isolation and triple replication.
Installation : and everything is simple: DEB package for Ubuntu, PEAR package for PHP.
License :
- Community License - free use. There are no restrictions when developing and testing, but a maximum of 6 running processes on production, i.e. one process on six servers, two on three, etc .;
- Enterprise License - unlimited, from $ 99 to $ 199, depending on the quality of support.
Some interesting projects were not included in this list due to the lack of support for PHP. Also, the projects Voldemort , Scalaris , ThruDB were not included . Due to poor performance, or poor documentation and due to the fact that since 2009, nothing has changed for the better.
Column Family (Bigtable) stores / Scalable Distributed Storage
The repositories presented in this section are mainly designed based on the design of the original Google Bigtable.
The main feature of these NoSQL is working with data whose volumes are measured in terabytes.
Instant access speed is not so important here, much more emphasis is placed on distribution, fault tolerance and the ability to process huge volumes of information.
Hbase
Description : Open Source development based on the original Google Bigtable design from Apache. Developed as part of the Hadoop project. Used by Facebook itself as the basis of the messaging service. HBase selects one indexed field. There is partial support for ACID, it turns out that transactionality seems to be there , but it is not supported in the most obvious way.
Installation : installed using a magic pill called Thrift, the installation and use process is well described in this habitat .
Performance : Field Testswith an unusual technique for measuring performance: on a cluster of 7 servers (16Gb RAM, 8x core CPU, HDD), operations were carried out in a table with 3 billion records. 300 processes of reading / writing were started simultaneously, the time spent on the operation was measured. As a result, the average recording time was 10ms , reading - 18ms .
License : Apache License 2.0 - use for any purpose for free.
Hypertable
Description : An interesting development similar to HBase. It has a bit more performance and is much more familiar with the HQL query syntax. Request example:
select * from QueryLogByUserID where row =^ '003269359' AND "2008-11-13 05:00:00" <= TIMESTAMP < "2008-11-13 06:00:00"
There are no transactions, which is clearly stated in the first lines of the documentation on the official website.
Installation : we connect to PHP using Thrift and the official ThriftClient ( github ).
Performance : several charts on the official website. As mentioned above, performance is similar to HBase.
License : GNU General Public License Version 3. - Use for any purpose free of charge. 24-hour support is available for an additional fee.
Cassandra
Description : Distributed storage originally developed on Facebook, subsequently transferred to Apache. Unlike the above, Cassandra is a distributed decentralized hash table (DHT) and is based on Amazon's Dynamo. It has a CQL query language that is very similar to SQL with some limitations. It is possible to build queries with sampling on several columns, add secondary indexes. In version 2.0, "transactions" have appeared that operate on the principle of "compare-and-swap".
The syntax of the transaction request will be noticeably this:
- Add Record
INSERT INTO users (login, email, name, login_count) values ('jbellis', '[email protected]', 'Jonathan Ellis', 1) IF NOT EXISTS - Record Update
UPDATE users SET reset_token = null, password = ‘newpassword’ WHERE login = ‘jbellis’ IF reset_token = ‘some-generated-reset-token’
Installation : there are several ways to establish the interaction between PHP and Cassandra (the same Trift, Cassandra-PHP-Client-Library, cassandra-pdo ). The last option seemed to me the most pleasant.
Performance : good comparative tests with graphs, according to the results of which, on 8 servers with a ratio of 50/50 read / write operations, Cassandra performs about 9.000 operations per second. HBase makes about 2,500 under the same conditions.
License : Apache License 2.0 - use for any purpose for free.
There are other BigTable solutions like Stratosphere , HPCC , Cloudera , Cloudata. They are not considered in detail for various reasons, for example: lack of PHP support, low prevalence, poor documentation.
Graph Stores / Graph DBMS
It was for their sake that this article was conceived. Recently, I discovered graphical NoSQL as a new version of the data storage structure and was delighted, because in a number of projects the basic functionality of graph DBMSs had to be implemented using not the simplest MySQL queries.
In a graphical DBMS, the structure of the stored data may look something like this:
If you put all the films in a graphical DBMS and associate with each actor who starred in it, you can easily find
фильмы,
в которых снимались актеры,
которые когда-либо снимались с актерами из "Матрицы",
и никогда не снимались с актерами из "Пиратов Карибского Моря"
Neo4j
Description : the most successful and sought-after development in the field of graph DBMSs. It fully supports ACID. It is easy to install and scales effortlessly. She has already formed a developed community, on most of the questions that arise, you can quickly find the answers. You can read about its capabilities in conjunction with PHP in this article .
Installation : installed from your repository, the Neo4jPHP client is used for PHP
Performance : due to its specificity, it seemed strange to me to give specific read / write speed indicators. It allows you to select complexly related data and does it many times faster than relational DBMSs.
License :
- Community Edition - GPL-licensed open source, free use
- Commercial Subscription - we have a high-performance cache, advanced features of horizontal scaling, support and some more goodies. The cost ranges from $ 0 (if you are a three-person startup with an annual project turnover of less than $ 100,000) to infinity (for very large companies)
In this section, I described only one DBMS, and its most interesting competitor, OrientDB, is located below. As it turned out, there are so many graphical DBMSs for the Web and for PHP in particular.
There is also Titan , which uses HBase, BerkleyDB or Cassandra as its back-end. There is not much information on his miracle; there is even less information on ways to make friends with PHP.
It is worth recalling FlockDB from Twitter, which can be connected to php using a Thirt-based client. But, again, due to the small amount of information about this DBMS, it is difficult to formulate a complete and objective opinion about it.
Document Stores / Document Oriented Storage
In this section we will consider document-oriented storages - DBMS for hierarchical data structures. These storages are universal: they have a high read / write speed, have a flexible approach to the formats of stored data, easily work with unstructured data and provide ample opportunities for scaling.
Mongodb
Description : Perhaps the most popular document-oriented NoSQL DBMS. Data is stored in JSON / BSON format. Good scaling, replication, indexes, Map-Reduce. Transactions are presented as compare-and-swap.
Installation : MongoDB from the repository, php-client from PECL.
Performance : there were slightly higher comparative tests , in which there were results on MongoDB.
License : GNU AGPL - open source, free use.
Couchdb
Description : development from Apache. Largely similar to MongoDB. It is distinguished by the absence of blocking during read operations, and the sharding technology that is more difficult to configure.
Installation : CouchDB from the repository, there are several options for the php client (PHPillow, PHP Object Freezer, PHP-on-Couch, extension from PECL).
Performance : according to the results of one test , it is noticeably slower MongoDB
License : Apache 2.0 - use for free.
There are many more developments in this area, but they seemed to me very monotonous. Although, perhaps, I just did not study them deep enough.
Oriententb
Description : document-oriented and, at the same time, graphical DBMS.
Its closest competitor as a document-oriented one is MongoDB. A separate page is dedicated to this comparison .
The main advantages of OrientDB:
- full ACID support
- the ability to use foreign keys in documents (the same as in relational DBMS)
- three types of indexes used (SB-Tree, Hash, MVRB-Tree) versus B-Tree in MongoDB
- high performance (OrientDB runs 150,000 w / s on regular hardware)
- simple query language similar to SQL
Separately, I want to note the query language, compare how identical update requests look:
- Mongodb
db.product.update( { “stock.qty”: { $gt: 2 } }, { $set: { price: 9.99 } } ) - Oriententb
UPDATE product SET price = 9.99 WHERE stock.qty > 2
Its main competitor as a graph is Neo4j. And I must say that it is much more difficult to master graph capabilities in OrientDB than in Neo4j. The first ideas about this can be obtained in this article .
Installation : you need to do a little magic with the installation, here is a quite working manual , and this library is recommended as a PHP client .
Performance : promise 150.000 of w / s , also has a Comparison of the graph database
LICENSE :
- Community Edition - Apache 2 license open source, free use for any purpose, including commercial
- Enterprise Edition - extended support and such benefits as Query Profiler, Metrics recording, Live Monitor with configurable alerts for £ 1,000 for the first server and £ 500 for each subsequent server. For startups, half the price.
Some conclusions
During the writing of the article, I found a lot of interesting useful and useful information, and I am glad to share it with the Habrovsk citizens.
I really liked such solutions as FoundationDB, Neo4j, OrientDB. I would like to devote a separate article to each of them.
In conclusion, I would like to share a fun picture that helps you quickly choose the NoSQL solution for your project. I saw the picture in a comment by 4dmonster , for which I thank him.
