New new thing (The new new thing)
The story of Silicon Valley
Databases are the backbone of the IT industry: unsung, invisible, but critical. And it is extremely harmful when they break or deform. This makes database specialists cautious. For years, only the Big Trinity — Oracle, IBM DB2, and maybe SQL Server — have been the only serious options. Then open source alternatives - MySQL, ProstgreSQL - became viable. And then, over the past five years, everything became very interesting ...
A bit of history.
At the turn of the millennium, more and more people began to realize that formal, structured, normalized databases requested through a subset of the SQL language more often made it difficult than simplified development. Over the past 10 years, many new databases have appeared, especially in the bowels of Google, which had a special need for distributed data warehouses - as a result of BigTable , Megastore and Spanner .
Meanwhile, Apache brought us Cassandra , HBase , and CouchDB ; Citrix offers plug-and-play distributed MySQL replacement; Redis has become a fundamental part of many Ruby on Rail (and similar) applications; and especially MongoDB became very popular among startups in spite of noisy criticism - in particular, write lock, which did not allow writing to the entire cluster in parallel. This, fortunately, will soon be fixed , which will lead to considerable rejoicing. (For reference: I am a developer, and I did some work with MongoDB, and I'm not a fan.)
Interestingly, these new developments - the so-called NoSQL “databases” - were innovative startups, and very few such dreamers took their work seriously. Databases are too critical, after all. If your database is deformed, then you are in a big ass. If your database does not guarantee the integrity of your data and your transactions (for example, if it does not support what is called an " ACID transaction "), then real database engineers will not take it seriously.
You may not need ACID guarantees but if a storage engine can't provide AD it's basically not a storage engine worthy of your time or data.- Kelly Sommers (@kellabyte) December 11, 2014
Yet we keep benchmarking storage engines that shit themselves and corrupt their own data with ones who don't. Why even benchmark brokenness.- Kelly Sommers (@kellabyte) December 11, 2014
MongoDB is not ACID compatible . Similar to Cassandra, Riak, Redis, etc. By the way, sometimes they even claimed that NoSQL databases are fundamentally ACID incompatible. It is not true. Google Megastore is actually ACID compatible , and their Spanner is even better . But you cannot use Megastore outside of Google, unless, of course, you are going to create your application completely on their peculiar App Engine platform.
That's why I was intrigued a couple of years ago when I came across a booth at TechCrunch Disrupt, whose tagline was “NoSQL, YesACID”. She was occupied by a company called FoundationDB , which created a great ACID compliant 1storage of key-values, over which access is granted through the so-called SQL layer . A week ago, they announced the release FoundationDB 3.0, which is 25 times faster than the previous version thanks to what they call "a heart transplant and lung" engine ( engine ).
This is a major feat in engineering . Citing their blog post, it's not just the 14 million records per second is 14 million records in a "completely streamlined, fully transactional database with 100% multi-core ( multi-key ) multi-node ( multi-node ) transactions [...] in the cloud [...]. In other words, FoundationDB can do 3.6 million records for 1 cent. ”
Impressive. Impressive enough to attract the attention of database engineers from the Enterprise field. And, of course, it is ideal for the upcoming Internet of Things, and a huge amount of data that will constantly be driven by various devices.
But, most importantly, it will spur their competitors to more achievements, which, in turn, will push a huge number of enterprises frozen in the Bronze Age, using Oracle and DB2, to, perhaps, a slow, accurate move to modern day. A day in which developers are spoiled by simple key-value repositories, powerful classic SQL queries, and distributed ACID transactions at the same time. AIn the long run, this will make life better. In the meantime, I’m taking my hat off to unsung database engineers who are pushing the engine of progress somewhere . You can’t imagine this, but they do us a great service.
If you follow the links, then they intentionally miss the description of the letter "C" in the acronym ACID. But for accuracy I’ll say that they mean strong consistency, not eventual consistency.
(From a translator)
Somewhere I saw a tweet that the guys are working on a Document Layer. Hope it will be MongoDB compatible.
This is the first article in the upcoming FoundationDB series of articles. Keep your hand on the pulse.