How researchers discover open MongoDB and Elasticsearch databases

Some time ago, it was very “fashionable” among security researchers to find improperly configured AWS cloud storages with all kinds of confidential information. Then I even published a small note about how Amazon S3 open cloud storage is discovered.

However, time goes by and the emphasis in research has shifted to the search for publicly available databases. More than half of the known cases of major data leaks over the past year are leaks from open databases (a review of leaks for 2018 here and here ).

Today we’ll try to understand how such databases are detected by security researchers ...

Required disclaimer: this article does not address or raise legal and ethical issues related to the search and identification of publicly available databases. All information below should be considered exclusively as educational and informative.

I won’t discover America if I say that the main tools for finding open databases are specialized search engines Shodan and Censys . I must say right away that these are not free services and you have to pay your honestly earned non-rubles for access to the full search results (in the case of Shodan it is $ 59 / month, and for Censys - $ 99 / month, and this is only for the minimum basic packages). Free versions of search engines greatly limit the number of results in the search results.

In addition to the classic Google-like search using the search bar, these search engines provide the ability to connect to them through the API. The lucky owner of a paid subscription will receive their API key. This makes it possible to significantly facilitate the task of raking up a pile of search garbage. I gave a list of useful scripts and search automation programs that use API keys at the end of this article.

Кстати говоря, используя небольшую хитрость и опять же поиск, только на этот раз по GitHub, можно найти некоторое количество API-ключей, оставленных в открытых репозитариях их неосторожными владельцами.

Let's look at the Shodan search engine as an example of how to find open MongoDB and Elasticsearch databases.

The simplest and most obvious query that begs is "MongoDB":

As can be seen from the screenshot, this request returned all indexed MongoDB servers to us (on default port 27017). Moreover, most of them (in the screenshot these are the first three) will be closed databases, requiring accounts for connection. And this is not quite what we are interested in. More precisely, this is not at all the case.

Let's complicate the query a bit using the search filters "all:" mongodb server information "all:" metrics "":

The result already looks much better. All found databases were freely available at the time when they were indexed by the search engine. With a high degree of probability, you can connect to them at the specified IP addresses using some manager for MongoDB (for example, NoSQL Manager for MongoDB or Studio 3T for MongoDB).

You can limit the search to a country. Let's say let's look for open MongoDBs in China (query "all:" mongodb server information "all:" metrics "country:" cn ""):

To search for open Elasticsearch databases it is convenient to use the query "port:" 9200 "all:" elastic indices "":

Unfortunately, I did not find a way how to set the search condition in the search bar according to the size of the database. If you know how to do this (set a filter on the "totalSize" or "sizeOnDisk" parameters in the search bar), then write in the comments.

You can apply more complex filters (restrictions on the size of the database, the date the server entered the index, etc.) to the search results using specialized scripts and programs. For this, you will already need paid access and an API key, as I wrote above.

Here is a short list of what might come in handy:

• cli.shodan.io is the official Shodan console. In Russian about its use can be read here .
• shodan-manual.com - a guide to Shodan in Russian.
• github.com/woj-ciech/LeakLooker is a python program for finding open databases (MongoDB, CouchDB, Elasticsearch, etc.).
• lampyre.io is a very powerful (and so far very buggy!) graphical interface for searching and visualizing what was found. It has built-in templates for searching MongoDB and much more. It is in beta testing with all the consequences ...
• t.me/dataleak (if blocked, then tele.click/dataleak ) is my modest Telegram channel in which I review information leaks and, in particular, write about what security researchers find in the public domain. It is interesting.