Deform: simple database as a service

Imagine a database that can validate JSON data using the JSON Schema , process events, and process data. And if the idea is implemented as a service using mongodb ?


We have developed just such a service. Our side projects use it:



Under the cut, a brief description of the main features of the service with examples.



This article contains sections:



Json


All entities in the service are JSON documents. Thanks to this, mongodb came up to us perfectly .


You can work with documents through HTTP REST Api, CLI and Python clients.


I prefer the CLI.


Here are some examples of its use:


  • Create document in collection venues

deform document create -c venues -d '{"text":"hello world"}'

  • Find out how many users are in the project

$ deform documents count -c my_project_users
108

Validation


I will say right away - the choice fell on JSON Schema draft v4. This diagram provides a lot of useful information.


Let's say that 2 documents come to you:


{
    "name": "Товар за 1 руб",
    "price": 1.0,
    "currency": "RUB"   
}

{
    "name": "1$ stuff",
    "price": 1.99,
    "currency": "United States Dollar"
}

And we need the second document not to go to the database - the currency (property currency) was transferred from him incorrectly .


Let's create a scheme according to which the first document passes, and the second does not.


{
    "description": "General goods schema",
    "type": "object",
    "properties": {
        "name": {
            "type": "string"
        },
        "price": {
            "type": "number"
        },
        "currency": {
            "type": "string",
            "enum": ["RUB", "USD", "CAD", "GBP"]
        }
    },
    "required": ["name", "price", "currency"]
}

Pay attention to currencyits property enum. Only documents in which the currency matches the lines "RUB", "USD", "CAD", "GBP" will be considered correct.


Files


Deform supports file handling. All your files will be documents in the collection _files. And in the document itself, where you attached the file, it will also be available as an object.


Let's add a new property to the goods scheme - the product image.


{
    "description": "General goods schema",
    "type": "object",
    "properties": {
        "name": {
            "type": "string"
        },
        "price": {
            "type": "number"
        },
        "currency": {
            "type": "string",
            "enum": ["RUB", "USD", "CAD", "GBP"]
        },
        "image": {
            "type": "file"
        }
    },
    "required": ["name", "price", "currency"]
}

Now we can upload a picture for this product.


Using the CLI, create a document:


$ deform document create -c goods -d '{
    "name": "Teapot",
    "price": 11.99,
    "currency": "GBP",
    "image": @"/tmp/teapot.png"
}'

As a result, we get


{
    "_id": "576ffce308888f000599ee17",
    "name": "Teapot",
    "price": 11.99,
    "currency": "GBP",
    "image": {
        "_id": "5772db5308888f000599f095",
        "collection_id": "goods",
        "content_type": "image/png",
        "date_created": "2016-06-25T18:31:12.356Z",
        "document_id": "576ec022bd4db46b765ae94a",
        "last_access": "2016-06-25T17:32:18.347Z",
        "md5": "a8eda376612338e0286ff1c1a725b111",
        "name": "teapot.png",
        "size": 16214
    }
}

You can get the contents of the file both from the collection with files and from the product document itself.


Link to the documentation section with files


Processing


Deform can process data. Full list of handlers .


Let's say that we need to change the size of the image of the product at the time of its creation.


The processor will help us resizewith this (at the moment it works only with images)


The scheme will turn into:


{
    "description": "General goods schema",
    "type": "object",
    "properties": {
        "name": {
            "type": "string"
        },
        "price": {
            "type": "number"
        },
        "currency": {
            "type": "string",
            "enum": ["RUB", "USD", "CAD", "GBP"]
        },
        "photo": {
            "type": "file"
        },
        "300x150": {
            "type": "file",
            "processors": [
                {
                    "name": "resize",
                    "in": {
                        "original_image": {
                            "property": "photo"
                        },
                        "size": {
                            "value": [300,150]
                        }
                    }
                }
            ]
        }
    }
}

Link to processing documentation section


Webhook


Our webhooks write history and they have events. They also have headers, methods and their own validation of matching documents.


Suppose we need a notification in slack that the product was created. The hook will look like this:


{
    "name": "Slack notification",
    "url": "https://hooks.slack.com/services/....",
    "method": "POST",
    "triggers": ["created"],
    "collection": "slack_notifications"
}

Pay attention to the property triggers- in this case only created documents will trigger the hook.


Now let's say that we only need notifications about goods where the price is higher 1000and the currency USD:


{
    "name": "Slack notification",
    "url": "https://hooks.slack.com/services/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",
    "method": "POST",
    "triggers": ["created"],
    "collection": "slack_notifications",
    "condition": {
        "type": "object",
        "additionalProperties": true,
        "price": {
            "type": "number",
            "minimum": 999
        },
        "currency": {
            "type": "string",
            "enum": ["USD"]
        }
    }
}

If the document is correct for the scheme inside condition, the hook will work out.


Link to the documentation section with hooks


Conclusion


This article is a short overview of the main features of the project.


If you are interested in it - I will be just any feedback, including objective criticism: D


A few links on the project:



Also popular now: