March 18, 2015 at 18:44Django DB Mailer - a simple and convenient battery for sending mail messages in your project
- Tutorial
The vast majority of projects use sending messages to the mailing addresses of clients / admins / departments. How tedious it is every time to create templates on the file system, set signals, use frequently duplicated code for rendering and sending.
In the end, the tasks arise of sending the background, changing the sender, prioritizing, re-sending the message in case of failure, often editing at the request of the client / marketers, attaching standard files to the letter in the form of some instructions for using the product, or standard documents, reports on clients, transfers, etc., etc.
In large projects, it is often necessary to visually divide the templates into categories, for filtering, so as not to waste time editing less important templates when refactoring, and to avoid confusion with new content, or when working with old ones.
For such solutions, there is a simple battery designed to solve most of these problems, and save you from unnecessary code, maintenance and hassle.
django-db-mailer is an open and easy-to-use battery for sending email messages, newsletters and pending notifications. It does not require special knowledge or custom settings. You just need to install the package, configure the Celery queue system in conjunction with Redis , and you can begin to do creative work.
1. Simple and convenient functional interface for sending letters.
2. Work in the specified queue (necessary when everything in the project is divided into different servers and workers).
3. The ability to resend the message in case of failure (it is possible to set the number of attempts in the database and in the code).
4. Timeouts
5. Logging (you can always see which messages were sent successfully / unsuccessfully. It is useful when investigating complaints from users).
6. Categories (quick filtering of heterogeneous templates. Cataloging by app, etc., etc.).
7. Recipient groups (convenient for dividing into departments within the company and setting recipient addresses in the admin panel).
8. Signals (sending messages on the event post_save / pre_save & etc. is configured directly in the admin panel).
9. Delayed sending messages (good for any promos, as well as when using transactions in views).
10. Built-in browser of fields in models (no need to go into the code in order to view the fields available for use. Do not drag you, and you yourself do not go into the code).
11. Prioritization system (the ability to set the priority of the message in the DB / Code).
12. Editing templates right in the admin panel (for support, managers - that’s it. It does not take the programmer’s time).
13. Templates are cached - each time a message is sent, there are no calls to the database (templates and settings are read from the cache, which reduces the load on the database. The backend is to your taste).
14. Visual editor TinyMCE / CKEditor (optional).
15. Versioning of changes in templates. You can always roll back (there are times when you can’t do without it).
16. Battery localization Russian / English.
17. Testing messages directly from the admin panel (it is possible to create a template and immediately get an example on the mailbox of the current user).
18. Out of the box, the popular django-grappelli theme is supported.
19. The types of letters html and text (the ability to specify the type of template).
20. The ability to use all the built-in features of the mailer of the junga.
21. Bcc - you can duplicate all letters to the specified addresses (it is useful for logging with text. By default, the body of the message is not logged in the database for security reasons).
22. API - for collaboration with applications in other languages / platforms.
23. An opportunity to set the “From” field to each template (useful when you have several departments. It is especially convenient when transferring responsibilities to another department).
24. SMTP settings for recipients (you can configure various smtp servers for sending letters).
25. Attaching standard files (various inserts, documents, instructions directly in the admin panel).
26. Multilingual patterns for localized sites (one pattern for multilingual sites. Depending on the locale, the message language is selected).
27. Disabling / Enabling alerts (convenient for debugging, or for temporarily disabling any promo campaigns of a company).
28. Easy integration with popular services such as postmark / aws ses (you just need to install python-postmark / django-ses).
29. Catching and sorting by Python exceptions (in the case of using a postmark, or something else - it is convenient to filter and quickly solve the problem).
30. Viewing the differences between versions (it is always necessary to see who changed what in the templates and what is the reason for the error).
31. Reporting on letters read by users (including information about geo. But there are no guarantees).
32. Sending SMS (all mailer features are also available for SMS notifications).
33. Text to speech (convenient for people with disabilities, or for urgent notifications through calls).
34. Push notifications (for mobile applications, through BaaS - parse.com, including Prowl for administrators).
35. Expanding battery capabilities through the use of proprietary backends and providers.
36. Updating templates in real time (there is no need to create files, make commits and roll out updates on servers).
37. Basic templates (you can easily inherit, and change them at any convenient time).
38. Pre_send / post_send signals (for example, if sending paid sms, for counting the number of characters and fixing in billing)
39. Subscriptions (by default there is a subscription implementation, with the ability to send / receive notifications, etc., including setting the preferred notification time)
Battery capabilities are closely related to Celery and Redis . More precisely, this is a background dispatch, a queue, priorities, and delayed dispatch.
Although most of the functionality works without them. That is, you can use the battery without any external dependencies.
This is at the discretion of the programmer. Difficulties in setting up those or other parts is not a problem. There is detailed documentation.
Instead of Redis, you can use RabbitMQ, MongoDB, and all Celery-supported backends.
Unfortunately, in Ceryry, priorities are only implemented for Redis. At the heart of quasi-priorities (not server side).
Let's look at a simple example for sending a message :
In this case, we set the message identifier, specify the recipient, and pass 2 models into the context that are automatically unpacked in the template + dictionary.
We can refer to the User model fields in the template as username or email. But there is also the opportunity to access them in the generally accepted form: user.username.
A more detailed example :
using docker:
using vagrant:
Run shell:
Create our first template:
We will send a letter:
For the first time, all queries to the database will be displayed in the console, as well as the letter itself. The second time there will be no queries to the database. Only log write requests.
Let's check what we have in the logs:
Now you can look into the admin panel:
History
To save the history of changes / rollback, you can install the django-reversion battery .
Editor
In order to edit templates in the visual editor, you can put the django-tinymce battery .
Themes
django-db-mailer - supports both the native theme and django-grappelli .
Queues
To send messages in the background with priority support, you need a django-celery battery .
Multilingualism
Multilingualism is possible in conjunction with django-modeltranslation .
Convert CSS
To convert and use lowercase CSS, you can preinstall the packagepremailer . A small but useful helper for creating cross-client messages.
Reports
Read reports and information about user data can be obtained by installing 2 simple batteries httpagentparser and django-ipware .
Basic templates
By default, there are ready-made basic templates that are optimized for desktop, mobile and browsers. In order to start using them, you only need to import them:
after which you can substitute your variables and blocks for use in letter templates.
Notes
In order for prioritization to work, you need a django-celery bundle with the Redis broker .
Older versions
If you do not need all the functionality, you can use a simpler version of the battery .
You can also use the DB_MAILER_ALLOWED_MODELS_ON_ADMIN variable to disable unnecessary functionality in the admin panel.
API
You can also use the battery in conjunction with a mobile application, or in internal projects in other languages. There are no restrictions. A project in this context is an external complement to absolutely any framework or project in languages other than Python.
A simple example:
We try to send an email using curl in the terminal:
Bandwidth API 1k + rps on a simple i7 2.3GHz
More examples for your programming language can be found in the documentation in the project repository
Support for Django starting from branch 1.4 to 1.9. Python 2.6, 2.7, pypy, 3.4+, pypy3
This battery is used in several large projects, and has saved a lot of time and nerves.
I hope it will be useful to the Django community.
Page on PyPi .
Project repository on GitHub .
Pull Requests are welcome (for those wishing to help, TODO.RU is at the root of the project).
In the end, the tasks arise of sending the background, changing the sender, prioritizing, re-sending the message in case of failure, often editing at the request of the client / marketers, attaching standard files to the letter in the form of some instructions for using the product, or standard documents, reports on clients, transfers, etc., etc.
In large projects, it is often necessary to visually divide the templates into categories, for filtering, so as not to waste time editing less important templates when refactoring, and to avoid confusion with new content, or when working with old ones.
For such solutions, there is a simple battery designed to solve most of these problems, and save you from unnecessary code, maintenance and hassle.
django-db-mailer is an open and easy-to-use battery for sending email messages, newsletters and pending notifications. It does not require special knowledge or custom settings. You just need to install the package, configure the Celery queue system in conjunction with Redis , and you can begin to do creative work.
What can a battery out of the box do?
1. Simple and convenient functional interface for sending letters.
2. Work in the specified queue (necessary when everything in the project is divided into different servers and workers).
3. The ability to resend the message in case of failure (it is possible to set the number of attempts in the database and in the code).
4. Timeouts
5. Logging (you can always see which messages were sent successfully / unsuccessfully. It is useful when investigating complaints from users).
6. Categories (quick filtering of heterogeneous templates. Cataloging by app, etc., etc.).
7. Recipient groups (convenient for dividing into departments within the company and setting recipient addresses in the admin panel).
8. Signals (sending messages on the event post_save / pre_save & etc. is configured directly in the admin panel).
9. Delayed sending messages (good for any promos, as well as when using transactions in views).
10. Built-in browser of fields in models (no need to go into the code in order to view the fields available for use. Do not drag you, and you yourself do not go into the code).
11. Prioritization system (the ability to set the priority of the message in the DB / Code).
12. Editing templates right in the admin panel (for support, managers - that’s it. It does not take the programmer’s time).
13. Templates are cached - each time a message is sent, there are no calls to the database (templates and settings are read from the cache, which reduces the load on the database. The backend is to your taste).
14. Visual editor TinyMCE / CKEditor (optional).
15. Versioning of changes in templates. You can always roll back (there are times when you can’t do without it).
16. Battery localization Russian / English.
17. Testing messages directly from the admin panel (it is possible to create a template and immediately get an example on the mailbox of the current user).
18. Out of the box, the popular django-grappelli theme is supported.
19. The types of letters html and text (the ability to specify the type of template).
20. The ability to use all the built-in features of the mailer of the junga.
21. Bcc - you can duplicate all letters to the specified addresses (it is useful for logging with text. By default, the body of the message is not logged in the database for security reasons).
22. API - for collaboration with applications in other languages / platforms.
23. An opportunity to set the “From” field to each template (useful when you have several departments. It is especially convenient when transferring responsibilities to another department).
24. SMTP settings for recipients (you can configure various smtp servers for sending letters).
25. Attaching standard files (various inserts, documents, instructions directly in the admin panel).
26. Multilingual patterns for localized sites (one pattern for multilingual sites. Depending on the locale, the message language is selected).
27. Disabling / Enabling alerts (convenient for debugging, or for temporarily disabling any promo campaigns of a company).
28. Easy integration with popular services such as postmark / aws ses (you just need to install python-postmark / django-ses).
29. Catching and sorting by Python exceptions (in the case of using a postmark, or something else - it is convenient to filter and quickly solve the problem).
30. Viewing the differences between versions (it is always necessary to see who changed what in the templates and what is the reason for the error).
31. Reporting on letters read by users (including information about geo. But there are no guarantees).
32. Sending SMS (all mailer features are also available for SMS notifications).
33. Text to speech (convenient for people with disabilities, or for urgent notifications through calls).
34. Push notifications (for mobile applications, through BaaS - parse.com, including Prowl for administrators).
35. Expanding battery capabilities through the use of proprietary backends and providers.
36. Updating templates in real time (there is no need to create files, make commits and roll out updates on servers).
37. Basic templates (you can easily inherit, and change them at any convenient time).
38. Pre_send / post_send signals (for example, if sending paid sms, for counting the number of characters and fixing in billing)
39. Subscriptions (by default there is a subscription implementation, with the ability to send / receive notifications, etc., including setting the preferred notification time)
Battery capabilities are closely related to Celery and Redis . More precisely, this is a background dispatch, a queue, priorities, and delayed dispatch.
Although most of the functionality works without them. That is, you can use the battery without any external dependencies.
This is at the discretion of the programmer. Difficulties in setting up those or other parts is not a problem. There is detailed documentation.
Instead of Redis, you can use RabbitMQ, MongoDB, and all Celery-supported backends.
Unfortunately, in Ceryry, priorities are only implemented for Redis. At the heart of quasi-priorities (not server side).
Let's look at a simple example for sending a message :
from dbmail import send_db_mail
# данные контекста
settings = Settings.objects.get(user_id=1)
user = User.objects.get(pk=1)
data = {'path': request.path}
# отправка простого сообщения
send_db_mail('welcome-notification', user.email, settings, user, data)
# отправка сообщения с вложенным файлом
send_db_mail('daily-report', user.email, user, data, files=[report_file])
In this case, we set the message identifier, specify the recipient, and pass 2 models into the context that are automatically unpacked in the template + dictionary.
We can refer to the User model fields in the template as username or email. But there is also the opportunity to access them in the generally accepted form: user.username.
A more detailed example :
send_db_mail(
# идентификатор шаблона в БД
slug='welcome',
# получатели могут быть списком или строкой. Например:
# 'user1@example.com' или 'user1@example.com, user2@example.com', или
# ['user1@example.com', 'user2@example.com'], или же идентификатор
# модели группы MailGroup. Например: developers, support, admins
recipient='user1@example.com',
# Все параметры *args доступны в качестве контекста в шаблоне
{
'username': request.user.username,
'full_name': request.user.get_full_name(),
'signup_date': request.user.date_joined
},
# Если передается модель в контекст, вы получаете доступ ко всем
# распакованным полям.
# Для m2m и fk, необходимо обращаться по module_name (mymodel.user.email).
MyModel.objects.get(pk=1),
# Необязательные дополнительные параметры **kwargs:
# from_email='from@example.com' # от кого
# cc=['cc@example.com'], # копия (надстройки те же, что и для recipient)
# bcc=['bcc@example.com'], # скрытая копия (надстройки те же, что и для recipient)
# user=User.objects.get(pk=1), # пользователь, для логирования в БД
#
# Описание данных опций доступны в официальной доке к Django
# attachments=[(filename, content, mimetype)],
# files=['hello.jpg', 'world.png'],
# headers={'Custom-Header':'Some value'},
#
# queue='default', # очередь
# retry_delay=300, # время до повторной отправки в случае неудачи
# max_retries=3, # максимальное кол-во попыток для переотправки
# retry=True, # разрешить попытку для переотправки
# time_limit=30, # максимальное кол-во времени для отправки
# send_after=60, # отправить письмо через N секунд
#
# use_celery=True, # использовать Celery
#
# Здесь могут быть любые дополнительные опции доступные в django.core.mail.message
)
Installation of the demo project:
$ sudo apt-get install -y virtualenvwrapper redis-server || brew install pyenv-virtualenvwrapper redis
$ source /usr/share/virtualenvwrapper/virtualenvwrapper.sh || source /usr/local/bin/virtualenvwrapper.sh
$ mkvirtualenv db-mailer
$ workon db-mailer
$ git clone --depth 1 https://github.com/LPgenerator/django-db-mailer.git db-mailer
$ cd db-mailer
$ python setup.py develop
$ cd demo
$ pip install -r requirements.txt
$ python manage.py syncdb --noinput
$ python manage.py migrate --noinput
$ python manage.py createsuperuser --username admin --email admin@local.host
$ python manage.py runserver >& /dev/null &
$ python manage.py celeryd -Q default >& /dev/null &
using docker:
$ git clone --depth 1 https://github.com/LPgenerator/django-db-mailer.git db-mailer
$ cd db-mailer
$ docker build -t dbmail .
$ docker run -it -d -p 8000:8000 --name dbmail dbmail
$ docker exec -i -t dbmail /bin/bash
$ cd /mailer/
using vagrant:
$ git clone --depth 1 https://github.com/LPgenerator/django-db-mailer.git db-mailer
$ cd db-mailer
$ vagrant up
$ vagrant ssh
$ cd /mailer/
Run shell:
$ python manage.py shell_plus --print-sql
Create our first template:
>>> from dbmail.models import MailTemplate
>>> from dbmail import send_db_mail
>>>
>>> MailTemplate.objects.create(
... name="Site welcome template",
... subject="Welcome",
... message="Welcome to our site. We are glad to see you.",
... slug="welcome",
... is_html=False,
... )
We will send a letter:
>>> send_db_mail('welcome', 'user@example.com', use_celery=False)
For the first time, all queries to the database will be displayed in the console, as well as the letter itself. The second time there will be no queries to the database. Only log write requests.
Let's check what we have in the logs:
>>> from pprint import pprint
>>> from django.forms.models import model_to_dict
>>> from dbmail.models import MailLog
>>>
>>> pprint([model_to_dict(obj) for obj in MailLog.objects.all()])
Now you can look into the admin panel:
$ xdg-open http://127.0.0.1:8000/admin/dbmail/ >& /dev/null || open http://127.0.0.1:8000/admin/dbmail/ >& /dev/null
Additional features:
History
To save the history of changes / rollback, you can install the django-reversion battery .
Editor
In order to edit templates in the visual editor, you can put the django-tinymce battery .
Themes
django-db-mailer - supports both the native theme and django-grappelli .
Queues
To send messages in the background with priority support, you need a django-celery battery .
Multilingualism
Multilingualism is possible in conjunction with django-modeltranslation .
Convert CSS
To convert and use lowercase CSS, you can preinstall the packagepremailer . A small but useful helper for creating cross-client messages.
Reports
Read reports and information about user data can be obtained by installing 2 simple batteries httpagentparser and django-ipware .
Basic templates
By default, there are ready-made basic templates that are optimized for desktop, mobile and browsers. In order to start using them, you only need to import them:
$ python manage.py load_dbmail_base_templates
after which you can substitute your variables and blocks for use in letter templates.
Notes
In order for prioritization to work, you need a django-celery bundle with the Redis broker .
Older versions
If you do not need all the functionality, you can use a simpler version of the battery .
You can also use the DB_MAILER_ALLOWED_MODELS_ON_ADMIN variable to disable unnecessary functionality in the admin panel.
API
You can also use the battery in conjunction with a mobile application, or in internal projects in other languages. There are no restrictions. A project in this context is an external complement to absolutely any framework or project in languages other than Python.
A simple example:
>>> from dbmail.models import ApiKey
>>> ApiKey.objects.create(name='Test', api_key='ZzriUzE')
We try to send an email using curl in the terminal:
$ curl -X POST http://127.0.0.1:8000/dbmail/api/ --data 'api_key=ZzriUzE&slug=welcome&recipient=root@local.host'
Bandwidth API 1k + rps on a simple i7 2.3GHz
More examples for your programming language can be found in the documentation in the project repository
Support for Django starting from branch 1.4 to 1.9. Python 2.6, 2.7, pypy, 3.4+, pypy3
Screenshots
This battery is used in several large projects, and has saved a lot of time and nerves.
I hope it will be useful to the Django community.
Page on PyPi .
Project repository on GitHub .
Pull Requests are welcome (for those wishing to help, TODO.RU is at the root of the project).