April 20, 2010 at 09:50We write functional / integration tests for a django project
In this exciting article, I will talk about tools with which you can write functional tests for a django project. There are a bunch of different other ways to do this, but I will describe one - the one that, in my opinion, is the easiest. In the meantime, we will create a beautiful report on code coverage (subjectively - nicer than what coverage.py does). And also, as a seasoning, there will be a bit of talk about testing.
Right off the bat.
WebTest is a library for functional testing of wsgi applications from Ian Bicking (author of pip and virtualenv). I added support for Unicode the other day (which is very important for us, right?), It has not yet been included in the main repository, so we put it from mine for now (you need installed mercurial, but you can also download zip- from the bitpack without it) archive, unzip and run setup.py install). Then, I think it will be possible to install c pypi. You can now install (pip install webtest) from there, only Unicode will not work in forms and links. ( UPD: everything has been included for a long time, you can install everything with pypi calmly )
Why WebTest, not twill? Twill also does not support Unicode (only it gets worse, not only Unicode, but even just non-Latin letters in utf8, as far as I can tell -UPD in the comments of meako writes that the lines just work, at least in conjunction with tddspry ), the last release was in 2007, there is a lot of code, outdated versions of the libraries in the delivery, and it seemed that to screw something there, it would take more effort . In general, twill is good, and html parsing is better there, so if that is, keep it in mind (and the django-test-utils package ( or tddspry? ) Then too).
Why not the built-in dzhangovsky test Client? WebTest has a much more powerful API; it is easier to write functional tests with its help. Read the docstring to django.test.client.Client:
Why not Selenium / windmill / ..? It does not interfere. They are testing another one though. For what you can use twill / WebTest, it is better to use twill / WebTest, because it will work much faster, have better integration with other code and easier setup.
A little tweaking is required for django-coverage. Don’t be scared, not big) You should:
1. add 'django_coverage' to INSTALLED_APPS and
2. in settings.py indicate where to save the html reports. It would be nice to create a folder for this business.
Here, for example, is a functional test for registration / authorization: We save it in the tests.py file of the desired application in the project. It seems that everything should be clear here. We check whether the registered person can leave the site, then go to it (by entering his email and password), can he register (will he receive the letter? Is the activation link correct?) And gets to the right page after activating the account. For convenience, a fixture was also used, in which the kmike user with email=example@example.com was already prepared (and which is used in other tests). It was possible to create this user directly in the test, this is not the point.
Pay attention to the API: we follow the links indicating their name (.click (u'Registration '), for example), i.e. what the user actually presses on (there are other possibilities). At each transition, WebTest automatically checks that the code 200 or 302 has returned to us (this is configured). To submit forms, you do not need to construct POST requests manually, forms are picked up from the html response code, it is enough to assign values to the necessary fields and execute the submit () method. Redirects after POST requests are done manually (and this is useful, because if there is no redirect, for example, an error filling out the form, the test will show this).
django_webtest.WebTest is the descendant from the Jungian TestCase, it can do the same. But the main thing is that it has a self.app variable of type DjangoTestApp available (this is the successor to webtest.TestApp) through which you can access the WebTest API. More information about what WebTest can do is best read on their website . There is a simple and pleasant API, you can follow the links, submit forms, upload files, parse the answer (much higher level and concise than the Dzhangovsky test client). django_webtest adds one feature specific to the junga to the API: the self.app.get and self.app.post methods accept an optional parameter
It is clear that here it was possible to test the most, and it was possible less, and here it is good to maintain some kind of balance: so that the tests are not difficult to write and maintain, that they check everything that is needed, but do not check what is not needed. Sometimes it will be wrong to click on a link through its name, sometimes it will not be easy to check whether there is text on the page, sometimes even this check will be superfluous. This, I think, is called experience, when you understand how best. The way I wrote these tests is not necessarily the best way in this situation (although imho is quite adequate), just consider it as an example, and not as an example to follow, think about what you are writing. One of the advantages of simple APIs is that the programmer begins to think what to write and how to write better, and not "how to add something at last ..".
We create the file test_settings.py (in the root of the project) with the following contents (syntax for django 1.1): And then run the tests:
You can do without test_settings (it’s easy to run
The code coverage report was saved in the previously specified folder. Open it (file cover / index.html) and see something like this:
We click on some link and see which code we executed during the tests, and which did not execute (and therefore could not be tested):
... many lines ...
... many lines ...
Yeah! It is immediately obvious that we did not check the situation when a person entered the email address of an already registered user.
It is important to remember that functional / integration tests are not a substitute for unit tests., but only an addition to them, and that 100% coverage does not guarantee the absence of errors. Unit tests are accurate, they say WHAT broke, they are extremely useful in refactoring and in difficult places in a project. Functional ones are rude, they only say “something seems to have broken somewhere” and they protect against stupid mistakes. But even if the tests just click on all the links on the site and check if an exception has occurred, then these will already be very useful tests that can save you from a bunch of troubles.
To illustrate the difference: in the unit test for the registration form, we would create an object of the EmailRegistrationForm class, pass different dictionaries with data into it and look at what exceptions are being raised, for example. Or they would test individual methods of this form. Unit tests are as close as possible to the code (although it makes sense not to let them go outside the public API), they test a separate piece of it, and allow us to verify that all parts of the system individually work correctly. Functional / integration tests help verify that they work together correctly too.
docs.djangoproject.com/en/dev/topics/testing
pythonpaste.org/webtest
bitbucket.org/ianb/webtest
bitbucket.org/kmike/webtest
bitbucket.org/kmike/django-webtest
bitbucket.org/kmike/django-coverage
github.com/ericholscher/django-test-utils
twill.idyll.org
nedbatchelder.com/code/coverage
Yes, all this can be used just as easily without django, WebTest is very easy to screw on to any framework that supports wsgi (and its support "all attention-worthy frameworks"), coverage.py works great for any tests. All of these django- ... applications are just to make installation and configuration as simple as possible. Well, django-coverage, if anything, has nothing to do with webtest, it just got it right here, to the heap.
1. install packages
2. add 'django_coverage' to INSTALLED_APPS
3. indicate in settings.py where to save the html reports. It would be nice to create a folder for this business.
4. write tests, inheriting our test case from django_webtest.WebTest and using self.app
5. run them:
If something does not work in webtest, django-webtest and django-coverage - write to Issues on bitbucket, I will try to help.
Right off the bat.
1. Install the necessary packages
$ pip install coverage> = 3.0 $ pip install -e hg + http: //bitbucket.org/kmike/webtest $ pip install django-webtest $ pip install django-coverage
WebTest is a library for functional testing of wsgi applications from Ian Bicking (author of pip and virtualenv). I added support for Unicode the other day (which is very important for us, right?), It has not yet been included in the main repository, so we put it from mine for now (you need installed mercurial, but you can also download zip- from the bitpack without it) archive, unzip and run setup.py install). Then, I think it will be possible to install c pypi. You can now install (pip install webtest) from there, only Unicode will not work in forms and links. ( UPD: everything has been included for a long time, you can install everything with pypi calmly )
Why WebTest, not twill? Twill also does not support Unicode (only it gets worse, not only Unicode, but even just non-Latin letters in utf8, as far as I can tell -UPD in the comments of meako writes that the lines just work, at least in conjunction with tddspry ), the last release was in 2007, there is a lot of code, outdated versions of the libraries in the delivery, and it seemed that to screw something there, it would take more effort . In general, twill is good, and html parsing is better there, so if that is, keep it in mind (and the django-test-utils package ( or tddspry? ) Then too).
Why not the built-in dzhangovsky test Client? WebTest has a much more powerful API; it is easier to write functional tests with its help. Read the docstring to django.test.client.Client:
This is not intended as a replacement for Twill / Selenium or the like - it is here to allow testing against the contexts and templates produced by a view, rather than the HTML rendered to the end-user.
Why not Selenium / windmill / ..? It does not interfere. They are testing another one though. For what you can use twill / WebTest, it is better to use twill / WebTest, because it will work much faster, have better integration with other code and easier setup.
2. Set up the project
A little tweaking is required for django-coverage. Don’t be scared, not big) You should:
1. add 'django_coverage' to INSTALLED_APPS and
2. in settings.py indicate where to save the html reports. It would be nice to create a folder for this business.
COVERAGE_REPORT_HTML_OUTPUT_DIR = os.path.join(PROJECT_PATH, 'cover')3. We write tests
Here, for example, is a functional test for registration / authorization: We save it in the tests.py file of the desired application in the project. It seems that everything should be clear here. We check whether the registered person can leave the site, then go to it (by entering his email and password), can he register (will he receive the letter? Is the activation link correct?) And gets to the right page after activating the account. For convenience, a fixture was also used, in which the kmike user with email=example@example.com was already prepared (and which is used in other tests). It was possible to create this user directly in the test, this is not the point.
Copy Source | Copy HTML
# coding: utf-8
import re
from django.core import mail
from django_webtest import WebTest
class AuthTest(WebTest):
fixtures = ['users.json']
def testLogoutAndLogin(self):
page = self.app.get('/', user='kmike')
page = page.click(u'Выйти').follow()
assert u'Выйти' not in page
login_form = page.click(u'Войти', index= 0).form
login_form['email'] = 'example@example.com'
login_form['password'] = '123'
result_page = login_form.submit().follow()
assert u'Войти' not in result_page
assert u'Выйти' in result_page
def testEmailRegister(self):
register_form = self.app.get('/').click(u'Регистрация').form
self.assertEqual(len(mail.outbox), 0)
register_form['email'] = 'example2@example.com'
register_form['password'] = '123'
assert u'Регистрация завершена' in register_form.submit().follow()
self.assertEqual(len(mail.outbox), 1)
# активируем аккаунт и проверяем, что после активации
# пользователь сразу видит свои покупки
mail_body = unicode(mail.outbox[ 0].body)
activate_link = re.search('(/activate/.*/)', mail_body).group(1)
activated_page = self.app.get(activate_link).follow()
assert u'Мои покупки
' in activated_page
Pay attention to the API: we follow the links indicating their name (.click (u'Registration '), for example), i.e. what the user actually presses on (there are other possibilities). At each transition, WebTest automatically checks that the code 200 or 302 has returned to us (this is configured). To submit forms, you do not need to construct POST requests manually, forms are picked up from the html response code, it is enough to assign values to the necessary fields and execute the submit () method. Redirects after POST requests are done manually (and this is useful, because if there is no redirect, for example, an error filling out the form, the test will show this).
django_webtest.WebTest is the descendant from the Jungian TestCase, it can do the same. But the main thing is that it has a self.app variable of type DjangoTestApp available (this is the successor to webtest.TestApp) through which you can access the WebTest API. More information about what WebTest can do is best read on their website . There is a simple and pleasant API, you can follow the links, submit forms, upload files, parse the answer (much higher level and concise than the Dzhangovsky test client). django_webtest adds one feature specific to the junga to the API: the self.app.get and self.app.post methods accept an optional parameter
user. If user is transferred, then the request (well, and all subsequent clicks, sending forms, etc.) will be executed on behalf of the dzhangovsky user with this username.It is clear that here it was possible to test the most, and it was possible less, and here it is good to maintain some kind of balance: so that the tests are not difficult to write and maintain, that they check everything that is needed, but do not check what is not needed. Sometimes it will be wrong to click on a link through its name, sometimes it will not be easy to check whether there is text on the page, sometimes even this check will be superfluous. This, I think, is called experience, when you understand how best. The way I wrote these tests is not necessarily the best way in this situation (although imho is quite adequate), just consider it as an example, and not as an example to follow, think about what you are writing. One of the advantages of simple APIs is that the programmer begins to think what to write and how to write better, and not "how to add something at last ..".
4. Run the tests
We create the file test_settings.py (in the root of the project) with the following contents (syntax for django 1.1): And then run the tests:
from settings import *
DATABASE_ENGINE = 'sqlite3'
DATABASE_NAME = 'testdb.sqlite'
$ python manage.py test_coverage myapp1 myapp2 myapp3 --settings = test_settings
You can do without test_settings (it’s easy to run
$ python manage.py test_coverage myappand don’t create any additional files), it’s more convenient: you can write any test-specific settings there, for example, use another DBMS to run tests faster or replace URLOpener for urllib2 so that tests don’t surfing the internet. It is convenient to wrap the command to run the tests in a shell script (or a bat file, if someone has the misfortune to write in python under windows)5. We look at pictures
The code coverage report was saved in the previously specified folder. Open it (file cover / index.html) and see something like this:
We click on some link and see which code we executed during the tests, and which did not execute (and therefore could not be tested):
... many lines ...
... many lines ...
Yeah! It is immediately obvious that we did not check the situation when a person entered the email address of an already registered user.
It is important to remember that functional / integration tests are not a substitute for unit tests., but only an addition to them, and that 100% coverage does not guarantee the absence of errors. Unit tests are accurate, they say WHAT broke, they are extremely useful in refactoring and in difficult places in a project. Functional ones are rude, they only say “something seems to have broken somewhere” and they protect against stupid mistakes. But even if the tests just click on all the links on the site and check if an exception has occurred, then these will already be very useful tests that can save you from a bunch of troubles.
To illustrate the difference: in the unit test for the registration form, we would create an object of the EmailRegistrationForm class, pass different dictionaries with data into it and look at what exceptions are being raised, for example. Or they would test individual methods of this form. Unit tests are as close as possible to the code (although it makes sense not to let them go outside the public API), they test a separate piece of it, and allow us to verify that all parts of the system individually work correctly. Functional / integration tests help verify that they work together correctly too.
6. All links
docs.djangoproject.com/en/dev/topics/testing
pythonpaste.org/webtest
bitbucket.org/ianb/webtest
bitbucket.org/kmike/webtest
bitbucket.org/kmike/django-webtest
bitbucket.org/kmike/django-coverage
github.com/ericholscher/django-test-utils
twill.idyll.org
nedbatchelder.com/code/coverage
Yes, all this can be used just as easily without django, WebTest is very easy to screw on to any framework that supports wsgi (and its support "all attention-worthy frameworks"), coverage.py works great for any tests. All of these django- ... applications are just to make installation and configuration as simple as possible. Well, django-coverage, if anything, has nothing to do with webtest, it just got it right here, to the heap.
7. Brief instructions
1. install packages
2. add 'django_coverage' to INSTALLED_APPS
3. indicate in settings.py where to save the html reports. It would be nice to create a folder for this business.
COVERAGE_REPORT_HTML_OUTPUT_DIR = os.path.join(PROJECT_PATH, 'cover')4. write tests, inheriting our test case from django_webtest.WebTest and using self.app
5. run them:
$ python manage.py test_coverage myapp1 myapp2 myapp3 --settings=test_settingsIf something does not work in webtest, django-webtest and django-coverage - write to Issues on bitbucket, I will try to help.