February 2, 2014 at 04:18The easiest way to add WebSocket to Django
- Transfer
Note of the translator: web sockets and Django is a rather complicated topic that has been raised more than once on habrahabr and the main idea is to write a parallel backend for web sockets. The author offers a rather concise solution to this problem, which truth still has a test of time.
TL; DR - I came up with a very simple solution for working with web sockets in Django. All you need to do is install django-websocket-request , run the script, and now your application supports websockets! This solution makes Django think it is receiving a normal (to some extent) HTTP request, so it will be compatible with almost all of your existing code. Solution works fine as with Django Rest Framework, and with the usual functions-representations and representations based on classes (Class Based Views).
We are developing Blimp 2 - this means that we constantly have a lot of changes in the code and infrastructure, and how we solve old and new problems. One of the decisions that we made regarding our application has significantly changed the interaction of the frontend and backend.
Blimp is currently running on Django. It serves our HTML code, processes our public and private API, all business logic. Until recently, most web applications were built like this, but Blimp has a thick JavaScript client. With normal requests, everything happens in the following way: you request a URL, some work is done on the backend side - database requests, caching, data processing, rendering HTML pages, loading third-party CSS and JavaScript libraries, loading our own JavaScript applications, some more data processing, and finally drawing the result.
After several months of practice and growth of our project, we found several key improvements that we can implement - the new version of the backend will have to do JSON generation, but not HTML rendering, and our front-end application, which is executed by the client, will simply consume data through the API. We decided that it would use websockets where possible, and regular XHR in all other cases.
The Django web framework and the like are built to work with the HTTP request / response cycle, so everything in them, both views and authentication mechanisms, accepts an HTTP request at the input and send an HTTP response at the output. On the other hand, the web socket server does not know anything about such a cycle.
Our main goals were:
1. The same mechanisms for serializing and deserializing data, both from the HTTP backend and from the web socket API.
2. The same business logic for everyone.
The first thought that came to mind was the transformation of all business logic into the methods of our models. This way we could write the code once and split it between the two backends. At that moment, this seemed like the right decision, but a few hours later it was proved that no. Since we are working on top of the Django REST framework, we would be forced to inherit and modify dozens of its classes and impurities, which actually didn’t sound so bad, but we were skeptical enough and decided to look for other ideas.
We knew that we wanted a single REST API that would work on two channels: HTTP and web sockets. The best option would be to avoid rewriting anything just to work with web sockets. At one point, it dawned on me. I remembered Sails.js , which does something similar with what we want to achieve.
Sails supports data exchange that is not tied to a specific mechanism, which will allow your handlers to automatically work with Socket.io and web sockets. In the past, you would have to write separate code to get such a result.
In simpler terms, we wanted to implement a data exchange mechanism that was not attached to anything specific, which would allow us to use everything that is from the Django request / response cycle to automatically generate web socket messages.
WebSocketRequest turned out to be an unexpectedly simple solution. This is a simple class that accepts a JSON string containing the following keys: method, url, data, and token. The method key can be any HTTP method: GET, POST, PUT, DELETE, PATCH, HEAD, or OPTIONS. The url key is an absolute URL without a domain name. The data key is an optional parameter - a dictionary containing data. The token key is also optional - used to recreate the HTTP authorization header, authorization via JSON Web Token, or for your own keys. You can look at my article to learn more about JSON Web Token, and if you use the Django REST framework, then you will probably like django-rest-framework-jwt .
WebSocketRequest works as follows:
Yes, yes, the request factory, you read it right. You may be familiar with it if you have ever written tests for Django applications, but if not, the request factory is committed to generating a request object that can be used as an argument for any view. The only negative is that such a request does not support the middleware mechanism , which for some may be a problem.
I would definitely like to hear about the possible problems of this approach. How can it be improved? What could break? What about industrial use?
Note that Django is not used at all in this example. Tornado issues a static HTML file and passes all the django-websocket-request web socket requests, in which all the magic happens.
I installed the demo application on Heroku: http://dwr-example.herokuapp.com/ . Be careful, data is erased periodically. If you find some kind of error, write to me on Twitter about it.
You can install WebSocketRequest using pip:
Source code:
https://github.com/GetBlimp/django-websocket-request
Demo application:
https://github.com/GetBlimp/django-websocket-request-example
TL; DR - I came up with a very simple solution for working with web sockets in Django. All you need to do is install django-websocket-request , run the script, and now your application supports websockets! This solution makes Django think it is receiving a normal (to some extent) HTTP request, so it will be compatible with almost all of your existing code. Solution works fine as with Django Rest Framework, and with the usual functions-representations and representations based on classes (Class Based Views).
More details
We are developing Blimp 2 - this means that we constantly have a lot of changes in the code and infrastructure, and how we solve old and new problems. One of the decisions that we made regarding our application has significantly changed the interaction of the frontend and backend.
Blimp is currently running on Django. It serves our HTML code, processes our public and private API, all business logic. Until recently, most web applications were built like this, but Blimp has a thick JavaScript client. With normal requests, everything happens in the following way: you request a URL, some work is done on the backend side - database requests, caching, data processing, rendering HTML pages, loading third-party CSS and JavaScript libraries, loading our own JavaScript applications, some more data processing, and finally drawing the result.
After several months of practice and growth of our project, we found several key improvements that we can implement - the new version of the backend will have to do JSON generation, but not HTML rendering, and our front-end application, which is executed by the client, will simply consume data through the API. We decided that it would use websockets where possible, and regular XHR in all other cases.
The Django web framework and the like are built to work with the HTTP request / response cycle, so everything in them, both views and authentication mechanisms, accepts an HTTP request at the input and send an HTTP response at the output. On the other hand, the web socket server does not know anything about such a cycle.
Our main goals were:
1. The same mechanisms for serializing and deserializing data, both from the HTTP backend and from the web socket API.
2. The same business logic for everyone.
The first thought that came to mind was the transformation of all business logic into the methods of our models. This way we could write the code once and split it between the two backends. At that moment, this seemed like the right decision, but a few hours later it was proved that no. Since we are working on top of the Django REST framework, we would be forced to inherit and modify dozens of its classes and impurities, which actually didn’t sound so bad, but we were skeptical enough and decided to look for other ideas.
We knew that we wanted a single REST API that would work on two channels: HTTP and web sockets. The best option would be to avoid rewriting anything just to work with web sockets. At one point, it dawned on me. I remembered Sails.js , which does something similar with what we want to achieve.
Sails supports data exchange that is not tied to a specific mechanism, which will allow your handlers to automatically work with Socket.io and web sockets. In the past, you would have to write separate code to get such a result.
In simpler terms, we wanted to implement a data exchange mechanism that was not attached to anything specific, which would allow us to use everything that is from the Django request / response cycle to automatically generate web socket messages.
Decision
WebSocketRequest turned out to be an unexpectedly simple solution. This is a simple class that accepts a JSON string containing the following keys: method, url, data, and token. The method key can be any HTTP method: GET, POST, PUT, DELETE, PATCH, HEAD, or OPTIONS. The url key is an absolute URL without a domain name. The data key is an optional parameter - a dictionary containing data. The token key is also optional - used to recreate the HTTP authorization header, authorization via JSON Web Token, or for your own keys. You can look at my article to learn more about JSON Web Token, and if you use the Django REST framework, then you will probably like django-rest-framework-jwt .
WebSocketRequest works as follows:
- Checks the received JSON string
- Creates an instance of the request factory (RequestFactory)
- Dynamically calls one of the request factory methods, which returns an instance of WSGIRequest
- Finds the appropriate URL representation
- Runs the found view along with all the data that came from the URL.
Yes, yes, the request factory, you read it right. You may be familiar with it if you have ever written tests for Django applications, but if not, the request factory is committed to generating a request object that can be used as an argument for any view. The only negative is that such a request does not support the middleware mechanism , which for some may be a problem.
I would definitely like to hear about the possible problems of this approach. How can it be improved? What could break? What about industrial use?
Demo
Note that Django is not used at all in this example. Tornado issues a static HTML file and passes all the django-websocket-request web socket requests, in which all the magic happens.
I installed the demo application on Heroku: http://dwr-example.herokuapp.com/ . Be careful, data is erased periodically. If you find some kind of error, write to me on Twitter about it.
You can install WebSocketRequest using pip:
pip install django-websocket-requestSource code:
https://github.com/GetBlimp/django-websocket-request
Demo application:
https://github.com/GetBlimp/django-websocket-request-example