November 23, 2015 at 11:22Codesign.io - architecture that works
Hi guys. My name is Pavel, I am a co-founder of Codesign service and I am responsible for its technical component. Codesign is a web service for creating and discussing feedback when working on websites, design layouts and presentations. If designers still write edits for you in full text in a letter, then it's time to tie it up and switch to Codesign.
We launched the service on July 22, 2015, and since then we have recruited 3,500 registered users, survived the launch on ProductHunt when 10,000 people visited us in 36 hours, received our first paid users and are discussing receiving funding from European ( I live in Italy) investors. In this article I will tell you how we built the service architecture and organized the development process.
When developing Codesign, we strove for modularity, considering that in this case it would be easier to make changes and replace one block with another. The first serious decision was to completely separate the API and the web application, so that they work on different servers. Since the team had experience with Python / Django , it was decided to build an API based on the Django Rest Framework . We initially planned to write a web application in AngularJS , but the hired front-end developer convinced us to implement it in React . The server side of the web application is implemented using the Express framework on Node.js.
Due to the fact that we wanted to spend as little time as possible on server configuration and be able to scale quickly (if we expect a large jump in traffic, for example, after publication on a popular resource), we decided to deploy our code on Heroku . Thus, on Heroku we have 2 servers for the API (production / development) and 2 servers for the web application (production / development). At Heroku at a free ratethe server falls asleep every hour and should be inactive for more than 6 hours a day. I had several servers raised several months before the advent of new tariffs, and there is no such restriction on them. The web application and API work on free servers, which we periodically scale when we expect traffic to flow. The main advantage of Heroku, I consider the simplicity of deployment and its GIT basis, which allows you to automatically deploy code from Github. This process is described in more detail below.
Heroku has a large number of different plug-ins , among which there are also database management systems (DBMS). Prior to the July 22 release, we had a prototype using MySQL and we decided to continue working within the framework of this DBMS and connected ClearDB . The free tariff has a database limitation of 5MB, so we first switched to the Punch tariff, and then Drift for $ 50 per month. ClearDB is now the most expensive component of technology services (only Intercom is more expensive , but I will not talk about it in this article) for which we pay. I thought that $ 50 a month was expensive and I thought that on Amazon RDS I will get higher performance for the same money. After a series of experiments, I realized that this is not so, because I only received similar query execution speed on RDS instances that cost more than $ 50, so we stayed with ClearDB.
Codesign is based on graphic files that users download for further discussion. Here we did not have a long discussion and stopped at Amazon S3 , where, with our traffic and the number of stored images, we pay $ 3-4 per month. On slow Internet, downloading images over 4MB took more than 30 seconds and there was a timeout error from Heroku . To solve this problem, we began to only generate a special link on the server, through which you can upload pictures directly to Amazon S3 from a browser . So the problem was resolved.
When one user makes changes to the board (a collection of pictures for discussion) on Codesign, other participants in this board receive notifications. To implement these notifications, we use two services: Heroku task scheduler to initiate sending notifications every hour, and Mandrill to send emails as such.
Django itself allows you to send reports of 500 errors that occur on the server. It's comfortable. But the Opbeat service turned out to be even more convenient . It collects information on errors, shows statistics of their occurrence, assigns each group of errors to the person who committed the code in which this error occurred. In addition, Opbeat monitors server requests and shows performance bottlenecks. There are many similar services., but Opbeat bribed up to 3 users with its simplicity and free of charge. Once we had a problem with the fact that the endpoint to receive the list of boards in the folder took 4 seconds. We could not understand what exactly the problem was. Opbeat showed that we are very non-optimal in making requests to verify user rights. We optimized the code and endpoint execution accelerated by 20 times.
Our development is currently fully built on GIT:
Further we plan to introduce the following tools:
This article is an example of how various third-party services can be integrated to simplify work on your own product. We did not immediately come to the architecture that you see in this article, but gradually - by trial and error. We do not pretend or agitate that this architecture is ideal. We are open to discussion and will be very happy to hear the feedback on the architecture that we use.
If you like Codesign (or don’t like it and you want to change it), come to work with us - we have 4 deleted jobs open .
We launched the service on July 22, 2015, and since then we have recruited 3,500 registered users, survived the launch on ProductHunt when 10,000 people visited us in 36 hours, received our first paid users and are discussing receiving funding from European ( I live in Italy) investors. In this article I will tell you how we built the service architecture and organized the development process.
Frameworks
When developing Codesign, we strove for modularity, considering that in this case it would be easier to make changes and replace one block with another. The first serious decision was to completely separate the API and the web application, so that they work on different servers. Since the team had experience with Python / Django , it was decided to build an API based on the Django Rest Framework . We initially planned to write a web application in AngularJS , but the hired front-end developer convinced us to implement it in React . The server side of the web application is implemented using the Express framework on Node.js.
Deployment
Due to the fact that we wanted to spend as little time as possible on server configuration and be able to scale quickly (if we expect a large jump in traffic, for example, after publication on a popular resource), we decided to deploy our code on Heroku . Thus, on Heroku we have 2 servers for the API (production / development) and 2 servers for the web application (production / development). At Heroku at a free ratethe server falls asleep every hour and should be inactive for more than 6 hours a day. I had several servers raised several months before the advent of new tariffs, and there is no such restriction on them. The web application and API work on free servers, which we periodically scale when we expect traffic to flow. The main advantage of Heroku, I consider the simplicity of deployment and its GIT basis, which allows you to automatically deploy code from Github. This process is described in more detail below.
Database
Heroku has a large number of different plug-ins , among which there are also database management systems (DBMS). Prior to the July 22 release, we had a prototype using MySQL and we decided to continue working within the framework of this DBMS and connected ClearDB . The free tariff has a database limitation of 5MB, so we first switched to the Punch tariff, and then Drift for $ 50 per month. ClearDB is now the most expensive component of technology services (only Intercom is more expensive , but I will not talk about it in this article) for which we pay. I thought that $ 50 a month was expensive and I thought that on Amazon RDS I will get higher performance for the same money. After a series of experiments, I realized that this is not so, because I only received similar query execution speed on RDS instances that cost more than $ 50, so we stayed with ClearDB.
Storage
Codesign is based on graphic files that users download for further discussion. Here we did not have a long discussion and stopped at Amazon S3 , where, with our traffic and the number of stored images, we pay $ 3-4 per month. On slow Internet, downloading images over 4MB took more than 30 seconds and there was a timeout error from Heroku . To solve this problem, we began to only generate a special link on the server, through which you can upload pictures directly to Amazon S3 from a browser . So the problem was resolved.
Notifications
When one user makes changes to the board (a collection of pictures for discussion) on Codesign, other participants in this board receive notifications. To implement these notifications, we use two services: Heroku task scheduler to initiate sending notifications every hour, and Mandrill to send emails as such.
Monitoring
Django itself allows you to send reports of 500 errors that occur on the server. It's comfortable. But the Opbeat service turned out to be even more convenient . It collects information on errors, shows statistics of their occurrence, assigns each group of errors to the person who committed the code in which this error occurred. In addition, Opbeat monitors server requests and shows performance bottlenecks. There are many similar services., but Opbeat bribed up to 3 users with its simplicity and free of charge. Once we had a problem with the fact that the endpoint to receive the list of boards in the folder took 4 seconds. We could not understand what exactly the problem was. Opbeat showed that we are very non-optimal in making requests to verify user rights. We optimized the code and endpoint execution accelerated by 20 times.
Development
Our development is currently fully built on GIT:
- First, the developer makes changes and uploads the code to the development branch in Github . It is impossible to push code directly into the master branch in Github, this is prohibited in the repository settings.
- At this point, a webhook is triggered and the repository code is redeployed to the Heroku development server.
- After making sure that the code works adequately, we create a pull request in the master branch, upon acceptance of which the code is automatically sent to the Heroku production server. If something went wrong and some kind of error was found, then Heroku makes it very easy to roll back to previous versions.
Future
Further we plan to introduce the following tools:
- Heroku review apps - for parallel testing of different features separately following git flow (now we have only one development server)
- Travis - to run unit tests with every push on Github
- CodeClimate - for automatic code analysis (will lead to high quality code)
- Amazon RDS - when we do not have enough tariffs that ClearDB offers, it will be necessary to deploy more powerful RDS instances
- Firebase - to implement real time updates of some blocks
- Stripe - to accept payments (now using Gumroad)
Conclusion
This article is an example of how various third-party services can be integrated to simplify work on your own product. We did not immediately come to the architecture that you see in this article, but gradually - by trial and error. We do not pretend or agitate that this architecture is ideal. We are open to discussion and will be very happy to hear the feedback on the architecture that we use.
If you like Codesign (or don’t like it and you want to change it), come to work with us - we have 4 deleted jobs open .