Running with obstacles or creating one project

What are you talking about

Chefonline.ru is a culinary portal, no matter how trite it may sound. We have a personal and general recipe book, it is possible to create shopping lists, plan menus, store favorites, you can write in a collective blog or just read it, there are other interesting and useful services for lovers and sympathizers.

The portal is quite young, although a lot has been done, but much is still under development or in the form of ideas, so I would like to talk not about the project itself, but about the process of its creation. I hope our experience will be useful to Habr-people.

Initial data

Apparently, you should not go into details how exactly I sat and thought, and how exactly the idea came to my mind to create a service for culinary and sympathizers. I’ll just say that the service did based on my needs and Wishlist, as I myself love to cook.

At the start of development, the following resources were available:

• The programmer, he is the layout designer (I)
• Designer
• Own under-framework, on which a dozen projects have already been implemented
• Shared hosting
• Some experience in web development and project management
• Quite a bit of free time

What was used during development:

• PHP5 fast-cgi + nginx
• MySQL5
• Smarty
• Jquery

I came up with a domain name and registered it without too much pain for an hour of enumerating possible options, reasoning that it’s not much to philosophize and come up with something tricky “start-up”. So, you can say, chefonline.ru was born.

At the time of launch, I had (and still have) the main work with a busy schedule, plus the autumn crisis of 2008 was gaining little momentum. So I had time for the project on weekdays for about an hour, plus several hours on weekends - to sit over the project at night and for the whole weekend I had neither strength nor desire. In general, we can only call what we did as a startup, only with a stretch and terminological reservations.

Start of development and architecture

Before starting development, I did not sit at night and do not design the architecture, did not suffer from the choice of technical solutions and did not get killed about the ideal database structure. Since the site was born out of fairly clear and understandable goals and needs for me, everything in my head was built quite clearly, at least at a high level and without going into too much detail - experience suggested that if you don’t design, you still have to then finish and remake during the project. I just painted a data interaction scheme in a couple of hours, threw a database structure and figured out big cases of cases. Many solutions have been tested for a long time and were not in doubt, the frame of almost all modules was pulled out of past projects. So I started writing code on the second day after the project idea arose.

The main difficulty was to think over a system of a book of recipes and user interactions within its framework - it could be either general, with one section structure, or personalized - with the ability to edit the structure, including creating it from scratch. Categories should have a tree structure. In my “framework” until that moment I used “NestedSets” and did not know grief, but the algorithm was not suitable for this specific task - constant inserts and updates could occur in the tree, and nested sets are not suitable for this.

Having break a bunch of material and having tried almost everything that is on tree structures, I finally came to the banal "id-parentId", but I did not select the recursion, but through JOIN (the methods are described on the dklab.ru forum ).

This method has some limitations and nuances (in particular, the level of nesting), but in general, after a synthetic test on a large base with a level of nesting 10 (and I can hardly imagine a recipe book with more nesting) this algorithm satisfied me and was eventually written class work with him. It was also decided to distribute the general catalog and personalized catalogs according to different tables in order to optimize the reading speed, in particular from the general recipe book - its categories are also used for the general recipe book displayed in the public part of the site.

In addition to the structure of the recipe catalog, I was worried about the use of Jquery, which I never used and actually combined the pleasant with the useful - the development and simultaneous study of the framework. Reading docks on Jquery and finding solutions took up most of the development time, writing code in PHP went pretty quickly and without any problems.

All ideas for the project were collected in a list (aka backlog, in terms of scrum), sorted by priority and divided into sprints, two months long. Thus, it was clear what the portal would turn into in 4-6-10 sprints, this simplified the adoption of some architectural decisions.

Design and usability

The design had the biggest problems. The designer worked on the project, but rather slowly, the design concept was obtained only 4 months after the start of work on the project, after that there were two more months of painful editing and elaboration of the internal pages. Usability was thought out together to the best of our abilities and knowledge, and we devoted maximum time to this issue and tried to strike a balance between beauty and functionality - it turned out or not, it was up to the users to judge.

Serious danger lurked after obtaining the first results on the concept - there were thoughts to accept what is, because there is nowhere to pull further - it may not be the way you want, but at least something. It is good that the excerpts were enough for disputes, and for abandoning the primary concept, and for further corrections and improvements. There were quite a few improvements - the concept was redone three times, the main page was completely redone two, and well, the trivialities were decent enough.

Involvement in the project had a huge impact on the development of the interface and its results - I myself as a user of the service could give feedback as quickly as possible - what and why do I need, what place to put and what effect this will have on the general interface of the site. It was quite difficult to ignore the fact that I, as a developer, go along the same paths and many things that seemed obvious to me turned out to be completely unobvious to users. The first beta testers helped me cope with this, and I express my great appreciation to them - many annoying bugs and interface problems were fixed with their help. It was difficult and a little disappointing to accept that the personal recipe book that we were so proud of, which we liked and seemed comfortable, According to the results of the first usability tests, it turned out to be completely unobvious to users. Resigned, of course, and redid. It was a good experience in acknowledging my mistakes, but there aren’t many of them.

As for the long rendering of models, practice has shown that no matter what personal relationships are, working relationships always need to be worked out, negotiated and agreed upon, working methods and interactions worked out in advance, as they say - on the shore. Hoping for consciousness or telepathy is pointless and harmful. After six months of unproductive work, we nevertheless came to an agreement on how and how we would work, after which the matter went much faster. In particular, every Sunday we met at my place and worked on the project almost all day in a calm atmosphere, and in the evening we tested new dishes, combining business with pleasure :)

Actually, the result of work on the visual component can already be estimated.

Performance optimization

Initially, it was obvious that a large amount of javascript and weak hosting would become bottlenecks, closer to the summer this was confirmed by performance tests and stress tests of an almost finished project. It was decided to carry out the following activities:

• Rent a dedicated server or at least VPS, where there is also the possibility of tuning and optimization
• Cache most statics and database queries in memcache
• Combine and compress javascripts and css

As a result, I transferred the project to VPS, and I had to configure everything myself (without any experience in administering * nix systems) - this is a separate fascinating story from which I learned a lot of experience and useful skills, as well as the false feeling that I am now king of the world. The result of my weekly efforts was the following bundle: nginx + php5-fastcgi + mysql5 + xcache + memcached. With this, I met my needs for server optimization (the bottleneck was still kernel settings that cannot be changed on VPS, as well as a rather weak provider channel) and switched to the client one.

Unfortunately, in terms of client optimization, not much has been done as planned. It remains to do the combination of javascripts and css, optimize the DOM, minimize the number of http requests, cache mechanisms are not fully implemented.

Total

A year has passed since the start of the project, during this time not so much was done as we would like, more mistakes were made than we would like, but a lot of experience was gained. This experience can be stated in several theses, which, in fact, are nothing more than once again confirmed by commonplace truths.

1. The involvement of project participants in the subject area is a big plus for the project.
2. It is not always worth putting off project implementation and diving too deep into planning and design. Iterative development, frequent demonstrations, getting maximum feedback - you need to strive for this.
3. Using your services (as an end user) from the very first prototype, you can avoid many problems in the future.
4. If the team members have a different level of motivation - communicate, develop common approaches, solve problems immediately after identifying them, do not delay or try to “hush them” - it will not resolve itself.
5. It is important to find a middle ground between perfectionism and pragmatism, in some cases painful refactoring will save a lot of time in the future, in some excessive pedantry can lead to the linking of the project in a swamp of improvements.
6. Less bicycles - everything has already been invented before us, and it’s good that I personally already realized this for myself, if I had started the project a couple of years earlier and I would have spent much more time thinking up my bicycles, rather than using existing material and other people's achievements.
7. Do not be shy to show the project, do not be afraid of negative feedback - this is a very important feedback and the more you get it in the early stages, the better the final product will be.
8. Development alone or in a small team of friends does not mean that the development process can be neglected. We use many scrum techniques and do not at all consider this a waste of time or excess, on the contrary, we constantly observe positive aspects.
9. Do not bet on the primary UGC, when developing a project, there should be a plan for filling it.

Update: I thank everyone who wrote about the errors found in the comments or in the feedback on the site. For mistakes, of course, I am ashamed, but it is better to fix them sooner rather than later. I also thank everyone who expressed useful and reasonable comments on the design, usability and technical part of the site.