The dark side of the hackathons
In the previous part of the trilogy, I examined several reasons for participating in hackathons. The motivation to learn a lot and win valuable prizes attracts many, but often because of the mistakes of the organizers or sponsor companies, the event ends unsuccessfully and the participants leave dissatisfied. So that such unpleasant cases occur less often, I wrote this post. The second part of the trilogy is devoted to the mistakes of the organizers.
The post is organized as follows: at the beginning I talk about the event, explain what went wrong and what it led to (or can lead to in the long run). Then I give my assessment of what is happening, and how I would act in the place of the organizers. Since I acted as a participant at all events, I can only assume the true motivation of the organizers. As a result of this, my assessment may turn out to be one-sided. I do not exclude that some of the points that I see as erroneous were actually conceived.
At some point, it may seem to the reader that the author decided to wave his fists after the fight. But I can assure you that this is not so. In some of the listed hackathons, I managed to take a prize, which, however, does not prevent me from saying that the event was poorly organized.
Due to respect for the organizers and participants, the post will not indicate specific companies. An attentive reader, however, can guess (or google) who they are talking about.
Hackathon No. 1. Strict framework
Six months ago, a large telecom company organized a data analysis hackathon. For the prize fund fought 20 teams. At the event, a dataset was provided for analysis, which contained information about calls to the company's support service, activity on social networks and encoded information about users (gender, age, etc.). The most interesting part of the dataset - user messages and operator response (text data) - was quite “noisy”, it needed to be cleaned for further work.
The organizers set the task - to do something interesting with the data provided, and it was forbidden to use additional open datasets from the network or parse the data yourself. It was also forbidden to offer ideas not related to the dataset. Unfortunately, the data provided were rather “poor”: it was difficult to get any interesting products from them, and from communicating with mentors it became clear that many of the ideas proposed were already being implemented (or would be implemented in the near future) in the company.
As a result, the vast majority of teams (15 out of 20) made chat bots. During the speeches, the decision of one team was not much different from the previous one. Unbearable, one of the members of the jury asked the next team on the stage: “What, guys, do you also have a chat bot?” As a result, of the three prizes, the first and second places went to teams that did not make chat bots.
For comparison, take the hackathon, organized by an international consulting company, for the Zvezdochka firm two years ago. Since the specifics of the Zvezdochka company was not familiar to many hackathon participants, at the beginning of the event, the organizers talked about the metrics that are used in the company. After that, six datasets of various orientations were provided: text, tables, geo-positions - there was room for maneuver for all participants. The organizers did not prohibit the use of additional datasets and even supported such undertakings. In the final of the competition, ten teams with different decisions fought for the main prize, and all the teams used the data provided by the company (despite the absence of prohibitions), which indicated good potential for obtaining quality products.
Morality
Do not limit the creative flow of participants. As an organizer, you must provide materials and trust their views and professionalism. If you are a member of a hackathon, any restrictions or prohibitions should alert you. Usually this is evidence of poor organization (an example from real life is a constant desire to stick a fence somewhere). If you still encounter restrictions, then be prepared for the fact that you have to create a project in the pool with great competition. In this case, you must take risks: do something fundamentally new or offer an unusual “killer feature” in order to stand out from the flow of monotonous projects.
Hackathon number 2. Impossible tasks
The hackathon in Amadore promised to be interesting. The sponsor company, a major manufacturer of telephones, began preparations 4 months before the date of the event. A social event was held in social networks, potential participants had to pass a technical test and write about their past projects in order to be selected for this event. The prize pool was pleasantly large. A few days before the hackathon, the mentors held a technical session so that the participants had time to penetrate the specifics of the industry.
At the event, the organizers provided an 8 Gb equipment dataset with a binary classification of breakdowns. They talked about the criteria for evaluating projects - the quality of classification, the creativity of creating features, the ability to work in a team, etc. That's just bad luck - for 8 GB of “features”, there were only 20 examples in the train and 5 in the test. The final nail in the lid of the hackathon’s coffin scored a face in the data: the equipment logs received on Wednesday contained an error in the equipment’s operation, and the ones created on Thursday did not contain (by the way, only two teams knew about it, and both were from Russia - the homeland of experienced datameners ) Although even knowing the true labels of the test did not help to tailor the answer, the task was unsolvable. The organizers did not get the desired result, the participants spent a lot of time solving a poorly composed task. The hackathon was a failure.
Morality
Perform technical examinations of assignments and check your assignments for adequacy. It’s better to overpay for a preliminary examination (in this case, any datacenter would immediately indicate that it is impossible to solve this problem) than to regret it later.
In this case, in addition to the time and money spent, the company lost a trust loan from potential candidates and it is possible to write about the results. By the way, not only the participants, but also the company should write about successful results, maximally realizing the hackathon from the point of view of PR. Unfortunately, not all companies do this, limiting themselves only to a post-announcement and a couple of photos from the event on Twitter.
Hackathon number 3. Take it or leave it
Most recently, our team participated in a hackathon in Amsterdam. Since I am an electrical engineer by training (in the field of renewable energy sources), the subject was just for us - energy. The hackathon was conducted in an online format: we were given a description of the assignment and a month to complete. The organizers wanted to see a finished project that will help increase the energy efficiency of Amsterdam houses.
We made a project in which electricity consumption is predicted (before that I participated in a competition on this topic where I got a near-sota solution, which can be read about here) and solar panel generation. Based on these predictions, the battery performance is optimized (this idea was partially taken from my master's diploma). Our project was in good agreement both with the task from the organizers (as it seemed to us then) and with the policy of the Amsterdam administration in the field of renewable energy for several years to come.
During the evaluation of projects, we, like many teams, were told that this was not what the customer expected, adding that we had to redo the project if we wanted to compete for a prize. We did not redo anything, resigned to defeat. Of the forty participating teams, we did not even go into the top 7, although the choice of organizers, it seems to me, was rather strange. For example, they missed the final team, which made an application for calculating wind speed and solar radiation (SI) according to smartphone sensors: microphone for wind, light sensor for SI. The feature killer had a classification of hotdog / not hotdog into three classes: the Sun, wind, water and the display of the corresponding Wikipedia article ( demo ).
Let us leave for a moment the moral side of the issue: blackmailing participants with the possibility of victory is simply unethical. Since one of the motivations for participating in hackathons (especially experienced developers) is the realization of their ideas, many strong participants can simply leave the event after hearing such feedback (which happened not only with our team, but also with a number of others who stopped updating the page of your project after listening to the mentor). Nevertheless, let's say we agreed with the organizers and redesigned our project to their requirements. What could happen next?
Since the organizers have their own understanding of the “ideal project”, all wishes (and, accordingly, changes) will lead us to this ideal. Contestants will spend their time and it will be harder and harder for them to refuse to participate further (since they have already invested their strength, and it seems that they won’t be able to win before). But in reality, competition for prizes will increase, and participants will increasingly have to redo the project for corrections from the organizers in the hope of receiving a prize. As a result, the guys who did not take prizes, looking back will realize that they participated in freelance without money: they made changes to the customer, but at the same time did not receive anything in return (except for the relevant experience, of course).
Morality
Often, wishes and feedback from the organizers come to the aid of the project. In this case, however, participants should not rely on the advice of mentors as a lame one on a cane. If you hear from the organizers the feedback on your project in the spirit of “take it away, we did not order it” - your participation in the hackathon on this can be considered completed.
If you are organizing a hackathon with a clear vision of the project, but without the skills or ability to implement it yourself, it is better to design your vision in the form of a technical task for a freelancer. Otherwise, you will have to pay twice - for the hackathon and for the services of a freelancer.