Is there any powder in the flasks? Hackathon Radio Canada 2018 (Part Two - Alertness)

    This is the second part of my story (sincerely thought that the latter).

    The first part is here .
    The third part


    So, formally, the team is formed. Although no one knows anyone yet. The purpose of the project is not defined. Radio Canada promises to provide some kind of API a week before the event, so that the participants have some time to familiarize themselves.

    As a person with certain experience in various projects, I understood from the very beginning that writing any working prototype in a couple of days is not an easy task. Considering that the team consists of participants with undefined skills and experience in various languages ​​and systems and never working together before, I realized that the likelihood of creating something working in general is not very real.

    During my attempts to organize a meeting of the whole team before the start of the hackathon, I first spoke with Zvezdochka, and then with Plato, about my own and, as it turned out, doubts about the success of the event.

    At this stage, we agreed that our goal is not victory, but participation is time.
    We want to get a working prototype that will not be ashamed to show each other, well, or Zvezdochka with Mercury in college - these are two.

    To reach point two, we need to create something minimally satisfying the conditions of the hackathon.

    And the conditions, if you remember the following:

    • Using the API from Radio Canada (it was logical to assume that it would be some kind of content: news, video, audio, possibly including additional / auxiliary data: region, date, artist, category, etc.). Total API from Radio Canada - some content.
    • Using Azure . This item seemed the simplest. I assumed that at least every developer would raise a server for development (each judge by itself, I was half right). Well, since the next paragraph will be AI, it was clear that we would need to take some kind of service related to this phrase.
    • As I just recalled, the hackathon theme is AI . So some use of AI needs to be hooked on the content that we get from Radio Canada
    • The interface must comply with WCAG 2.0 AA . I did a couple of searches for this standard. I realized that in practice, our task will be: format the HTML according to some rules there. But, for myself, I decided that we would seriously do this if there was time, and the prototype itself would exist and somehow function. Otherwise, somehow it turns out that the cart is in front of the horse.
    • Teams may use other public services and data sources. At first glance, this paragraph does not sound like a requirement, but rather as a resolution. But I would probably interpret it as a call not to use any proprietary software, i.e. Avoid any opportunity to receive future licensing and copyright issues. (Long live the open source!)
    • The entire prototype , from architecture design to code and other elements, must be created during the hackathon . On the one hand, the desire of the organizers to level the chances of the teams is understandable. On the other hand, I understood very well that the organizers did not have the opportunity to trace how many lines of code were written before, and how many in time. Therefore, I was not strongly opposed if we had something to bring “with us” and had no illusions that some teams would also come a bit more prepared than others.

    Idea for a project
    I do not know what ideas arise in your head. But I was steadily arising a chain: a lot of content - received - sent to the AI ​​for analysis - received the results of the analysis - visualized. I twisted it and that way. More so than that, since the main task is simply to do something that will work, and not to create something impressive, excuse me. In total, it turned out that we will have some content analyzed by the AI, which for some reason we must somehow visualize.

    It is worth noting that having come to one of the main pages of Radio Canada we found one wonderful fact - they do not have a Search String!


    From that moment, everything worked out. Everyone agreed that we would do some kind of search string.

    And everything would be fine if we didn’t have to stick the AI ​​somewhere, which we don’t really need to search. They agreed that we would take some kind of audio (perhaps pick it out from the video) - send it for recognition in the AI, we will add the result to the database, and only then do the search in the database by text (which was recognized by the AI? ;-) ), which allows us to talk about satisfying the requirements of the hackathon.

    The asterisk bluntly stated that by and large, the fact that she generally subscribed to it was already a big step. Let not a favor, but, let’s say, definitely an adventure. On the one hand, she will do her best during the hackathon if she is more or less clear on what to do. But, on the other hand, she does not promise that she will have time to familiarize herself with what clouds, Azure, AI, and other things that would be nice to know before getting involved in a fight.

    From Mercury, she also said that everything is fine, but there is no time. So - see you at the hackathon!

    The phaeton came out of contact with Zvezdochka irregularly, constantly promising something inaudibly, but in fact it was also clear that the person was busy with his own affairs and, God forbid, he would free himself to the hackathon.

    Plato also said that he did not have much time. But he, like me, understood how important it was to get as prepared as possible for an event. So we both signed up for Azure to play a little. I had little experience with AWS , but the keyword is “small.” I'm just more or less familiar with AWS terminology, I understand that there are EC2, S3, Route53, Lambda, etc. Plus, there is a security system with roles, keys, groups, subnets, and other goodies. So, when I went into Azure Dashboard, everything seemed pretty simple and clear to me (but I still had to figure it out a little deeper and longer).

    As far as I understand, Plato, for his part, spent his "homework" at 5 points. He was primarily interested in Azure Cognitive Services and the possibilities of creating and training his neural network ( Machine Learning Studio ).

    In total, a week before the event, we met with Plato only once, for just a couple of hours. He tried to send a curl (REST API mode) request to Azure Speech-to-textservice using Python. He faced several obstacles: the REST API accepts chunks of audio for no more than 15 seconds, if the paramatra (Recognition mode, Language, Output format) is incorrect, it does not return any error, but returns HTML with either 4xx or 5xx status. But, by the end of the evening, Plato already had a script for slicing audio into pieces for 15 seconds and successfully sent them to the API and received some intelligible text in French. Which of course pleased us as researchers.

    I spent several hours stupidly trying to build some kind of JavaScript server backend from scratch. I took the bare Debian 8 Jessie machine as a basis and using google tried to pick up Node.js and screw the library to itbut everything somehow fell apart. Some libraries were not installed, something was working wrong. The point was that I wanted to check the same Speech-to-text service, but in client library mode, which, for example, has no limit of 15 seconds. Javascript was chosen to the extent that C # and Java, in my understanding, are much more difficult to deploy (I can make a lot of mistakes) and require some knowledge on how to call the runtime (JVM? CLR?). Since I have no real experience on either one, I thought that I could handle JS in the microvolume needed to connect the necessary library. There was no time for a full study of the issue.

    In general, nothing good came out of my experiments. The only useful conclusion I made for myself is that at the hackathon I will not try to use some new language for myself, but I will work with PHP that I seem to know and, at least, understand when, what, who and why is missing.

    In the week preceding the hackathon, around March 13, we received an invitation by mail to fill out an online application form.

    Profile and a couple of comments about her


    • Name, Surname, Email, Phone Number
    • Team name
    • Will you be present at the Radio House (la Maison de Radio-Canada) on March 19 at 18.00.
      Answer options: “Yes, I will attend”, “No, but I will attend and participate in the live broadcast on YouTube”, “No, but I will check the recording of the broadcast later on YouTube”
    • Will you be present at the Radio House (la Maison de Radio-Canada) March 23-25 ​​during the hackathon?
      Answer options: “Yes, I will be present”, “No, I would like to work remotely.”
    • Presentation of the prototype. If you have chosen the “remote” mode of participation, then for the presentation of your prototype you can choose any place of your choice, or you can use one of the regional representations of Radio Canada.
      Answer options: “At my discretion”, “Take advantage of regional representation”
    • Do you need a "children's room". If you need to provide care for a child / children (6 months - 11 years old), then upon request such a service will be provided to participants: Saturday, 8:45 - 22:00 and Sunday, 8:45 - 19:00.
      Answer Options: Yes, No
    • How did you hear about us?
      Answer options: “Website”, “Friends”, “Newsletter”, “Announcement”, “Other” (field to fill)
    • Questions or comments?
      Answer options: (field to fill)

    Thinking out loud

    I don’t really understand why Agorize or Radio Canada reassembled the names, phone numbers, mailing addresses and team names. We already provided all this information when we registered our team and, as you recall, we even uploaded a presentation.

    Information about the preliminary meeting with the participants on March 19 was good news. It only remained to make time to get on it.

    Information about the possibility of teams participating remotely gave some scope to the upcoming event. And, even to confess, it was a little scary for the possibility of the appearance of some AI gurus from somewhere in the USA or Europe against the background of which we would be ashamed to show our project, if any. On the other hand, it was reassuring that absolutely all correspondence and information about this hackathon was in French. This factor sharply narrowed the potential circle of international participants. (The idea of ​​an international scale came to me after my first visit to the Agorize website , since hackathons from different countries were published there.)

    The nursery is our Canadian style. Like the WCAG 2.0 AA standard mentioned in the requirements, the “extension” for children illustrates the permanent desire of Canadian society to level the rights, and most importantly the capabilities of all members of the society. So that no circumstances could cause discrimination in any form.

    Day 0 - Monday, March 19th. On your marks!


    On Monday afternoon, we received a letter from the organizers through the Agorize system with a username and password to an Azure account specially created for our team. This excited us a little, but since Monday is a difficult day, and we all, as I have repeatedly said, are busy people, we did not have the opportunity to seriously dig and “touch” this account. In addition, Plato and I already got acquainted with Azure and at that time we were more interested in accessing the Radio Canada API, since a lot depended on it too and this part was a kind of “white spot” in our architecture.

    Meanwhile, by 6 p.m. I had to get to the presentation. I left work a little earlier, thank God I have 30 minutes to get to work from the Radio House, well, 40 maximum, or 10-15 minutes by bus. The asterisk also promised to be at 17:45. Plato said that would do too. As a result, Plato was a little late for the start of the presentation and was out of place at about 18:35.

    I was there by 17:50. There were quite a lot of people in the hall. The asterisk introduced me and Mercury to each other. Mercury came a little earlier and already managed to communicate with some future participants. The asterisk and Mercury were quite positive about our chances of winning. Mercury spoke out about the fact that many people in the audience are not experienced developers, but are just designers, or students, the same as they are with Zvezdochka or even just business students who might have some good business ideas, but that’s not great advantage for hackathon. We sincerely believed: there is no prototype - no victory.

    I want to note that at the time of our registration, I expected that from 4 to 10 teams would compete maximum. Since there were very few teams that passed the “full” registration. At the time of March 19, I already knew that the number of teams was definitely more than 10. But I did not do the calculations, there was no time for it, and what does it change in the end?

    The presentation began a little late, at about 18:10.

    You can watch it on YouTube in full.

    Opening remarks were first given by Maxime St-Pierre- Executive Director, Head of Digital, Radio Canada. He greeted those present and said, in particular, that the reason for this hackathon, like the previous year, was the rapidly developing world of technology, which should be kept pace with. That the first hackathon brought good results and practical results. Therefore, Radio Canada is optimistic about such events.

    Then the host of the evening Dominique Gagné took over the floor
    - Directrice, Intelligence d'affaires numériques, Médias numériques, Radio Canada. She once again went according to the schedule of the hackathon for the upcoming weekend.

    Microsoft Azure Session
    The next speaker was David Chapdelaine - Technology Solutions Professional (Azure) at Microsoft. At the very beginning of his speech, he asked everyone to tweet something with the tag #hackathonRC for further analysis at the end of his speech. In general, the presentation was dedicated to the fact that using Azure we can do anything, faster and more convenient than before. He went on to a small presentation on Azure Dashboard. For me, this could not be useful from a practical point of view. Those who understood what he was talking about, they already knew that. And those who saw it for the first time, all the same, could not remember or understand, as with any instrument of this kind. This is just like trying to explain how to work with MS Word or MS Excel in 5 minutes to someone who has never seen it, and even in presentation mode.

    From David the baton was taken by Jean-Francois Gamache - Cloud Solution Architect at Microsoft. Jean spoke about the serverless architecture ( Serverless ) that can be implemented on the basis of Azure. And also went through the list of AI services available through the Azure platform.

    Then Jean showed a touching video about how AI is changing people's lives. Please take a few minutes of your time and watch this snippet .

    Personally, such things inspire me. I like our age of technology. I like when technology brings real benefits to people. I like to feel part of these changes. I like the fact that something that science fiction writers wrote about a couple of decades ago is being implemented right before our eyes. Perhaps these feelings are pushing me not to stop and learn new things again and again.

    Further, Jean demonstrated several AI-related projects taken from the Microsoft public repository on GitHub and a little from here .

    After that, Jean proceeded to the prototype server-side application, which pulls all tweets according to a specific hashtag in real time (apparently through the Twitter API). It then sends the contents of the tweet for analysis to the Text Analytics API. As a result of the analysis, the text score is returned from 0 to 1, where 0 is a sharply negative content, and 1 is an absolutely positive text. Next, depending on the assessment, the following action is selected. So for negative tweets an email is generated, and positive ones are sent to the Slack chat channel. And also, regardless of the analysis result, all evaluation results are sent to some kind of utility for monitoring and visualization. Jean repeated several times that such a server solution can be created in 10-15 minutes using Azure tools. The demonstration included the very #hackathonRC hashtag that we were asked to tweet at the very beginning. This apparently should confirm that the application is working in real time. The asterisk even managed to notice her tweet flashing on the screen somewhere.

    Concluding the Azure part, Jean and David answered several questions from the audience.

    After that, a 10-minute break was announced to go to the toilet and / or snack.

    We took a break to get to know each other again. We exchanged impressions of the presentation. The whole team agreed that the feed is quite lively and interesting.

    Once again we talked about the idea for our prototype. And they assigned roles:
    Plato - working with the Radio Canada API - pulls data (which is still unknown) - sends it for processing to the Azure API (it is still unknown which) - receives the analysis result and puts it into a database.

    I - get data from the database - send the results as a kind of search microservice to the front-end.

    Asterisk and Mercury - create HTML for visualizing search results and JS functionality for dynamic search (when entering each character) for a prettier effect on the presentation.

    Phaeton, since it is constantly disappearing, the role is not defined. But in case of manifestation of the initiative should help the guys with the frontend.

    All team members were happy with their roles. Everyone felt that the delivered part was within their power, although many questions were in the air, but until we start doing something with our hands, it makes no sense to ask them.

    So, after the break, it's the turn for Radio Canada experts.

    Radio Canada Session
    Quoc-Viet Nguyen - Solutions Architects and R&D at Radio Canada introduced the long-awaited API from Radio Canada. As it turned out, 4 APIs were introduced. One of which is public (although I’m not sure how public it is, but let's start from the fact that, firstly, it was said so, secondly, it is available without any authorization, and thirdly, if Radio Canada feels negative consequences of the habr effect, they can always close access to it). And three more APIs open only to hackathon members.

    • Neuro - a public API is used on the news portal
    • Validation Media - API only for hackathon participants, used to work with media files.
    • Sitesearch - API only for hackathon participants, used on two music sites ICI MUSIQUE and CBC MISUC
    • Via Foura - external API only for hackathon participants, used to work with comments.

    Since most of the APIs do not have detailed documentation, Viet promised to provide all sample requests as a collection for Postman .

    Postman - I liked it
    Before the hackathon, I saw this program once or twice, but for the most part I used online utilities for building and testing the REST API. After I started using Postman I fell in love with it.

    It seems to me that this is a convenient tool that allows you to not only generate requests, but also collect them into collections (this allows you to organize the storage of requests by source, destination, user cases, test cases or whatever else you want, as well as share with colleagues who are already prepared and prepared tested queries).

    Plus, Postman just records the history of my requests. This makes it easier to debug requests and not lose successful or unsuccessful attempts, and much more, until which I have not reached my hands.

    Before passing the floor to the next speaker, Viet also answered several questions from the audience, not only in the audience, but also remotely watching the broadcast via YouTube.

    Further, the floor was given to the specialist in UI and Web accessibility (sorry, again I can not google the correct translation for this term into Russian) Olivier Fortin . Olivier drew the attention of the audience that approximately 13.7% of the adult population of Canada has some difficulties for normal perception.

    Personal Opinion about Disabled People in Canada
    In Canada, it is considered bad form to pronounce the word - disabled. There are no disabled people, there are people with disabilities.

    I have been living in Canada for 7 years, but, so far, I can neither agree with this approach nor reject it. On the one hand, I like to use clear terms to express my thoughts. And I love when others also use words for their intended purpose. On the other hand, the word “disabled person” does not give us an understanding of what kind of person is behind this term. Based on my philistine perception of the world, it could be equally likely to be a person without an arm or leg and a diabetic, and a lame and oblique one.

    Moreover, if you play the game of "association", then blind or deaf, it comes to mind is not the first or second, in response to the "disabled person". So in this sense, I agree with the Canadians and am ready to ban the word.

    Another interesting fact is the perception of the world and words (maybe only me like that?). Continuing the game of association with the word invalid, I will not immediately have the image of an elderly person. But if we take the statistics, then among the elderly there are a lot of people with "disabilities" ;-) I think that the Canadian statistics of 13.7% includes this category of elderly citizens.

    The attitude towards pensioners and the elderly is very respectful. They, unlike Russian pensioners, live in many ways even better than the younger and more able-bodied population. A decent pension, decent savings and a paid mortgage, various benefits and allowances allow pensioners to have a much more advantageous balance of income-taxes-expenses compared to working citizens. I’m not talking about 100% of pensioners, I’m just voicing what everyone thinks, knows and discusses. And, probably, we even strive for exactly this - to a well-to-do and well-fed old age.

    If you are in Canada in the summer, pay attention to expensive gigs on the roads of the country - in most cases you will find older people driving. Quite often you can see gray-haired bikers. The first years upon arrival, we were still surprised at this, but now we hardly notice it - it has become the norm for our perception.

    But on the other hand, “a person with disabilities” sounds, although politically correct, but, in my subjective opinion, also does not reflect something specific. That is, we change the flea.

    Based on these statistics, it turns out that in our province of Quebec alone we have about 1 million people with disabilities. This is why Web accessibility is so important in this age of technology and the Internet to ensure equal access to information, products and services for all, without exception.

    Olivier said what you should pay attention to first when developing HTML pages:

    • alt - text attribute for images
    • legend - tag for forms (as I understand it)
    • tracking the status of the page state and in particular focus. The page should be designed in such a way that it is convenient to navigate through it using only the keyboard
    • all links / links must have text labels
    • error messages within forms should be clear and unambiguous. (I think that it should be clear which field caused the error, and the text of the error should be descriptive. And not just highlight the incorrect value with a red frame, since a potential visitor may not be able to see this frame)
    • Pay particular attention to semantics and test / verify / validate your HTML code. This will help you get not only a working page or website, but also avoid problems and improvements in the future.

    Olivier noted that compliance with and following these rules will allow not only to comply with the standards as such, but will also improve the SEO of your site, as well as its ergonomics.

    He then recommended several Web accessibility testing applications.

    • Wave - Chrome extension
    • aXe - Chrome extension
    • Tota11y - Chrome extension
    • NVDA - screen reader for Windows
    • VoiceOver - screen reader for MacOS and iOS

    I was a little amused by the question of one of the participants who directly asked the question of why Web accessibility is so important and how important is it to follow all these rules? (implying, as it seemed to me: what the hell do we need to spend our precious time on all these dances with a tambourine?). To which she received the answer that if she carefully read the technical requirements for this hackathon, then WCAG 2.0 is part of these technical requirements.

    But the participant was not satisfied and asked another question “on the forehead” (which is very rare in Canada, usually everyone tries to read between the lines and not to ask uncomfortable questions): will someone check each line of code for compliance with these high requirements. Olivier and Dominic replied as politely as possible that during the hackathon the specialists of Radio Canada, Microsoft and other partners will always be there to help if you have any questions. And that, in truth, the jury will certainly not have time to check every line of code in each project.

    Toward the close, there were a couple more questions about using the APIs given by viewers of the YouTube broadcast.

    They thanked us all, wished them success at the upcoming hackathon and, so to speak, see you on Friday.

    Our small team once again gathered in the lobby. They exchanged a few words. I must say that despite the fatigue after a hard day and 2 hours in a sitting position, everyone had smiles on their faces. And we were enthusiastic in anticipation of the upcoming adventure called "hackathon."

    While working on this second part of my story, I watched the video of March 19 several times, looking for links, names, details, etc. I want to note and I would like you to understand that going out in the evening after the presentation in my head did not have the whole picture just described. 2 hours of presentation, without the ability to rewind or pause or ask to repeat what I did not hear. This is not the same as having an already published video and text that can be re-read and reviewed. There was porridge in my head. All this had to be reviewed, comprehended, tried, analyzed. Highlight what can be used quickly and in practice, and which again will fall into the big backlog of what else needs to be studied in the future.

    We all live in a wild rhythm. There is no time to stop and study any question that interests us or to comprehend and analyze some information that has appeared.

    I don’t like the speed with which the river of time carries me, preventing me from making my own decisions, but only forcing me to follow the course of events.

    This is one of the reasons that pushed me to this hackathon. She became the reason that this article was born. I'm just trying to analyze and document what is happening to me. I believe this is important. And I hope that this is interesting to readers.

    Let me put it on this ... “to be continued”

    I do not want to lose the interest of those who are waiting for the continuation of the first part. And immediately I begin to write the third, I think the final part.

    UPDATE:The third part

    Also popular now: