Student Supercomputer Competitions: Instructions for Use

Hello, Habr! I am the captain of the St. Petersburg State University team, which took part in the ASC competitions. Last week, stealapanda published an article on the experience of working with Sunway Taihulight, the world's most powerful supercomputer. It became clear that many have heard for the first time about such an event. In my article I want to talk generally about HPC competitions, how they are held and what skills will come in handy if you want to fit into this fascinating adventure. Also on the example of ASC I will describe how it all goes.


What are HPC competitions and how do they differ from regular sports programming


HPC-competitions - competitions of student teams to solve problems using computing clusters. Many people know sports programming competitions, conducted according to the rules of ACM ICPC, so for convenience I will conduct a comparison with them. In ACM ICPC competitions, the solution comes down to writing an algorithm that solves the problem within the given time and memory limits. You need to understand which algorithm is suitable for solving the problem and encode it as soon as possible. In HPC competitions, you don’t have to read fascinating task texts trying to figure out which algorithm is hidden in them. But you will find a meeting with the Linux console, the long-term legacy code of scientific applications and the latest technologies (yes, all together), parallel and multi-threaded programming technologies, and task launch schedulers.

Instead of writing trees and sorting with your eyes closed, you will have to master the speedy writing of make files, learn compilation flags and make friends with vim, ssh and screen.

I did not meet any restrictions on the number of participation of the same person, as in the ACM ICPC competitions. Now in the world there are 3 such competitions: ASC, ISC, SC. Their main difference is the venue. The rules and the set of tasks are very similar. Separately, only the Chinese (ASC) can be singled out who conduct a preliminary qualifying round with solving problems and provide equipment to teams in the final and, for some tasks, also give access to computing resources during the qualifying round. On ISC and SC there is no preliminary qualifying stage with tasks. Teams are selected on the basis of applications with a description of the cluster configuration that the team is going to use in the competition and the relevant HPC topics of the experience of the team members. In all 3 competitions, they are awarded for the best Linpack (one of the tests,

Now I’ll tell you a little about each of the competitions.

ASC(Asian Supercomputer Challenge). Organized by China, the main sponsor is Inspur. In 2012, this competition was held only for the Chinese teams, and since 2013 it entered the international level. Due to the availability of a sponsor and the preliminary stage of selection is the most massive. From 2013 to 2017, the number of participating teams increased from 43 to 230. The quota for admission to the final during this time increased from 10 to 20 places. The first stage (preliminary contest) is held in absentia for two months, from January to March. According to the results of the first stage, teams are selected who are invited to participate in the final in full at the end of April. A low entry threshold is determined not by the ease of tasks, but by the fact that you do not need some crazy amount of money and a super-fashionable effective cluster to participate. The team should consist of 5 “non-graduated” students (bachelors, Masters) and 1 coach. During the finals, the coach must not be at the workplace of the team.

ISC is held as part of the student section of the ISC (International Supercomputer Conference) in Germany on the 20th of June. The team should have 6 students and 2 coaches. Acceptance of applications for participation takes place in October-November. There is no preliminary selection stage with solving problems, only an application for participation with a description of the cluster configuration and the experience of team members and the University in the field of HPC. The vendor who will provide the equipment the team must look for on their own. The organizers can write a letter on behalf of the organizing committee with a proposal to the vendor to assist the team, but no one provides the finished equipment, as in the case of ASC. The objectives of the competition are the launch of benchmarks and several scientific applications.

SCHeld in America in November, also in conjunction with a supercomputer conference. The competition takes place in a hackathon format and lasts 48 hours. The team also consists of 6 people, but they do not work at the same time - one student is forbidden to stay in the competition area for more than 12 hours a day. During this time, the team should launch 2 benchmarks (Linpack, HPCG) on pre-assembled and configured clusters, try to reproduce the results of the selected article from last year's conference and launch 3 more applications, one of which is kept secret until the start of the contest. This year's challenges can be viewed here .

And what to do with all this?


Here I will try to describe how the selection occurs, the technologies that I will probably encounter during such competitions and the tasks of the past years. Our team has experience only in ASC competitions, so all particular features will be described in detail only with respect to this competition.

Competitions involve working with a computing cluster. Here a subjective moment appears - the success of the team to some extent (strongly or weakly depends on the task) is influenced by the hardware that is used for calculations, so one of the primary tasks is to find a sponsor who is ready to provide equipment (it’s profitable from a marketing point of view ) I also heard that some teams (it seems that it was one of the Chinese) use the systems of their University, but this is not about Russia. I can hardly imagine anyone letting the University equipment be taken to another country. Spoiler: our team does not even have access to the University Center. For preparation, we use the educational cluster of the department 10 years ago and are constantly looking for any opportunities to gain access to any third-party systems. In SC and ISC contests, teams assemble the cluster completely independently, taking into account only 3000W power limitations. ASC puts the participants in slightly more equal conditions - the cluster for all participants is assembled on the basis of the same server, which is provided by the main sponsor of the competition by Inspur. Teams can add their accelerators (video cards, Xeon Phi, FPGA), install SSDs, add memory, but at their own expense (or at the expense of attracted sponsors).

In the application, the team indicates the configuration of their cluster and the background of the participants and their University in the field of high-performance computing. At ASC, this is part of an in-depth qualifying round along with problem solving. In ISC and SC it is just immediately submitted as an application for full-time participation in the competition.

A stack of technologies and skills that are likely to be useful during the contest: programming languages: C / C ++, Python, Fortran, bash; multithreaded technologies: OpenMP, MPI, OpenCL, OpenACC, CUDA, Intel Cilk Plus, Intel TBB, etc.

The ASC qualifying stage begins around the 10th of January and lasts 2 months. During this time, it is proposed to solve 3 problems. Typically, the goal is to speed up the code provided (the last time there was one task that did not require acceleration). Maximum for the qualifying stage, you can score 100 points, which are distributed as follows:

  • 10 points (5 + 5) are given for representing the team and describing the activities of the university / department in the field of HPC. The presentation of the team includes the name, slogan, team photo, a story about the members of the group and the organization of work in the team. The description of the activity includes the presentation of available computing resources, experience in solving HPC problems, articles on this topic, as presented by HPC at the university (courses, etc.). Honestly, it is not clear why points are awarded; there is no clear scale (as for all other tasks of the qualifying stage). For the cool team? For the completeness of the story? Or just everyone at 10 points per description?

    The remaining 90 points are given for completing assignments. Part of the tasks implies that participants must run them on their equipment, others need to be run on a remote cluster, the same for all teams. If the team has access to powerful computing resources, then it is precisely for such tasks that it is worth betting.
  • 1 task, up to 15 points - collect the cluster configuration, justifying the use of the selected elements. It is understood that this configuration will be used by the team in the final, but in fact it can be changed. You need to build the solution on the basis of the provided server (in 2017 Inspur NF5280M4), the switch (Infiniband or Ethernet), the corresponding cable and card are also available on the list of available equipment, until 2016 inclusive Xeon Phi was available. A limiting condition is that the theoretical power of the entire cluster must meet 3000W.
  • 2 task, up to 15 points - launch of the benchmark. Over the years, HPL (Linpack), HPCG (2016), HPCC (2015) (HPL + other tests) were presented . In 2017, it was necessary to run the benchmark on a cluster provided by the organizers with KNL processors (2nd generation Intel Xeon Phi), earlier the participants launched benchmarks on the equipment that they had. The task is to squeeze the maximum performance. In the report, it is necessary to describe the configuration of the cluster on which the benchmark was launched, the results of the launches and the steps to configure it.

    3 and 4 tasks previously gave 20 and 40 points, respectively. In 2017 - 30 points each.
  • Task 3 - code optimization, in order to speed up work on a certain data set. In different years, these were scientific applications in C, C ++, Fortran.
    2013 - Gromacs
    2014 - Quantum Espresso Test
    2015 - NAMD
    2016, 2017 - MASNUM_WAVE
  • Task 4 - optimization for Xeon Phi and launch on it. (until 2017). In 2017, the 4th task was devoted to the forecasting of traffic by a neural network, and it was no longer the speed of work that was estimated, but the accuracy of the prediction.
    2013 - BSDE option pricing
    2014 - 3D-EW
    2015 - Gridding (Square Kilometer Array project)
    2016 - neural network optimization (Chinese DNN program)

When solving problems, it is worth considering that the Chinese servers provided for some tasks during the Chinese New Year in early February will be turned off (within a week). After that, the excitement begins on them and the task can stand in a queue for execution for a long time. Tasks that require running on these servers must be addressed first.

After about one to two weeks, the results become known. What is characteristic, the organizers do not upload any results table and even their points are not personally reported to the teams. The only thing you can focus on is the order in which the teams are listed. According to our assumptions, it corresponds to the rating. In addition to the final teams, a list of teams that have shown good results is also announced.

There are 2 months before the finals, so what to do with them? In the final, you will need to assemble and configure the cluster yourself. This is given 2 days. They bring you a rack, the ordered number of servers, cables and two monitors. Forward!

image

image

So, if you have never before built and configured a cluster - it's time to learn. After that, there are 2 days to launch applications. These will be the same tasks as in the qualifying round, but with different input data. 3 new tasks are also added: two become known along with an invitation to the finals, another one is issued directly on the day of the competition. On each day of the competition, you need to run 3-4 tasks, the best result is indicated in the final card, which is dealt at the end of the day. A special screen (and a web page) indicates the energy consumption of the cluster of each team. If it exceeds 3000W, then the university logo is highlighted in red and the siren begins to rattle wildly throughout the room. The result of such a run will not count towards the team.

image

In contrast to the preliminary stage, the rules for holding the final prescribe a point system and in the hall on the wall there is a large table where the results for each task are entered.

image

After the mysterious scoring in the qualifying round, such transparency is extremely pleasing. 7.5 points are given for benchmarks, 15 points for the remaining tasks, 10 points for the presentation of the team, in total - 100. In general, points are awarded as follows: the team that showed the best result gets the maximum, the remaining teams get points based on the ratio of their result to the result the leader.

Prizes and awards. A cup and a diploma are awarded to all teams that qualified for the finals. There are rewards for 1st and 2nd places, for the best result on Linpack, for the best result for some other separate applications. Moreover, if the best result is achieved by a team that has already received one of the above prizes, then the next one will be awarded. You can also actively PR your team and contest on social networks (Twitter or Chinese WeChat) and get the Best Popularity Prize.

image

By the way, the amount for the Best Popularity Prize has not yet arrived. I hope that transfers within China are still faster and at least the Chinese teams have received their awards.

Due to the fact that all 3 competitions have common roots, teams that took 1st and 2nd places in the ASC are automatically invited to the competition in Germany.

Conclusion


Competitions are interesting in that they provide an opportunity to work with “large computers”, scientific developments and look at another world different from the commercial enterprise IT. I would be glad if someone was interested and this year new Russian universities will appear in the list of participating teams. Every year, Russian teams put everyone on the shoulder blades in ACM ICPC competitions and, I hope, after a couple of years of training, they can put them in HPC :)

Links for those who are interested:
Website blog of an American journalist covering each of these competitions
Registration for ASC 2018

Also popular now: