October 27, 2015 at 10:37How we did HD Wi-Fi at the stadium in Krasnodar for 34 thousand people
It may seem that Wi-Fi in the stadium is not needed at all. I came to be sick - so watch the game, and not post a selfie and read the mail. But no, Wi-Fi is needed for two important reasons. Firstly, with such a crowd of people, the channels of the cellular network will be dismantled extremely quickly, and most of the fans will be left without a phone for a while. Secondly, in fact, the stadium owners really wanted to give a special mobile application for all fans.
Inside - the layout of players on the field, statistics on the match and players, plus streaming video from cameras so that you can see in close-up on the faces of the tablet or phone what is happening. Well, immediately the repetitions of important points from different angles.
At a radio
examination Membrane inspection result
All of this requires fast high-density Wi-Fi with minimal interference. There are no ordinary hotspots here - you need a good examination of the radio survey, plus special equipment. The result is 150 tricky access points with beamforming after some pretty trivial jobs.
High density in relation to Wi-Fi is a rather vague and conditional concept. To simplify this situation, it’s such a state of the network where there are as many active users as it is unable to digest a wireless network designed according to the classical design (access points with omnidirectional antennas that work comfortably with a maximum of 20-30 clients on the radio interface and a coverage radius of 15 meters in typical office). Accordingly, special coverage is required (so that subscribers and points do not interfere with each other), and a large number of customers impose bandwidth requirements on all network elements: controller, switches, routers, firewalls and, in fact, Internet access channels.
When creating a wireless network for objects with a high density of subscribers, which is a stadium, the following factors must be taken into account:
The wireless network, built according to the traditional design, will not cope with the tasks. The result would be large packet losses, a large number of retransmissions, and, as a result, extremely low data transfer speed, unstable operation of services.
Reducing the coverage of access points is achieved by:
That is not all. Still required:
At the Krasnodar FC stadium, we used the Cisco Wi-Fi solution, which includes new Cisco narrowly-targeted antennas, high-performance dual-band Cisco access points in combination with software algorithms to automatically configure a wireless network depending on a constantly changing radio frequency environment, as well as CleanAir technology, which includes access points with built-in intelligent hardware features that identify, classify and reduce the effects of radio frequency interference. To support network monitoring and management, the solution includes Cisco WLC multi-function wireless network controllers and Cisco Prime Infrastructure infrastructure management software.
Cisco Antenna:
A narrowly targeted sector antenna with MIMO support. It has 6 elements, 3 for operation in 2.4 and 5 GHz. The beam width of the DND in the 2.4 GHz range is 36 ° in the horizontal plane and 36 ° in the vertical plane, in the 5 GHz range - 55 ° and 48 °, respectively. The antenna housing provides protection against external dust and moisture in accordance with IP65. Designed specifically for use with a Cisco access point in a high-density client environment. It is used to cover the stands of the stadium.
Cisco Antenna:
Sector antenna with MIMO support. It has 4 elements operating simultaneously in 2.4 and 5 GHz. The beam width of the DND in the 2.4 GHz range is 65 ° in the horizontal plane and 105 ° in the vertical plane, in the 5 GHz range - 55 ° and 110 °, respectively. The antenna housing provides protection against external dust and moisture in accordance with IP54. Used in conjunction with a Cisco access point. It is used to cover the stands of the stadium, VIP-boxes, press zones, areas close to entering the field and rooms with a high density of subscribers.
Cisco Access Point:
"Stadium" dual-band access point, specially designed for use in places of high concentration of users. The access point is designed and certified to work with narrowly directional antennas having a gain of up to 13 dB (2.4 GHz) and 7 dB (5 GHz), for example, such as the Cisco antenna described above. It has advanced settings for the radio transmitter, which is indispensable for such installations. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. It supports the IEEE 802.11n standard and a 2x3 MIMO configuration with two spatial streams.
Cisco Access Point:
A dual-band access point with IEEE 802.11ac support and a 4x4 MIMO configuration with three spatial streams. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. It is used in conjunction with a sector antenna for the first rows of the stands of the stadium, VIP lodges, press zones, zones close to entering the field and rooms with a high density of subscribers, as well as in areas with a low density of subscribers together with dipole omnidirectional antennas.
Cisco Wireless Controller:
Centralized access point management is provided by the high-performance Cisco WLC. This model of controllers supports simultaneous control of up to 6000 access points. It provides 10 Gb / s bandwidth and supports simultaneous operation of up to 64,000 users, which is indispensable for such installations. System fault tolerance is achieved by combining two controllers in a cluster. Switching time from the active controller to the standby is less than a second without breaking client sessions.
The wireless network controller is responsible for such functions as providing secure seamless roaming, automated radio control (controlling the carrier frequency and power of access point radio transmitters), eliminating “holes” in the coverage when access points fail, switching users from loaded points to less loaded ones, Detection and avoidance of mutual influence of wireless devices and interference; Detection of intrusions into a wireless network; Providing security policies and QoS policies.
We were lucky, and the customer turned in at that moment when concrete had not yet been poured, and the stadium was only being designed. This meant that we could pre-lay places for equipment, cable routes and other infrastructure. This is important, because in addition to directly Wi-Fi, we also made IP TV and Digital Signage - in other words, huge screens with what is happening on the field.
In total, about 50 subsystems had to be combined at the stadium.For my department, it all started with the design of our infrastructure. The problem is that the exact location of the radio equipment can be revealed only after understanding what kind of shape and materials the stadium will be in. It's like a sound - you need to measure to get an accurate picture of the propagation of waves. Accordingly, in the project, we thought out what we could fixed (in particular, it was especially good that we managed to lay the tracks and these special slots for equipment in the photo below). At this stage, there is no understanding, for example, who will do the roofing at all, so there is little accurate data.
During the design, we agreed on the installation location for a long time: for example, in the aisles where people walk, it is impossible to install equipment that will bulge. These are the requirements of firefighters. We proposed options with gating and holes in reinforced concrete structures, but then the operation decided that the object would begin to crumble ahead of time from such holes. Also, on this site, the architect had aesthetic requirements - and the project laid more strength. The combination of aesthetics and increased durability allowed us to convince the operation, and we designed the strobes for the hidden installation of antennas and access points in thermal boxes.
The next stage is the stage of working documentation.When it became clear who made what system, and it was approximately clear which solutions would be applied (first concrete and structures were worked out, after which there was preparation of working documentation for engineering and automated systems). The peculiarity of the stage is that changes occur at the construction site and sometimes it is necessary to make editions in the working documentation. The main task is to provide for everything necessary. That there was no such thing, “And here we’ll demolish the partition.” It is important to think in advance also because the project must be within the budget. The most important part for us here is to conduct a live radio survey, when concrete is only laid. That is, in such a state of the stadium:
At a radio
survey The next stage of construction and installation work- the most fun part, on which you have to argue a lot and defend decisions with contractors on related systems. You also have to cooperate with contractors who, for example, make a roof, on which it is necessary to provide fasteners, which should make it easy to operate the system. We have run into this stage more than once at large facilities, we gained a lot of experience specifically in the stadiums - we did the IT infrastructure and the sound of CSKA and Spartak stadiums (30 and 44 thousand seats, respectively).
Here, as usual - there are many contractors, and no one wants to wait for the rest at work. This also affected us - one contractor laid the stove without our thermal boxes, and we had to look for new thermal boxes for installation. We have added a new rule - to carry out the procurement after creating the test unit, additional components may be needed. No matter how well the site was designed by all organizations.
The main task is to evenly cover the bowl, taking into account that users will sit very tightly. To increase the density of coverage, antennas with a small coverage area are used, due to this, more access points are installed without negative interference.
To do this, you need:
I remind you that we did it when only a part of the structures was ready:
Just like that, we are testing on Scotch tape.
It was not possible to examine the upper platform due to the lack of a winch and crane. More precisely, they were, but they could not get under the roof. I had to conduct a survey on the lower platform and make a projection on the upper platform.
It was very difficult to determine the distance from the antenna on the crane to the stands in the vertical. We measured the distance with a laser. The radio operator did not work for the crane operator, I had to scream and use gestures. The process was long. We moved the cradle of the crane during the day to inspect the quarter, after which we projected the results onto the upper platform.
Measuring equipment is attached to the cradle, and then ...
It rises ...
And depicts a piece of the roof.
When the roof was partially ready, the high-altitude workers were called to check the result. For them we instructed, gave them radios. In general, they were quickly taught to do the procedure, which they themselves could not do without tolerances. Thus, we checked the results of the radio survey and the transfer of the projection. As we understood, if the inclination of the rostrum of both the upper and lower is the same, then the measurement results will coincide - and this theory was confirmed.
Here are the following schemes:
Tribunes
Membrane Sectional
view
Cisco Wi-Fi is used not only in Krasnodar. These systems are supplied to American stadiums. Specifically, on the technical side, there is accumulated experience. It was much more interesting for the customer to look at the financial and practical results of introducing such an infrastructure. Probably one of the best examples is the implementation of the Connected Stadium system at the CONSOL Energy Center stadium (for the Pittsburgh Penguins NHL football team). There was also HD Wi-Fi, many screens of the type like these:
And split screens, visible from the stands. So, the number of advertisers has doubled, and sponsorship revenue has tripled. The stadium brand has become more recognizable and memorable. 80% of viewers saved the content that they watched at the stadium, relaying it to social networks.
The stadium is completed. According to our subsystem - now we are implementing a system for broadcasting video traffic via Wi-Fi for a special mobile application. The peculiarity is that data is transmitted to devices using multicast. The use of streaming video streaming via multicast-mailing significantly reduces the load on communication channels, radio environment resources and network equipment resources. This effect is achieved due to the fact that the data from the source is not transmitted individually to each recipient (unicast), which produces copies of the uniform traffic, but because the source sends a single copy of the data along the general route to those recipients who subscribed to the newsletter. The advantage of this approach: the addition of new users does not entail the need to increase network bandwidth along a common route to consumers of the service. Respectively,
I will answer questions in the comments or by mail [email protected]
Inside - the layout of players on the field, statistics on the match and players, plus streaming video from cameras so that you can see in close-up on the faces of the tablet or phone what is happening. Well, immediately the repetitions of important points from different angles.
At a radio
examination Membrane inspection result
All of this requires fast high-density Wi-Fi with minimal interference. There are no ordinary hotspots here - you need a good examination of the radio survey, plus special equipment. The result is 150 tricky access points with beamforming after some pretty trivial jobs.
Why exactly HD, and what are the difficulties?
High density in relation to Wi-Fi is a rather vague and conditional concept. To simplify this situation, it’s such a state of the network where there are as many active users as it is unable to digest a wireless network designed according to the classical design (access points with omnidirectional antennas that work comfortably with a maximum of 20-30 clients on the radio interface and a coverage radius of 15 meters in typical office). Accordingly, special coverage is required (so that subscribers and points do not interfere with each other), and a large number of customers impose bandwidth requirements on all network elements: controller, switches, routers, firewalls and, in fact, Internet access channels.
When creating a wireless network for objects with a high density of subscribers, which is a stadium, the following factors must be taken into account:
- The total number of subscribers - the stadium, when fully loaded, accommodates 34,000 potential users. Accordingly, our network in peak may be loaded with approximately 30 thousand connections to video streaming.
- The density of subscriber devices - fans sitting in the stands are very densely placed, on average 10 people are accommodated on 10 square meters. A key indicator by which a sufficiently suitable network architecture is checked is the required transmission capacity per subscriber. Each access point has a certain bandwidth passing through the radio channel, evenly divided between customers, and the coverage radius. The higher the client density, the more clients will connect to one access point, and the less bandwidth per client. The logical way out is to put more points, but we are limited in free radio resources in the 2.4 GHz band (there are only 3 disjoint channels - 1, 6 and 11), in the 5 GHz band it is simpler - there are 11 channels, but not all mobile devices support 5 GHz. Therefore, accordingly, we can’t refuse 2.4. Accordingly, it is necessary to reduce the radius of action of the points or more accurately generate radiation patterns in order to reduce the negative impact on neighboring points operating at the same frequencies and optimally distribute radio resources.
- The characteristics of the implanted object, affecting signal propagation — the stadium bowl — is a large open space in which inevitable interference between access points;
The wireless network, built according to the traditional design, will not cope with the tasks. The result would be large packet losses, a large number of retransmissions, and, as a result, extremely low data transfer speed, unstable operation of services.
Reducing the coverage of access points is achieved by:
- the use of specialized narrowly directed antennas (more on this below);
- shutdown of low channel speeds (up to 12 Mbps);
- disable processing of packet packets of subscribers with a low signal level (RX-SOP);
- lowering the power of the transmitter;
That is not all. Still required:
- Protect your HD coverage from spurious Wi-Fi. Negative effects on the coverage of the spectator area of the stadium bowl I can have my own points located in the VIP-zones, lobbies near the exits on the field. The recipe is the same - power control and radio coverage.
- Reduce the utilization of the radio channel: do not use more than four SSIDs (ideally use one) in the visual zones, as each SSID requires sending a separate Beacon packet and each broadcast SSID responds to a null probe request; fight against client-provoked interference by inviting viewers to connect to a Wi-Fi network, as the connected device starts sending a probe request at times less; minimize extraneous access points and Ad Hoc devices from the coverage area of the wireless network such as interference sources and beacon, probe request, probe response packet generators and refuse to use “adjacent” Wi-Fi networks and use a single wireless network.
What equipment was used
At the Krasnodar FC stadium, we used the Cisco Wi-Fi solution, which includes new Cisco narrowly-targeted antennas, high-performance dual-band Cisco access points in combination with software algorithms to automatically configure a wireless network depending on a constantly changing radio frequency environment, as well as CleanAir technology, which includes access points with built-in intelligent hardware features that identify, classify and reduce the effects of radio frequency interference. To support network monitoring and management, the solution includes Cisco WLC multi-function wireless network controllers and Cisco Prime Infrastructure infrastructure management software.
Cisco Antenna:
A narrowly targeted sector antenna with MIMO support. It has 6 elements, 3 for operation in 2.4 and 5 GHz. The beam width of the DND in the 2.4 GHz range is 36 ° in the horizontal plane and 36 ° in the vertical plane, in the 5 GHz range - 55 ° and 48 °, respectively. The antenna housing provides protection against external dust and moisture in accordance with IP65. Designed specifically for use with a Cisco access point in a high-density client environment. It is used to cover the stands of the stadium.
Cisco Antenna:
Sector antenna with MIMO support. It has 4 elements operating simultaneously in 2.4 and 5 GHz. The beam width of the DND in the 2.4 GHz range is 65 ° in the horizontal plane and 105 ° in the vertical plane, in the 5 GHz range - 55 ° and 110 °, respectively. The antenna housing provides protection against external dust and moisture in accordance with IP54. Used in conjunction with a Cisco access point. It is used to cover the stands of the stadium, VIP-boxes, press zones, areas close to entering the field and rooms with a high density of subscribers.
Cisco Access Point:
"Stadium" dual-band access point, specially designed for use in places of high concentration of users. The access point is designed and certified to work with narrowly directional antennas having a gain of up to 13 dB (2.4 GHz) and 7 dB (5 GHz), for example, such as the Cisco antenna described above. It has advanced settings for the radio transmitter, which is indispensable for such installations. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. It supports the IEEE 802.11n standard and a 2x3 MIMO configuration with two spatial streams.
Cisco Access Point:
A dual-band access point with IEEE 802.11ac support and a 4x4 MIMO configuration with three spatial streams. Equipped with a built-in spectrum analyzer (CleanAir technology), which allows you to detect and avoid radio frequency interference of any origin. It is used in conjunction with a sector antenna for the first rows of the stands of the stadium, VIP lodges, press zones, zones close to entering the field and rooms with a high density of subscribers, as well as in areas with a low density of subscribers together with dipole omnidirectional antennas.
Cisco Wireless Controller:
Centralized access point management is provided by the high-performance Cisco WLC. This model of controllers supports simultaneous control of up to 6000 access points. It provides 10 Gb / s bandwidth and supports simultaneous operation of up to 64,000 users, which is indispensable for such installations. System fault tolerance is achieved by combining two controllers in a cluster. Switching time from the active controller to the standby is less than a second without breaking client sessions.
The wireless network controller is responsible for such functions as providing secure seamless roaming, automated radio control (controlling the carrier frequency and power of access point radio transmitters), eliminating “holes” in the coverage when access points fail, switching users from loaded points to less loaded ones, Detection and avoidance of mutual influence of wireless devices and interference; Detection of intrusions into a wireless network; Providing security policies and QoS policies.
Progress
We were lucky, and the customer turned in at that moment when concrete had not yet been poured, and the stadium was only being designed. This meant that we could pre-lay places for equipment, cable routes and other infrastructure. This is important, because in addition to directly Wi-Fi, we also made IP TV and Digital Signage - in other words, huge screens with what is happening on the field.
In total, about 50 subsystems had to be combined at the stadium.For my department, it all started with the design of our infrastructure. The problem is that the exact location of the radio equipment can be revealed only after understanding what kind of shape and materials the stadium will be in. It's like a sound - you need to measure to get an accurate picture of the propagation of waves. Accordingly, in the project, we thought out what we could fixed (in particular, it was especially good that we managed to lay the tracks and these special slots for equipment in the photo below). At this stage, there is no understanding, for example, who will do the roofing at all, so there is little accurate data.
During the design, we agreed on the installation location for a long time: for example, in the aisles where people walk, it is impossible to install equipment that will bulge. These are the requirements of firefighters. We proposed options with gating and holes in reinforced concrete structures, but then the operation decided that the object would begin to crumble ahead of time from such holes. Also, on this site, the architect had aesthetic requirements - and the project laid more strength. The combination of aesthetics and increased durability allowed us to convince the operation, and we designed the strobes for the hidden installation of antennas and access points in thermal boxes.
The next stage is the stage of working documentation.When it became clear who made what system, and it was approximately clear which solutions would be applied (first concrete and structures were worked out, after which there was preparation of working documentation for engineering and automated systems). The peculiarity of the stage is that changes occur at the construction site and sometimes it is necessary to make editions in the working documentation. The main task is to provide for everything necessary. That there was no such thing, “And here we’ll demolish the partition.” It is important to think in advance also because the project must be within the budget. The most important part for us here is to conduct a live radio survey, when concrete is only laid. That is, in such a state of the stadium:
At a radio
survey The next stage of construction and installation work- the most fun part, on which you have to argue a lot and defend decisions with contractors on related systems. You also have to cooperate with contractors who, for example, make a roof, on which it is necessary to provide fasteners, which should make it easy to operate the system. We have run into this stage more than once at large facilities, we gained a lot of experience specifically in the stadiums - we did the IT infrastructure and the sound of CSKA and Spartak stadiums (30 and 44 thousand seats, respectively).
Here, as usual - there are many contractors, and no one wants to wait for the rest at work. This also affected us - one contractor laid the stove without our thermal boxes, and we had to look for new thermal boxes for installation. We have added a new rule - to carry out the procurement after creating the test unit, additional components may be needed. No matter how well the site was designed by all organizations.
What is the difficulty of the task?
The main task is to evenly cover the bowl, taking into account that users will sit very tightly. To increase the density of coverage, antennas with a small coverage area are used, due to this, more access points are installed without negative interference.
To do this, you need:
- Antennas with high gain at frequencies of 2.4 and 5 GHz.
- Mandatory - with multi-channel receive-transmit (MIMO) 4 * 4 to increase the bandwidth in the radio channel (but it happens that the receivers do not have MIMO or 1 * 1). The data transfer speed depends on the type of MIMO on the client side.
- Further, we believe that on this project, the receiver (subscriber terminal, for example, telephone) and the transmitter (our antenna) should be on the line of sight. It is necessary to create all the conditions for this to be done in most cases.
- In the future, it is good to use the 802.11ac standard, but not all user devices will support it. As well as not all devices of users work at 5 GHz. This is a very good solution for the future - the 802.11ac standard will have support for MU-MIMO. Multi user MIMO, unlike the usual Single user MIMO, sends a frame to multiple recipients, not just one. That is, an access point with 4 antennas can send one stream to 3 user devices at one time.
Radio Survey
I remind you that we did it when only a part of the structures was ready:
Just like that, we are testing on Scotch tape.
It was not possible to examine the upper platform due to the lack of a winch and crane. More precisely, they were, but they could not get under the roof. I had to conduct a survey on the lower platform and make a projection on the upper platform.
It was very difficult to determine the distance from the antenna on the crane to the stands in the vertical. We measured the distance with a laser. The radio operator did not work for the crane operator, I had to scream and use gestures. The process was long. We moved the cradle of the crane during the day to inspect the quarter, after which we projected the results onto the upper platform.
Measuring equipment is attached to the cradle, and then ...
It rises ...
And depicts a piece of the roof.
When the roof was partially ready, the high-altitude workers were called to check the result. For them we instructed, gave them radios. In general, they were quickly taught to do the procedure, which they themselves could not do without tolerances. Thus, we checked the results of the radio survey and the transfer of the projection. As we understood, if the inclination of the rostrum of both the upper and lower is the same, then the measurement results will coincide - and this theory was confirmed.
Here are the following schemes:
Tribunes
Membrane Sectional
view
Foreign experience
Cisco Wi-Fi is used not only in Krasnodar. These systems are supplied to American stadiums. Specifically, on the technical side, there is accumulated experience. It was much more interesting for the customer to look at the financial and practical results of introducing such an infrastructure. Probably one of the best examples is the implementation of the Connected Stadium system at the CONSOL Energy Center stadium (for the Pittsburgh Penguins NHL football team). There was also HD Wi-Fi, many screens of the type like these:
And split screens, visible from the stands. So, the number of advertisers has doubled, and sponsorship revenue has tripled. The stadium brand has become more recognizable and memorable. 80% of viewers saved the content that they watched at the stadium, relaying it to social networks.
Next steps
The stadium is completed. According to our subsystem - now we are implementing a system for broadcasting video traffic via Wi-Fi for a special mobile application. The peculiarity is that data is transmitted to devices using multicast. The use of streaming video streaming via multicast-mailing significantly reduces the load on communication channels, radio environment resources and network equipment resources. This effect is achieved due to the fact that the data from the source is not transmitted individually to each recipient (unicast), which produces copies of the uniform traffic, but because the source sends a single copy of the data along the general route to those recipients who subscribed to the newsletter. The advantage of this approach: the addition of new users does not entail the need to increase network bandwidth along a common route to consumers of the service. Respectively,
I will answer questions in the comments or by mail [email protected]
