Surround Sound - AMD TrueAudio
With the advent of the Volcanic Islands series of GPUs with Mantle and TrueAudio technologies, the mention of AMD can be seen more and more in the context of the discussion of the topics of sound processing. Along with the importance of the Mantle API, designed to increase graphics performance, TrueAudio technology involves spatial separation of sound, as well as reducing the load on the processor. But that is not all.
The beginning of the development of the idea of high-quality sound in AMD products was laid in 2006 by ATI, which then still remained an autonomous corporation. Radeon R600 video cards have received hardware decoding technology UVD, HDMI support and an integrated sound card.
The approach was a real gift for those who used the computer as a home multimedia system. A whole bunch of different wires were easily replaced with a single HDMI cable. To this end, special DVI-HDMI adapters were even supplied with some models of video cards. By and large, at this stage, users received only practical benefits, since in addition to getting rid of extra wires, you could forget about a separate sound card.
Subsequent improvement in direction continued in 2008, when similar support for audio transmission became available through DisplayPort. Then, in 2009, the forces were thrown to improve the sound quality thanks to the support of the DTS-HD Master Audio format - transmission of audio data in high bitrate.
In parallel, AMD Eyefinity technology began to spread. Technology support provides connection of several displays directly to one video card. Along with the development of Eyefinity, an increase in resolutions and support for various multi-monitor configurations, there is support for outputting multi-channel DDMA (Discrete Digital Multi-Point Audio).
If before the output via HDMI or DisplayPort was limited to only one audio stream, now it is possible to output several independent channels. This has become useful for more convenient configuration of multitasking and video conferencing. Of course, in games such an approach also looks extremely attractive.
Despite the modernization of work with sound, AMD set aside the idea of developing hardware solutions using already known products from specialized companies. Against this background, the problem of the notorious EAX hardware for gaming three-dimensional sound remained unresolved. However, many users are still satisfied with the lack of hardware acceleration, being content with codecs on the motherboard.
This is largely due to the release of Windows Vista, where sound processing by hardware was set aside. From Vista to Windows 8, DirectSound does not have access to the sound card driver and works in emulation mode on the processor. Despite the fact that this approach to implementing a software mixer solved some problems with the sound drivers, this also led to a loss of compatibility with EAX and DirectSound ( they wrote about the problem and its solution in 2011). So, a whole galaxy of large sound companies including Creative, in order to optimize sound in operating systems, switched to OpenAL, which, by and large, became a universal solution.
In this equation of optimization and sound processing, there is another denominator in the form of a processor, which, in essence, is responsible for delivering the intent of the sound entourage of game developers to the end user. The processor load is well demonstrated in the diagram below, where a reverb effect is added to the sample.
If you simulate a rich audio component of a modern game, then the CPU load will increase tenfold. At the same time, the processor has other tasks, and on average, 10% of resources are allocated for sound processing. Of course, this ratio literally binds the hands of developers.
And what will happen if the GPU is equipped with a signal processor for real-time sound processing? Developers will get full access to the technology, data processing will take place directly in the GPU and will not depend on the sound card and processor. The latter also means an increase in CPU performance. This is exactly the hardware solution implemented in the Volcanic Islands video cards (available for graphics processors with the GCN 1.1 architecture), and the technology was named TrueAudio.
Despite the fact that TrueAudio is the first programmable “audio engine”, in the field of technology to improve the sound there are already quite authoritative figures. Therefore, in order to avoid patent wars, AMD attracted the cooperation of the company Tensilica and GenAudio, and also received a license to use the signal processors Xtensa. In this case, there is widespread cooperation with companies involved in game engines and libraries.
So, the key function of AMD TrueAudio is reserved for the HiFi EP Audio DSP cores from the above-mentioned Tensilica company. Directly for sound processing, there are three cores of Xtensa HiFi 2 and the core of advanced logic Xtensa HiFi EP. The latter is responsible for data processing, helps to increase memory performance and is built on a 24-bit architecture. To do this, each core provides 32 kilobytes of cache memory for storing instructions for processing, 8 kilobytes of Scratch memory, and 32 kilobytes for data.
Kernels also use algorithms to use instructions and algorithms to reduce power consumption. By the way, the cores are also equipped with a multiply-addition unit, which is intended to increase performance with low power consumption in the DTS Master Audio decoder. The logic of its own production is able to use up to 64 megabytes of video card memory. The above feature in the form of no load on the CPU is associated with the built-in DMA engine, which interacts with the built-in memory of 384 kilobytes.
The sound that passes through TrueAudio will be outputted not only via HDMI and DisplayPort, but also via USB, a 3.5 mm audio jack and even through a codec integrated into the motherboard.
The use of the above architecture allows you to create 3D surround sound, as well as achieve the so-called “positional sound”. Approximately under similar descriptions we have long known such systems as DTS and Dolby. However, AMD TrueSound is positioned as a solution other than the listed technologies. The differences lie in the fact that the same Dolby Digital involves the use of multiple channels, as well as all sources of sounds in one plane, and the effect of presence is provided by the illusion of movement of sound in this plane. RealSpace 3D
technology is moving in a direction similar to AMD TrueSoundfrom VisiSonics, which, by the way, is also intended for games and virtual reality. The common feature of technologies is that they provide for spatial separation of sources and sound modeling in both horizontal and vertical planes.
To create a positional sound, AMD uses AstroSound technology created by psychophysical research from the above-mentioned GenAudio studio. The technology is a block of algorithms that is used as a plugin in some DSPs. The library is not only available for all popular operating systems including mobile ones, but is also compatible with various hardware. Sound modeling occurs from 0 to 359 degrees in the horizontal plane and from -90 to 90 degrees in the vertical plane. Surround sound can be output through multi-channel configurations, as well as through headphones or two speakers.
In addition, AMD TrueAudio supports Convolution Reverb reverb, which the company compares with “space photography”. Reverberation is based on pre-prepared records in various spaces.
As a result, TrueAudio, coupled with Convolution Reverb, is able to work and process more than 100 streams in real time and provide positional sound in any space. The peculiarity of using reverberation in the context of TrueAudio is that its support will not load the processor, which was the main problem of the implementation.
A key feature of AMD TrueAudio is that in addition to the corresponding hardware (video cards based on processors with the GCN 1.1 architecture), products with technology support are needed.
The first game with TrueAudio support was Thief, which supports the reverb effect. It is noteworthy that all other audio channels are still processed by the CPU, and the processor is unloaded just due to the support of Convolution Reverb.
Another product example of using TrueAudio is the Tuscany demo application created by GenAudio and Oculus VR. In this case, the AstroundSound extension is used for the spatial separation of sound sources.
AMD TrueAudio deserves attention not only because it is one of the first solutions for improving sound in a long time, but also because of the possibility of a drastic change in the audio level on a PC. Moreover, the PlayStation 4 and Xbox One have AMD APUs and are built on the x86 architecture, which means that AMD TrueAudio may soon be ported to the console. It remains only to wait for the presence of the full implementation of technology in products for virtual reality and fresh games.
Prerequisites for TrueAudio
The beginning of the development of the idea of high-quality sound in AMD products was laid in 2006 by ATI, which then still remained an autonomous corporation. Radeon R600 video cards have received hardware decoding technology UVD, HDMI support and an integrated sound card.
The approach was a real gift for those who used the computer as a home multimedia system. A whole bunch of different wires were easily replaced with a single HDMI cable. To this end, special DVI-HDMI adapters were even supplied with some models of video cards. By and large, at this stage, users received only practical benefits, since in addition to getting rid of extra wires, you could forget about a separate sound card.
Subsequent improvement in direction continued in 2008, when similar support for audio transmission became available through DisplayPort. Then, in 2009, the forces were thrown to improve the sound quality thanks to the support of the DTS-HD Master Audio format - transmission of audio data in high bitrate.
In parallel, AMD Eyefinity technology began to spread. Technology support provides connection of several displays directly to one video card. Along with the development of Eyefinity, an increase in resolutions and support for various multi-monitor configurations, there is support for outputting multi-channel DDMA (Discrete Digital Multi-Point Audio).
If before the output via HDMI or DisplayPort was limited to only one audio stream, now it is possible to output several independent channels. This has become useful for more convenient configuration of multitasking and video conferencing. Of course, in games such an approach also looks extremely attractive.
Despite the modernization of work with sound, AMD set aside the idea of developing hardware solutions using already known products from specialized companies. Against this background, the problem of the notorious EAX hardware for gaming three-dimensional sound remained unresolved. However, many users are still satisfied with the lack of hardware acceleration, being content with codecs on the motherboard.
This is largely due to the release of Windows Vista, where sound processing by hardware was set aside. From Vista to Windows 8, DirectSound does not have access to the sound card driver and works in emulation mode on the processor. Despite the fact that this approach to implementing a software mixer solved some problems with the sound drivers, this also led to a loss of compatibility with EAX and DirectSound ( they wrote about the problem and its solution in 2011). So, a whole galaxy of large sound companies including Creative, in order to optimize sound in operating systems, switched to OpenAL, which, by and large, became a universal solution.
In this equation of optimization and sound processing, there is another denominator in the form of a processor, which, in essence, is responsible for delivering the intent of the sound entourage of game developers to the end user. The processor load is well demonstrated in the diagram below, where a reverb effect is added to the sample.
If you simulate a rich audio component of a modern game, then the CPU load will increase tenfold. At the same time, the processor has other tasks, and on average, 10% of resources are allocated for sound processing. Of course, this ratio literally binds the hands of developers.
And what will happen if the GPU is equipped with a signal processor for real-time sound processing? Developers will get full access to the technology, data processing will take place directly in the GPU and will not depend on the sound card and processor. The latter also means an increase in CPU performance. This is exactly the hardware solution implemented in the Volcanic Islands video cards (available for graphics processors with the GCN 1.1 architecture), and the technology was named TrueAudio.
TrueAudio architecture
Despite the fact that TrueAudio is the first programmable “audio engine”, in the field of technology to improve the sound there are already quite authoritative figures. Therefore, in order to avoid patent wars, AMD attracted the cooperation of the company Tensilica and GenAudio, and also received a license to use the signal processors Xtensa. In this case, there is widespread cooperation with companies involved in game engines and libraries.
So, the key function of AMD TrueAudio is reserved for the HiFi EP Audio DSP cores from the above-mentioned Tensilica company. Directly for sound processing, there are three cores of Xtensa HiFi 2 and the core of advanced logic Xtensa HiFi EP. The latter is responsible for data processing, helps to increase memory performance and is built on a 24-bit architecture. To do this, each core provides 32 kilobytes of cache memory for storing instructions for processing, 8 kilobytes of Scratch memory, and 32 kilobytes for data.
Kernels also use algorithms to use instructions and algorithms to reduce power consumption. By the way, the cores are also equipped with a multiply-addition unit, which is intended to increase performance with low power consumption in the DTS Master Audio decoder. The logic of its own production is able to use up to 64 megabytes of video card memory. The above feature in the form of no load on the CPU is associated with the built-in DMA engine, which interacts with the built-in memory of 384 kilobytes.
The sound that passes through TrueAudio will be outputted not only via HDMI and DisplayPort, but also via USB, a 3.5 mm audio jack and even through a codec integrated into the motherboard.
Positional sound
The use of the above architecture allows you to create 3D surround sound, as well as achieve the so-called “positional sound”. Approximately under similar descriptions we have long known such systems as DTS and Dolby. However, AMD TrueSound is positioned as a solution other than the listed technologies. The differences lie in the fact that the same Dolby Digital involves the use of multiple channels, as well as all sources of sounds in one plane, and the effect of presence is provided by the illusion of movement of sound in this plane. RealSpace 3D
technology is moving in a direction similar to AMD TrueSoundfrom VisiSonics, which, by the way, is also intended for games and virtual reality. The common feature of technologies is that they provide for spatial separation of sources and sound modeling in both horizontal and vertical planes.
To create a positional sound, AMD uses AstroSound technology created by psychophysical research from the above-mentioned GenAudio studio. The technology is a block of algorithms that is used as a plugin in some DSPs. The library is not only available for all popular operating systems including mobile ones, but is also compatible with various hardware. Sound modeling occurs from 0 to 359 degrees in the horizontal plane and from -90 to 90 degrees in the vertical plane. Surround sound can be output through multi-channel configurations, as well as through headphones or two speakers.
In addition, AMD TrueAudio supports Convolution Reverb reverb, which the company compares with “space photography”. Reverberation is based on pre-prepared records in various spaces.
As a result, TrueAudio, coupled with Convolution Reverb, is able to work and process more than 100 streams in real time and provide positional sound in any space. The peculiarity of using reverberation in the context of TrueAudio is that its support will not load the processor, which was the main problem of the implementation.
Conclusion
A key feature of AMD TrueAudio is that in addition to the corresponding hardware (video cards based on processors with the GCN 1.1 architecture), products with technology support are needed.
The first game with TrueAudio support was Thief, which supports the reverb effect. It is noteworthy that all other audio channels are still processed by the CPU, and the processor is unloaded just due to the support of Convolution Reverb.
Another product example of using TrueAudio is the Tuscany demo application created by GenAudio and Oculus VR. In this case, the AstroundSound extension is used for the spatial separation of sound sources.
AMD TrueAudio deserves attention not only because it is one of the first solutions for improving sound in a long time, but also because of the possibility of a drastic change in the audio level on a PC. Moreover, the PlayStation 4 and Xbox One have AMD APUs and are built on the x86 architecture, which means that AMD TrueAudio may soon be ported to the console. It remains only to wait for the presence of the full implementation of technology in products for virtual reality and fresh games.