Sound games: an invisible market awaits heroes

A fragment of the official poster for the game A Blind Legend .

Surround binaural sound from the headphones creates the effect of presence and forces the player’s brain to draw “graphics”. Audio quests, sound RPGs, and blind shooters are sold on GooglePlay, iTunes, and Steam for between $ 2 and $ 20. It is all the more interesting that in Russia, sound games and interactive audio books are an empty niche where only a few people try themselves. What is this invisible market, who needs it and how to approach it - found out the author of GeekBrains Maria Lisyanskaya.

The article consists of four parts:

• find out who needs sound games and why
• We talk about what technologies audio games are based on,
• We’re talking with a player who buys audio games and monitors their development,
• Talk about tools useful to a sound game developer.

Audio Games - Why and Why

Computer games that you can play with your eyes closed attracted the attention of professional developers in the early 2000s, when the technological base ripened for such projects. 3D sound positioning systems made it possible to navigate in the playing space by ear. This presented new ideas for gameplay and instantly interested people with visual impairments. Search for enemies by sound was implemented back in 1996, in the graphic horror movie Enemy Zero, where you had to look for invisible monsters. But then a full-fledged audio panorama was out of the question.

Later, a new incentive appeared for the development of audio games - it is convenient to play them on mobile devices. Here you have non-standard gameplay, and hearing training, and the opportunity to play without straining your eyes after a working day. Put on your headphones - and go. At the same time, you can be in a public place and not lose control over what is happening around.


Photo: StockSnap There

are few sound games so far - hundreds of people are counting. There are even fewer high-quality products with professional voice acting, so each new product attracts attention. To verify this, take a look at the forum forum audiogames.net , which discusses new releases. The bulk of the audience here is blind players. Thanks to screen access and braille displays, totally blind people use the computer like everyone else, but from mainstream graphic games they have little access to it - mostly fighting games where you can hear which side the opponent is from (Mortal Kombat IX and X, Injustice 2, etc. ) This situation encourages the blind to take up the development of games themselves and, of course, support the dollar and euro for those who release something worthwhile.

The threshold for entering the development of sound games is lower than in a graphic igrostroy. No need to hire artists and animators, no need to measure polygons or shaders with anyone. Everything that you can express in words and sound is possible in your game world. A feasible task for small teams and enthusiasts.

How it works

Some sound games are completely voiced by living people, others by a speech synthesizer. Screening technologies for many are associated with Windows accessibility, TalkBack application on Android and programs for creating audio books from text (ICE Book Reader, “Balabolka”, Demagog, etc.). The basic principle is the same everywhere: the speech synthesizer reads aloud to you.

Audio games with professional acting voice acting can produce some of the information through a synthesizer. This allows you to listen to the text in your desired voice, with adjustable pitch and speech speed. For example, a person who listens to a text in a foreign language may slow down reading to parse every word. And the one who saves time, on the contrary, will set one and a half or double speed. If you can take notes or rename objects in the game, again, a synthesizer will come in handy. For amateur authors, artificial dubbing is a saving on sound recordings, the ability to reduce the weight of the game and change dialogs on the fly without the need to dub voice announcers.

To voice themselves on Windows, audio games typically use Text-to-speech (TTS) technology and the MS SAPI speech interface. Under this interface, different companies produce voice synthesizers, which are also called speech engines. Each engine has its own set of votes, as a rule, for several languages.

By default, you have at least one English SAPI voice installed on your system. Russian, French and other voices can be downloaded from the Internet. For example, on GitHub there is a free voice engine RHVoice and several Russian voices to it. Esperanto, English, Georgian and Ukrainian are also supported. Compared to commercial products from the Nuance Vocalizer Expressive line, RHVoice voices are not so realistic, but legible and with pronounced intonations.

If you dive a little deeper into the world of sound games, you will find that some of them are voiced not through SAPI or its analogues for other OSs, but using an external screen access program (“screen reader”). Such software allows you to control the operating system and the applications running in it by ear. Among blind Windows users, the popular NVDA screen access program is popular. Making friends with your game is easy with the NVDA Controller API library . We will also touch on the topic of screen readers in the section “Engine for audio games”.

By genre and organization of gameplay, sound games cover a lot of directions: from simple puzzles to large-scale RPGs, simulators, races and strategies. Let's get some examples.

Conversation with the player

In order not to get stuck on theories and my own subjective impressions, I interviewed Vladimir Erokhin, the person who moderated the site for sound games blind.games for a while, still plays a lot of audio games and knows the community of blind gamers from the inside.

- What are your favorite audio games and why? Share your impressions.

- My undisputed favorite is “A Hero's Call”. I love RPG, and this is not just a party RPG, but a game with an open world, convenient navigation, a developed system of skills, an interesting story, a lot of quests. What a cool ambient and surround sound - it's just necessary to play and feel. You walk past the fountain and clearly feel that it is behind you, hear how far it is, hear the drops hit the stone. The developers somehow screwed the technology from the Oculus Rift virtual reality helmets. I understood this from the oculus.dll file and the names of other libraries in the game folder. Voiced even passing characters. All sound weighs more than a gigabyte. Few games without graphics weigh so much.

And the music! This is a separate topic. The trick is that each character from among your party members has his own musical theme. And each character has his own story and quests that reveal his character and motivation. In the dialogs, you play the image of the main character, and your choice really affects both the relationship with the NPC and the skills that are revealed to you and your group. Through dialogs, you can even pump some of the satellites, if you motivate them, help them become more confident in themselves. You can talk with the characters of the group at any time, and sometimes it promotes you in the story. But it’s difficult to predict the consequences of a choice. I’ve done a good deed - I returned to the owner the ring stolen by a thief-sopartiyets. I didn’t know that then my companion would lead me to the theft customer, which in the end will remain offended and will block our access to the best item store in the game. Well, the thief, of course, was upset ...

All dialogs are voiced by actors. I normally understand English by ear, and if a person does not keep up with the announcer, he can re-read the same thing several times with text through SAPI and then use the up and down arrows to select the answer option. To whom it is very difficult, in the settings they can enable Google translation in real time.

- Do you have any disadvantages of this game?

- She's short. Developers talk about 17 hours of gameplay. I played in fits and starts, so it took about a month and a half to complete. I have never regretted neither time nor money: I put 9 out of 10 to the game. I would like the authors to continue in the same vein.

- It is only for Windows, there will be no mobile release?

“Of course it won't!” There, the engine is not even sharpened for mobile devices. To transfer the entire interface would have to be rewritten. Unless there may still be a release on the Mac. They are now making a version with graphics support for the sighted to play more and to make money on it.

- Is there anything interesting among mobile games?

- There is! But here we must look at what platform we are talking about. Most mobile audio games are now released under iOS. Under Android - very little.
My second favorite game, A Blind Legend, is for PC, iOS, and Android. I went through it on PC and iOS at different difficulty levels. Very dynamic gameplay! I have never seen a game in which, running away from the guards, it would be necessary to jump on the roofs and, in order not to crash, put up a shield. A very interesting plot concept: a daughter helps a blind knight in his travels - he warns of dangers, tells where to go. Unusual and high-quality combat system, cool bosses. If you fight with a troll, you fight with a troll, and if with an underwater monster, it's completely different. Because everyone has their own behavior, their own system of punches and tactics. Before attacking someone, you need to learn how he fights, and only then catch him on a long swing, for example, and strike.

- How long is it?

- This game is shorter in duration than “A Hero's Call”. They also say that of the two versions of voice acting, French is better. I can’t appreciate this, let those who know or study French appreciate it.

- How is the gameplay on the mobile device different?

- In my opinion, the game feels sharper with the touchscreen, because it is very simple from the computer, for example, to put the block with the “F” key. And it’s impossible to put blocks so mechanically from the cell phone, you’ll still be hurt because the screen is not responding so fast. You feel the connection of game actions with your movements more, you feel the weapon better. In fact, I'm covering the touchscreen glitches right now (laughs). On a smartphone, this game is an excellent time killer, especially if you need to pass the time on the road. Although, in principle, it claims to be much more. The voice acting is great, the plot is decent.

- Does the mobile version have paid content?

- I don’t remember exactly, but, in my opinion, if you are killed in the mobile version and you want to immediately continue the game, you need to buy something else. Otherwise, you will be able to return to the game only after some time. Therefore, in fact, I prefer the PC version and often played it. For a computer in general, games are many times more and I spend more time behind it.

- What games other than “A Hero's Call” did you buy?

- Entombed (this is a dungeon crawler, I did a stream on it), The Nightjar, zombie shooter Audio Defense, two parts of Papa Sangre - they are awesome just! Audio Defense and the second Papa Sangre use an accelerometer. That is, you control the character, turning with the phone in your hand. In a shooter, this is very convenient, because when zombies twist at you from different sides, you kill one - you quickly turn around and shoot the second. In close combat, you can still strike with a bat or stick from top to bottom. By the way, in Audio Defense you have a lot of different weapons, well voiced, which can be pumped. Now I’m already used to it, but in the first place it was a very interesting experience. So I casually named some more favorite games.

- You haven’t really talked about Papa Sangre yet.

- This is an adventure. They don’t give a fight. At each level, in order to solve the problem: save someone or find something - you need to distract or bypass monsters. If the monster is sleeping, and you start to stomp strongly, touch the glass or rustle, he will wake up. And the big question is whether you can run away from him. Therefore, it is better to carefully pass. And you rearrange the legs with two buttons on the touchscreen: one foot at the top of the display, the other at the bottom. You put both hands on the smartphone and step. If you run too fast, you will stumble, fall, make a noise, and while you get up, they will reach you. Still, when you run away, you can crash into the wall. While you will be deployed, you will definitely be devoured. And not all monsters sleep. Some walk on the level, some still fly. And dying in this game is very unpleasant: the sound when the character is devoured is too realistic, binaural,

- Is the screen reader useful for this game?

- Only in the menu, and the levels are fully voiced. Voice Over, which is responsible for screen access in iOS, has to unload during the game, because otherwise it intercepts clicks.

- How does the second part of Papa Sangre differ in the plot?

- In the second part, the hero after death falls into the kingdom of memories. To regain the human body, you need to collect memories in the form of musical fragments, and so interesting stories gradually develop that are not always associated with the main character. This may be, for example, the memories of a sailor about a sunken submarine. And in the second part, a system of achievements appeared. Get out of a burning house in so many seconds, never raise the alarm and another 100,500 achievements. If a person suffers from Manchkinism, he will hang for a long time with this game until he collects all the achievements.

“Have you ever regretted buying an audio game?”

- Only once when I bought Blind Gladiator. They did not provide a demo. “A Hero's Call” was also not provided, but there at least sound previews could be heard. And then I bought it and regretted it. In other cases, on the contrary, I was pleased to encourage the work of the authors, to encourage them to do something else.

- What annoys you in sound games?

- Many games sin with a bad sound, so bad that I don’t play. They also torment me with stupid stories of the same type, especially in side scrollers, where you run and kill with packs of monsters. It’s rare when games are addictive for a long time. In crafts on BGT annoying an abundance of bugs.

- Which genres are best suited for audio games?

- Action can be well implemented. The Road to Rage - there is such an action shooter, and it is written in PureBasic. And it’s written very well: if I didn’t know, I would not have thought that such a thing is possible on PureBasic. You can also implement a strategy very well in sound where active actions are not needed. I would not refuse new high-quality races of the Top Speed ​​level or from a simulator.

- How do you like the Eurofly flight simulator?

- I love this game for its scale and deep study. By the way, my friend, also a blind player, took part in adding real-life airports to this game. Thanks to him, for example, Kemerovo Airport appeared there. The map shows the vicinity of the airport, so if you live close by, there is a good chance of manually examining the map to see your area.

- What game are you missing? What more do you expect from audio games?

- They would have made a tactical audio game such as the sighted "Jazz. Work for hire ”or“ Jagged Alliance ”, which I played with the help of friends. Here I would be stuck there! This could be done as a mod pack for the Tactical Battle sound game. But I would prefer a full-fledged separate game. So that there were mercenaries, arms suppliers, so that it was necessary to capture roadblocks. I would also play an audio game in the Warhammer universe.

- And what about multiplayer games?

- I play a lot of text MUDs. These are not audio games, but they are accessible and some of them can be set up with sounds that play during certain events. I was surprised at first that the sighted still play in madi, where there are no graphics. A friend told me that he plays for the sake of an adequate company. Because you go into games like WOW and hear from schoolchildren about mothers and so on.

“People who like to read gather in madas.”

- Yes, there is a lot to read. And they love to fantasize, to represent the game world. In audio games, there is also such a moment.

- And what, in your opinion, can audio games be interesting for the sighted?

- Some sighted people with whom I communicate play audio games for the sake of new sensations. Even if they don’t get out of the habit and often lose, they still have fun: “How cool I crashed now, ahh ... What a sound!” etc.

- How do you look for games for yourself?

- Through the site audiogames.net and its forum - there you can track interesting projects long before the release. Through friends with whom we share games for the sighted and the blind. Even through the AppleVis website, where all iOS sound games appear.

- Thank you for the story! I feel that much remains behind the scenes, but you can’t discuss it all at once.

- You are welcome. Yes, the same Mady - this is a separate big topic, worthy of a separate article.

Genres and examples of sound games

We have compiled the TOP player, and now we will add links and facts so that you better understand what it is about.

Sound Action and Adventure

A Blind Legend is a mobile hack-and-slash with realistic binaural sound and real-time battles. The game in English and French can be downloaded from PlayMarket , iTunes Store and Steam . Developers from the Dowino studio (with the participation of Radio France) raised this project on crowdfunding and raised $ 53,469.

Papa Sangre for iOS is a survival horror infused with the spirit of Mexican mythology. The only way to survive is to walk quietly and not wake up famously, and in which case, run away. The popularity of the first part (2010) inspired the developers of Something Else to continue, which was released in 2013. The sound quality in the game can be estimated here.. The studio also released other audio games: The Nightjar and Audio Defense. Alas, all of them have already disappeared from the AppStore. But the Papa Engine , which allows you to make real-time games with binaural sound, has healed its life. Some time ago, developers announced that they were redoing it for compatibility with Unity.


Source: TheVerge.com . There - in detail about the game (English).

3D Sound Shooters

Shades of Doom is a first-person shooter for Windows. We focus on the terrain using a computer, as well as on the echo of our own steps, the draft in the doorways and the noise from objects nearby. The game simulates the Doppler effect - the frequency of sound changes as the source moves. The latest version of Shades of Doom is sold on the developer's website for $ 25.

Swamp is a multi-user FPS. The action takes place during a zombie invasion of a town near a swamp. You have to survive (this is not easy) and complete missions together with other players. An English review of the game can be heard here .

Audio RPG

Sound role-playing games are expanse for a novice developer and a chance to realize epic ideas without spending money on graphics.


Source: OutOfSightGames.com .

A Hero's Call is an RPG with six character classes and 70 types of enemies. It runs on Windows XP, 7, 8, 10, and with the Wine program on Linux and MacOS. The game was created using the tools of FMOD Studio . There are sound demos for navigation and control. Developers ask for $ 20 per game, and they pay this price, including users from Russia.

Sound simulators

Eurofly is a free civil aviation simulator. The game uses real GPS coordinates and a detailed digital atlas, by which you can study the nature of the terrain and the location of settlements in the areas of interest. Fly, refresh your knowledge of geography and get acquainted with the airports of the world, each of which has its own convergence height and its own characteristics. You can go on a “free flight” or sequentially complete missions and open aircraft models.

The game covers all aspects of pre-flight preparation and flight: boarding passengers, refueling, talking with airport managers, announcements in the cabin, etc. Time is running at a real speed: if the flight takes 5 hours, you fly so much. When the fuel runs out, you can board at the nearest airport, unless prohibited by mission conditions. For example, an airline may prevent you from crossing Austrian airspace on the way from Slovakia to the Czech Republic.

Many mods have been released for Eurofly : airplanes, airports, stewardesses (there is a Russian steward Andrey), dispatchers, sound effects. On the developer's site, you can track the “current traffic” - where and from where game flights fly, where each aircraft is now and how many passengers it has on board.

The project exists and is being developed through donations from players. The game is available in English, Slovak, Czech, Italian and Spanish. According to some reports, work is underway on Russian localization.

And for fans of war there is a GMA TankCommander - a tank simulator that fully voices itself. You are attacked by tanks, helicopters, cars, trains, artillery and troops. During maneuvering, one must not land in a mine or in a swamp. The game is in English, there is an unofficial crack.

Side scrollers

You can’t say “side view” about the sound game, but this is exactly how it works in the organization of the playing space: to move forward, you press the “right” key. In Angel gift , an angel girl tries to return love, joy and peace to a village captured by demons. To do this, Liriel sits on the roofs of houses, listens to what is happening inside, and gives people what they need, and when necessary, they fight demons.

Augmented Reality

Visual aids are primarily associated with augmented reality, but sound can play an equally important role. The Audiogames project (3D-sound game installation)
is an example of an interactive three-dimensional sound environment with which the user interacts to solve conditional gaming tasks.

Interactive audiobook

Audio wave is the simplest version of a sound game, but with high-quality voice acting and a well-thought-out script, it captivates like a good book. The plot screensavers are not separated from the gameplay - the narrative develops continuously, so you need to carefully listen to each scene. For a player, this is a good way to pump knowledge of a foreign language : English, Japanese or French - given the countries in which sound games are most often released.

In 1997, exclusively in Japan, a sound novel was released for the Sega Saturn"Real Sound: Kaze no Regret." Everything here, starting with the main menu, was voiced by the actors. The player heard music, background sounds, character dialogs - and chose action options using a gamepad. Real Sound was one of the first commercial non-visual games.

The audio quest works on similar principles, but is more interactive: we move around locations, listen to their descriptions, communicate with characters, use inventory items. Sometimes action elements are interwoven into the quest, as in the mysterious Chillingham audio quest, which has disappeared from the Internet, where there was a battle system. The key role in the games of the genre is played by the plot and atmosphere. Authors can rely on horror or humor - as in Grail to the Thief , where a rogue from the future moves in a talking time machine in the era of King Arthur.

Sound Game Tools

An audio game can be made on Unity or another engine with support for 3D sound positioning. But there are specialized tools for this task. It's time to talk about them, and about libraries that come in handy if you want to create your own engine for audio games or to understand the topic more deeply.

Work with sound

Photo: theglassdesk .

We talked about the Doppler effect in Shades of Doom. In the days of Windows XP, this was done using DirectSound3D. In Windows 8 and 10, the X3DAudio program interface and the Xaudio2 library are responsible for working with positional sound .

You describe sound sources using the X3DAUDIO_EMITTER structure, where there is a Doppler Scaler parameter that is responsible for the severity of the Doppler effect. The parameter can take values ​​in the range from 0.0f to FLT_MAX. A source can have multiple channels. Sound from multiple sources realistically blends into the overall scene.

To control the "listener" - the object relative to which the sound stage is being rendered - use the X3DAUDIO_LISTENER structure.

The easiest way to work with XAudio2 is in C ++, but there are also wrappers.

• For Python, there is Pyrilla - a wrapper for the Gorilla Audio C library that runs on Windows, Linux, and OS X. Pyrilla is included in the PyPI package directory and you can install it without dependencies - just pip install.
• On the .NET platform, you can create positional sound using the XAudio2 HRTF wrapper with GitHub . The plus is that you can use the binaural synthesis system (HRTF) without Unity and other powerful tools. Solution sharpened under Windows 10.

FMOD . Anyone who is even a little familiar with the theme of sound in games, probably knows about this library. It is used in commercial projects of the level of StarCraft II, Just Cause 3 and MotoGP17. Through the API, FMOD can be used in Unity, UnrealEngine and with other game engines. A description of all the features of the library requires a separate brochure. In short, FMOD allows you to create 3D sound, recognize the properties of a user's sound card, and use WMM and A3D sound systems. The library is easy to distribute with the game - just copy the .dll to the user's hard drive.

Bass. The BASS cross-platform library will provide you with sound not only on Windows (x86 and x64), but also on Android, iOS, Linux and MacOS. There are tools for 3D sound positioning and adding effects (echo, reverb, distortion, flanger), support for A3D and EAX, as well as the ability to play tracker music, record samples and stream audio streams to the Internet. A plus for a beginner igrodela will be the presence of an API for using BASS with Python (pybass), C # (Bass.Net), Java (NativeBass), C / C ++. Fans of old school and hardcore can also experiment with PureBasic, FlatAssembler, Delphi, VisualBasic.

Audio Game Engine

Blastbay Game Toolkit (BGT) is designed exclusively for creating sound games for Windows. There is support for multi-user mode and there are no genre restrictions. The game code is written in the scripting language AngelScript . Since 2014, BGT has been free for commercial and non-commercial use. Users of the engine released dozens of libraries for it, which can be found on the official forum.

BGT “out of the box” works with MS SAPI and all popular screen access programs: NVDA, JAWS, Window eyes and System access. You can offer the player a choice: use one of the SAPI voices installed on the computer or screen reader. You can also force the game to automatically check if the player is running a screen reader, and if not, enable SAPI output.

Unfortunately, the author stopped supporting the engine in 2016, but according to the capabilities of BGT, it is still not outdated and is suitable for the first projects that will give you starting experience in developing audio games.

Examples of games on BGT:

• Kringle Crash is a little arcade game where you are Santa Claus and you must protect fragile Christmas presents from unbelted elves.
• Obsessive Compulsive is a logic simulator that forces you to sort, group and match objects. There are no correct answers, the main thing is not to contradict yourself in anything. The game is not compiled and requires the installed BGT engine. But you can study its code!
• Damage extreme is a sound arcade game that fully voices itself using MS SAPI. The player will have to defend the base alone from tanks, aircraft and landing. This is another partially open source game that you can explore.

Work with voice interfaces

To synthesize and recognize speech in the game, you can use TurboPower Async Professional tools , which provide direct access to the Microsoft Speech API, TAPI and serial ports - from the C ++ Builder and ActiveX environments.

Work with Augmented and Virtual Sound Reality

In an interview, Oculus’s spatial sound positioning technology was mentioned above . A detailed description of it is here . Google Resonance Audio surround sound

system allows you to build complex sound environments for games, augmented applications and virtual reality. The system runs on Android, iOS, Windows, macOS, and Linux. The presence of an open-source API and SDK (licensed under Apache 2.0) allows you to make friends with a system with different programming languages: Java, JavaScript, C / C ++, Objective-C. In addition, Resonance Audio integrates with Unity, Unreal Engine, FMOD, Wwise and DAW (Digital Audio Workstation).

Conclusion

So far, interested users are buying audio games from foreign developers. The lack of competition from domestic teams is an unfortunate gap, because the creation of sound games is relatively simple and inexpensive. Instead of making a hundred thousandth clone of minecraft or a typical casual for a set of experience, it makes sense to try yourself in a new direction.

Even if you do not want to completely abandon the graphics, you can make the game available by ear using hot keys and optional sound comments.

I would be glad if this article leads someone to new thoughts: inspire you to create a sound game or encourage you to use speech interfaces in the application. Questions can be left in the comments - I will try to answer.