Kwiki - control your hands! augmented reality game

    Good afternoon, Habrasociety

    This publication will be of interest to those who are not indifferent to innovations in user interfaces, as well as enthusiastic Augmented Reality (Augmented Reality) .

    If we take a closer look at the work of the human brain, try to recreate the tasks that it performs, in particular those that we take for granted, you can see how huge the number of calculations performed by the brain is. For example, motion detection and tracking, or the difference of more than one object, their movement relative to each other, their speed and distance between them.

    I note right away that Kwiki The Bird and the development of such games is not the main activity of our company. The aim of the game was to test the capabilities of various algorithms for determining / tracking movements of a computer's webcam.

    What did we get:

    You can play the game here:

    Marketing part:

    To promote the game, we have chosen platforms for distributing free indie games - like kongregate . The best in our opinion were: , . The number of views / games on the day of publication was 500 people, without additional promotion methods.

    Game concept:

    Motion recognition technology, the algorithms of which exist now ( below the link ), enough cheese. Inaccuracy of tracking. At first, we thought about hiding such inaccuracies with the original idea of ​​the game - a drunk bird, which is difficult to control.

    Our task was not to keep the user in the game for a long time. Since this is impossible for such games, since controlling the camera (in most cases built into the top panels of laptops, iMacs and just standing on the monitor), your hands get tired very quickly. Namely, this is your controller, constantly in the air. Therefore, I wanted to see the viral spread of the game among the audience.

    Actually, this turned out, according to statistics (each user who played the game brought his friend, as can be seen from the photographs taken during the game).

    Therefore, I want to call this item more or less successful. Still, games with a new control interface, namely a web camera, are viral and can spread themselves.

    Then, after watching a popular video about the Kiwi bird, which does not know how to fly and strives into the sky, - Kiwi (YouTube Most Adorable video of 2006). We decided to make a kind of sequel, using the idea of ​​a kiwi bird, which was supposed to hide some of the shortcomings in technology.

    Technical part:

    Designer's work - 20 hours augmented reality game

    1. Creating the image of the main character - Quick.
    2. Design 20 game levels.
    3. Design of trophies (3 for each subgroup of levels - Earth, Above ground at night, Sky, Space). Trophies do not carry any special significance, but increase the user's interest, which, in general, stimulates the game to continue and fly higher.
    4. “Bad objects”, their own for each level, for the capture of which the user is taken away points.
    5. "Deadly" enemies, a collision in which means the end of the game.

    The flash programmer's work (game logic) - 40 hours.

    With logic, there were a small number of alterations. In the beginning there was one screen and objects simply fell from top to bottom. We decided to add a sense of flying, every time Quickie eats a good object, he moves n pixels up to his dream, and, accordingly, when he eats a bad one, he goes down.

    Running the first test of the game among our colleagues / friends, surprisingly, we saw that many people manage to control the bird despite the lack of technology and different lighting conditions (dark / bright room). And so it was decided to add “deadly objects” on the senior level groups: augmented reality game
    This is a thundercloud, an airplane and a UFO, not habranlo of course, but quite impressive on the last subgroup of levels - Cosmos.

    Moreover, some testers even managed to read, cheat, controlling the bird only at the top of the screen and collecting all the bad / good objects, gaining the maximum number of points.

    Job Programming (facebook) - 20 hours

    Because We already had experience in creating Facebook applications, with this game we decided to try to enter the Facebook market (although it is said loudly, given the goals that we set for ourselves).

    The main aspects were added:

    • facebook connect for stand alone application (games from the site),
    • function of automatic publication of results in a news feed,
    • The ability to view the activity of the game directly from the stand alone website and the I Like button, which affects the display of results in a search on Facebook, is most of all.

    Work with gesture recognition algorithms - 80 hours

    Here, perhaps, the most interesting thing I want to share with you. What algorithms we used, what we changed, and most importantly, why:

    Inspired by the work of Grant Skiner and of course Mario Klingermann aka Quasimondo . The latter, by the way, has a fairly large number of examples ported from C, openCV, to flash, for face recognition and tracking, as well as what is called NFT (Natural Feature Tracking) and tracking of colored objects.

    The last algorithm in the list, but not the last for us, but rather the first one that I want to note is the Justin Windle motion recognition algorithm .

    Because Our bird should fly, we should not only determine the movement, but also do its tracking.

    Actually, the similarity of our algorithms was in dividing the screen into a grid, we average the colors within each cell / segment with a regular interval. The difference in brightness within each segment (from one calculation to the next) will determine whether there really was a movement in this area of ​​the screen. augmented reality game

    After analyzing all the work, it was this approach that turned out to be the most accurate for us. Although, in reality, it takes a lot of action to get Pixel () , set Pixel (), as well as the conversion of these results into color, saturation, brightness of objects, each of which is heavy enough for the brain of our computer - the processor. A good computer, of course, copes with this easily, for example, the iMac 3.06GHz copes with this with a bang.

    We summarize how this works in Flash:

    • After the camera object was created, it passed to the MotionTracker class , which extends the Point object . The MotionTracker class contains two BitmapData objects, one representing the current data from the webcam, and the other is used to store the previous frame.
    • When updating the method is called, a new frame is drawn on top of the previous one using the filter difference.
    • The results are obtained by ColorMatrixFilter , increasing the contrast of the image and dropping dark areas (with less movement) further into the background.
    • The blur filter is then applied by us in order to further reduce the noise level and form the “Fate of Movement” from areas where more movement occurs.
    • The trashhold is called as a result of BitmapData , in fact, as a result of which the grid next to the black pixels should be ignored, and all light pixels are assigned “motion detected”.

    The level of the trashhold can be adjusted in the game settings, with a simple slider that moves from a relatively bright room to a dark one.

    The parameters that make up are the brightness, blur (contrast), the contrast and the spacing of the segments of the same mesh.

    Future plans

    1. Starting VKontakte after creating the second beta version and fixing all bugs.
    2. Add game mode training or how to.
    3. Automatic detection of the active camera.
    4. Adding more design and sound effects.
    5. Change Quicki's action animation.
    6. Posting photos of the user on the walls of friends on facebook, increasing the sharing parameter.
    7. Change background music :).

    Thank you for attention.

    Lastly, a fan of the video “how to play Quickie”:

    Also popular now: