Input Delay on Retro Consoles and Emulators



    TL; DR The article describes the known input lag problem, which manifests itself when trying to play old games on modern hardware: emulators of retro consoles, bluetooth gamepads, etc. Sometimes the delays are so big that it becomes impossible to play. I will describe my way of finding an acceptable configuration to run my favorite games.

    With the wave of popularity of RaspberryPi single-board computers, OrangePi many acquaintances bought for themselves their bundles. Without deciding what to do with them, they began to sculpt from them retro-console based on the RetroArch emulator and Recallbox distribution. When I tried to play my favorite childhood games on this, I was surprised: " How could I play this?". The physics of the games seemed somehow wrong, the feeling is disgusting. After some time they told me that the whole thing is the input delay, which at first glance does not feel like a delay, but just like other physics.

    It turned out that the problem has long been known among console geeks and They are actively exploring it, I will analyze other people's research and try to measure the delay of my setup.

    What is the problem?


    Entry delay is the time from the moment the button on the controller is pressed to the reaction of the character on the screen. Here many factors play a role at once:

    • Monitor / TV Delay - Some modern TVs have an image output delay of more than 100ms. This is due to image post-processing, buffering, filters, etc. This compares the CRT and LCD TVs with high latency. Typically, manufacturers of monitors indicate the speed of the matrix in the specifications, referring to the speed of switching between colors, but this does not have a bearing on the delay in image output.

      One analog signal from the set-top box on two televisions. There is a significant delay on the LCD TV compared to CRT.
    • Computer delay - a game console or a computer with an emulator does not process the signal instantly, it takes time. The computer needs to receive a signal from the input device, process it and display the image on the screen. The delay is also introduced by the vertical synchronization of the image when displayed on the screen. Here , RetroArch developers try to solve this problem.

      Demonstration of a big delay on the emulator. Video Denis Major
    • Controller Delay — The input device may also introduce a delay. For example, if connected to a loaded USB bus, via Bluetooth, etc.

    Emulators


    Everyone wants to play on modern hardware, normal TVs and monitors with HDMI, and also run all possible games on one device. For this, emulators of popular consoles that run on regular X86 or ARM computers are used. Here are the most popular ones:

    Retroarch



    The most popular and advanced emulator. Able to emulate PlayStation1, SNES, NES, GameBoy, Sega Genesis / CD and other consoles. It works on desktop OS Windows, Linux, MacOS and on modern consoles Xbox, Android, PlayStation2 / 3/4 / Vita, Nintendo Wii / Switch and others . Based on it, the popular Recallox distribution kit is made .⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

    Openemu


    image
    Very convenient and simple emulator for macOS. It supports most popular consoles and has a convenient game gallery sorted by platform. Out of the box, it supports playstation 4 gamepads via bluetooth. It is inferior in capabilities to RetroArch, but for my taste it is much more convenient to use. For my measurements, I will use it, since it immediately works on macOS without perdoling with gamepad settings. ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀

    Measurement procedure


    The most affordable way to measure the delay is to record on the video screen and the moment you press the joystick button. Modern smartphones easily shoot video at 240 FPS or more, which is quite enough for measurements. To accurately record the moment of pressing, you can solder the LED to the contacts of the button, which will light up at the time of pressing. The LED lights up in nanoseconds, which allows you to accurately capture the moment of pressing.

    Brunnis from the libretro project forum has done a lot of research on input lag. I consider his studies to be the most complete and objective.

    image
    USB gamepad with a soldered LED to record the moment the button was pressed

    image
    The button is not pressed

    image
    The button is pressed

    image
    Start of animation on the screen after 24 frames (100 milliseconds)

    Calculation formula


    In a second, 1000 milliseconds.
    The phone camera takes pictures at 240fps.
    So 1 frame of video = 1000/240 = 4.16ms.

    It is also important to consider the frame rate at which the monitor and computer are running. For example, for an operating mode of 30fps, one frame of the monitor will be 33ms, and for 60fps - 17ms. This should be taken into account, since the time between frames cannot be less than these values, which means that events occurring during the frame update period on the monitor will be rounded to the last frame.

    I will not give here all the results of the measurements of the dude, who are interested in suggesting to get acquainted independently.
    The main thing to remember: original consoles had an average delay of 50 to 70ms, depending on the console and PAL / NTSC mode. A well-tuned emulator on the PC adds 60-80ms of delay, depending on the emulated platform. The raspberry pi emulator can have up to 150ms of delay in total.

    My searches


    As a child, I had a Sega Mega Drive, it is her games that cause the strongest nostalgia. Therefore, I will describe my search for a comfortable setup specifically for this console. I want to say right away that my research does not claim to be an absolute measurement accuracy.

    My requirements:

    • Wireless gamepads - console geeks may condemn me, but I want to play comfortably while sitting on the couch in any position. Wired gamepads are terribly inconvenient, everyone hurts about them and you won’t play for a long time.
    • Output via HDMI or DisplayPort - there are no analog inputs on my monitor, so I can’t buy the original Sega Mega Drive. USB adapters for analog video capture introduce their delay and are inconvenient. I also don’t want to buy a separate CRT-TV for the set-top box

    Macbook + Dualshok 4 + OpenEMU


    I really like the gamestation from Playstation 4, I consider it the most convenient of the existing gamepads. It can work both via bluetooth and via USB cable. Fully supported in OpenEMU out of the box with no settings.


    Despite all the convenience of OpenEMU, its main problem is FPS drawdown in full screen mode. I'm not sure what the reason is, and maybe this is fixing, but it didn’t work out for me. Therefore, I play in windowed mode.


    OpenEMU emulator interface. All games are sorted by platform.

    Measurements


    I didn’t want to dissociate the gamestation from the Playstation to solder the LED, so I had to do the countdown of the moment the button was pressed by the sound. The error of this method is approximately 8-10ms. The monitor uses the LG 24UD58 in 4K @ 60fps mode connected to the MacBook via DisplayPort. Game Contra Hard Corps.


    The delay on the MacBook with a Dualshock4 controller connected via bluetooth is approximately 150ms. It's a lot!
    MacBook + Dulashock gives approximately 150ms of delay. This is a lot for a comfortable game, such a delay is noticeable and hardcore platformers like Contra Hard Corps play uncomfortable.
    Despite the fact that the delay in such a bundle is unforgivably large, it still deserves the right to life. Since it is enough to throw a gamepad in the backpack to the laptop and get a game console anywhere. Even before the measurements, it was clear to me that this was not the best option and I continued to search.

    Clone Sega Genesis


    Now many different pirate clones of retro consoles with pre-installed games are sold. Some of them are made from Android set-top boxes with console emulators inside, which means they suffer the same problems with delays. But some promise a full console without an emulator . I found the following console on Avito for mowing: retrogenesis.ru model Retro Genesis HD Ultra. They write on the site that this is not an emulator, but inside is really a hardware console. There are pre-installed games and the ability to insert cartridges. At the same time, there is an HDMI output and wireless gamepads!


    Wireless gamepads


    Most of all I was bothered by gamepads operating at 2.4GHz. Knowing the problems of bluetooth and WiFi in this range, the first thing I tried was to measure the delay of the gamepads themselves. The console supports the simultaneous connection of wired and wireless gamepads. It turned out that the wireless part was made using a separate module that closes the contacts in the same way as a wired gamepad. That is, in fact, it emulates an ordinary wired one with the same pinout.

    image
    A radio module that emulates the clicks of a wired gamepad

    I connected the probe 1 of the oscilloscope to the LED on the controller, and the other to the pin corresponding to the Up key on the console. When the signal is processed by the radio module, it simulates a keystroke, and I will see it on the probe 2. By comparing the time difference between signals 1 and 2, I could very accurately get the delay that the wireless gamepad adds. On an oscilloscope, the scale of one cell is 5ms, which means that the delay of wireless gamepads is 25ms.


    Comparison of signals on the gamepad and console


    Wireless module adds 25ms delay

    Measurements



    The delay on the Sega Retro Genesis is about 70ms. Subtracting the

    25ms delay of the radio module, it turns out that the console itself has a delay of 50ms, which corresponds to the original Sega. When connecting wired gamepads, the delay will obviously be less by 25ms. In general, the gameplay feels like on the original console, and I'm satisfied. Only the inability to download my games (ROMs) upsets me, but I will try to solve this problem with a rewritable cartridge, or try to find the opportunity to upload games to the built-in flash drive myself.

    Conclusion


    My measurements do not take into account the delay in the output of the monitor, but all measurements were taken on the same monitor, so it does not affect the results. Each measurement I repeated 10 times, sometimes getting a spread in readings of about 4 frames, that is 16ms. I believe that the point is the delay in vertical synchronization when forming an image on the console.

    In general, I think that my measurements give a general idea of ​​the delays on the two setups and may be useful to those who choose a method.

    Runahead option on RetroArch


    A year ago, RetroArch developers announced that they achieved fewer delays on their emulator than on the original consoles. I have not tried this option myself, but I need to keep it in mind.

    References


    Thanks to Denis Major for his research. Thanks to this video, I first found out that the problem is not in the modified physics of the games, but in the input delays.

    A lot of research on delays across platforms .

    Another study .

    Pro Hux Space Neuron



    In our Hackspace Neuron, we regularly play retro consoles, reverse the iron, and do many other things. With us, you can use a soldering iron, an oscilloscope, a frequency analyzer and chat with funny nerds. Come to visit.

    All news about the life of Huxpeys in the telegram channel: @neuron_news
    Tags:

    Also popular now: