Since the release of the Recalbox RGB DUAL and even more so since the launch of the Recalbox RGB JAMMA campaign, many of you have been asking questions about input lag.

Input lag is a serious and highly technical subject, which for some remains an eternal debate.

Many talk about it, but few really master the subject. In the midst of fantasies and preconceived ideas, we felt it was vital to provide you with a transparent, factual, technical and rational explanation. That's what we're going to try to do in this blog post!

Before we get down to the technical side of things, let's talk a bit about semantics, to make sure we understand what we're talking about.

What is input lag?

Input lag is a term that's become a bit of a catch-all when it comes to defining a game's reaction time after an action on your part.

Here, we prefer to use the more appropriate term global lag.

To put it simply: global latency is the time it takes for your character to react, when you press a button on your controller.

But behind the term overall latency, there are actually 3 very distinct phases :

• input lag, in the literal sense of the term, which corresponds solely to the time it takes for the emulator to register the press of a key. This time can vary enormously depending on the controller, protocol (wired, USB, i2c, Bluetooth...), emulator configuration or operating system you're using.
• process lag, which is the processing time taken by the emulator to generate a new frame according to the event received
• video lag, which is the time between your console sending the video signal and its display on screen. This is your TV's processing time. Video lag is considered to be non-existent on CRT screens, although it can vary depending on the configuration of your HDTV.

So, you might ask: how can we measure all this in a factual way? Well, quite simply with precise tools and methodologies!

• input-lag-tools will enable us to specifically measure controller-related input lag.
• an oscilloscope will enable us to measure the sum of input lag and process lag.
• finally, the Latency Bro will enable us to measure the famous global latency as a whole.

1 - Measuring input lag

First, we're going to concentrate on input lag. To do this, we're going to try out different controllers and arcade kits, and compare the results obtained according to the systems used.

To do this, we've created a loop: a button on the joystick or arcade panel is connected to one of the Raspberry Pi's GPIO pins.

The Raspberry Pi will then be able to press the button itself, just as a gamer would. All it has to do is time the time elapsing between pressing the button... and receiving information from the system that the button has been pressed.

Of course, each of these measurements was taken dozens of times, to obtain a precise average input lag for each of the controllers tested.

Here's the results table.

In the left-hand column, you'll find the controller or arcade stick used, then its connection type, followed by the card and operating system used, to finally find the average input lag, and the percentage chance of missing a frame (of having a frame lag).

There are a number of interesting findings to be gleaned from these results.

As you can see, even wired controllers are not all in the same boat, with some generating more input lag than others.

What's more, the tests show that the connection protocol used has very little influence on the results. It all comes down to controller design and operating system optimization.

Even if the results are generally convincing for the USB protocol, it nevertheless shows its limits. It is indeed very difficult to go under 1ms with a USB connection. This is a limitation directly linked to the USB protocol: USB communication takes place in the direction PC/RPi to controller. The Raspberry sends a request to the controller at a given frequency to retrieve the status of the buttons: this is known as "polling".

It's precisely to free ourselves from this technical limitation that we have chosen to use another process on the Recalbox RGB JAMMA: interrupts.

When the controller detects a change of state on a button, it instantly notifies the Raspberry Pi of this change.

This is one of the reasons why the Recalbox RGB JAMMA performed so well in this test!

2 - Measuring process lag

With the first step, we've determined the time between pressing a button and it being taken into account by the system.

Now let's move on to measuring process lag, i.e. the time it takes for the system to process the event. Or, to be more precise the time taken for the system to generate a frame, i.e. an image, after receiving a button press event, such as a jump.

Using an oscilloscope, we can measure the time between pressing the button and sending the RGB signal to the screen.

The test protocol is as follows:

• a probe (yellow line) is placed on the button that will be used to change the image, and will therefore pick up the falling edge (the button press)
• a second probe (green line) is placed on one of the RGB signals. It reflects the RGB signal transmitted to the monitor. In RGB, the signal of a completely black frame will be transmitted as a sequence of 0v values, i.e. a flat line. For a white screen, with pixels at their maximum brightness, the signal goes wild and rises to 0.7V.
• For a better interpretation of the results, we're going to use the 240p Test Suite, which allows us to switch from an all-black to an all-white screen.

We'll then be able to measure the time between pressing the button and the image change.

The measurement in video :

It's in this situation that we can understand the effort made on Recalbox RGB JAMMA to reduce input lag to less than 0.5ms.

It's time to solve a little math problem:

If the 20.6ms of latency above contains 0.5ms of input lag on the Recalbox RGB JAMMA, what latency will a JAMMA controller with 10ms of input lag suffer, at the very least?

However, these measurements are laborious and difficult to repeat. That's why the Latency Bro was born.

3 - Measuring overall latency - Switching to arcade and JAMMA solutions

Measuring input lag and process lag is all well and good, but everyone will agree that what we're really interested in is overall latency!

So we're going to measure overall latency, i.e. the complete time that elapses between pressing a button and the corresponding change in image, whether on a CRT or LCD screen.

To achieve this, we have designed the Latency Bro, an electronic circuit capable of pressing a button itself, and measuring the time until a change of image is detected on your TV, thanks to a photosensitive cell. This cell is capable of detecting a change in brightness created by the CRT screen beam or by the brightness of pixels on an LCD screen.

)

It is therefore possible to measure overall latency accurately and universally, both on original hardware and on emulation systems.

We had three objectives when we designed the Latency Bro and measured latency on the hardware that follows:

• to answer your questions about the latency of Recalbox products (RGB DUAL and RGB JAMMA)
• take stock of latency on the arcade
• configure and optimize your Recalbox system using precise measurements
• and once and for all, put an end to this misguided notion of "debate" on a problem that is clearly measurable and improvable.

Here's how we went about measuring latency on the various systems:

• we start by accessing the service menu
• place the cursor on the "GRID" menu entry
• press button 1 to display the grid
• we'll measure latency
• then we'll press the START button and button 1.

Disclaimer: For the sake of objectivity, all measurements were made by @gtranche from HFSPlay, on his New Astro City with MS9 29 board. Many thanks to him and to HFSPlay for their time and support.

Here's a look at the latency bro in operation:

Boards tested :

We tested these jamma boards:

Test number 1: CPS2 + Super Puzzle Fighter 2 X

We start this procedure with the original hardware, here a multi CPS2 on which we've "written" the Super Puzzle Fighter 2 X rom (spf2xj.zip). We'll use this to establish a reference time, enabling us to compare other solutions and measure the latency they add.

Result: 80.7 milliseconds! It seems a long time to go from one screen to another, but it doesn't matter, because we now have our reference value on which to base the rest of our measurements.

For each of the Raspberry Pi to JAMMA boards, we decided to take the first measurement with a vanilla installation (without touching any configuration), as many users don't risk modifying the advanced lag options.

However, if options to reduce input lag are available in the system concerned, we activated them to see their impact on latency.

Here's the result, sorted in ascending order of latency:

As you'd expect, in first place in the solutions compared with the original hardware, the mister is very close to the original latency. Beware, however, of the joysticks you use, some of which may add milliseconds you could do without.

In second place, the Recalbox RGB JAMMA adds only 6.70ms or less than half a frame of delay. The default configuration of Recalbox RGB JAMMA, to which we've added run-ahead, brings us within a few milliseconds of the original CPS2 experience.

By deactivating Run Ahead, the Recalbox RGB JAMMA is positioned in 3ᵉ position with an addition of 20.80ms, or 1.25 frames of delay on average in its Vanilla configuration!

The RGB Pi Jamma, meanwhile, is +43.90ms, or 2.63 frames late, which can start to make itself felt on the most nervous games.

Finally, RPI2JAMMA, which was tested on a Raspberry Pi 3 (as it doesn't support RPi4), adds over 4 frames of latency. This can undoubtedly be improved with emulator configuration, but lag improvement options were non-existent at the time of testing.

Technical note: what is Run Ahead?

Run Ahead is a retroarch option that "pre-calculates" any frames that may be generated as a function of controller events. When an event is received, the pre-calculated frame is used directly, rather than having to be calculated on the fly.

So it's really useful, but not compatible with all games/systems.

About the Raspberry Pi

To make it easier for you to find your way around and compare solutions on the Raspberry Pi, we've simplified the results tables.

The first table simply compares the frames added in the fastest configuration for each solution:

And the second compares frames added by vanilla latency:

Test number 2: CPS3 + Street Fighter 3 Third Strike

No original hardware here, and the mister is out of the running as he doesn't (yet?) support CPS3.

So we measured latency on different Raspberry Pi solutions.

First observation: the latency to switch from one screen to another is much closer to what we expected on a menu as simple as the menu service: 20ms for the Recalbox RGB JAMMA with Run Ahead, which is very close to the display time of a frame at 60HZ (16.66ms).

And it's the Run Ahead coupled with the Recalbox RGB JAMMA's very low input lag that puts it in first place. The measurements will be updated once we've managed to recover an original CPS3 :)

The lack of vanilla configuration for JAMMA SD or RPI2JAMMA places them much further down the table, with delays of between 3 and 4 frames compared with Recalbox RGB JAMMA in Run Ahead.

Conclusion

We're delighted to have been able to put numbers, methodologies and test protocols in place for this blog post, but our mission to streamline input lag has only just begun.

And in the next steps, we'll have to :

• continue to improve the Recalbox system to reduce latency on CRT as on HDMI
• measure other systems and consoles, other controllers, and why not gather all this information on a specific site
• send a few prototypes of the latency bro to anyone who'd like to have them, so that we can double-check the results.

If you've read this far, well done! All that's left is to thank you once again for your support in the Recalbox project, which would be nothing without you!