Introduction

Definition: in gaming, the audio latency is the delay between a user action and the sound being heard.

This latency can be split into two parts:

UI Latency is the time between a user’s tap or click on the screen and the processing of the event on the software side.
To reduce UI latency, you can use either the new Unity input system or interface like OnPointerDown from UnityEngine.EventSystems. Look for TestTapMidi.cs source in TestOboe scene (Maestro Pro).
Hardware Latency origin is mainly related to the time need by the audio component (FMOD by default) to process and play the sound build by the software (like Maestro MPTK). The latency depends on the buffer size (DspBufferSize) and the synth sample rate.
It’s easy to calculate the hardware latency:
- latency_ms = (BufferSize / SynthRate) * 1000
- Example: 10,67 ms = (512 bytes / 48000 Hz) * 1000
Sadly, it’s not possible to defined small buffer with FMOD. Audio Glitches (Buffer Underruns) will be audible due to the overrun buffer. If the CPU cannot fill the buffer in time, the system may run out of audio data to play, resulting in audible pops, clicks, or dropouts.

This page explain how to reduce the hardware latency by changing parameters in the audio settings in Unity/FMOD (first option) or using audio component Oboe/AAudio/OpenSL (second option).

Android platform limitation: It’s quite difficult to guarantee a stable and accurate MIDI playing. Android threading is not accurate. It is largely sufficient for games, but should be avoided for more musical applications. Check with the demo before or start with a quick MVP.

First Option: Modify Unity Audio Settings

This the recommended method.

According to your need, you could have only to change some properties on the Unity setting to resolve your latency issue.

From Unity menu Edit → Project Settings → Audio

Set DSP Buffer size to Best latency.
Try with higher Sample Rate or let to zero.
Use audio effects with caution, as they use up a lot of CPU.

You can also change these parameters from the MPTK prefab inspectors.

Tips:

Audio latency is not a specific issue with Android but also on Windows, IOs, Mac, … Obviously, these setting are useful also for them.
Depending on your system, a too low buffer size can generate weird sound with some material.
Also, forget Bluetooth headset because of a high latency (for now!).

Second Option: Replace FMOD by Oboe

FMOD is the standard audio engine for Unity. It’s a powerful engine with a lot of features, well tested on a lot of hardware, but not compliant with very low buffer size. However, use the FMOD audio engine if latency is not critical to your creation.

Oboe is an low latency audio library that uses OpenSL and AAudio to handle the audio data on Android. Its requires Maestro Pro and some (simple) configuration.

Some important notes

Unity effects and 3D sound are disabled (these features are based on FMOD).
Oboe is working only on Android.

How to use Oboe with your MPTK application ?

Recommended Unity version: 2022.3 and above.
Recommended Maestro version: 2.12 and above.
Create a new Unity project.
Oboe is working only for Android, select build platform to Android in “Build Settings”.
Download and import in your project the Oboe package from here (version tested:1.6.2).
Download and import in your project Maestro Pro from here.
Go to Unity menu Edit → Project Settings → Player
- add symbol UNITY_OBOE (just remove this symbol to restore the legacy FMOD mode)
- add symbol DEBUG_HISTO_DSPSIZE to get buffer statistic (for dev step only)
- allow unsafe code

Starting with Maestro MPTK v2.12, the definition of assemblies has been added. This reduces the build time. Assemblies are perfect for isolating each component, the downside is that you have to manually declare which assemblies are used by the MPTK assembly.

Open the Midi Player Run assembly definition, allow unsafe code (search MidiPlayer.Run in MidiPlayer/Scripts), add reference to Oboe and apply.

Same action for Midi Player Demo assembly definition, allow unsafe code, add reference to Oboe and apply.

Build and deploy your APK and that’s all folks!!! MPTK has automatically switched to the Oboe audio engine.
To restore the legacy FMOD mode. just remove UNITY_OBOE symbol from Project Settings → Player

Demo

As usual with Maestro MPTK, all sources are available with the package on the Unity store.

The Oboe Test is a simple demonstration of MIDI and direct note playing. To play notes, tap or wipe on the magenta zone. You can adjust the synth settings in real-time to improve latency and sound quality:
Download APK with Oboe This demo is also useful to check the compatibility with the device. Look at the Frame length historic. If some frame are not a multiple of 64, the sound will be degraded.
Download APK with FMOD

MIDI Euclidean Rhythms
- This demo is useful for experimenting with the MPTK synthesizer voices, runtime effects, and running multiple players simultaneously. There are two players running simultaneously: one for the drums, which is a MidiStreamPlayer prefab, and one for the midi, which is a MidiFilePlayer prefab.
- Download the APK