Definition: in gaming, the audio latency is the delay between a user action and the sound being heard.
This latency can be split into two parts:
- UI Latency is the time between a user’s tap or click on the screen and the processing of the event on the software side.
To reduce UI latency, you can use either the new Unity input system or interface like OnPointerDown from UnityEngine.EventSystems. Look for TestTapMidi.cs source in TestOboe scene (Maestro Pro). - Hardware Latency origin is mainly related to the time need by the audio component (FMOD by default) to process and play the sound build by the software (like Maestro MPTK). The latency depends on the buffer size (DspBufferSize) and the synth sample rate.
This page explain how to reduce the hardware latency by changing parameters in the audio settings in Unity/FMOD (first option) or using audio component Oboe/AAudio/OpenSL (second option).
First Option: Modify Unity Audio Settings
According to your need, you could have only to change some properties on the Unity setting to resolve your latency issue.
From Unity menu Edit → Project Settings → Audio
- Set DSP Buffer size to Best latency
- Try with higher Sample Rate or let to 0.
- Minimize audio effects as much as possible
You can also change these parameters from the MPTK prefab inspectors.
Tips:
- Audio latency is not a specific issue with Android but also on Windows, IOs, Mac, … Obviously, these setting are useful also for them.
- Depending on your system, a too low buffer size can generate weird sound with with some material.
- Also, forget Bluetooth headset because of a high latency (for now) !
Second Option: Replace FMOD by Oboe
FMOD is the standard audio engine for Unity. It’s a powerful engine with a lot of features, well tested on a lot of hardware, but too general for efficiency. However, use the FMOD audio engine if latency is not critical to your creation.
Oboe recqires Maestro Pro and some (simple) configuration.
How to use Oboe with your MPTK application ?
- Recommended Unity version: 2021.3 and above.
- Recommended Maestro version: 2.12 and above (in progress).
- Create a new Unity project.
- Oboe is working only for Android, so select build platform to Android in “Build Settings”.
- Download and import in your project Oboe (version tested:1.6.2) from here.
- Download and import in your project Maestro Pro from here.
- Go to Unity menu Edit → Project Settings → Player
- add symbol UNITY_OBOE (just remove this symbol to restaure the legacy FMOD mode)
- allow unsafe code
Starting with Maestro MPTK v2.11, the definition of assemblies has been added. This reduces the build time.
- Open the Midi Player Run assembly definition, allow unsafe code, add reference to Oboe and apply.
- Same for Midi Player Demo assembly definition, allow unsafe code, add reference to Oboe and apply.
- Build and deploy your APK and that’s all folks!!! MPTK has automatically switched to the Oboe audio engine.
Some notes
- Many different devices use Android, it’s not possible to check all smartphone and tablet hardware.
- MPTK Unity effects and 3D sound are disabled but soundfont effect filter works well.
- Oboe is working only on Android.
- To restore the legacy FMOD mode. just remove UNITY_OBOE symbol from Project Settings → Player
Demo
As usual with Maestro MPTK, all sources are available with the package on the Unity store.
- The Oboe Test is a simple demonstration of MIDI and direct note playing. To play notes, tap or wipe on the magenta zone. You can adjust the synth settings in real-time to improve latency and sound quality:
- Important note: like any DSP software, and fluidsynth in particular, audio is processed by block of 64 values. It’s important that the audio buffer size is greater than or equal to 64 and a multiple of 64, otherwise you’ll get bad sound. Try different values to get the good buffer size to improve latency and keep a good playing!
- Download APK with Oboe
- Download APK with FMOD
- MIDI Euclidean Rhythms
- This demo is useful for experimenting with the MPTK synthesizer voices, runtime effects, and running multiple players simultaneously. There are two players running simultaneously: one for the drums, which is a MidiStreamPlayer prefab, and one for the midi, which is a MidiFilePlayer prefab.
- Download the APK
Have fun!