MusIT

Oboe: Fight Android Audio Latency

Audio latency is the delay between the time an application sends a buffer of audio data to the APIs and the time it is heard.

The latency can be split into two parts:

  • UI Latency is the time between a user’s tap or click on the screen and the processing of the event on the software side.
    To reduce UI latency, you can use either the new Unity input system or interface like OnPointerDown from UnityEngine.EventSystems. Look for TestTapMidi.cs source in TestOboe scene (Maestro Pro).
  • Hardware Latency origin is mainly related to the time need by the audio component (FMOD for example) to process and play the sound build by the software (like Maestro MPTK). This time is directly dependant of the buffer size (DspBufferSize) and the synth sample rate.

There are also solutions to reduce hardware latency by decrease the size of this buffer: changing parameters in the audio settings in Unity/FMOD or using another audio component (Oboe/AAudio/OpenSL)

First Option: Modify Unity Audio Settings

According to your need, you could have only to change some properties on the Unity setting to resolve your latency issue.

From Unity menu Edit → Project Settings → Audio

With Unity 2019
  • Set DSP Buffer size to Best latency
  • Try with higher Sample Rate
  • Minimize audio effects as much as possible

You can also change these parameters from the MPTK prefab inspectors.

Tips:

  • Audio latency is not a specific issue with Android but also on Windows, IOs, Mac, … Obviously, these setting are useful also for them.
  • Depending on your system, a too low buffer size can generate weird sound with with some material.
  • Also, forget Bluetooth headset because of a high latency (for now) !

Second Option: Audio Library with Oboe and MPTK Pro

The default audio engine for Unity is FMOD. It’s a powerful engine with a lot of features, well tested on many hardware but too general for efficiency. However, use this Unity default audio engine if latency is not crucial for your creation.

Oboe replaces the default Unity audio engine.

How to use Oboe with your MPTK application ?

  • Recommended Unity version: 2021.3 and above.
  • Recommended Maestro version: 2.12 and above (in progress).
  • Create a new Unity project.
  • Oboe is working only for Android, so select build platform to Android in “Build Settings”.
  • Download and import in your project Oboe (version tested:1.6.2) from here.
  • Download and import in your project Maestro Pro from here.
  • Go to Unity menu Edit → Project Settings → Player
    • add symbol UNITY_OBOE
    • allow unsafe code
With Unity 2021.3
Android setup used for testing
  • Open the Midi Player Run assembly definition, allow unsafe code, add reference to Oboe and apply.
Add reference to Oboe assembly
Add reference to Oboe assembly
  • Same for Midi Player Demo assembly definition, allow unsafe code, add reference to Oboe and apply.
Add reference to Oboe assembly
  • Build and deploy your APK and that’s all folks!!! MPTK has automatically switched to the Oboe audio engine.

Some drawbacks (world is not perfect!)

  • Many different devices use Android, it’s not possible to check all smartphone and tablet hardware.
  • MPTK Unity effects and 3D sound are disabled.
  • Oboe is working only on Android.

Demo

APK demos are available for those who want to test on their Android: APK for Euclidean Rhythm demo (not yet available). Demo are build with or without Oboe. The No Oboe with a 128 bits buffer size provides a good latency but could produces weird sound on some hardware.

Of course, I would be very happy to have your return.

This demo is useful to test latency but also the MPTK synthesizer voices run time effects and running different players at the same time. So, there is two prefabs MidiStreamPlayer for the drums and one MidiFilePlayer for the midi running at the same time.

Video from a Windows desktop