Maestro MPTK – Sound Spatialization with Unity

Sound spatialization in a nutshell! On this page, you’ll find detailed instructions on setting up spatialization in Unity, along with how Maestro MPTK can assist you.

What Spatialize sound is?

Audio Spatializers use “physical” characteristics of a scene, such as the distance and angle between an AudioListener and an AudioSource, to modify the properties of sound transmitted to the user. Spatialization can improve the perception that a sound originates from a specific location in a scene.

Why Spatialize sound?

Sound Spatialization is a key component of the player’s immersion. Our ears and brains are highly aware of microscopic delays between the sound received from a source at the left and right ears respectively. Furthermore we are capable of unconsciously interpreting a change in the balance of high frequencies to tell if an object is in front of, behind or even above or below us: Sound is extremely important in our daily navigation, we just maybe don’t notice it that much! (extracted from Unity Doc, see here for all the text).

Setting Up Spatialization

• Setup a Spatializer is straightforward!
• First of all, choose and install an audio Spatializer: Unity – Manual: Audio Spatializers (unity3d.com). Maestro has been tested with these plugins:
  • Steam Audio. Tested on MacOS and Windows. It’s a very interesting plugins with a lot of features available on Windows, MacOS, Linux, Android, iOS and a good documentation. Ice on the cake, easy to install. A must have!
    Check that the Scripting Define Symbols STEAMAUDIO_ENABLED is defined after the Steam Audio installation (see screen shot bellow).
  • Oculus Spatializer Unity Tested only on Windows
• Define the spatialization mode in the Maestro Prefab (MidiFilePlayer, MidiStreamPlayer, …). This mode is easy to deploy because is not requiring coding, just some settings to adjust in the Maestro prefabs inspector.
  • You can add multiple Maestro prefabs (MidiFilePlayer, MidiStreamPlayer, etc.) to the scene.
  • Each prefab hosts its own MIDI synth, meaning each audio source has a unique 3D position.
• All these usages are available with Maestro Free!
• But for more advanced usage, you can use the MidiSpatializer prefab.
  • This feature is available with the Pro version and allows you to apply 3D positions by channels, instruments, or tracks from a single MIDI File.
  • Also, this will require scripting.
• Look here how to build a Multi Sound Spatialization solution with Maestro.

Special note for Steam Audio:

Tips for the Maestro Free :

• Associate Maestro prefabs with your graphical Game Object to localize the sound in 3D space. When the Game Object is moving, the sound will move with it.
• See here a quick tuto on the Maestro Discord site.
• Define the Spatializer in the Unity menu “Edit / Project Settings / Audio”.
• To fully appreciate sound spatialization, we recommend using headphones!

Free demo TestMidiSpatialization

• With just a MidiFilePlayer and spatialization setting enabled, you get 3D music.
• By script, others MidiFileplayer can be instantiated and are moving in the scene.
• As usual, all sources are available.

It’s better to uncheck Pan Change to avoid specific settings with the Presets/Instruments SoundFont attributs to affect 3D position.

Note: When Spatialization is checked, Unity spatializer is enabled. It’s not necessary to install a spatializer plug-ins to get some (very) simple spatial effects but you will get a warning message from Unity:

Audio source failed to initialize audio spatializer. An audio spatializer is specified in the audio project settings, but the associated plugin was not found or initialized properly. Please make sure that the selected spatializer is compatible with the target.

And, you will still have these effects: volume depending on distance and left/right panning (but limited to panning specified in the SoundFont). I encourage you to install a spatializer, they are generally free!

Script

In fact, no code is needed for Spatialization with Maestro. This script is just useful to animate the Game Object in the 3D Space.

using UnityEngine;
using MidiPlayerTK;

namespace DemoMPTK
{

    /// <summary>
    /// Add this script to the Game Object you want add Music.
    /// A sphere in the demo. Can be of course any kind of game object
    /// </summary>
    public class TestMidiSpatialization : MonoBehaviour
    {
        // Material of the gameobject, useful for random color.
        public Material material;
        // Start position
        public Vector3 StartPosition;
        public float Radius;
        public float smoothTime = 0.3F;
        public Vector3 RotateAmount;  // degrees per second to rotate in each axis. Set in inspector.

        private Vector3 velocity = Vector3.zero;
        private Vector3 Target;

        private void Start()
        {
            Random.InitState(System.DateTime.Now.Millisecond);
            SetTarget();
        }

        /// <summary>
        /// From the UI, add a gameobject which holds a MidiFilePlayer
        /// </summary>
        public void AddSphere()
        {
            // Create a gameobject from this gameobject (sphere + MidiFilePlayer)
            GameObject goCreated = Instantiate(gameObject);

            // Random color of the gameobject
            Renderer renderer = goCreated.GetComponent<Renderer>();
            renderer.material = material;
            renderer.material.color = new Color(Random.value, Random.value, Random.value);

            // Get the MidiFilePlayer component attached to this GameObject
            MidiFilePlayer mfp = goCreated.GetComponentInChildren<MidiFilePlayer>();

            // Random selection of the MIDI ... and play!
            mfp.MPTK_MidiIndex = Random.Range(0, MidiPlayerGlobal.MPTK_ListMidi.Count);
            mfp.MPTK_Play();

        }

        private void Update()
        {
            if ((transform.position - Target).sqrMagnitude < 0.01f)
                // Defined a new random position when gameobject reach the target (distance < 0.01f)
                SetTarget();

            // At each update, move the Game Object to the target ... smoothly
            transform.position = Vector3.SmoothDamp(transform.position, Target, ref velocity, smoothTime);

            // And rotate, why not?
            transform.Rotate(RotateAmount * Time.unscaledDeltaTime);
        }

        /// <summary>
        /// Defined a new random position
        /// </summary>
        private void SetTarget()
        {
            Target = Random.insideUnitSphere * Radius + StartPosition;
        }
    }
}