MusIT

Maestro MPTK – Sound Spatialization with Unity

Sound spatialization in a nutshell! On this page, you’ll find detailed instructions on setting up spatialization in Unity, along with how Maestro MPTK can assist you.

What Spatialize sound is?

Audio Spatializers use “physical” characteristics of a scene, such as the distance and angle between an AudioListener and an AudioSource, to modify the properties of sound transmitted to the user. Spatialization can improve the perception that a sound originates from a specific location in a scene.

Why Spatialize sound?

Sound Spatialization is a key component of the player’s immersion. Our ears and brains are highly aware of microscopic delays between the sound received from a source at the left and right ears respectively. Furthermore we are capable of unconsciously interpreting a change in the balance of high frequencies to tell if an object is in front of, behind or even above or below us: Sound is extremely important in our daily navigation, we just maybe don’t notice it that much! (extracted from Unity Doc, see here for all the text).

Setting Up Spatialization

  • Setup a Spatializer is straightforward!
  • First of all, choose and install an audio Spatializer: Unity – Manual: Audio Spatializers (unity3d.com). Maestro has been tested with these plugins:
  • Define the spatialization mode in the Maestro Prefab (MidiFilePlayer, MidiStreamPlayer, …). This mode is easy to deploy because is not requiring coding, just some settings to adjust in the Maestro prefabs inspector.
    • You can add multiple Maestro prefabs (MidiFilePlayer, MidiStreamPlayer, etc.) to the scene.
    • Each prefab hosts its own MIDI synth, meaning each audio source has a unique 3D position.
  • All these usages are available with Maestro Free!
  • But for more advanced usage, you can use the MidiSpatializer prefab.
    • This feature is available with the Pro version and allows you to apply 3D positions by channels, instruments, or tracks from a single MIDI File.
    • Also, this will require scripting.
  • Look here how to build a Multi Sound Spatialization solution with Maestro.

Tips for the Maestro Free :

  • Associate Maestro prefabs with your graphical Game Object to localize the sound in 3D space. When the Game Object is moving, the sound will move with it.
  • See here a quick tuto on the Maestro Discord site.
  • Define the Spatializer in the Unity menu “Edit / Project Settings / Audio”.
  • To fully appreciate sound spatialization, we recommend using headphones!
Define your Spatializer in Unity project settings

Free demo TestMidiSpatialization

  • With just a MidiFilePlayer and spatialization setting enabled, you get 3D music.
  • By script, others MidiFileplayer can be instantiated and are moving in the scene.
  • As usual, all sources are available.
Define simple spatialization with Maestro and Unity

It’s better to uncheck Pan Change to avoid specific settings with the Presets/Instruments SoundFont attributs to affect 3D position.

uncheck pan change for better result

Note: When Spatialization is checked, Unity spatializer is enabled. It’s not necessary to install a spatializer plug-ins to get some (very) simple spatial effects but you will get a warning message from Unity:

Audio source failed to initialize audio spatializer. An audio spatializer is specified in the audio project settings, but the associated plugin was not found or initialized properly. Please make sure that the selected spatializer is compatible with the target.

And, you will still have these effects: volume depending on distance and left/right panning (but limited to panning specified in the SoundFont). I encourage you to install a spatializer, they are generally free!

Script

In fact, no code is needed for Spatialization with Maestro. This script is just useful to animate the Game Object in the 3D Space.

C#
using UnityEngine;
using MidiPlayerTK;

namespace DemoMPTK
{

    /// <summary>
    /// Add this script to the Game Object you want add Music.
    /// A sphere in the demo. Can be of course any kind of game object
    /// </summary>
    public class TestMidiSpatialization : MonoBehaviour
    {
        // Material of the gameobject, useful for random color.
        public Material material;
        // Start position
        public Vector3 StartPosition;
        public float Radius;
        public float smoothTime = 0.3F;
        public Vector3 RotateAmount;  // degrees per second to rotate in each axis. Set in inspector.

        private Vector3 velocity = Vector3.zero;
        private Vector3 Target;

        private void Start()
        {
            Random.InitState(System.DateTime.Now.Millisecond);
            SetTarget();
        }

        /// <summary>
        /// From the UI, add a gameobject which holds a MidiFilePlayer
        /// </summary>
        public void AddSphere()
        {
            // Create a gameobject from this gameobject (sphere + MidiFilePlayer)
            GameObject goCreated = Instantiate(gameObject);

            // Random color of the gameobject
            Renderer renderer = goCreated.GetComponent<Renderer>();
            renderer.material = material;
            renderer.material.color = new Color(Random.value, Random.value, Random.value);

            // Get the MidiFilePlayer component attached to this GameObject
            MidiFilePlayer mfp = goCreated.GetComponentInChildren<MidiFilePlayer>();

            // Random selection of the MIDI ... and play!
            mfp.MPTK_MidiIndex = Random.Range(0, MidiPlayerGlobal.MPTK_ListMidi.Count);
            mfp.MPTK_Play();

        }

        private void Update()
        {
            if ((transform.position - Target).sqrMagnitude < 0.01f)
                // Defined a new random position when gameobject reach the target (distance < 0.01f)
                SetTarget();

            // At each update, move the Game Object to the target ... smoothly
            transform.position = Vector3.SmoothDamp(transform.position, Target, ref velocity, smoothTime);

            // And rotate, why not?
            transform.Rotate(RotateAmount * Time.unscaledDeltaTime);
        }

        /// <summary>
        /// Defined a new random position
        /// </summary>
        private void SetTarget()
        {
            Target = Random.insideUnitSphere * Radius + StartPosition;
        }
    }
}

Get MPTK from the Unity store

If you like Midi Player Tool Kit, please leave a review on the Asset Store. It’s very appreciated!!!

Maestro MPTK on ChatGPT!

From various MPTK documentation sources, @DarkSky42 has created a custom LLM based on ChatGPT. You can now ask all the questions you want and get a good level of answer: request code example, verify your source code, explain the MPTK demo …

Contact

If you have questions, please don’t hesitate to contact us via the dedicated Unity forum or our Discord  channel.

Reach the Discord archive by topic.

We are always happy to discuss your projects!

Add MIDI Music With 3 Clicks for Free

Sound Spatialisation, MPTK is ready for Virtual Reality [Pro]

Sound Spatialisation, MPTK is ready for Virtual Reality [free]

Midi Synth : Real Time Voice Effect Change

Euclidean Rhythm demo

The Deezer playlist that helped me create Maestro