Prefab MidiSpatializer

Sound Spatialization is a key component of the player’s immersion. Our ears and brains are highly aware of microscopic delays between the sound received from a source at the left and right ears respectively. Furthermore we are capable of unconsciously interpreting a change in the balance of high frequencies to tell if an object is in front of, behind or even above or below us: Sound is extremely important in our daily navigation, we just maybe don’t notice it that much! (extracted from Unity Doc, see here for all the text).

MidiSpatializer is a Prefab available with the Pro version which is able from one MIDI file to apply 3D sound position by each channel, instrument, or track.

• This Prefab is available with MPTK Pro.
• Scripting is necessary to use this Prefab to defined your rules for calculating positions.
• Be ready for your AR/VR application!
• See an example with source code at the end of this page and the demonstration available in the package. Below a video of the demo.

Setup Spatialization with Unity

Read absolutely:

Unity plugins must be deployed before playing with spatialisation. Here, how-to and the plugins tested with Maestro MPTK.

Inspector parameters

This prefab is based on the MidiFilePlayer Prefab, so the inspector is quite the same. See here the MidiFilePlayer prefab. The main differences is with the foldout “Spatialization”.

• Spatialization is forced to true.
  • AudioSource setting is modified for playing spatialization algo if a spacializer plugins is defined.
  • A Maestro specific function is able to pause the midi reader if the distance between the gameobject (which hold this prefab) and the AudioListener is greater than Max Distance.
• Channel Spatialization : each Midi channels are assigned to an instantiated MPTK Midi Synth.
• Track Spatialization : each Midi tracks are assigned to an instantiated MPTK Midi Synth.
• Max Spatial Synth :
  • Between 16 and 50.
  • Generally set to 16 for the Channel Spatialization. Try to define the lower value possible as each Midi Synth have an impact on DSP performance.
  • This setting can’t be changed when running because MPTK Midi Synths are instantiated at start.
• Max Distance: distance for pausing the midi reader.

Tips :

• You need to defined the position on the 3D space of each Midi Synths because Maestro  is unable to know this information. See demo.
• Unfortunately, DearVR don’t works with this prefab but its perfect with the MidiFilePlayer prefab. Any reason found, we hope to correct this issue ASAP.

Integration of MidiSpatializer in your script

See TestSpatializerFly.cs and events associated in the canvas gameobjects of MidiSpatializer scene for the whole example.

using MidiPlayerTK;
...
// Arrange each players depending of the current instrument 
// associated to the channel
public void ArrangeByInstrument()
{
  isPositionByInstrument = true;
  // Exec from the UI, applied to each MidiFilePlayer (MidiSynth)
  // 16 synths (one by channel) has been created at the start 
  // of the main synth (where the Midi is read) and stored
  // in SpatialSynths
  // Below, we calculate a position for each. 
  foreach (MidiFilePlayer mfp in MidiFilePlayer.SpatialSynths)
  {
    // Get the synth associate to this channel
    TestSpatializerFly tsf = mfp.gameObject.GetComponent<TestSpatializerFly>();
    int preset = mfp.MPTK_ChannelPresetGetIndex(mfp.MPTK_DedicatedChannel);
    // Appply a 3D position depending the current preset
    tsf.PosSynth = PositionByInstrument(tsf.PosSynth, preset, mfp.MPTK_DedicatedChannel);
  }
}

/// <summary>
/// Calculate the position based on the GM Instrument Families, see here: http://www.music.mcgill.ca/~ich/classes/mumt306/StandardMIDIfileformat.html
/// Try to deploy instrument in inverse V
/// 1-8	    Piano	
/// 9-16	Chromatic Percussion	
/// 17-24	Organ	
/// 25-32	Guitar	
/// 33-40	Bass	
/// 41-48	Strings	
/// 49-56	Ensemble	
/// 57-64	Brass	
/// 65-72	Reed
/// 73-80	Pipe
/// 81-88	Synth Lead
/// 89-96	Synth Pad
/// 97-104	Synth Effects
/// 105-112	Ethnic
/// 113-120	Percussive
/// 121-128	Sound Effects            /// 
/// </summary>
/// <param name="preset"></param>
/// <returns></returns>
Vector3 PositionByInstrument(Vector3 posSynth, int preset, int channel)
{
   float range = 950f;
   float x, z;
   if (channel != 9)
   {
     // left to right
     x = Mathf.Lerp(-range, range, (float)preset / 127f);
     // at left:ahead, center:bottom, at right:ahead
     z = preset < 64 ? z = Mathf.Lerp(0, range, (float)preset / 64f) : Mathf.Lerp(range, 0, ((float)preset - 64f) / 64f);
    }
    else
    {
      // Special case for drum: set to center at bottom
      x = 0f;
      z = range;
    }
    return new Vector3(x, posSynth.y, z);
}