A-Frame Resonance Audio Component

Rich, Immersive, Audio 👉 Live demo 👈

OS	Browsers	Status
Linux	Firefox, Chrome
macOS	Safari
Windows	IE

With Resonance Audio, bring dynamic spatial sound into your A-Frame VR, AR experiences at scale.

Google Resonance works with audio rooms and audio sources. The acoustics of each source are affected by the room it is in. For that purpose, this package provides three components:

• resonance-audio-room
• resonance-audio-room-bb
• resonance-audio-src

These can be used as primitives (using the a- prefix) or as attributes, except resonance-audio-room-bb: it calculates the bounding box of an entity and considers that as the room. It can thus not be used as a primitive.

The audio source can also be a MediaStream.

Audio rooms and audio sources don't necessarily have to have a parent-child relationship. See resonance-audio-src's room property.

Declarative usage

Basic usage is quite simple and works just like any other A-Frame component. Don't forget to point the src attribute to the correct element or resource.

Example with basic usage with primitives:

<a-scene>
  <a-assets>
    <audio id="track" src="assets/audio/track.mp3"></audio>
  </a-assets>
  <a-resonance-audio-room
    position="0 0 -5"
    width="40"
    height="4"
    depth="4"
    ambisonic-order="3"
    speed-of-sound="343"
    left="brick-bare"
    right="curtain-heavy"
    front="plywood-panel"
    back="glass-thin"
    down="parquet-on-concrete"
    up="acoustic-ceiling-tiles"
    visualize="true">
    <a-resonance-audio-src
      position="-10 0 0"
      src="#track"
      loop="true"
      autoplay="true"
      visualize="true"></a-resonance-audio-src>
    <a-resonance-audio-src
      position="10 0 0"
      src="#track"
      loop="true"
      autoplay="true"
      visualize="true"></a-resonance-audio-src>
  </a-resonance-audio-room>
</a-scene>

Example with entities, models and a bounding box room:

<a-scene>
  <a-assets>
    <a-asset-item id="room-obj" src="assets/models/room/room-model.obj"></a-asset-item>
    <a-asset-item id="room-mtl" src="assets/models/room/room-materials.mtl"></a-asset-item>
    <audio id="track" src="assets/audio/track.mp3"></audio>
  </a-assets>
  <a-entity
    obj-model="obj: #room-obj; mtl: #room-mtl"
    position="0 1 -0.5"
    resonance-audio-room-bb="
      visualize: true;
      left: brick-bare;
      right: transparent;
      front: transparent;
      back: brick-bare;
      down: parquet-on-concrete;
      up: transparent">
    <a-entity
      position="-1.5 -0.5 1.8"
      resonance-audio-src="
        visualize: true;
        src: #track;
        loop: true;
        autoplay: true;"></a-entity>
  </a-entity>
</a-scene>

Dynamic usage

All properties can be changed dynamically. This allows for MediaStreams to be set as audio sources.

Take the following example. A more comprehensive version is available in examples/primitives-using-mediastream/.

<a-scene>
  <a-assets>
    <audio id="track" src="assets/audio/track.mp3"></audio>
  </a-assets>
  <a-resonance-audio-room
    position="0 0 -5"
    width="4"
    height="4"
    depth="4"
    ambisonic-order="3"
    speed-of-sound="343"
    left="brick-bare"
    right="curtain-heavy"
    front="plywood-panel"
    back="glass-thin"
    down="parquet-on-concrete"
    up="acoustic-ceiling-tiles"
    visualize="true">
    <a-resonance-audio-src
      id="audioSource"
      position="0 0 0"
      src="#track"
      loop="true"
      autoplay="true"
      visualize="true"></a-resonance-audio-src>
  </a-resonance-audio-room>
</a-scene>

When this is loaded, track.mp3 plays indefinitely. Running the following javascript changes the source from #track to the passed MediaStream. A practical example of stream is a WebRTC stream.

document.getElementById('audioSource').setAttribute('resonance-audio-src', 'src', stream)

Installation

npm

npm install --save @digaverse/aframe-resonance-audio-component
# Or
yarn add @digaverse/aframe-resonance-audio-component

import '@digaverse/aframe-resonance-audio-component'

from source

1. Clone this repository.
2. Run npm install followed by npm run build.
3. Add build/aframe-resonance-audio-component.js to your html page.
4. Use the component like in one of the above examples.

API

resonance-audio-room

Properties

Property	Description	Default value	Values
width	Width of the audio room (in meters).	0
height	Height of the audio room (in meters).	0
depth	Depth of the audio room (in meters).	0
ambisonicOrder	Ambisonic order of the audio room.	1
speedOfSound	Speed of sound within the audio room (in meters per second).	343
left	Material of the left room wall.	brick-bare	room materials
right	Material of the right room wall.	brick-bare	room materials
front	Material of the front room wall.	brick-bare	room materials
back	Material of the back room wall.	brick-bare	room materials
down	Material of the room floor.	brick-bare	room materials
up	Material of the room ceiling.	brick-bare	room materials
visualize	Show a wireframe box visualization of the audio room. Access using el.getObject3D('audio-room')	false	true

Members

Accessible via entity.components['resonance-audio-room'].<member>.

Name	Description	Type
audioContext	The audio context used by Google Resonance.	AudioContext
resonanceAudioScene	The reference to the Google Resonance instance managing the audio in this room.	ResonanceAudio
sources	A collection of the attached audio source components.	array of resonance-audio-src component instances
el.audioSources	Returns sources.	array of resonance-audio-src component instances
el.sounds	A collection of the origins of the attached audio sources.	array of HTMLMediaElement and/or MediaStream objects

resonance-audio-room-bb

This component's only purpose is to instantiate a resonance-audio-room component with the dimensions of the bounded object. For any interaction, use the resonance-audio-room component after it has been fully loaded.

Events

Event	evt.detail property	Description
bounded-audioroom-loaded	room	The audio room element.

resonance-audio-src

Properties

Property	Description	Default value	Values
src	The source of the audio. This can be a HTMLMediaElement (<audio /> or <video />), MediaStream, an ID string pointing to a HTMLMediaElement or a resouce string. If a resource string is passed, the resource is loaded in the <audio /> element accessible via the defaultAudioEl member. Note that you can not set the same HTMLMediaElement on multiple entities with the resonance-audio-src component (globally!).
room	The audio room of this audio source. Use this if the room and source aren't parent and child, respectively. This can be a string selector (used with document.querySelector or a HTMLElement).	el.parentNode
loop	Whether to loop the element source. Overwrites the value set by the input element.	true
autoplay	Whether to autoplay the element source. Overwrites the value set by the input element.	true
gain	Source's gain (linear).	1
maxDistance	Source's maximum distance (in meters).	1000
minDistance	Source's minimum distance (in meters).	1
directivityPattern	Source's directivity pattern where the first number is the alpha and the second is the sharpness. An alpha of 0 is an omnidirectional pattern, 1 is a bidirectional pattern, 0.5 is a cardiod pattern. The sharpness of the pattern is increased exponentially.	0 1 or in THREE.Vector3 format: {x:0, y:1}
sourceWidth	The source width (in degrees), where 0 degrees is a point source and 360 degrees is an omnidirectional source.	0
rolloff	Source's rolloff model.	logarithmic	oneOf(logarithmic, linear, none)
position	The position in local coordinates. If set, this position will be used instead of the entity's position component. Revert to the entity's position by setting this property to any invalid position (such as null).	{ }
rotation	The rotation in local degrees. If set, this rotation will be used instead of the entity's rotation component. Revert to the entity's rotation by setting this property to any invalid rotation (such as null).	{ }
visualize	Show a wireframe sphere visualization of the audio source. Its radius equals the minDistance. If this audio source is not in an audio room, the sphere turns red. Access using el.getObject3D('audio-src').	false	true

Members

Accessible via entity.components['resonance-audio-src'].<member>.

Name	Description	Type
room	The audio room component.	AComponent
connected.element	Whether the audio of this source comes from an HTMLMediaElement.	boolean
connected.stream	Whether the audio of this source comes from a MediaStream.	boolean
sound	The current connected HTMLMediaElement or MediaStream instance.	HTMLMediaElement or MediaStream
resonance	Reference to the Google Resonance audio source	Source
defaultAudioEl	The audio element used when a resource string is set as the source.	HTMLAudioElement
mediaAudioSourceNodes	A collection of references to sources (so they can be reused).	mapping of HTMLMediaElement to MediaElementAudioSourceNode and MediaStream to MediaStreamAudioSourceNode

Events

Event	evt.detail property	Description
audioroom-entered	room	The audio room element that was entered.
	src	The audio source element that entered the room.
audioroom-left	room	The audio room element that was left.
	src	The audio source element that left the room.

Notes

Support for custom positioning and rotation for the audio room is omitted due to the necessity to propagate positioning and rotation calculations to its audio source children and the complexities involved with that.

The visuals are now a simple box wireframe for the audio room and a simple sphere wireframe for the audio source. The box is how Google Resonance actually considers the room and all involved calculations, so other or more complex shapes are not possible (yet). Future work for the audio source visualization might take into account its parameters, such as the directivity pattern and source width.

The audio seems to continue to propagate to infinity outside of an audio room when the default directivity pattern is set.

The bounding box feature isn't perfect yet. It currently takes the dimensions of the entity's bounding box and assumes the center point of the entity is the same as the center point of the audio room. This is correct for simple shapes, but might not be correct for more complex models.

Audio rooms (entities with component resonance-audio-room) and audio sources (entities with component resonance-audio-src) have a one to many relationship, and only that relationship. Rooms do not have any influence on eachother. The same goes for audio sources and rooms that they are not descendants of, even if they are physically positioned within another audio room. Do not nest rooms. Furthermore, source entities don't have to be immediate room childs: in that case use the room property to point to the audio room.

Dynamically changing positioning and rotation of audio source or room container elements is not fully supported.

Future work

• Hook the HTMLMediaElement.srcObject interface so no changes to original code are necessary (except for adding the components).
• Take scaling into account.
• Find a better method of calculating the bounding box in resonance-audio-room-bb.

---

Credits:

_{Original author Etienne Pinchon.}
^{aframe-resonance-audio-component was forked from etiennepinchon/aframe-resonance}

_{Google Resonance Audio project.}
^{aframe-resonance-audio-component is based on Google Resonance Audio project}