Sound With No Audio Files

The world at the top of this site has a Sound button. Turn it on and you hear wind, the waterfalls, the campfire, a wind chime somewhere far off now and then, and a note wherever you click. The mill, the whales, the koi and most of the other things that react to a click have sounds of their own. Under it all, a slow song plays, and it grows as you find things in the world.

None of it is a recording. Every sound is built from oscillators and noise with the Web Audio API while the page runs. The code for all of it is about 7 KB gzipped, and the page only downloads it when you turn sound on.

Off until you ask

Browsers don't let a page play sound until the visitor has done something on it, and an AudioContext made outside a click starts out suspended. So the context is made inside the click on the Sound button, before anything else happens. Then the sound code loads, and the context is handed to it.

soundBtn.addEventListener(
  "click",
  () => {
    // ...
    const job = ++soundJob;
    // ...
    const ac = new AudioContext();
    import("./sound").then(({ makeSound }) => {
      if (job !== soundJob || signal.aborted) return ac.close();
      sound = makeSound(ac, P.night, occasion);
      sound.song(songLevel());
      sound.active(visible && !document.hidden);
    });
  },
  { signal },
);

The job counter covers someone clicking the button twice quickly. If sound was turned off again before the code arrived, the context is closed and nothing plays.

Every click in tune

A click rings a bell. Its pitch depends on where the click lands: low on the left, high on the right, across two octaves. The notes come from a pentatonic scale, D major at dusk and A minor at night:

const KEYS = {
  dusk: {
    scale: [62, 64, 66, 69, 71],
    chords: [
      [0, 2, 3],
      [-1, 0, 2],
      [1, 3, 4],
      [-2, 0, 1],
    ],
  },
  night: {
    scale: [57, 60, 62, 64, 67],
    chords: [
      [0, 1, 3],
      [1, 3, 4],
      [-1, 1, 2],
      [2, 4, 5],
    ],
  },
  // ...
};
const hz = (m) => 440 * 2 ** ((m - 69) / 12);

The numbers are MIDI notes, and hz turns them into frequencies. A pentatonic scale has no two notes a half step apart, so any notes from it sound fine together. However fast someone clicks, it comes out as a tune and not a mess. Switching the theme plays a soft chord in the new key, and every click after that is in the new key too.

The lanterns in the tree use the same scale. Drag across them and each one rings the next note up from left to right, so a swipe plays a run, like wind chimes. In Halloween week both keys switch to a darker Japanese scale on the same roots, and everything, clicks and lanterns and song, turns a little spooky.

The bell is a sine wave plus three quieter overtones that die away faster than it does, which is roughly what a struck metal bar does. The other sounds are made the same way, from a few simple parts. A whale's call is a triangle wave that slides up and back down with a wobble, through a filter that keeps it low. Knocking on the arch is two short sine tones that drop in pitch, each with a tap of noise on top.

A hall made of noise

Synthesized sounds on their own sound small and close, as if they are right next to your ear. Reverb gives them a room. Most reverb uses a recording of a real space, called an impulse response. Here the impulse response is 3.4 seconds of random noise, smoothed to take the edge off and faded out over its length. Run through a ConvolverNode, that sounds like a big, soft hall, which suits a world in the sky.

Each sound decides how much of itself goes into the hall. A whale's call is mostly hall, so it seems to come from far off. A knock on the arch is mostly dry, so it sounds close. Everything then goes through a gentle compressor, so when several things happen at once, the output can't clip.

Sound from a worker

The world is drawn in a web worker, and web workers can't play audio. So when something makes a sound, the renderer adds its name and where it is across the world to the message it already sends the page each frame. The page plays it, panned left or right to match where it happened on screen.

Room for the waterfalls

Under everything is the wind: noise through two band-pass filters at about 420 and 640 Hz, drifting slowly from side to side. The gusts come from the world itself. When one crosses the island, bending the grass and pushing the swing as it goes, the renderer sends it like any other sound, with how strong it is and how long it takes to cross. The page plays a rush of air that swells, rises in pitch and travels from the side the gust comes from to the other, so what you hear moves with what you see. Waterfalls are noise too, and the obvious way to add them, more noise, only thickens the wind into a roar. So they are kept apart from it in three ways.

• In pitch. A waterfall's hiss is filtered to above 2.2 kHz, well over the wind. On top of it are drops: tiny sine blips that slide up in pitch as they fade, four to six a second, at random moments. The drops are what make it sound like water and not static.
• In place. Each waterfall is panned to where it is on screen, with very little reverb, so it sounds close and in one spot, while the wind is wide and far off.
• In level. Together they sit about 4.5 LU under the wind. You hear them when you listen for them.

The upfall, the waterfall at the far mill that runs up into the sky, is quieter again, and its drops slide down instead of up.

The same idea gives the rest of the world a voice. The campfire ticks and now and then pops. The tree's leaves stir as each gust reaches it. At dusk a bird calls from one side and another answers from the other, and at night two crickets chirp, a little out of step. Each one sits in its own part of the spectrum, and none of them is loud, so together they make something you could leave on in the background.

The renderer tells the page where the waterfalls, the campfire and the tree are whenever the layout changes, so each sound pans to its own place. A timer every 250 ms works out what falls in the next quarter of a second: how many drops and crackles, each at a random moment so they never settle into a beat, and whether a bird or a chirp is due.

A song that grows

The field guide lists the things to find in the world, and the song follows it. The first find brings in a low drone and a little tune rung on bells. A quarter of the guide adds a soft chord, half adds plucked notes, and three quarters adds two whales calling to each other. A finished guide adds the far mountains, echoing the tune back from one side and then the other.

The tune uses the same scale as the clicks, so clicking along to it makes a tune of your own over the top.

The song is four chords, one bar of four seconds each, taken from the chords in the key. The tune is eight bars long and goes round the chords twice. A timer runs every 250 ms and schedules any bar that starts within the next second, at an exact time on the audio clock:

function schedule() {
  if (nextBar < ac.currentTime) nextBar = ac.currentTime + 0.1;
  while (nextBar < ac.currentTime + 1) {
    playBar(nextBar, bar++);
    nextBar += BAR;
  }
}

The timer itself can run late when the page is busy, and that doesn't matter: it only has to run once in the second before a bar is due, and the audio thread plays the bar on time. Each voice fades in over a bar and out into the next, so the bars blend into one sound. A new layer joins at the start of the next bar, never in the middle of one.

Quiet when you look away

When the world scrolls out of view, or you switch tabs, the sound fades out. About a second later the audio context is suspended, so the browser stops running audio for it at all. It resumes when you come back.

Checking levels without listening

Forty-nine sounds and a song, in four keys, are hard to balance by ear. So I rendered every sound offline. An OfflineAudioContext runs the same code as fast as it can and hands back the samples instead of playing them.

For each sound, in both themes, I checked three things:

• the peak level
• that no sample came out as NaN
• that no two neighboring samples were far enough apart to make an audible click

The peak levels made the balancing a matter of numbers. Every sound now peaks between about -14 and -32 dB, so nothing jumps out over the rest.

Peaks say little about a sound that never stops, so the waterfalls, the fire and the rest were balanced by loudness instead, measured in LU over a long render and set against the wind's.

The song was harder to render, because its timer reads ac.currentTime, and an offline context's clock stands still until it renders. The test gives the context a fake clock, steps it forward a second at a time while the timer schedules the bars, and then renders. Even with the whole guide found, the song never peaks above about -16 dB, so a knock on the arch still stands out over it.

The source is in lib/world/sound.js (opens in a new tab) if you want to read it.