Hearabouts / how it works
How Hearabouts synthesizes rain, wind, fire and crickets
Hearabouts is a web app that plays places — rain on a veranda, a pine wood in the wind, an autumn night of crickets — synthesized in real time from small physical models. Not a second of it is recorded. It never repeats, and nothing loops. The first load, engine included, is about 350 KB gzipped (the voxel scenes load later), and it works offline.
Try it: https://hearabouts.kosame.work/
This note goes through how the sound is built, one source at a time: the idea behind each model, and what had to change after comparing it with real recordings. There is no code here, and very little maths.
Each example below is 12 seconds of one sound on its own, rendered by the same engine. They are brought to the same loudness so you can hear them on laptop speakers; the real difference in level is given under each one.
Why synthesize?
Most ambience is a few minutes of a field recording, joined end to end. Leave it on for an hour and you start to recognise the join, or the same bird singing the same phrase. A recording also keeps whatever happened to be there: a road in the distance, a plane.
Synthesis has no end and never comes back to the same moment. A place holds only the sounds it is meant to have. Rain and wind can be changed in physical units, millimetres per hour and metres per second. And the whole first download is smaller than one minute of MP3.
The price is that whether it sounds real depends entirely on listening and measuring.
Rain: the rain rate sets the drops
You choose a rain rate in mm/h, and the drops follow from it: how many there are of each size (the Marshall–Palmer distribution) and how fast each size falls.
Distant rain is a crowd of impacts, from about 120 to 30,000 a second, landing 2 to 35 metres around you. There are eight surfaces — leaves, water, a tin roof, a window, a tent, an umbrella, stone and wood — and each rings in its own way.
Big drops landing right next to you are made separately. A tin roof, a window pane, a tent's fly sheet and an umbrella have so many closely spaced resonances that each one is modelled as a small plate reverb, struck by every drop. A tin roof literally is a plate reverb. From underneath, you hear the sheet's own vibration rather than the splash on top.
Real recordings changed this more than once. In a recording of rain on a porch, about 13 impacts a second stand 6 dB above their surroundings; the first version had 50, and the rain sounded too grainy. Distant rain on leaves and stone is now a finer grain of smaller impacts, with the same total energy.
Wind: you hear what it hits
Moving air makes almost no sound. What you hear is what the wind hits, so the wind here is not filtered noise but a field of wind speed driving many things that sound. Pine needles, wires, twigs and rock edges sing at a pitch set by their thickness and the speed of the wind (Aeolian tones). Leaves flip over in clusters; grass washes against the ground.
Gusts come from three random drifts with different time scales (Ornstein–Uhlenbeck processes), which quicken as the wind rises. Gusts arrive sharper and rarer than lulls.
Against a recording of a pine wood, the top end fell away too slowly. It now drops at the measured 9 to 10 dB per octave.
Fire: one flame moves everything
A slowly changing flame signal drives the low roar, the lapping of the flames and the hiss of escaping gas together. How fast a flame flickers depends on its size: about 4.7 times a second for a fire 10 cm across, 1.9 for one 60 cm across. Flare-ups, settling logs and fresh wood come every few minutes.
The biggest fix after listening to recordings was an always-on "frying" sizzle. In a real fire the gaps between pops are quiet, and in the high bands the pops stand 30 to 45 dB above them.
Crickets: one insect at a time
Each insect is modelled on its own. A cricket sings by drawing a toothed file on one wing across a scraper on the other. Every tooth strikes the wing's resonant "harp" once, like a clock's escapement, so each pulse is close to a pure tone: a bell cricket (suzumushi) strikes 62 teeth in 13.4 milliseconds, about 4.6 kHz.
Species differ less in pitch than in rhythm: how pulses group into chirps, and chirps into phrases. The singing rate follows the air temperature (Dolbear's law).
From sounds to a place
Sounds side by side are not yet a place. Where you are matters: outside, under the eaves, in a tent, by a window or inside a room. Sounds from outside reach you through walls and glass and lose their highs. Rain on your own tent or window is on your shelter, so it is not muffled. There are two reverbs, one for the space outside (a forest, a valley, a garden, a temple hall, a cave) and one for the room.
Weather, gusts and animal activity drift on natural time scales, and at night sudden sounds are softened. The clock can run in real time, and a "journey" can carry a place through a storm that builds and clears. There are thirty places and five journeys.
Checking against real recordings
So that nothing is judged by ear alone, real recordings are measured and the synthesis is compared with them: the long-term spectrum in third octaves, how peaky the waveform is, how often and how regularly events start, and the envelope statistics of 30 frequency bands — their moments, correlations and modulation (the "sound texture" statistics of McDermott and Simoncelli).
The recordings came from Wikimedia Commons, in the public domain or under Creative Commons licences, read one by one: 24 of 93 were dropped, among them songs called "Rain", a speech filed under rivers and a wind quintet. A commercial sound-effects library served as a second ruler. None of these recordings is in the app.
Small and steady
All of the sound is computed in plain TypeScript inside the browser's AudioWorklet. To keep an iPhone from dropping out, the audio code allocates no memory while it runs, because a garbage-collection pause is an audible gap. One sound costs about 2% of a CPU (an M1 Mac running Safari's JavaScript engine); a whole place, about 5%.
Run offline on one core, the same engine writes audio 16 to 58 times faster than real time, depending on the place.
Describing a place in words
Type "light rain on a tent, a creek nearby, crickets", and that place plays. It understands ten languages, uses no AI model and sends nothing off your device. Words become physical values: "drizzle" is 0.6 mm/h, "heavy rain" 15, "a downpour" 40; a distant waterfall is 180 metres away; "tent" puts the rain on the fly sheet and you inside it.
The picture follows the sound
The voxel diorama on screen draws the events you are hearing. A drip from the eaves is announced by the audio engine a second ahead, and the picture lets it fall in time.
How it was made
Most of the synthesis code was written by an AI coding assistant under my direction; every sound was chosen by ear. No generative-AI audio model is involved: what you hear comes from the physics calculation.
Listen
Hearabouts: https://hearabouts.kosame.work/
The same engine also renders sound libraries for video and games (10-minute WAV takes and 60-second loops): https://kosamehi.gumroad.com/
— kosame