At least once a year I attempt to write more complex audio programs but invariably give up because
I always thought it'd be great if I could just boot directly into an "audio REPL" in Scheme/K/whatever and simply manipulate sound bytes directly without constructing and interpreting a shoddy DSL first. I have something approaching that in the following proof of concept. It almost addresses my complaints.
This wound up being a fun exercise because it involves some uncommon Scheme topics.
Code.
(import (chicken foreign) (chicken random) (chicken repl) (srfi 4) (srfi 18))
#>
#include <ao/ao.h>
#include <string.h>
ao_device* open_audio(int rate) {
int driver;
ao_sample_format fmt;
ao_initialize();
driver = ao_default_driver_id();
memset(&fmt, 0, sizeof(fmt));
fmt.bits = 16;
fmt.channels = 2;
fmt.byte_format = AO_FMT_LITTLE;
fmt.rate = rate;
return ao_open_live(driver, &fmt, NULL);
}
void close_audio(ao_device *d) {
ao_close(d);
ao_shutdown();
}
<#
(define-foreign-type ao (c-pointer (struct "ao_device")))
(define open-audio (foreign-lambda ao "open_audio" int))
(define write-audio (foreign-lambda void "ao_play" ao nonnull-u8vector int))
(define close-audio (foreign-lambda void "close_audio" ao))
(define (make-audio-writer sample-rate)
(let ((ao (open-audio sample-rate)))
(lambda (buffer) (write-audio ao buffer (u8vector-length buffer)))))
(define (audio-output sample-rate buffer-size-samples audio-writer)
(letrec*
((start (time->seconds (current-time)))
(sleep (/ 1 (/ sample-rate buffer-size-samples)))
(bytes (/ 16 2))
(chan 2)
(buffer-size (* bytes chan buffer-size-samples))
(buffer (make-u8vector buffer-size 0 #f #f))
(loop (lambda (n)
((thread-specific (current-thread)) buffer)
(audio-writer buffer)
(thread-sleep! (seconds->time (+ start (* n sleep))))
(loop (+ n 1)))))
(thread-specific-set! (current-thread) (constantly (void)))
; write n blocks of lead buffer?
(loop 0)
(release-number-vector buffer)))
(define SAMPLE-RATE 48000)
(define BUFFER-SIZE 960)
(define audio-writer (make-audio-writer SAMPLE-RATE))
(define audio-thread
(make-thread (lambda ()
(audio-output SAMPLE-RATE BUFFER-SIZE audio-writer))))
(define repl-thread (make-thread repl))
(define (set-f! f) (thread-specific-set! audio-thread f))
(define noise (compose random-bytes u8vector->blob/shared))
(thread-start! audio-thread)
(thread-start! repl-thread)
(thread-join! repl-thread)
My compiler couldn't find libao by default so I ran this for Mac OS
brew install libao
csc -I/opt/homebrew/include -L/opt/homebrew/lib -L -lao -o noise noise.scm
It might be different for you (computer bullshit).
Once you run it
rlwrap ./noise
you are dropped into a Scheme REPL accompanied by complete silence, but this is more powerful than first meets the ear.
There are two loops taking place in parallel.
Even during silence, the second loop is always writing a buffer full of bytes to the sound card. The buffer is zeroed out by default, hence the lack of noise, but the user can specify arbitrary Scheme code to populate this buffer with audio data. It's useful to explore the low level setup first though.
Libao is refreshingly simple.
ao_initialize().ao_default_driver_id().ao_sample_format struct.ao_device pointer with ao_open_live(driver_id, format, options).ao_play(ao_device *, bytes, size).ao_close(ao_device *).ao_shutdown().I've simplified this process into three foreign functions.
(open-audio): Steps 1-4 with default 16 bit stereo output.(write-audio): Step 5. Run as many times as needed.(close-audio): Steps 6-7. Not actually used here.There are multiple ways to leverage Chicken's C FFI. I also referred to this writeup for advice. Since I only need the slightest bit of outside code, I found it most tolerable to declare all my C functions (including includes) in a #> … <# block, then Schemeify them with (foreign-lambda). I think this is rather straightforward.
Just as a C function is
return-type function_name(arg-type, arg-type)
a foreign lambda is
(foreign-lambda return-type "c_function_name" arg-type arg-type)
where the "c_function_name" can refer to a function declared in the above block, or simply available from one of the #includes. You'll notice that ao_play isn't declared anywhere in the code because it's used unmodified directly from libao.
Declaring a foreign type can simplify these signatures.
(define-foreign-type ao (c-pointer (struct "ao_device")))
(define open-audio (foreign-lambda ao "open_audio" int))
(define write-audio (foreign-lambda void "ao_play" ao nonnull-u8vector int))
(define close-audio (foreign-lambda void "close_audio" ao))
These map well to their C equivalents for the most part.
ao_play actually expects a char * for its byte buffer, so nonnull-u8vector will throw a warning. Using a c-string instead will satisfy the compiler, but result in a runtime error. I'm fine with the warning for now.
It seems like there's no native straightforward way to manipulate C structs in Scheme. One of the articles I've linked discusses some alternative eggs, but it's enough to just pass around the ao_device struct as a pointer on the Scheme side of things.
For an audio signal with n byte samples, c channels, and a sample rate of r, n×c×r bytes must be written to the soundcard every second. Channels are L/R interleaved, so for a 16 bit stereo signal, you'd have
(left-byte-1 left-byte-2 right-byte-1 right-byte-2 left-byte-1 …)
This signal is most effectively modelled by a statefully-manipulated u8vector whose length is a perfect fraction of n×c×r. Smaller buffer sizes allow more user interactions per second. In production code I would work with slices of this slice for even greater responsiveness. This example uses a sample rate of 48,000 and writes audio every 960 samples, meaning it can represent 50 parameter changes per second: good for a REPL, but maybe not ideal for piano keys.
Audio is written with the foreign function ao_play. It uses a foreign pointer to an ao_device. This can all be encapsulated into a closure so that the rest of the code only needs to think in terms of Scheme.
(define (make-audio-writer sample-rate)
(let ((ao (open-audio sample-rate)))
(lambda (buffer) (write-audio ao buffer (u8vector-length buffer)))))
This wraps ao_play as an audio-writer, which is a u8vector → void function that takes a buffer full of fresh bytes and writes it. It will run in the core audio output loop. The rest of the program must keep the buffer full to avoid stuttering.
Most loops will go as fast as the CPU permits. This is a problem with realtime audio, where the soundcard pipeline will back up with eager data that doesn't reflect temporal reality. The jam will cause actual interactions to occur seconds later than they should. This raises the interesting problem of waiting. OS specific sound libraries like OpenBSD's sndio know when to stop, but libao doesn't appear to have any sync mechanism.
If there are 50 writes per second, it may seem like enough to sleep for 1÷50 seconds after every write, but this will cause a compounding drift, since the actions taken while awake also cost fractional seconds. Luckily Chicken allows sleeping/waking in terms of time rather than raw seconds. This is relatively straightforward.
As seen here.
(thread-sleep! (seconds->time (+ 1 (time->seconds (current-time)))))
This will sleep for exactly one second without drift. Say it took 0.05 seconds to process the sleep instructions in the first place. The thread will only sleep for 0.95 seconds to account for this drift, since it wakes up at a precise timestamp rather than after an elapsed duration.
Chicken uses green threads, which provide the illusion of parallel activity while running on the same core. This allows the REPL to wait for user input while audio constantly broadcasts in the background. In C—where only heavyweight pthreads are available—I keep this as one big loop with polling IO.
Every thread has an arbitrary (thread-specific) value, which can be set by the thread itself, or the parent thread. If the child audio thread's (thread-specific) is the buffer-filling u8vector → void function, then the REPL user can change this to any other Scheme function during runtime and hear its effects instantly. This eliminates all the redundant DSL parsing and makes the program infinitely extensible. Even extremely complicated DSP can be written ahead of time and (load)ed from disk.
You can see it applied during every audio write cycle with
((thread-specific (current-thread)) buffer)
The interface of u8vector → void might seem restrictive at first, but recall that arbitrary configuration data can be curried in ahead of time. An exercise for the reader? I don't know if replacing closures across threads is an atomic operation. Maybe mutexes will enter the picture eventually. It may not make an audible difference. I also don't know if it's practical to replace a closure for every parameter change (think a filter sweep); luckily Chicken has pointers.
The default silence of the audio thread is represented by a K combinator that returns nothing.
(thread-specific-set! (current-thread) (constantly (void)))
This runs against the audio buffer 50 times per second and ignores it.
Now all the code in (audio-output) should make sense. It takes the audio-writer closure, allocates a buffer, and just writes it endlessly to the soundcard.
In order not to block, both the REPL and the audio must run in children threads of the main one. Waiting on the REPL thread will allow both to run indefinitely.
(define audio-thread
(make-thread (lambda ()
(audio-output SAMPLE-RATE BUFFER-SIZE audio-writer))))
(define repl-thread (make-thread repl))
(define (set-f! f) (thread-specific-set! audio-thread f))
(define noise (compose random-bytes u8vector->blob/shared))
(thread-start! audio-thread)
(thread-start! repl-thread)
(thread-join! repl-thread)
All values declared in the main thread will be available to children. Crucially, the REPL has access to (set-f!), which itself is a closure around the audio thread. The user runs this to redefine the u8vector → void audio output function.
I've included noise, which fills the vector with random bytes without any copying. Turn your system volume as low as possible, since there is no amplitude parameter exposed. Then run
(set-f! noise)
and you should instantly hear white noise. If you run the default function
(set-f! (constantly (void)))
again, you'll get an interesting stutter effect, since the buffer is now being ignored again, but it is full of random bytes instead of zeroes. You'll need to write your own zeroing function to return to silence.
Still, a promising first step away from C.