a synesthetic exploration
of color theory and
acoustic frequency relationships

the colour of sound explores the phenomenon of synesthesia, particularly the chromesthesia variant where sounds automatically evoke experiences of color. while true synesthesia is a neurological condition, this project creates an algorithmic approximation of this experience that allows anyone to explore potential mappings between sound and color.

drawing inspiration from both scientific research on cross-modal perception and the experiences of synesthetes, the project maps spectral qualities of sound (timbre, frequency, amplitude) to visual qualities (hue, saturation, brightness) using several different mapping strategies derived from psychological research.

the core of the colour of sound utilizes real-time audio analysis and spectral processing to extract meaningful features from sound, which are then mapped to visual parameters. the system uses machine learning to develop more sophisticated mappings based on psychological studies of cross-modal associations.

    technologies = {
        sound processing: 'web audio API';
        visualization: 'three.js, GLSL shaders';
        ML: 'tensorflow.js';
        analysis: 'FFT analysis';
    }

The project employs multiple mapping strategies:

• frequency-to-hue mapping based on newtonian color wheel
• amplitude-to-luminance correlation
• spectral centroid to color saturation
• harmonic content to color complexity
• customizable mappings based on user preferences