Take a look at the two shapes below. One of these shapes is named ‘Maluma’ and one is named ‘Takete’. Can you guess which is which?
If you said that the sharp, angular shape was ‘Takete’ and the smooth and round shape was ‘Maluma’, then you just experienced one of the crossmodal correspondences first found by German psychologist Wolfgang Köhler in 1929.
Crossmodal correspondences are associations that people make across the different senses. In the example above, most people will associate the sharp image (vision) with the sound ‘Takete’ (hearing). Researchers in the field of crossmodal correspondences are always on the hunt to try to find new pairings and better understand this phenomenon.
Some common associations people make include pairing light-coloured and smaller objects with higher-pitched sounds, and darker-coloured and larger objects with deeper sounds. Additionally, the colours ‘rose’ and ‘white’ are often paired with ‘sweet’ tastes. Perfume companies consider crossmodal correspondences when designing the packaging for different scents, as it helps give customers an idea of what smells to expect before they step into the store (for example, you can probably picture in your mind what a pink perfume would smell like over a blue perfume).
This is not to be confused with synesthesia, a phenomenon where there are atypical associations being made across the different senses. This only occurs in 4.4 per cent of individuals, including artists such as Kanye West, Billie Eilish, and Vincent Van Gogh. The key difference is that those with synesthesia (also referred to as synesthetes) typically have unique pairings which only they experience, whereas crossmodal correspondences are consistent across large groups of individuals. Synesthesia can present in many different ways, including tasting words or seeing music (you can learn more about it in this video).
One of the most commonly researched areas of crossmodal correspondence is related to audio-visual pairings. While qualities of sound like pitch and volume have established pairings, there hasn’t been quite as much research on timbre. Timbre is often described as the quality of the sound, or the ‘tone colour’, and is one of the characteristics which help us separate a note played on the tuba from a note played on a saxophone or flute. Our lab asked participants to rate how similar different shapes were to different sounds. These sounds included harpsichords, trumpets, and organs with different effects applied, including distortion and reverb.
We found that people associated distorted instruments with jagged shapes and softer instruments with smooth shapes. Additionally, sounds which had reverb applied were matched with hollow shapes more often (which makes sense, as reverb makes it sound like you are in a big empty room).
While these crossmodal correspondences are fun to discover, they also have interesting potential applications: researchers and designers are working to provide ‘sensory substitutions’ for those who may be blind or deaf. For example, a camera attached to a pair of glasses could capture the image in front of an individual without vision and relay the image’s qualities through sounds that have been determined to be most commonly associated with the different shapes, patterns, and colours. Some sensory substitution devices do exist, but more work discovering crossmodal correspondences and representing them effectively will help them become more practical and sophisticated.
