2003-12-19 On The Phenomenology of Synaethesia Some people "experience sensations in multiple modalities in response to stimulation", 'see' printed numbers as different colours (5 as green; 2 as red) -- synaethesia. My are my notes on The Phenomenology of Synaethesia, Ramachandran and Hubbard's summary paper of their research. http://www.imprint.co.uk/pdf/R_H-follow-up.pdf -- Phenomenology of Synaesthesia http://www.psychiatry.cam.ac.uk/isa/whatis.html -- Good summary of synaesthesia http://dalston.ku24.com/cluster/archives/000280.html -- link to paper (when I think of it, it's a bit like code highlighting in BBEdit.) # There are some fantastic findings, like whether the font of the number changes the colour sensation: a font like Gothic may sometimes evoke a stronger colour: "We suggest that such fonts might serve as 'hyper-normal' stimuli that evoke even larger responses from the grapheme neurons than a more 'prototypical' font might." (Which, from an interface perspective, is wonderful: that the brain has a shape that binds around sensations, and this shape itself can be gamed in other ways.) # Looking for correspondances across synaesthesiacs: "For instance, it may not initially be obvious why the arbitrary vowel sequence AEIOU should map in a non-arbitrary manner to a certain sequence of colours (and we have seen some hints of this happening). But the point is that the sequence AEIOU may not be arbitrary. It may reflect progressively anterior mouth and tongue articulations which in turn might be mapped in a topographically organized phoneme space." (At which point I have to mention both the Great Vowel Shift (15th-18th century) and the more recent Northern Cities Shift: a vowel-sound moves to disambiguate from another, this causes a knock-on effect; the consequence is that internally to a dialect people can understand each other better, but between dialects (between cities in North America) people have trouble comprehending one another. I believe bread-related products also underwent a similar shift moving across the Atlantic: each of biscuit/cookie/rusk/scone/bun/cake rotates along the axis.) http://alpha.furman.edu/~mmenzer/gvs/what.htm -- Great Vowel Shift http://www.ic.arizona.edu/~lsp/Northeast/ncshift/ncshift.html -- Northern Cities http://interconnected.org/home/mini/archive/2003/03/16/ -- more ling. links # A clever-but-simple experiment showing that synaesthesia does indeed occur on the sensory level (it's not imagination or just a strong memory association): """ For example, if several 2s are scattered among a matrix of randomly placed 5s, the global shape formed by the embedded 2s is very hard to discern; normal subjects take several seconds to find the shape because the 2s and 5s are composed of the same features three horizontal lines and 2 vertical lines. But if a grapheme-colour synaesthete looks at it he instantly ั or very quickly ั sees the global shape as a red triangle or square against a background of green 2s. """ (So if comprehension == navigation, is synaesthesia a good metaphor for gaining expectations of a system? Once you understand how a scientific paper is organised (its semantic shape) then you don't just read the paper, you 'see' its shape, concentrate on divergences, etc. It's an unrelated symbol that's fired in addition to just your reading symbols: a multimodal perception.) # And linking the asymmetry of synaesthesia (graphemes to colour but not vice versa) with the nonarbitrariness of metaphor ("For example, you say 'loud shirt' but you rarely say 'red sound'; you say 'sharp taste' but rarely 'bitter touch'.") -- is this to do with evolution and neural constraints? Correspondingly we share facial expressions from being infants and smells are talked of as tastes; are these too metaphors? "There are also hints that patients with right hemisphere lesions show problems with metaphor. It is possible that their deficits are mainly with spatial metaphors, such as 'He stepped down as director'." (And then later, "It may be because puns are in some ways the opposite of metaphors; a metaphor reveals a deep similarity whereas a pun is a superficial similarity masquerading a deep one, hence its comic appeal.") # This is startling: """ We offer an empirical solution to the question that has long puzzled philosophers: How does the activity of neurons in the sensory (and other areas) in the brain give rise to the subjective quality ั the 'qualia' of sensations like red or green or pain? The neurons in V4 (and other colour centres) are not that different physically from (say) the neurons in auditory cortex concerned with hearing, so why does their activity feel so utterly different? One tool for probing this question is the 'martian colour effect', an unusual tinting of colours evoked synaesthetically, which is most obvious and pronounced in a colour-blind synaesthete we have tested, but occurs in 'regular' synaesthetes as well. We attribute this colour distortion to the fact that the colours evoked by cross-activation in the fusiform 'bypass' earlier stages of colour processing, and therefore may confer an unusual ('martian') tint to the colours evoked. This is an important finding, for it suggests that the qualia label ัthe subjective experience of the colour sensation ั depends not merely on the final stages of processing but on the total pattern of neural activity, including the earlier stages. """ (which only goes to show that models aren't the system, and that a model will always have (a) parts of the system they can't imitate, and (b) the model will be able to imitate [things? what? qualia?] that the system can't. Synaesthesiacs can perceive colours they can't sense!) # Imagined grapheme-colours are brighter: """ The reverse side of the coin from being 'blind' to a character is to imagine a number or letter that is not physically presented. Many synaesthetes reported that when they visualised a number in front of them it was, surprisingly, more strongly coloured than when they looked at a real number. We suggest this is because when you imagine something visually there is activation of the same brain areas ั such as the grapheme area in the fusiform ั that are driven by actual physical colours, but without the real black (or white) number coming in from the retina and competing with the colour experience, this imagined number actually evokes a stronger experience of the corresponding colour. """ # So synaesthesia is hardwired metaphor, and who knows why it's there, but means we must have pre-determined strong associations between certain symbols already. What do we always use space for, what is it a strong-metaphor for? What does happen if that lacks? Can anything like this happen in reverse (two modalities combining, what's the effect of breaking an expectation?)? # This is an interesting form of exaptative feature: we develop a particular way to relate to space, say, a series of expectances and the-way-the-world-works, then we notice that another system works in a same way and apply the same model to it, and that becomes hardwired in the brain. So we can treat the world of ideas like space, and abstraction like vision (seeing only surface features).