I’m back from the Beyond Belief II conference at the Salk Institute in La Jolla, which packed an extraordinary amount of intellectual stimulation into a few short days. Any conference where you wander into the opening reception, get drawn into a conversation about reductionism and meaning with Stuart Kauffman, Rebecca Goldstein, and Sir Harold Kroto, and end up closing down the bar, is bound to be a good one, and this did not disappoint. (The title notwithstanding, much of the conference had little to do with atheism or religion — the subtitle “Enlightenment 2.0” gave a better flavor.) The talks provided fodder for at least ten to twenty blog posts, of which I’ll probably get around to writing one or two.
One of the talks was by local neuroscientist V.S. Ramachandran, or “Rama” to his friends. (Like any good neuro person, his web page includes a fun collection of optical illusions.) He talked about his experiments with synesthesia, the phenomenon in which people see graphemes (e.g. numbers or letters) as associated with colors. I do that a little bit — five is certainly yellow, seven is red, and eight is blue — but it’s closer to a vague association than a vivid experience. Some people report very strong synesthetic reactions, and for a long time researchers have wondered whether the experience was mostly metaphorical or something stronger.
To test synesthesia, Rama and collaborators designed an experiment where they could measure the vividness of the colors associated with the numbers 2 and 5. They chose those because you can make them look almost identical, although reversed, by choosing a boxy font. Then they made up a picture (on left) of mostly fives, with a few twos scattered within there. Then they asked people to pick out the twos. Most ordinary folks could do it within about twenty seconds or so.
But true synesthetes could do it immediately. That’s because to them, the twos popped out as a brightly colored triangle (right). This established beyond much doubt that synesthesia was “real,” and more particularly that was a measurable phenomenon with real consequences.
This, in turn, strengthened the hypothesis that the origin of synesthesia was to be found in the structure of the brain. Indeed, it turns out that the region of the brain responsible for processing graphemes lies adjacent to the region responsible for processing colors.
It’s fairly easy to imagine that small alterations in “normal” brain wiring — strengthening some pathways, inhibiting others — could be responsible for synesthesia. To test this idea further, Rama had synesthetes look at alternative representations of the numbers — using Roman numerals, for example — and found that the colors did not appear. That’s perfectly consistent, as the relevant bits of brain are responsible for processing the graphical representation of the number 5, not the abstract mathematical concept of “five.” Further studies using functional magnetic resonance imaging have found that the brains of synesthetes and non-synesthetes really do light up in different ways when they look at numbers. (There are other ways to be synesthetic — associating colors with sounds or smells, for example — which may also be traced to connections between different parts of the brain.)
But then here’s my favorite part. They found a synesthete who was color blind. That may seem strange, but what it really means is that the subject had problems with his retina that left him able to distinguish only an extremely narrow range of wavelengths when looking at most images in the world — his brain was fine, but his eyes weren’t quite up to the job. But when he saw certain numbers, he experienced colors that he otherwise never saw. Here is Ramachandran in Scientific American:
We also observed one case in which we believe cross activation enables a colorblind synesthete to see numbers tinged with hues he otherwise cannot perceive; charmingly, he refers to these as “Martian colors.” Although his retinal color receptors cannot process certain wavelengths, we suggest that his brain color area is working just fine and being cross-activated when he sees numbers…
The effect is most obvious and pronounced in the colorblind synesthetes, but occurs in “regular” synesthetes as well. The colors evoked by cross activation in the fusiform gyrus “bypass” earlier stages of color processing in the brain, which may confer an unusual tint to the colors evoked. This is important for understanding the phenomenon of synesthesia, because it suggests that the qualia label–that is, the subjective experience of the color sensation–depends not merely on the final stages of processing but on the total pattern of neural activity, including earlier stages.
Martian colors! How awesome is that?
Very cool.
There is a series of lectures from 2003 by V.S. Ramachandran called “The Emerging Mind” online at the BBC website:
http://www.bbc.co.uk/radio4/reith2003/lecturer.shtml
He is an engaging speaker, and they are well worth a listen. Synesthesia features in lecture four: “Purple Numbers and Sharp Cheese”.
I don’t quite get the idea. A synesthete should first read the two as a two before she could associate it with the color red. This seems pretty obvious to me: If I tell a synesthete “tell me the color of the number I’m thinking of” he won’t be able to tell before I tell him what number it is. But then, if she has to read each five to know it’s green, and each two to know it’s read, finding all the twos should take her about the same time as it would take any other person.
I think being able to experience color sensations you never had or could have before (because not made from addition of retinal input, as per RGB signals) would be really cool. (PS: If you are a misguided naive realist philosopher who doesn’t understand the concept of color sensations because you think we just “see colors” (light I guess, “directly”), I can’t help you very much.)
Interesting! I’m a synesthete with respect to sound: I see music and other sounds as some kind of flow of colored patterns, but I never thought of numbers as having colors or anything. Is there some relation between these types of synesthesia? I did see the 2’s fairly easily…
As a side note, two outstanding examples of synesthetes. First, there’s a superb poem by Arthur Rimbaud about synesthesia titled “Vowels” (“Voyelles” in the original French) that begins “I invented the colors of vowels”. He conjures up weird images and colors in connection with (French) vowels. Maybe that explains part of his incredible linguistic abilities.
Second, I don’t remember in which of his books Richard Feynman refers to the colors of Bessel functions, and tells how for him special functions had that kind of perceptual properties. Maybe someone else can be more precise about this than I…..
The idea is that it’s not a conscious effort. The color version of the 2/5 diagram really is an accurate analog of how synesthetes see it. I think for people like us who have never experienced it, it is almost impossible to imagine, it just sounds so weird and counterintuitive, but the experiments prove that it really happens that way.
Another ultra-bizarre effect V.S. talks about in the BBC Reith Lectures (link in my previous comment) is one where a person can recognize the face of his mother, but is unable to emotionally accept the fact that it *is* his mother. Instead of accepting her, he accuses her of being an impostor (sadly distressing for both parties).
The brain is truly a strange device 🙂
rusell before you become aware of the information to process it your mind has already done an enormous amount of preprocessing. As the quote says, the colors are associated to the shapes not the numbers. Thus when looking at the picture it will appear differently colored even without processing the fact that there are different numbers. The “preprocessor” has already associated colours to the different shapes, and then when picking out the different numbers the synesthets merely have to pick out the different colours as opposed to the different shapes, which, as you can see yourself in the provided images, is enormously more simple and quick.
Here’s what Feynman is quoted as saying, from Wikipedia http://en.wikipedia.org/wiki/Grapheme-color_synesthesia:
On the other hand, many synesthetes never realize that their experiences are in any way unusual or exceptional. For example, the Nobel prize winning physicist, Richard Feynman reports:
“When I see equations, I see the letters in colors – I don’t know why. As I’m talking, I see vague pictures of Bessel functions from Jahnke and Emde’s book, with light-tan j’s, slightly violet-bluish n’s, and dark brown x’s flying around. And I wonder what the hell it must look like to the students.”
Feynman, Richard. 1988. What Do You Care What Other People Think? New York: Norton. P. 59.
The beyond belief website is a pain to load
While its a great story, the sad truth is it’s only a case study, and unfortunately the ‘pop-out’ search effect for synesthetes doesn’t replicate with 14 synesthetes (as I recall, none of them pass a well controlled version of the task). Its hard to imagine how it could, since, as a previous commenter mentioned they would have to be able to read the 2’s and 5’s as 2’s and 5’s before they would be colored — and there are thousands of experiments on visual search suggesting thats not possible.
Reference:
Edquist, J., Rich, A.N., Brinkman, C. & Mattingley, J.B. (2006). Do synaesthetic colours act as unique features in visual search? Cortex, 42, 222-231.
(see also Mattingley, J.B., Rich, A.N., Yelland, G., & Bradshaw, J.L. (2001). Unconscious priming eliminates automatic binding of colour and alphanumeric form in synaesthesia. Nature, 410, 580-582.)
The biggest issue is that, as Tim pointed out, this isn’t a replicable effect. There are a few reasons why:
1) The visual search paradigm that produced those pretty greens and reds had a problem with viewing angle. From my discussion on the topic: However, when Laeng and colleagues performed a similar test with synesthete PM, they found that the speed of visual search seemed to depend more on the location of the stimulus in the array (Laeng, Svartdal, & Oelmann, 2004). That is, if the target were within 6° of the center of PM’s visual field, she performed better than the control participants, but if the target was in the periphery, her search performance was the same as that of the control participants.
2) There seems to be two variants of color-graphemic (colored-letter) synesthetes. One variety, called “associators”, has their synesthetic percepts (experiences) in their “mind’s eye”. It’s more a visualization, colloquially (and maybe neurologically, depending on who you talk to). The other form of synesthete, or “projector”, has their synesthetic percepts out in perceptual space, where the percepts can interact with other visual phenomena by, for example, overlaying on a stimulus.
*goes back to her visual crowding chapter*
Hey, it’s great to have people who know what they are talking about chiming in. But I’m not sure that what Veronica and Tim say is convincing that the effect isn’t real. Certainly I can imagine that some part of my brain recognizes the shapes before “I” can do so — in other words, before the meaning of the shape recognition percolates up to my consciousness. (Not that I know what I’m talking about, but I can imagine it.) Likewise, the fact that the viewing angle was crucial seems to be an interesting feature of the effect, but not evidence that it’s not real.
That’s sort of what I was wondering… Is it that any object designated as “3” or “5” or whatever, is also designated green or red or whatever color, is it the concept of that number that creates teh color or is it that pattern of lines. for example if you were walking through the woods would a synasthete see giant red threes where tree branches intersect to form threes, or could the person make it appear red by looking at it and deciding yeah, that’s a three. Just curious if anybody knows
I was talking to one of the other graduate students here at UC Davis a little while back, and she mentioned that she was synesthetic. It was rather interesting to listen to her talk about how when she memorized telephone numbers, she sees them in her mind’s eye as being multi-colored blobs.
BBC’s Horizon had a program about this subject, see here
There are also two online synesthesia tests that website.
Oops, a typo in the link. Let’s try again:
This is the link
Following up on Kevin’s comment: I’d like to ask some synesthetes what hybrid characters look like, that are intermediate between other ones. Also, if “five” looks green, what does “V” look like, if you know it is Roman numeral for five, etc.
There’s more to this than meets the eye. I spotted the 2’s in less than 3 seconds, and some scanning was involved. I think this method of perception is camouflage or pattern decoding, spotting the leopard in the tree.
Seeing the synesthete way, the red-green example, is a more holistic form of pattern recognition. This seems to be more like shape decoding, a red triangle on a green ground, less about 2 and 5.
In either case, I suspect that separating the 2 from the 5 is simple pattern recognition, the number, or the color, or both, are associated after the fact.
FWIW, seeing is all about the preprocessing, we don’t ‘see’ RGB or wavelengths and turn them into a color sensation. The brain ‘computes’ the ‘color’ from the environmental information. If you look out of a room, into daylight, your perception of the colors are relatively continuous and logical. Do the same thing with a camera, everything outside is bluer because the film or sensors respond to specific wavelengths. The eye-brain adjusts for those differences.
Sean, the most basic problem is the effect doesn’t replicate, regardless of whether we have an intuitive theory for how it could or couldn’t work.
The main claim, that the recognition happens “pre-attentively” (visual search researchers love to break things up into pre-attentive and post-attentive processing, something even I have done on occassion) and can therefore guide us in a search task, has been shown to be false in at least half a dozen studies I know of, including the two I cited in my previous post — one even published in Nature! Using several methods, from visual search to the attentional blink, suggests that people only experience the synathesia after they recognize and attend to the letters. It’s not as cool a story as Ramachandran’s and so it doesn’t get talked about much, but science is science.
The best popular exposition of this is “The Man Who Tasted Shapes” (Bradford Books) by Richard E. Cytowic.
He’s also written the world’s only college level textbook on the subject, “Synesthesia: A Union of the Senses.”
Cytowic does a pretty good job of fielding most of the hanging questions posed in the blog comments, above.
There will likely be a LOT more confirming evidence before this natural child gets too enthused about synesthesia. Then again, even if it is a valid effect for some people, so what? The human brain has been known to be capable of wacky effects long before this particular finding.
For the subjective record, I picked out all six of the 2s in about 3 seconds (totally subjective time measurement) and I ain’t never seen no colorified numbers, equations, or other mathematical representations in my 59.333 years of trying to interpret reality. I also have bad eyesight, and have never been known as especially perceptive. I tried it on a friend, timing him with my watch. He did it in well under four seconds. His history (I’ve known him for 40 years) is not that of a particularly perceptive genius either. If it’s taking other persons 20 seconds to find the six twos then we hereby announce ourselves as candidates for the Perception Hall of Fame, and freely offer our precious bodily fluids to all genius gene banks (at a ridiculously small fee; donations to individuals of the opposite sex, using traditional methods of implantation, may be even cheaper).
Tim, that makes sense; the fact that the “pre-attentive” idea had been tested and ruled out wasn’t quite clear from the previous comments. That’s what I get for writing about stuff on which I’m not an expert. (Not that I’m likely to stop.)
Yeah, Ramachandran’s not exactly known for producing studies that end up being replicated. Carefully controlled designs are not his forte (that would be publicity). But there is some really cool research on the psychological reality of synesthetic color (like Randy Blake’s).
The problems for Ramachandran’s “model,” and his data, are multitude. Perhaps the most obvious is that it’s not a neurally plausible model. The “pop out” phenomenon that Ramachandran claims to have observed, and which is the hallmark of many visual search paradigms, is almost certainly occurring in the primary visual cortex (V1), as all of the single features that lead to “pop out” phenomena are processed in V1 (this includes color). So, “pop outs” are themselves thought to be V1 phenomena. However, the potential color-grapheme connections are way up in V4. By that time, you’ve either seen the differences, or you haven’t, and all sorts of more complex attentional issues come into play (David Gilden’s done some cool work on this stuff). In other words, if you don’t get the color differences in early visual processing, you’re not going to get them, and since the grapheme processing comes later than V1 anyway, you might as well just go on that.
Put yet another way, synesthetic colors (for color-grapheme synesthetes, at least), aren’t individual features separable from the letters themselves. Instead, they’re inseparable feature conjunctions, so that if you don’t recognize the grapheme, you don’t recognize the color, and since the processing of the grapheme comes pretty late in visual processing, if you recognize the grapheme, you don’t need to recognize the color to perform the visual search task. This is, I believe, the conclusion of the Equist et al. paper that someone referred to earlier (the one with 14 synesthetes). Ironically, if Ramachandran were paying attention to the data from “color-blind” synesthetes, he’d have realized this too, because it implies that the color-grapheme connections occur way too late to benefit visual search.
P.S. Did Scott Atran’s talk rock as hard as it did last year?
If there are difficulties with these test, why not do more sophisticated tests like PET scans where you can really see if the people really see colored letters instead of black and white letters?