Do you and I see colors the same way?

Suppose that Mary and Twin Mary live almost exactly duplicate lives in an almost black-and-white environment. The exception to the duplication of the lives and to the black-and-white character of the environment is that on their 18th birthday, each sees a colored square for a minute. Mary sees a green square and Twin Mary sees a blue square.

Intuitively, Mary and Twin Mary have different phenomenal experiences on their 18th birthday. But while I acknowledge that this is intuitive, I think it is also deniable. We might suppose that they simply have a “new color” experience on their 18th birthday, but it is qualitatively the same “new color” experience. Maybe what determines the qualitative character of a color experience is not the physical color that is perceived, but the relationship of this color to the whole body of our experience. Given that green and blue have the same relationship to the other (i.e., monochromatic) color experiences of Mary and Twin-Mary, it may be that they appear the same way.

If this kind of relationalism is correct, then it is very likely that when you and I look at the same blue sky, our experiences are qualitatively different. Your phenomenal experience is defined by its position in the network of your experiences and mine is defined by its position in the network of my experiences. Since these networks are different, the experiences are different. Somehow I find this idea somewhat plausible. It is even more plausible some experiences other than colors. Take tastes and smells. It’s not unlikely that fried cabbage tastes differently to me because in the network of my experiences it has connections to experiences of my grandmother’s cooking that it does not have in your network.

Such a relationalism could help explain the wide variation in sensory preferences. We normally suppose that people disagree on which tastes they like and dislike. But what if they don’t? What if instead the phenomenal tastes are different? What if banana muffins, which I dislike, taste differently to me than they do to most people, because they have a place in a different network of experiences, and if banana muffins tasted to me like they do to you, I would like them just as much?

In his original Mary thought experiment, Jackson says that monochrome Mary upon experiencing red for the first time learns what experience other people were having when they saw a red tomato. If the above hypothesis is right, she doesn’t learn that at all. Other people’s experiences of a red tomato would be very different from Mary’s, because Mary’s monochrome upbringing would place the red tomato in a very different network of experiences from that which it has in other people’s networks of experiences. (I don’t think this does much damage to the thought experiment as an argument against physicalism. Mary still seems to learn something—what it is to have an experience occupying such-and-such a spot in her network of experiences.)