Occurrent memory of color experience

Many people when occurrently remembering what a tomato looks like have a mental image, with a degree of vividness and precision differing from person to person, of a red round thing. But not everyone. Visual aphantasiacs don’t have visual images at all in their memories. Presumably, too, there are partial cases: some may have visual images of shapes but without color. Nonetheless, it seems correct to say that such people still know what red is like.

I conjecture that most people have some kinds of experience where they know what the experience is like, but when they remember what the experience is like their memory does not contain a phenomenal copy of the experience. Maybe for you it’s certain kinds of tastes, or smells, or feeling hot water on your body.

For definiteness, suppose that for Alice it’s redness. She has seen red tomatoes. She knows what it’s like to see red, and she remembers what a red tomato looks like, but when she makes her memory occurrent, it does not contain a phenomenal copy of the experience.

What’s going on? Here is one possibility. Even though the memory of a colored object has no visualized color in it, it has some other distinctive phenomenal quality that encodes the specific color. Thus, while neither remembering a red tomato nor a blue plum involves visualizing a color, remembering a red tomato involves a non-color phenomenal quality that encodes the redness, while remembering a blue plum involves a different non-color phenomenal quality that encodes the blueness. It is by having that encoding non-color quale that one knows what the encoded color quale is. The encoding non-color quale is presumably not a sound, or a smell, or any other sensory quale, but a distinctive memory-of-color quale.

(One might conjecture that this is so even for those aphantasiacs who deny any phenomenal quality to their memories, because perhaps it is possible to mistake not having sensory qualia in one’s memories for not having any qualia in them.)

(One might also wonder if it goes to meta levels. If you remember remembering a red tomato, do you have a memory-of-memory-of-red quale, rather than just a memory-of-red quale? I don’t think it needs to do that. If you’re capable of having a memory-of-red quale, that can just figure in your memory of remembering.)

It seems very plausible that on this suggestion, the memory-of-X qualia can differ from person to person, even if the X qualia do not. One reason to think this is that some people do visualize colors, and for them a memory-of-red quale will just be a red quale (or a visualized-red quale, which might not be exactly the same). But a more interesting question is this: Can two people be such that one person’s memory-of-X quale is just like the other person’s memory-of-Y quale? The answer could be negative even if the memory-of-X qualia differ from person to person, for it could be that nonetheless there is a person-independent way to recover the X quale from its encoding in the memory-of-X quale. There can be multiple ways of encoding the same thing while yet each one can be uniquely decoded.

Maybe. But what if it turns out that there is no person-independent mapping from a memory-of-X quale to an X quale? I think this endangers the idea that having a memory-of-red quale in conjunction with one’s remembering is what makes one count as occurrently knowing what a ripe tomato looks like.

Is there a different way to account for what happens when one occurrently remembers what it’s like to see a red tomato when one cannot visualize colors, without positing some kind of memory-of-X qualia?

Here’s a suggestion inspired by David Bernhardt’s comment (but perhaps with I misunderstood the comment). Our thoughts can make de re reference to objects and events and the like. Perhaps when I think “Ripe tomatoes look like this”, the “this” makes a de re reference to my past experience of red. But just as the word “this” does not reflect the specific character of its referent, and indeed sounds and looks the same regardless of what it refers to, neither do the intrinsic features of the thought reflect the specific character of the past experience of red. Imagine you’re an almost exact twin of mine, except that you always wore glasses that swapped red and blue. Then you could have an intrinsically exactly similar “Ripe tomatoes look like this” thought to mine, but in your case the “this” would refer to an experience of blue.

Could something like this be how we have occurrent memories of how tomatoes look, a mere demonstrative de re thought? If so, then the knowledge argument against materialism is in trouble. For Mary, raised in her black and white room, could have a de re thought that ripe tomatoes look like this, where her “this” refers de re not to her own experience but to mine! After all, if you and I are both pointing to the same horse, and you think “This animal is lovely” and I think “This animal is lovely”, we seem to be thinking the same thought, precisely because our “This” has the same referent. So if Mary thinks “Tomatoes look like this” and I think “Tomatoes look like this”, and the “this” in both cases refers to my experience of red, then we seem to be thinking the same thing.

This suggests to me that we have three options:

1. People who can’t visualize colors don’t know what colors are like when they aren’t currently perceiving colored things.

2. They know it by means of memory-of-color qualia, which have an unambiguous mapping onto color qualia.

3. They know by means of de re thoughts that get their content from past experiences that are not reflected in the intrinsic character of the thought.

I think (1) and (2) leave the knowledge argument against materialism in place, but (3) endangers the argument.

I don’t know what the right story is.

Do you and I see colors the same way?

Suppose that Mary and Twin Mary live almost exactly duplicate lives in an almost black-and-white environment. The exception to the duplication of the lives and to the black-and-white character of the environment is that on their 18th birthday, each sees a colored square for a minute. Mary sees a green square and Twin Mary sees a blue square.

Intuitively, Mary and Twin Mary have different phenomenal experiences on their 18th birthday. But while I acknowledge that this is intuitive, I think it is also deniable. We might suppose that they simply have a “new color” experience on their 18th birthday, but it is qualitatively the same “new color” experience. Maybe what determines the qualitative character of a color experience is not the physical color that is perceived, but the relationship of this color to the whole body of our experience. Given that green and blue have the same relationship to the other (i.e., monochromatic) color experiences of Mary and Twin-Mary, it may be that they appear the same way.

If this kind of relationalism is correct, then it is very likely that when you and I look at the same blue sky, our experiences are qualitatively different. Your phenomenal experience is defined by its position in the network of your experiences and mine is defined by its position in the network of my experiences. Since these networks are different, the experiences are different. Somehow I find this idea somewhat plausible. It is even more plausible some experiences other than colors. Take tastes and smells. It’s not unlikely that fried cabbage tastes differently to me because in the network of my experiences it has connections to experiences of my grandmother’s cooking that it does not have in your network.

Such a relationalism could help explain the wide variation in sensory preferences. We normally suppose that people disagree on which tastes they like and dislike. But what if they don’t? What if instead the phenomenal tastes are different? What if banana muffins, which I dislike, taste differently to me than they do to most people, because they have a place in a different network of experiences, and if banana muffins tasted to me like they do to you, I would like them just as much?

In his original Mary thought experiment, Jackson says that monochrome Mary upon experiencing red for the first time learns what experience other people were having when they saw a red tomato. If the above hypothesis is right, she doesn’t learn that at all. Other people’s experiences of a red tomato would be very different from Mary’s, because Mary’s monochrome upbringing would place the red tomato in a very different network of experiences from that which it has in other people’s networks of experiences. (I don’t think this does much damage to the thought experiment as an argument against physicalism. Mary still seems to learn something—what it is to have an experience occupying such-and-such a spot in her network of experiences.)

Red cars and playdough

A red chunk of playdough needs to be red through and through. A red car need only be red on the outside. Peanut butter to be smooth must be smooth all the way through. But a mattress needs to only be smooth on the upper side to be smooth.

In other words, predicates like “is smooth” and “is red” apply to objects in different ways. A seemingly arbitrary decision needs to be made to how to apply them to a particular kind of object.

But perhaps this is only the case because chunks of playdough, cars, blobs of peanut butter and mattresses are not substances. Perhaps we can hope that for substances such decisions do not need to be made? But that hope is quickly dashed when we realize that a decision has to be made whether to call an electron a wave or a particle or both or neither, and that a decision has to be made which of a horse’s muscles are relevant to saying that the horse is strong (does it need to have strong eyelid muscles? tail muscles?).

Maybe when we descend to the level of applying fundamental predicates to substances, then the problem disappears. But that’s not clear. Position predicates seem to be fundamental but there is arbitrariness in deciding how to apply them to quantum objects when they are not in an eigenstate of position.

Perhaps where the arbitrariness disappears is when we consider cases where a fundamental predicate fundamentally applies to a substance. A fundamental predicate might non-fundamentally apply to a substance: thus, a dog might be negatively charged, and “is negatively charged” might be fundmental, but the dog is not fundamentally negatively charged—rather it is charged in virtue of mathematical facts about the overall distribution of fundamental charge properties throughout its body.

Towards a Thomistic theory of fundamental distributional properties

Recently, various metaphysicians (e.g., Parsons, and Arntzenius and Hawthorne) have tried to give an account of spatially nonuniform properties that would work for extended simples or gunky objects (i.e., ones that have no smallest parts). I think there is an interesting account that has in an important way a Thomistic root, and that's no surprise because Aquinas did not believe that substances had substantial parts, so he faced the problems that people thinking about extended simples face. I will develop a partial account for shape, location and color. The version I will give in moderate detail is Pythagorean, because mathematical objects are involved in physical reality itself. The Pythagorean account is easier to wrap one's mind around. I think it may be possible to use Category Theory to de-Pythagorize the account, but I will only sketch the beginning of that line of thought.

A basic insight Aquinas has is that material objects have a special accident called "dimensive quantity", which accident in turn provides a basis for further accidents, such as color. Moreover, objects normally are located in a place by having their dimensive quantity be located there.

On to the Pythagorean account. Suppose that each extended object O has fundamentally associated with it a manifold G of some appropriate smoothness type (we may in the end want to generalize this, perhaps to a metric space, perhaps a topological space, but let's stick to manifolds for now). This manifold I will call the object's (internal) geometry. The fundamental relation between the object and the manifold that associates the manifold to the object is being geometrized by: the object is geometrized by the manifold. This manifold is a purely mathematical object existing in the Platonic heaven. Nonetheless, which manifold an object is geometrized by significantly affects its nomic interaction with other objects. The shape properties of an object are grounded in the fact that the object is geometrized by such-and-such a manifold.

Next, we need location. There is a fundamental relation between an object O and a function L from the object's geometry G to another mathematical manifold called "spacetime", which relation I will call being located by. The function L describes how the object's geometry is located within spacetime. We can now say that two objects O₁ and O₂ overlap if and only if there are L₁ and L₂ such that O_i is located by L_i, for i=1,2, and the ranges of the functions L₁ and L₂ overlap.

Now, let's add some color into the picture. There is an abstract object which is a colorspace. Maybe it's some kind of a three-dimensional manifold. There is a fundamental relation between an object O and a function c from the object's geometry to the colorspace, which we may call being colored by. This function describes the distribution of color over the object's geometry.

This is the Pythagorean version of the view. Now we should de-Pythagorize it. Suppose a fundamental determinable of objects: being geometrized. A maximally specific determinate of being geometrized will be called a geometrization. And then—this is getting sketchier—one makes the geometrizations, and maybe other Platonic things like geometrizations, into a category isomorphic to an appropriate category of manifolds. I don't know what, if any, classical ontological category arrows correspond to. Maybe some kinds of token relations. If we're substantivalists about spacetime, we can suppose a special object, S, the spacetime. And then there is a fundamental relation of being located by between an object O and a morphism L of the category of geometrizations whose domain is O's geometrization and whose codomain is S's geometrization. It's harder to bring colors into the picture. This is far as I got. And even if I finish the de-Pythagorization, I will still want to de-Platonize it.

More generally, the de-Pythagorization proceeds by replacing mathematical objects associated with an object with maximally specific determinates of a determinable that, nonetheless, stand in the same structural relations as the mathematical objects did. Category Theory is a promising way to capture that structural sameness, but it might not be the only way.

Colors

One might think that all there is to an object's being red is that object's appearing red to one in standard circumstances. Here is a potential counterexample. I gave my four-year-old son some red gelatin dessert (probably not of the Jell-O brand) and asked him what color it was. He said: "It looks red" (his slight emphasis). Now, he wasn't expressing a doubt about whether the circumstances were standard. Rather, because he is red-green colorblind, he knows there is a gap between an object's appearing red to him in standard circumstances, and its actually being red. (He agreed later that it looked "reddish green" to him. But once I told him that it was actually reddish orange, he accepted that, and from then on was very firm that it was reddish orange.) I suppose "standard circumstances" could include "standard observer", in which case there is no counterexample. In any case, the point here is that color terms for my son have a serious intersubjectivity, and perhaps objectivity, to them.

Interestingly, I think that occasionally my son bridges this gap inferentially—the object looks more like red objects do than like green objects do, so it's probably red—and sometimes he bridges it non-inferentially through an internalization of stereotyped colors. Thus, when asked about a green leaf or green grass what color it is, he instantly responds that it's green—I think he may be seeing it as green in the way in which I can see a person as old. Does he have the quale of green in the case of the green leaf or green grass, when he sees it (visually!) as green? I have no idea. If to have a quale of green is just to be non-inferentially visually appeared to greenly, then he does, since he is visually appeared to greenly, except that in his case the green appearance of the leaf depends also on shape. If the green appearance depends also on shape, it's hard to say that it has the quale of green. But maybe it does. (Suppose I heard an object as green—maybe because I could hear the molecules vibrating as they reflect green light—would that have the same quale as seeing it as green?)