Phantom of the escalator

Very cool paper with the awesomest title ever given to a scientific report:

Fukui T, Kimura T, Kadota K, Shimojo S, Gomi H (2009) Odd Sensation Induced by Moving-Phantom which Triggers Subconscious Motor Program. PLoS ONE 4(6): e5782. (it's a free access journal)

Odd sensation? Induced by moving-phantom? Which moreover triggers subconscious motor program?? You got to be kidding me, that's a Nobel prize winner!

So let's check it out. The premise is extremely simple: have you ever stepped onto a stopped escalator? How did it feel? Yeah, I'm sure you know what I mean. I also have it. Well, that's what the authors of this paper set out to understand. They call it an "odd sensation". (One of the charms of the paper, I should mention, is that it is written in Japanglish).

Here's how the problem is stated in the introduction:

"When we encounter an escalator that is out of service, we recognize the escalator is certainly stopped and is not going to move; have the definite intention to step onto the stopped escalator (action to peculiar physical dimensions of stairs, see Fig. 1A), not a moving one (very familiar action); and then step onto it. After doing so, despite the proper awareness and intention, we frequently feel clumsy, as if our posture is awkward, our bodies heavier, and the steps are sometimes hard to climb. This clumsiness is associated with some kind of odd sensation." (emphasis added)

If you have no idea at all what these guys are talking about or what this "odd sensation" is supposed to be, then the paper will make no sense at all to you. It's really one of these things you have to experience firsthand to understand what it's all about. But still, even if you're like me and had this "odd sensation" dozens of times, you might ask why this is interesting at all. Well, the crucial point with this peculiar illusion resides in the interaction between top-down and bottom-up processes involved in visuo-motor integration. As the authors put it:

"...when we step onto a stopped escalator even with perfect knowledge that it is stationary, we are often surprised by the strange feeling accompanied by our own action. This feeling implies the involvement of subconscious processes against conscious awareness in the emergence of the odd sensation. Thus, this phenomenon raises an intriguing question as to how implicit motor programming escapes from conscious top-down control and offers an opportunity to study it."

In other words, it is definitely most interesting that such an effect should exist even when people know full well that the escalator is stopped. Why does it feel still feel so weird? What's going on, and how can we study this?

The authors decided to use an actual escalator to investigate this "escalator illusion". They set out three hypotheses that might explain it:

1) the effect is merely due to our unfamiliarity with stopped escalators. It coincides with specific body movements, of course, but the "odd sensation" only comes from "unfamiliarity" and that's it.

2) the effect is primarily due to the specifics of the escalator: it is true that an escalator's steps are deeper and higher than most normal stairs, and when it is stopped we have to accomplish nonuniform movements to climb it, which induce clumsiness (i.e. the first step taken is notably shorter than the following ones)

3) the effect emerges against our will, as our brain somehow triggers the motor program required to step onto a moving escalator even as it is perfectly evident that it is a stopped one. (in their words: "the subconscious emergence of the habitual escalator-specific motor program leads to the inappropriate motor behavior, which leads to the odd sensation".

To differentiate these 3 hypotheses (and it is clear that they favour the third one), they tested 3 conditions: subjects had to step onto either a moving escalator, a stopped escalator, or a wooden stair not looking like an escalator but having the same physical dimensions. There were only 7 subjects, and during all trials they wore movement trackers on their back and their heels. The kinematics of upper and lower body was the main measure studied, and after each trial they also were asked to rate on a 1 to 5 scale "the extent to which they felt some kind of odd sensation".

I will spare you an account of the detailed results of this study, which are quite elaborate. The analyses I must say are very original and smart, but I wonder how many journals have turned down this study before it was published in PLoS One. I have the feeling that many reviewers would have been put off by the overkill nature of all these kinematic data and path analysis statistics for the sole purpose of investigating an "odd sensation". Well, you ask me, this should be published in Science or Nature, no less.

The discussion is a little bit redundant, but it can be conveniently summarized as follows:

"the prerequisite for the emergence of the odd sensation is an inappropriate motor behavior against the current situation resulting from the habitual motor behavior for a moving escalator. People may not be able to suppress this behavior despite completely understanding the current stopped escalator situation.

(...)

The conflict between the motor intention and the sensory outcome would be the important factor for the emergence of some kind of strange/peculiar feeling."

Well, this is the trouble with habits. They only work insofar as the context doesn't change too much. But the authors cannot really explain where this "subconscious" motor program specialized for escalator climbing comes from in the first place. How many escalators do you have to use in a lifetime to acquire the "subconscious motor program" leading to this "odd sensation"? What is more, it is clear to me that the "sensation" is there as well when you have to climb down a stopped escalator, and when you walk on one of these very long airport treadmills that happen to be nonfunctional. Does the interpretation based on the kinematics of climbing up still hold? I would also like in the future to know more about exactly what the "odd sensation" is. In the introduction, they mention clumsiness, heaviness, and awkwardness. Are these all synonymous, or different dimensions of the sensation? Can these be dissociated, for instance if we modulate the speed of the escalator (very slow instead of stopped, and so forth)? Finally, I'd like to ask why the word "phantom" is used in the title, but not at all in the text? This is very bizarre and disappointing, as it appears also in the abstract:

"moving escalator-specific motor actions emerged after participants had stepped onto the stopped escalator despite their full awareness that it was stopped, as if the motor behavior was guided by a ‘‘phantom’’ of a moving escalator"

Well, as a self-proclaimed phantomologist, I can only say that I perfectly agree with the use of the word (although it is not clear if we're talking of a "phantom moving escalator" or "phantom self on a moving escalator"), but it really boggles the mind that the concept should be brought about in the title and the abstract, and then be entirely forgotten. Another case of vanishing phantoms, I guess.

Despite these very few caveats, I'd like to repeat that this is a very delightful paper. And I'd like now to share with you something it reminded me about. The authors have this to say in the introduction:

"when going up or down stairs, we sometimes extend the foot in expectation of an additional step at the end of the stairs, which causes us to stagger."

This they say is entirely different from the escalator illusion because in this case we realize that we made a mistake, "a self-prediction error caused by our thinking there was an additional step", whereas in the escalator case there is no error, we know and see that it is stopped. Well, this brought to my mind a very peculiar account from a very peculiar book. It allows me to demonstrate that the use of the word "phantom" by the authors of the escalator study was perfectly appropriate, as there could be a nice continuity from the "odd sensation" they talked about, to the situation where we lose our footing while climbing down the stairs as we expect an additional step that is not there, and then onto the full-blown "phantom" projection, meaning disembodiment as in out-of-body experiences. This is in my opinion all part if phantomology.

The demonstration is found in the account by a man named Sylvan Muldoon of his repeated out-of-body experiences (see Blackmore's account of this interesting fellow). In his main book, The Projection of the Astral Body, Muldoon explains in some excruciating details the many ways and situations to induce an "astral projection". Of particular interest here is a subsection of chapter 2 that explains how an OBE can be induced by mistakenly expecting an additional staircase step.

A projection caused by a mis-step

Here is an example of how an unexpected jar will throw the phantom out of coincidence. One night, a few years ago, I was coming down the stairs of my home. I had been sleeping, and was still quite drowsy. There were fifteen steps, and I had trodden up and down them hundreds of times, having lived in the house all my life. Just why I do not know, but, as I reached the bottom step, I tried to take one step more (many of us have done this), and the impetus jarred me vigorously.
A breath-taking sensation shot through the pit of my stomach, and even before the physical dropped to the floor, I found myself projected from it, in a perfectly conscious state. (...) I both saw the physical mechanism fall to the floor and also felt it fall, while standing several feet from it. Let me analyse this, and see what actually happenened, for by doing so we shall discover the fundamental law of astral projection.
It must be understood that it is not the conscious mind which causes projection, but it is the subconscious Will.
(...)
Which brings us to the bed-rock rule of astral projection: IF THE SUBCONSCIOUS WILL BECOMES POSSESSED OF THE IDEA TO MOVE THE BODY (COINCIDING BODIES) AND THE PHYSICAL COUNTERPART IS INCAPACITATED, THE SUBCONSCIOUS WILL MOVES THE ASTRAL BODY INDEPENDENT OF THE PHYSICAL." (The Projection of the Astral Body, Chap. 2, p.64)

There goes some "odd sensation", isn't it? Note how he calls the OBE a "phantom thrown out of coincidence", which is actually a wonderful and accurate way to put it from a cognitive perspective I think. Note also how Muldoon uses almost exactly the same terminology than the Japanese authors of the escalator study, positing a conflict between the "subconscious will" (the "motor programs") and the physical outcome of the event. In modern words, we would say that the body schema and the body image are not coinciding, hence the induction of an "odd sensation", which could perhaps manifest itself in more extreme situations (and possibly quite rare individuals) as an OBE.

Ok, I refrain to talk here about similar effects found in elevators. Remind me to discuss someday the wonderful paper by Schilder on the topic (Parker, S. & Schilder, P. (1930). Das Körperschema im Lift. Zeitschrift für die gesamte Neurologie und Psychiatrie, 128: 777-783).

Also, if he agrees to let me do so, I might also discuss in the future a nice idea for a project that my colleague Pär Halje proposed in less than 5 minutes when he was pressed to come up with something for a PhD class. He called it "The trampoline after-effect", and I think the idea is just too wonderful to be left alone. More phantomology induced by cool artifacts.

And one last word about escalators: French readers might want to learn more about the international plot of escalator repairmen. We know what you are doing, bastards, so stop it already.

The Tom Cruise effect

I just found this picture from one of my favourite blogs, the Canadian Cynic (originally at the Photoshopping site Freakingnews.com).
In the interest of cognitive science and skepticism, I just had to use it as a mini-primer on the Thatcher effect.
Look at it:


Well, that's creepy enough, isn't it? I mean, first of all, that's Tom Cruise, and as if that's not enough, he's upside-down but his face is not. Brrrrr. Something is definitely wrong with that picture, so wrong that I would have a very hard time locating it, even approximately, anywhere in the uncanny valley. Ok, but now look at what happens when that picture is further turned upside-down:


So, I ask you: which one is creepiest? Just look at them side by side, maybe this can help:

I would say the second one, on the right, looks even worse than the first one. If I had any readers on this blog, I would put on an online poll to settle the matter. But I'm all alone here, so I'll have to trust my own feelings.

Anyway, let's flesh out some background now. I think this is a rather extreme instance of the so-called Thatcher illusion ([1]; find the original, an hugely influential, paper by Peter Thompson here). This is very well known, but it's still an extremely efficient classroom demonstration of how the visual system deals with special objects like faces.


Look first at both pictures of Maggie in the lower row. The one on the left is normal, the one on the right is grotesquely deformes (eyes and mouth have been selectively inverted). In the upper-row, the images are simply inverted, and the striking discovery is that the "grotesque" Thatcher is much less grotesque in this orientation. The effect is generally interpreted as a demonstration of the fact that inverting a face disrupts configural processing, meaning that it's extremely hard to detect incongruencies in the location, size, distance, and overall relationships of isolated parts when the face is rotated 180°. There is actually some ongoing debate about the exact mecanisms involved in face processing and the real meaning of the Thatcher illusion, but I won't go into the details here.

Now, if you wonder what happens when the images slowly rotate or are rotated by 90°, you can test it yourself on Michael's Bach excellent website. If you want more theory of what's going on behind the Tatcher illusion, check this old post of the now-probably-defunct-but nevertheless-missed blog Mixing Memory.

Here's a variant of the Thatcher effect that I like very much, but which actual relevance to the effect discovered by Thompson can perhaps be questionned ([2]; it's a tribute to Thompson by Stuart Antis, which you can download here):

Again, see Mixing Memory for some explanation. But here's the caption by Anstis:

(a) Unretouched negative photograph of Tony Blair. This is analogous to an upside-down face. (b) Face is negative except for the eyes, teeth, and hair, which are positive. This is equivalent to an upside-down thatcherized face, but looks barely more grotesque than (a). (c) Reversing the contrast of (a) yields a normal photo. (d) Reversing the contrast of (b) yields a grotesque portrait, with the whole face positive except for the negative eyes, hair, and teeth. This is equivalent to an erect thatcherized face.

From the text, some sentences that deserve full re-transcription here:

I have done for negative faces what Thompson did for inverted faces. Figure 1a is a contrast-reversed portrait of Mrs Thatcher's latest successor, the much-loved Tony Blair, well-known for his gleaming eyes and teeth. Next to it is a modified negative (figure 1b), which looks similar to, or perhaps more lifelike than, 1a. In fact, the mouth, eyes, and hair in figure 1b were individually contrast-reversed before the whole picture was contrast-reversed, so these features are positive whilst the rest of the face is negative. This is analogous to thatcherizing the face in the contrast domain.
When these two pictures are viewed in positive, figure 1c is clearly normal, whereas the grotesqueness of figure 1d is vampirish enough to scare little Tory children.

And the glorious conclusion:

This would suggest that there are similarities between the Blair illusion and the Thatcher illusion (in much the same way as there are similarities between Blair and Thatcher).

So, how does all this relate to the Tom Cruise pictures? Well, I guess the Thatcher explanation will say that the head-inversion somehow disrupts the grotesque-ness of the face-inversion. As humans and their heads are overwhelmingly perceived in the upright orientation in daily life, and the visual system evolved to quickly make sense of facial identitites and expressions, then the manipulation in the second picture is even more obvious than the one in the first picture. Anyway, I can add another example so that you can make up your own mind, and then I'll leave it at that:













Which one is the creepiest?

[1] Thompson P, 1980, "Margaret Thatcher: a new illusion" Perception 9(4) 483 – 484
[2] Anstis, S.M. (2005). Homage to Pete Thompson: The Tony Blair illusion. Perception, 34, 1417

Creating phantom fingers

The Phi phenomenon (or Phi effect, or Phi movement, of Phi illusion, or apparent motion...) is a classic demonstration of how the brain just makes up stuff in your mind that are actually not out there. Typically, you see one dot flashing on your left, and when it turns off, another dot appears on the right. Given the right circumstances, and these are pretty accomodating anyway (i.e. it works under a lot of conditions), what you see are not two stationary flickering dots, but a single dot moving back and forth. It is indeed quite hard not to perceive the nonexistent intermediary dots in between the flickering ones: they are, as it were, phantom dots.

The effect was discovered by Max Wertheimer, one of the founders of Gestalt psychology, and first published as: Experimentelle Studien über das Sehen von Bewegung. Zeitschrift für Psychologie 1912, 61, 161-265, 1912. (see a list of Wertheimer's publications, and much more, in this excellent website).

Now, this simple effect as been hugely influential and useful for theories of movement perception, and vision and cognition in general. There are hundreds of variations that you can think of (and that have been most probably already studied, make no mistake), as well as thousands of papers on the phenomenon. It has become quite hard to do anything original in this field, at least since Shiffrar and Freyd's excellent study on "apparent motion of the human body" [1], which I plan to discuss someday in the context of phantom limbs. One of the questions raised by the Phi phenomenon is of course its mechanisms: is it primarily a bottom-up thing, based on mindless "motion detectors", or is it modulated by top-down factors, like beliefs, knowledge, expectancies, etc.?

Well, I now see that the journal Perception has a short and intriguing piece [2] that might bring something new to the topic. Whereas Shiffrar and Freyd focused on the perception of apparent motion for other person's body parts, Ekroll and Scherzer, the authors of this new paper, propose a demonstration of the Phi illusion for one's own body. Here's their abstract:

We describe how illusory apparent motion of one’s own fingers and other body parts can easily be experienced in spite of grossly conflicting proprioceptive signals. This simple illusion may be used to shed light on the crossmodal integration of visual and proprioceptive signals. Our preliminary observations suggest that the visual and proprioceptive signals do not merge into a common crossmodal percept. Instead, mutually inconsistent visual and bodily sensations are experienced simultaneously.

So, how do you induce apparent motion using your own body? Here's what the authors say:

To experience the phenomenon, hold one of your hands with the fingers pointing
away from you (...). Only the index finger should be stretched out and visible from your point of view. Then lower the index finger and lift the middle one instead. Keep the hand steady while alternating swiftly and cyclically between these two finger positions, ie move the two fingers up and down in counterphase. When this is done swiftly enough, you will have the visual impression of a single finger tip moving horizontally instead of two fingers moving up and down. (...) One may initially find it difficult to move the fingers swiftly enough while preserving the appropriate coordination of the two fingers. (...) The effect seems to be more easily experienced
when the fingers are viewed monocularly with a half-closed eye. (...) Furthermore, viewing the fingers in peripheral vision, say, by looking at the wrist instead of directly at the fingers, also seems to enhance the effect.

Taken from Perception's website, the figures below of a hairy hand illustrates the idea. From my experience, you have to do this rather fast, and it's not so easy to be consistent (it is very important that you always see the same "amount" of finger sequentially, otherwise it seems to me that any effect there might be is destroyed).































The animation can be seen here. It is really a matter of subjective opinion whether there is a strong feeling of apparent horizontal motion or not while looking at this (I mean, looking at the animation, i.e. not doing it yourself). The authors also recommend trying this with the feet, as in the follwing figures:


































The animation can be seen here. This works even less, I think, but try it anyway.

Ok, good. Neither these effects work so well for me, but I accept the general idea (and maybe I should practive a little bit more, while trying I did have brief glimpses of a quite effective phi, but this was very transient). Theoretically, when this works, it should induce the perception of intermediate phantom fingers in between the real fingers, just like phantom dots are created by the brain in between two flickering dots. I'll keep trying.

Well ok, this is all fun, and makes for a very cool classroom demonstration. But one might ask, from a theoretical point of view, what's the point? Well, the authors valiantly present three theoretical perspectives arising from this funny experience:

1) "proprioceptive signals from the hand could have dominated over visual motion signals, making the subject perceive the veridical vertical finger motion, although a second passive observer perceives illusory horizontal motion"

Well that's a good idea. The problem is that it will be hard to gather good data from this illusion, as subjects have to do it themselves, and therefore it can hardly be controlled, plus the feeling is by definition subjective, and so forth. What is more, if you want a "second passive observer", then this guy would have to see the fingers from someone else's point of view, hardly a practical experimental setting. Another approach to test hypothesis 1) would be to induce in the fingers some kind of proprioceptive disturbance, to see if this enhances the perception of phi. But I have no idea how to do this without disturbing motor processes. It seems we reached an experimental dead-end. But proprioception is not the only thing getting in the way of vision here. As I said, it is very difficult to produce consistent, identical and repetitive finger movements while looking at them from a specific angle, so the visual information itself is quite unstable, which might be enough to reduce any phi effect.

2) vision may dominate over proprioception, and the active subject may have the
bodily sensation of a horizontally moving finger.

Sure. I mean, this is sort of trivial, for if the illusion that is described works at all, then it implies that vision does indeed, somehow, dominate over proprioception.

3) "proprioceptive and visual information may fail to merge into a coherent crossmodal percept. In our experience, the latter seems to be the case."

I'm not sure what this last hypothesis means. In the author's words: "one experiences a visual illusion of horizontal motion although the fingers are felt to move vertically". What would it feel like if a "coherent crossmodal percept" between proprioception and vision was achieved? How is hypothesis 3) different from hypothesis 2)? If vision dominates over proprioception, well then they have not "merged" appropriately, right? On the other hand, it might be a sort of figure-ground phenomena, akin to what happens during binocular rivalry or the Necker cube: there are times, when doing this illusion, where you perceive either proprioception, or vision, but not both at the same time (and when vision is in the foreground, then phi is experienced). But again, it's close to impossible to actually test this thorougly. It gets also quite confused in the conclusion, where the authors write:

"the proprioceptive and visual signals, which in many other cases of proprioceptive/visual conflict merge into a common percept, fail to do so in the present case. A possible explanation for this would be that the horizontal apparent motion is so discrepant from the vertical motion that would be expected on the basis of the efferent motor commands that the visual percept is more plausibly attributed to an object not belonging to the observer's own body."

This they link to the literature on experimental disturbances of agency (of the Jeannerod type). But obviously this goes a step too far. I'm of course speaking on my own experience (or lack thereof) of this illusion, but it certainly does not feel like the moving fingers, whether they seem to move vertically or horizontally, are someone else's! I don't think there's any disturbance of agency or ownership in this illusion, as far as I'm concerned I think this is the wrong approach.

But then I don't really have anything better to propose, and I think overall that the publication is very cool and intriguing.

[1] Shiffrar M & Freyd JJ (1990). Apparent motion of the human body. Psychological Science, 1 (4): 257-264.
[2] Ekroll V, Scherzer T R, 2009, "Apparent visual motion of the observer’s own limbs" Perception 38(5) 778 – 780