THE LENS IN YOUR EYE casts an upside-down image on your retina, but you see the world upright. Although people often believe that an upside-down image in the eyeball gets rotated somewhere in the brain to make it look right side up, that idea is a fallacy. No such rotation occurs, because there is no replica of the retinal image in the brain—only a pattern of firing of nerve impulses that encodes the image in such a way that it is perceived correctly; the brain does not rotate the nerve impulses.

Even leaving aside this common pitfall, the matter of seeing things upright is vastly more complex than you might imagine, a fact that was first pointed out clearly in the 1970s by perception researcher Irvin Rock, then at Rutgers University.

Tilted ViewLet us probe those complexities with a few simple experiments. First, tilt your head 90 degrees while looking at the objects cluttering the room you are in now. Obviously, the objects (tables, chairs, people) continue to look upright—they do not suddenly appear to be at an angle.

Now imagine tipping over a table by 90 degrees, so that it lies on its side. You will see that it does indeed look rotated, as it should. We know that correct perception of the upright table is not because of some “memory” of the habitual upright position of things such as a table; the effect works equally well for abstract sculptures in an art gallery. The surrounding context is not the answer either: if a luminous table were placed in a completely dark room and you rotated your head while looking at it, the table would still appear upright.

Instead your brain figures out which way is up by relying on feedback signals sent from the vestibular system in your ear (which signals the degree of head rotation) to visual areas; in other words, the brain takes into account head rotation when it interprets the table’s orientation. The phrase “takes into account” is much more accurate than saying that your brain “rotates” the tilted image of the table. There is no image in the brain to “rotate”—and even if there were, who would be the little person in the brain looking at the rotated image? In the rest of the essay, we will use “reinterpret” or “correct” instead of “rotate.” These terms are not entirely accurate, but they will serve as shorthand.

There are clear limits to vestibular correction. Upside-down print, for instance, is extremely hard to read. Just turn this magazine upside down to find out. Now, holding the magazine right side up again, try bending down and looking at it through your legs—so your head is upside down. The page continues to be difficult to read, even though vestibular information is clearly signaling to you that the page and corresponding text are still upright in the world compared with your head’s orientation. The letters are too perceptually complex and fine-grained to be aided by the vestibular correction, even though the overall orientation of the page is corrected to look upright.

Let us examine these phenomena more closely. Look at the square in a. Rotate it physically 45 degrees, and you see a diamond. But if you rotate your head 45 degrees, the square continues to look like a square—even though it is a diamond on the retina (the tissue at the back of the eye that receives visual inputs); vestibular correction is at work again.

The Big PictureNow consider the two central red diamonds in b and c. The diamond in b looks like a diamond and the one in c looks like a square, even though your head remains upright and there is obviously no vestibular correction. This simple demonstration shows the powerful effects of the overall axis of the “big” figure comprising the small squares (or diamonds). It would be misleading to call this effect “context” because in d—a square surrounded by faces tilted at 45 degrees—the square continues to look like a square (though perhaps less so than when isolated).

You can also test the effects of visual attention. The figure in e is a composite. In this case, the central red shape is ambiguous. If you attend to the vertical column, it resembles a diamond; if you view it as a member of the group forming the oblique line of shapes, it seems to be a square.

Even more compelling is the George W. Bush illusion, a variant of the Margaret Thatcher illusion, which was originally developed by psychologist Peter Thompson of the University of York in England. If you look at the upside-down images of Bush’s face on this page (f), you see nothing odd. But turn the same images right side up, and you see how grotesque he really looks. Why does this effect happen?

The reason is that despite the seamless unity of perception, the analysis of the image by the brain proceeds piecemeal. In this case, the perception of a face depends largely on the relative positions of the features (eyes, nose, mouth). So Bush’s face is perceived as a face (albeit one that is upside down) just as an upside-down chair is readily identified as a chair. In contrast, the expression conveyed by the features depends exclusively on their orientation (downturned corners of the mouth, distortion of eyebrows), independent of the perceived overall orientation of the head—the “context.”