Why do virtual objects close to my face appear blurry when wearing a VR headset? My vision is fine!

And why does the real world look strange immediately after a long VR session?

These are another two (relates ones) of those frequently-asked questions about VR and head-mounted displays (HMDs) that I promised to address a while back.

Here’s the short answer: In all currently-available HMDs, the screens creating the virtual imagery are at a fixed optical distance from the user. But our eyes have evolved to automatically adjust their optical focus based on the perceived distance to objects, virtual or real, that they are looking at. So when a virtual object appears to be mere inches in front of the user’s face, but the screens showing images of that object are — optically — several meters away, the user’s eyes will focus on the wrong distance, and as a result, the virtual object will appear blurry (the same happens, albeit less pronounced, when a virtual object appears to be very far away). This effect is called accommodation-vergence conflict, and besides being a nuisance, it can also cause eye strain or headaches during prolonged VR sessions, and can cause vision problems for a short while after such sessions.

Now for the long answer.

Any optical system using lenses needs to adjust to bring objects at different distances into focus. Cameras adjust by changing the distance between the lens and the imaging plane, and the eyes of humans, all other mammals, birds, and reptiles adjust by changing the optical power of the lens (see Figure 1). This latter process is called accommodation. There is a ring of muscle (the ciliary muscle) around the eye’s lens that can contract and relax to change the shape of the lens.

Figure 1: An eye focuses on objects at three different distances (near, medium, infinite). Depending on an object’s distance, light rays from that object diverge at different angles when reaching the eye. The eye counters that by accommodating, i.e., changing the shape of the lens.

How does the eye know how much to squeeze the lens to focus on an object that it wants to see? There are two independent mechanisms. The first one is based on blurriness-detecting cell clusters directly in the retina. If an object is out of focus, these cells will detect that, and send signals to the ciliary muscle to adjust focus until the object appears sharp (see Figure 2). This mechanism works reliably, but it is not particularly fast. It is like the autofocus mechanism in many cameras, where the lens has to move back and forth and back and forth a few times before it finds the correct focus distance.

Figure 2: An eye accommodating based on blurriness response. Left: the eye’s lens has the wrong shape, causing incoming light from an object not to converge into a single point, but into a small disk. Right: receptors in the retina detect the resulting blurriness, and cause the ciliary muscle to adjust the shape of the lens until the object comes into sharp focus.

The second mechanism is an improvement exploiting the fact that there are two eyes, and that there is a very small area of high resolution in the center of the retina, the fovea. Because the resolution, in pixels or light-sensitive elements per degree, of the eye is surprisingly low outside the fovea, definitely not high enough to discern small detail or let alone read text, there is a natural tendency to swivel the eye so that light from the current object of interest travels straight through the lens and lands in the fovea. This process is called vergence.

Now, given that the two eyes are a certain distance apart, they have to swivel to different orientations to foveate the same object (specifically, they have to rotate different angles around an axis that is orthogonal to the line connecting the two eyes’ centers, which, for most people, is horizontal). A mathematician or geometrically inclined person would be able to take the fixed distance between the two eyes, the so-called inter-pupillary distance (which you should measure before using VR), and the two eyes’ vergence angles, and from those data triangulate the position of the object currently being foveated, and then adjust both eyes’ optical power to the distance from each eye to that triangulated position to quickly bring that object into sharp focus.

Figure 3: A pair of eyes looking at an object. Left: eyes are parallel, and lenses are accommodated for infinity distance. The object appears outside the fovea (red), and is blurry. Right: the eyes verge inwards to bring the object into their respective foveas, and automatically adjust accommodation, based on their vergence angles, to form a high-resolution and sharp image of the object.

Amazingly, that’s exactly what our eyes do, or rather, what our brain does (see Figure 3). Of course, our brains don’t “know” geometry or triangulation, per se. This coupling between accommodation and vergence, unsurprisingly called accommodation-vergence coupling, is a learned reflex developed during very early childhood. As infants and babys look around their environments, their eyes will naturally foveate on objects at different distances. At that time, the inborn blurriness reflex will adjust their eyes’ lenses until those objects appear sharp. While this is going on, their brains remember, at a neural pathway level, what level of accommodation corresponds to what amount of vergence, and as time progresses, accommodation based on vergence, which is faster than accommodation based on blurriness, will become the predominant mechanism.

And then VR comes along and screws it all up.

As explained in this article, the screens of all currently available VR headsets have a fixed optical distance, typically a few meters away from the user. On the other hand, VR, or at least good VR, does a proper job of simulating vergence. When looking at virtual objects at different virtual distances, the VR user’s eyes foveate and verge towards those objects just as in reality. But, due to the headset’s fixed optical plane, the vergence-based triangulated distance to a virtual object no longer matches the optical distance from which the light comprising the virtual image actually comes, the user’s eyes accommodate to the wrong distance, and the virtual object will appear blurry (see Figure 4). This mismatch is called accommodation-vergence conflict, and it has several ill effects. For one, as already mentioned, it will lead to blurry vision, for example making it hard to read text up close — that’s a problem for current VR, as due to current headsets’ low angular resolution, text isn’t very readable at farther distances, either, unless it is very large.

Based on this explanation, there is a simple trick to clearly see close-by objects in VR: close one eye. Accommodation-vergence coupling requires both eyes to be open; if one is closed, the blurriness-based mechanism takes over — and it isn’t fooled by VR.

Figure 4: A pair of eyes looking at a virtual object in a head-mounted display. The eyes verge towards the virtual position of the virtual object, guided by the stereoscopic imagery on the virtual screen. However, the light comes from farther away, causing blurriness due to wrong accommodation. (Details of HMD lens effects ignored for clarity.)

But the problems go further than that. There is still the other, blurriness-based, focusing mechanism. In real-world vision, the two mechanisms agree with each other. In VR, they fight. Vergence tells the eye to accommodate to one distance, the blurriness receptors tell it to accommodate to another. This can lead to eye strain and headaches, as the eyes’ lenses may rapidly accommodate back and forth trying to make sense of the situation. After a longer period of conflict, the accommodation-vergence reflex might just throw up its hands and give up, causing accommodation-vergence decoupling. At that point the other mechanism takes over, and vision tends to improve: while blurriness avoidance is slower to focus, it is, as I mentioned above, not fooled by VR.

However, when the user takes the headset off after a long VR session, accommodation-vergence coupling does not come back immediately. This means that right after using VR for an extended period of time, users might experience strange-feeling vision problems. They might not be able to focus very well, or the real world might look oddly unreal, or they might have slight double vision. This is a concern, and a reason why it is usually recommended to take a short break from driving or operating heavy machinery or performing brain surgery after using VR. Fortunately, accommodation-vergence coupling does come back after some time, the amount of which varies by person.

Accommodation-vergence conflict is the one remaining aspect of vision that is not simulated by current VR headsets. While it is not as big a deal as simulator sickness induced by poor tracking, high latency, or artificial locomotion, it is a nuisance with potentially problematic short-term after-effects. It is also the main reason that commercial VR HMDs have age limits: it is possible that prolonged use of VR (or other stereoscopic display systems) may interfere with the formation of the a-v coupling reflex in very young children. Since it would be highly unethical to study this scientifically, we will probably never find out for certain. However, it is generally believed that a-v coupling has set in permanently at around two years of age and is elastic afterwards, meaning it will snap back after being temporarily disabled.

The way to solve accommodation-vergence conflict is to create headsets that can present images at different optical distances. This can be achieved in multiple ways: by dynamically changing the distance between the headset’s screen(s) and lenses, essentially inverting the way a camera focuses; by developing flexible lenses that invert the way the eye focuses; by using near-eye light field displays, which mimic the way light arriving at the eye from different distances diverges at different angles; or by true (interference-based) holographic displays, which are a specific form of light field displays.