Just the other day, I jumped on the wayback machine and posted an article about our work in immersive tele-collaboration, featuring research (and a video) from about four years ago. The shame! I figured it would be excusable that one time, and I would never do it again. Oh well, here we go.

Keeping VR users from hurting themselves

… or their expensive VR equipment.

It’s a pretty big deal. Virtual Reality, especially its head-mounted implementation, is quite good at overriding its users’ sense of place and space. “Presence,” or the feeling of bodily being in a place where one knows to be not, is a powerful and compelling experience, but it has a downside: users experiencing it lose touch with their real physical environments. Exhibit A: Figure 1 (granted, there are some concerns that the following video clip was staged, but let’s pretend it’s for reals).

Figure 1: When instinct takes over. Source: imgur

To prevent this kind of thing from happening — at least in most cases — Valve implemented a system called “Chaperone” into the SteamVR run-time framework that runs their and HTC’s Vive VR headset (and potentially other headsets, through Valve’s OpenVR layer).

At its core, Chaperone is a combination user interface / software / API system that brings relevant parts of users’ physical environments into their virtual environments. Concretely, during environment setup, users can sketch out the horizontal boundaries of their “play space” by tracing them with a tracked controller. At run-time, Chaperone constantly monitors users’ head and controller positions, and alerts users when they come close to the defined boundaries by drawing a wireframe grid representing said boundaries. It’s a highly important, and very clever, system.

So important and clever, in fact, that Valve recently filed a patent about it. Here are the three claims made in that patent application:

We claim:

1. A method for warning a user of a head-mounted display of potential collisions with real-world obstacles, comprising the steps of: defining the location of said obstacles relative to a reference point; continually monitoring the location of said user relative to said reference point; performing a first detection to determine when the distance between said location of said user relative to the location of one of said obstacles is smaller than a predetermined alarm threshold; and in response to said first detection, displaying a visual alarm on said head-mounted display to provide an indication to said user of a potential collision with one of said obstacles.
2. The method of claim 1, wherein said location of said user is determined by continually monitoring the location of a portion of said head-mounted display.
3. The method of claim 1, further comprising continually monitoring the location of an object held by said user, performing a second detection to determine when the distance between said location of said object relative to the location of one of said obstacles is smaller than said predetermined alarm threshold, and, in response to said second detection, displaying a visual alarm on said head-mounted display to provide an indication to said user of a potential collision between said object and one of said obstacles.

Given the importance of a Chaperone-like system for VR, especially “room-scale VR,” where users can walk around freely inside a limited physical space, this patent would grant Valve a powerful position in the emerging consumer VR market.

And that’s why I had to fire up the time machine again, and dust off an even older video of mine:

Figure 2: VR Quake 3 Arena Map Viewer, uploaded to YouTube on February 26th, 2007 (but filmed about a year earlier than that, and showing software developed in mid-2005 — holy cow, I still had some hair back then!).

I’d like to direct viewers’ attentions to time point 0:52 in the video. It shows a green grid popping up as the user (me) moves his hand-held controller too close to one of the CAVE’s rear-projected walls. This (very brief and unintentional) part of the video shows two things (and implies a third):

1. A user interface to define the boundaries of a physical space (not shown, but implied).
2. A system that continually monitors the position of a user’s head, and the position(s) of one or more hand-held controllers.
3. A system that alerts a user of close proximity of the user’s head or hand-held controller(s) to previously defined physical space boundaries, by displaying a virtual 3D representation of said boundaries (via a green wireframe grid).

There are a few differences between this system, which I prosaically call “screen protector” or “screen saver,” and Chaperone:

1. The grid is green instead of cyan (at least by default).
2. The grid pops in and out of existence at full opacity instead of smoothly increasing opacity depending on distance.
3. The screen protector only draws the part(s) of the boundary that is/are about to be violated, not the entire boundary.
4. The concrete VR system shown in the video is a projection-based VR environment, not a head-mounted environment.
5. The user interface to define physical boundaries is based on editing configuration files, not tracing out the physical space using a tracked input device.

I argue that differences 1-3 are rather minor, and that difference 4 is simply due to this particular video having been filmed in a projection-based environment instead of a head-mounted one. As the screen protector is implemented as a module of the underlying Vrui VR toolkit, it is entirely hardware-agnostic. The exact same VR toolkit, and the exact same applications running on this toolkit, also work for head-mounted displays, simply by changing some (OK, quite a few) parameters in a run-time configuration file. Figure 3 shows the same toolkit and application, running on an Oculus Rift DK1. Alas, the screen protector didn’t make an appearance in that video — it was continually monitoring for boundary violations, but there were none.

Figure 3: Same toolkit and application as Figure 2, running on an Oculus Rift DK1. Video filmed on August 14th, 2013.

The only remaining difference between Chaperone, as defined by the three claims in Valve’s patent application, and the screen protector, is the ingenious and very user-friendly way Chaperone allows users to trace out their play spaces. I wish I had had the need to come up with something like that, and the creativity to then actually come up with it.

So the bottom line is: please, Valve, if you get this patent granted, don’t sue me for infringement for the use of the screen protector in the Vrui VR toolkit. That would be mean; I didn’t steal it from you. And dear reader: put down the torches and pitchforks. Please don’t misread this article as trying to imply that Valve stole Chaperone from me. I am convinced they did not. My screen protector and Chaperone are independent inventions of the same obvious solution to an obvious and very important problem. My run-in with the problem just predated theirs, to the best of my knowledge.

Closing thought: why did I name this module “screen saver?” Because it literally saves screens, that’s why. Duh. I got hands on my first VR environment, a Fakespace Immersive Workbench, in April 1998. It had a single back-projected approximately 6’x4.5′ screen, angled like a drafting table. The user was head tracked, was wearing two position-tracked Fakespace Pinch Gloves, and was holding a position-tracked Polhemus stylus with a single button. About a few months into using the workbench, I tried reaching for a virtual object virtually located behind the real screen, and banged the stylus (which had a sharp metal tip) right into the screen. It left a permanent scratch, fortunately in a not-so-noticeable location. The next day I implemented the screen saver.