Text entry in virtual environments is one of those old problems that never seem to get solved. The core issue, of course, is that users in VR either don’t have keyboards (because they are in a CAVE, say), or can’t effectively use the keyboard they do have (because they are wearing an HMD that obstructs their vision). To the latter point: I consider myself a decent touch typist (my main keyboard doesn’t even have key labels), but the moment I put on an HMD, that goes out the window. There’s an interesting research question right there — do typists need to see their keyboards in their peripheral vision to use them, even when they never look at them directly? — but that’s a topic for another post.

Until speech recognition becomes powerful and reliable enough to use as an exclusive method (and even then, imagining having to dictate “for(int i=0;i<numEntries&&entries[i].key!=searchKey;++i)” already gives me a headache), and until brain/computer interfaces are developed and we plug our computers directly into our heads, we’re stuck with other approaches.

Unsurprisingly, the go-to method for developers who don’t want to write a research paper on text entry, but just need text entry in their VR applications right now, and don’t have good middleware to back them up, is a virtual 3D QWERTY keyboard controlled by a 2D or 3D input device (see Figure 1). It’s familiar, straightforward to implement, and it can even be used to enter text.

Figure 1: Guilty as charged — a virtual keyboard in the Vrui toolkit, implemented as a GLMotif pop-up window with rows and columns of buttons.

The problem is that a straight-up virtual keyboard is, in my opinion, pretty much the worst possible way of going about it. While the old legend about QWERTY being specifically designed to slow down typists is mostly that, a legend, the truth is that keyboards rely on ten fingers and tactile feedback. In VR, where the current reality is no tactile feedback and at most two 3D devices tracking the user’s hands instead of fingers, users cannot touch-type, but instead have to virtually pound the virtual keyboard with their fists. In other words, virtual keyboards are an implementation of hunt&peck typing: find virtual key with input device by visually scanning the UI (“muscle memory” doesn’t work), press button on device to press virtual key, repeat. Granted, that can actually be quite fast, but it is very repetitive and annoying. Having to type extended text by pressing the same real button for each character in turn is a one-way ticket to carpal tunnel town. On top of that, virtual keys have to be fairly large — there is a direct relationship between the size of a button and how quickly it can be selected — and as a result, virtual keyboards take up a lot of display real estate. That large size leads to the need for large movements across the keyboard when typing, and then Gorilla Arm says hi, too.

So what else is out there? A lot, actually, and I’m not even aware of most of it because text entry isn’t my main focus. It just so happens that I think Vrui has a pretty neat way of entering text, and because I got involved in a discussion about text entry, I made a video about it:

The method shown in this video is called Quikwriting or Quikwrite (not QuikWrite as in the video title, my mistake), and it was developed by Ken Perlin (yes, that Oscar-winning Ken Perlin) and patented by New York University. The three main benefits of Quikwrite over virtual keyboards are: (1) entering text of any length is a single stroke, without the need to press/release a button or otherwise signal an event for each character, (2) the Quikwrite UI is very compact and naturally see-through, and (3) the UI can be used without having to look at it at all times.

The main reason why Quikwrite is fast and can be used almost without looking at the UI is that characters are not selected by locating a small rectangle in a sea of similar rectangles, but by moving an input device in one of only eight directions. Specifically, a character is selected by moving the input device from the UI’s center into one of eight major regions arranged around the center along the edges and corners of a square, then potentially from that major region into an adjacent minor region, and then back to the center, which fluidly links to the sequence for the next character.

How fast is it? I haven’t done any formal studies, but it’s faster than using a virtual keyboard. For me, it’s not much faster at this point, but then I’ve used keyboards for 30 years, and Quikwrite for maybe 30 minutes total while developing and debugging it in Vrui  (yeah, I don’t use VR applications to write novels). It obviously takes some training to commit the position of characters and special symbols to muscle memory, but after extended use I expect it to be a lot faster. Speed isn’t even the main reason why I like Quikwrite: for me, it’s not having to play staccato on an input device button. The UI being compact is a nice benefit, too.

But back to the video: the fluid single-stroke nature of Quikwrite enables using it even when there is no button at all on the input device driving it. Such as when using gaze direction for hands-free text entry. With a virtual keyboard or similar methods, there would have to be some way to select the key you’re currently looking at, such as holding gaze while a timer runs out. That’s a problem because the timer can neither be too short, or there will be spurious characters, nor too long, or typing will be slow (and annoying). In Quikwrite, users can type as slowly or quickly as they want (or can), and pause at any time, without having to perform any special actions.

While we’re on the topic: Quikwrite lends itself well to implementation via thumb stick, too. Always start and end characters from the center? Check. Only need to move in one of eight directions? Check. Once in a major region, the minor region is along the same square edge? Check. I really need to implement that (update: I did). And what about Valve’s way of entering text using a game controller? It’s a similar idea, but I find it a bit more clunky. The character positions on the “flower” take just as long to learn, but they’re alphabetic, not optimized by frequency analysis, and there’s still that button mashing going on. In the end it’s going to be a matter of taste, as always in VR.

Finally: what about Swype (pretty much the only other single-stroke method I’m aware of)? It supposedly works well on smart phones, but I’m not so sure about it in VR. It requires a large UI and constant visual focus just like a virtual keyboard, and I am guessing it only works as well as it does due to having a physical surface against which to slide one’s finger. It also needs to be backed by a dictionary to decide which characters have actually been entered, so that’s a bit of a drag.