It has been way too long that I have publicly released a new version of the Vrui VR toolkit. The main issue was that I had been chasing evolving hardware, from the Oculus Rift DK1 to the Oculus Rift DK2, and now to HTC’s Vive. During that long stretch of time, I was never happy with the state of support of any of these devices.
That’s finally changed. I have been working on full native support for HTC’s Vive head-mounted display over the last few months (with the first major break-through in May), and I think it’s working really well. There are still a lot of improvements to make and sharp edges to sand off, but I feel it is worthwhile releasing the software as it is now to get some early testing done. So without much further ado, here is Vrui-4.2-004.
Figure 1: Vrui’s ClusterJello toy application running on an HTC Vive head-mounted display. Recorded using a second-generation Microsoft Kinect camera (Kinect-for-Xbox-One).
A lot has changed since the last major public release, Vrui-3.1-001, almost three years ago (there was a minor update last November, but still…). I’ll write up detailed release notes at some later time, but for now, let’s do a quick run-down on how to download and install Vrui, and, most importantly I assume, how to get it to work with the Vive.
Vrui uses the lower levels of Valve’s OpenVR SDK and SteamVR run-time to receive 6-DOF tracking data and controller events from the Vive and its Lighthouse tracking system. Before installing Vrui, one therefore has to install the Linux versions of Steam and SteamVR.
How to get Steam depends on the Linux distribution. On Fedora, there is a “steam” package in the RPMFusion repositories, so getting it is as simple as
$ sudo dnf install steam
Ubuntu and Linux Mint most probably have Steam packages as well, but I don’t know right now what they’re called. Hit the Googles.
After Steam is installed, start it, wait for updates to complete, and then log in with a valid Steam account. Next, download SteamVR from the “Tools” menu. When that’s complete, you’re done with Steam for the moment. Don’t try starting SteamVR, it doesn’t work right now anyway.
Downloading and Installing Vrui
Vrui has a handy installation script, one version for Fedora, and one for Ubuntu/Linux Mint. Here is how to get and run it from a terminal window:
$ cd $ wget http://idav.ucdavis.edu/~okreylos/ResDev/Vrui/Build-Ubuntu.sh $ bash Build-Ubuntu.sh
The script (if you’re on Fedora, replace Build-Ubuntu.sh with Build-Fedora.sh) will download and install a bunch of tools and libraries (such as g++ or libjpeg) from your distribution’s repositories, and then download, build, and install Vrui itself. It might ask for your password to make system-level changes at a few points — specifically, to install dependency packages, to install Vrui into /usr/local, its default installation directory, and to install custom device permission rules in /etc/udev/rules.d (see below). The entire process usually takes only a few minutes. The entire Vrui source package is downloaded into ~/src/Vrui-<version>-<release>, e.g., ~/src/Vrui-4.2-004. If you want to uninstall Vrui later, run from a terminal:
$ cd ~/src/Vrui-4.2-004 $ sudo make uninstall
Non-Standard SteamVR Locations
The Build-* scripts assume that SteamVR is installed somewhere inside the current user’s home directory, and contain code to find it automatically. If they fail, or if SteamVR is installed somewhere else, support for Vive tracking will be disabled (this is indicated towards the end of the configuration messages printed when building Vrui).
If SteamVR support was disabled, you can enable it manually by providing the path to SteamVR’s root directory when running “make” and “make install”:
$ cd ~/src/Vrui-4.2-004 $ make STEAMVRDIR=/path/to/SteamApps/common/SteamVR $ sudo make STEAMVRDIR=/path/to/SteamApps/common/SteamVR install
This will only rebuild the OpenVRHost device driver module, and be very quick.
Installing Udev Rules
Talking to non-standard devices such as the Vive usually requires administrator privileges. That’s annoying and unnecessary, and Vrui contains rules to relax the permissions on such devices for the user currently logged in at the console. These rules are installed automatically by the Build-* scripts, but if you built Vrui manually, you install them by running from a terminal:
$ cd ~/src/Vrui-4.2-004 $ sudo make installudevrules
(and enter your password if prompted). This will create a new file, 69-Vrui-devices.rules, in /etc/udev/rules.d. If you do not install these rules, you need to run the RunViveTracker.sh script as root, e.g., through sudo. This is not recommended.
After installing the rules, temporarily unplug the power connector to the Vive’s breakout box, and plug it back in. This will apply the new rules to it.
Testing the Tracking Driver
During installation, Vrui created a helper script, RunViveTracker.sh, and installed it into /usr/local/bin. Vrui’s actual tracking driver is called VRDeviceDaemon, and it is also installed in /usr/bin/local, but do not run VRDeviceDaemon directly. Loading OpenVR’s Lighthouse driver module requires a special environment, and RunViveTracker.sh sets that up. Open a new terminal window, and run:
If all goes well, this will print a rather large volume of debug and diagnostic messages, including a ton of “ioctl (GFEATURE): Broken pipe” messages that are safe to ignore, and then just sit there. To stop the tracking driver later, press Ctrl-C. To test tracking, open another terminal window and run:
$ DeviceTest localhost:8555 -t 0 -p
DeviceTest is a Vrui utility that connects to a running tracking server (called VRDeviceDaemon, and listening to TCP port 8555 on the local machine) and prints tracking and button/valuator (analog axis) data. When run as above, it will continuously print the position of the Vive HMD. To exit from DeviceTest, press any key.
To test controller tracking, turn on one or both of the controllers, and run
$ DeviceTest localhost:8555 -t 1 -p -b
which will print the position of the first controller, and the states of both controllers’ buttons.
Setting Up the Tracking Environment
Before you can run Vrui applications using the Vive, you need to set up the tracking environment. This is basically the same step as “room-scale setup” in SteamVR. From a terminal, run
$ RoomSetup Vive
This starts a graphical configuration utility. In the beginning, it will show some random environment definition, and a dialog window with multiple pages. Room setup proceeds in a sequence of steps:
On the “Controller” page, select your controller type from the drop-down menu. You’ll probably want “Vive.” This defines measurement offsets for the later steps.
The “Floor Plane” page sets up the environment’s “up” direction and its floor plane. First, push the “Reset” button, and then place your controller on the ground in the middle of your VR space, so that the tracking “donut” touches the ground, and the buttons and touchpad face up. Then press any button on the controller. The dialog window will update to show the position of the new center point, and a default “up” direction of 0.000, 1.000, 0.000. In most cases, that’s all it takes.
Should you later notice that virtual environments appear tilted, you can calibrate the “up” direction by re-doing the floor plane step, and measuring additional points from around your floor by touching the floor with the front of the tracking donut, and pressing a controller button. This will calculate the plane that best fits the points you collected, and calculate the “up” direction such that it is orthogonal to that plane.
Vrui needs a notion of a “forward direction,” which is the typical direction you’re facing when standing in your VR space. To define it, point a controller in the direction you want, and press a button.
The final setup step is to define the boundaries of your physical VR space. While VR applications are running, Vrui will continuously check the positions of the HMD and all controllers against those boundaries, and show a “screen saver” grid when any come close.
To define a polygon, first press the “Reset” button, and then walk around your VR space and touch every corner of the accessible space with the front edge of one controller’s tracking donut, and press some button on the controller. You can see the boundary polygon updated in RoomSetup’s display window.
You can zoom and pan inside RoomSetup’s window to inspect parts of your polygon. Roll the mouse wheel to zoom, and press and hold the “z” key, and then move the mouse without pressing any buttons, to pan. Should the display ever get out of whack, press Win+Home, or right-click the mouse and select “View” -> “Reset View” from the Vrui System Menu, to reset it.
Saving the Current Setup
You can go back and fix other components of the setup, such as floor plane or forward direction, any time during setup. Once you are satisfied with your setup, save it by pressing the “Save Layout” button, and exit RoomSetup by closing the window, or pressing Esc.
Turning on the Vive’s Display
The Vive’s display should have turned itself on when you plugged it in. If it didn’t, open your display control panel (if you have an Nvidia graphics card, run nvidia-settings), and set it up manually. Make sure that you select a resolution of 2160×1200 pixels, and a refresh rate of 90Hz.
Vertical Retrace Synchronization
It is very important that rendering to a head-mounted VR display is synchronized with the display’s vertical retrace period, otherwise there will be tearing and horrible judder. How to enable this depends on the graphics driver used, and the way how the Vive’s display is set up in the X server. An easy way to check whether synchronization is working is to put on the HMD — unless everything is smooth, it’s not synchronized. An additional way is to press Win+Pause while a VR display window has input focus. This will collect frame rate information and print a summary when Win+Pause is pressed again. The printed frame rate should be close to 89.5 Hz (the Vive’s true frame rate).
If the Vive is connected as a secondary X screen, vertical retrace synchronization should work out-of-the-box.
If the Vive is connected as an extended display (“TwinView” in Nvidia parlance), there may need to be some additional steps. Using Nvidia’s proprietary drivers — which I am using on all my computers — there is an environment variable that needs to be set to tell OpenGL to which display to synch (OpenGL could figure this out by itself based on the position of the window asking for synch, but Nvidia never implemented that). First, find out the display connector name of the Vive’s display, most easily through nvidia-settings. This is going to be something like DFP-1 or DFP-2. Then, before running a Vrui application from a terminal window, enter
$ export __GL_SYNC_DISPLAY_DEVICE=DFP-1
into that terminal window, replacing DFP-1 with the actual name from nvidia-settings.
That was basically it. You can now run any Vrui application in the Vive. Vrui contains a number of small example programs, mostly meant to get developers started on creating Vrui applications. The Build-* scripts automatically built and installed them, but if you built Vrui manually, go into the ExamplePrograms directory and run make:
$ cd ~/src/Vrui-4.2-004/ExamplePrograms $ make INSTALLDIR=/usr/local $ sudo make INSTALLDIR=/usr/local install
Then run one of them like this:
$ ClusterJello -rootSection Vive
The -rootSection Vive option tells Vrui to run in the Vive. If you leave it off, Vrui will run in desktop mode (you can make Vive mode the default later).
ClusterJello is a silly toy application. It shows a block of lime green Jell-O on the floor. To interact with the block, press some button on a controller, and select “Dragger” -> “6-DOF Dragger” from the tool selection menu that pops up. Then reach down and grab the block by pressing and holding the same button.
You can change the behavior of the Jell-O by pressing and holding the controller’s menu button, and selecting “Show Settings Dialog.” Use the menu button to interact with that dialog, and press the “X” button to dismiss it.
To exit from the program, select “Quit Program” from the Vrui system menu.
Note: Do not run the “Jello” application. Virtual Jell-O comes in three flavors, to highlight different aspects of the Vrui VR toolkit. Jello is a very simple application that will run the Jell-O simulation inside the application’s main loop, which means it will not hit target frame rate in most cases, and look and feel very choppy. ClusterJello is a more advanced version of the same that runs the Jell-O simulation in a background thread, so it will always run at the display’s native frame rate. SharedJello is a distributed application that allows multiple users, each from their own VR environment, to play together with the same block of Jell-O. It is client/server based, with SharedJelloServer being another Vrui example program.
This is a preliminary release of Vrui with Vive support. There are still a lot of improvements to be made. Latency, smoothness, tracking stability, etc. are not necessarily up to snuff. Probably most importantly, Vrui does not yet receive notifications when the headset loses tracking. Meaning, should this happen, the VR display will get very wonky. I’m working on that part.
Figure 2: Another Vrui application running on an HTC Vive head-mounted display. Recorded using a second-generation Microsoft Kinect camera (Kinect-for-Xbox-One).