I am a research computer scientist at the University of California, Davis. My research areas are scientific visualization, particularly in immersive ("virtual reality") environments, human/computer interaction in immersive environments, and 3D computer graphics.
My primary work is software development, from architecture over design to implementation and coding. I am the primary software developer for the UC Davis W.M. Keck Center for Active Visualization in the Earth Sciences (KeckCAVES). Some of my released packages are Vrui (a VR development toolkit), CollaborationInfrastructure (a tele-collaboration plug-in for Vrui), Kinect (a driver suite to capture 3D video from Microsoft Kinect cameras), LiDAR Viewer (a visualization package for very large 3D point clouds), 3D Visualizer (a system for interactive visual analysis of 3D volumetric data), Nanotech Construction Kit (an interactive molecular design program), and SARndbox (an augmented reality sandbox).
I also dabble in VR hardware, in the sense that I take existing custom or commodity hardware components (3D TVs, head-mounted displays, projectors, tracking systems, Wiimotes, Kinect cameras, ...) and build fully integrated immersive environments out of them. This includes a fair share of driver development to support hardware that either doesn't have drivers, or whose vendor-supplied drivers are not up to par.
I’ve been spending all of my time over the last few weeks completely rewriting Vrui‘s collaboration infrastructure (VCI from now on), from scratch. VCI is, in a nutshell, the built-in remote collaboration / tele-presence component of my VR toolkit. Or, in other words, a networked multi-player framework. The old VCI was the technology underlying videos such as this one:
Figure 1: Collaborative exploration of a 3D CAT scan of a microbial community, between a CAVE and a 3D TV with head-tracked glasses and a tracked controller.
A funny thing has been happening for the last few days. There is a link farming or SEO or whatever site out there that is currently doing a deep clone of this blog (I’m obviously not going to provide a link here), but unlike all the other times that’s happened, these guys aren’t just straight-up copying, they’re running the text through what looks like a game of Google Translate Telephone (or a simple synonym replacer, but I’m going to pretend it’s the former). I’m guessing the reason is to obfuscate the source and throw off plagiarism detectors, but the result is unintentionally(?) hilarious.
For a fuller experience, here are two paragraphs from the first article I mentioned above:
“The quick reply is after all that this relies on your mannequin of the headset. However should you occur to have an HTC Vive, view the graphs in footage 1 and a couple of (the opposite headsets behave in the identical method, however the precise numbers differ). These figures present the display decision, in pixels / °, alongside two traces (horizontal and vertical, respectively) that undergo the middle of the suitable lens of my very own Vive. The purple, inexperienced and blue curves present the decision for the purple, inexperienced and blue main colours respectively, this time not on the idea of my very own measurements, however by analyzing the show calibration knowledge measured for every particular person headset on the manufacturing unit after which saved within the firmware.
At this level it’s possible you’ll surprise why these graphs look so unusual, however for that you’ll have to learn the lengthy reply. Earlier than I’m going into that, I need to throw away a single quantity: proper in the course of the suitable lens of my Vive (on pixel 492, 602) the decision for the inexperienced shade channel is 11.42 pixels / °, each horizontal and vertical instructions. When you needed to cite a single decision quantity for headphones, that will be the one I used to be going to, as a result of it’s what you get once you have a look at one thing instantly in entrance of you and much away. As figures 1 and a couple of clearly present, no quantity can inform the entire story.”
Now, having posted this article, I am waiting with bated breath for what kind of hash their cloning bot will make out of it.
Apparently, there were good sales numbers for VR equipment prior to the holiday season, and therefore a host of new VR users are coming in just about now. This meta-post collects a bunch of stuff I’ve written (or presented) in the past that might be of interest to some of those new users. These questions/answers are not hardware-specific, meaning they apply to any current-generation VR system (Oculus Rift, HTC Vive, all the Windows Mixed Reality headsets, PlayStation VR, …), and go beyond basic tech questions such as “how do I plug this in, install drivers, …).
How does VR actually work? (presentation at VRLA ’16 Expo, 25 minutes) In other words, how does a collection of screens and lenses and tracking sensors etc. create the illusion of a virtual space that feels real?
I heard I need to dial in my IPD, or Inter-Pupillary Distance. Do I need to go to (and potentially pay) an optometrist to have it measured, or can I do that myself at home? (This post is also referenced in the presentation I linked above.)
What are the problems with artificial VR locomotion, i.e., moving through a virtual space via a joystick/touchpad/button or other means that does not involve actually walking? (presentation at a VR meet-up, 40 minutes) (This presentation is also referenced in the presentation I linked above.)
There is one other issue for which I do not have a full article, but it’s quite important for new users: VR sickness (aka motion sickness, simulator sickness, …). Today’s VR headsets, at least the ones doing full head tracking (that means Rift/Vive et al., and not Gear VR, Oculus Go, Google Cardboard, …) should not cause VR sickness per se. These days, it is primarily caused by artificial locomotion in games or applications, as I explain in the second presentation I linked above.
The important message is: do not attempt to fight through VR sickness! If you try to stomach it out, it will only get worse. Stop using VR the moment you feel the first symptoms, take a long break, and then try again if you want to continue with the application/game that made you sick. If you try to power through repeatedly, your body might learn to associate sickness with VR, and that might cause you to get sick even when merely thinking about VR, or smelling the headset, or similar triggers. Just don’t do it.
That’s about it; now go ahead and enjoy your shiny new VR systems!
Want to Know More?
Here are a couple of other, more hardware-specific, topics:
How exactly does Oculus’s Constellation tracking system work? A deep dive into the Oculus Rift DK2’s camera tracking system — which works basically the same as the commercial Rift’s, in three posts:
Good news, everyone! This here blog, Doc-Ok.org, hit 1,000,000 (that’s one million) total page views early in the morning of Christmas Day, 12/25/2018, just about six years and four months after I started it.
I have no idea whether that’s something to be embarrassed or proud of, given the narrow range of topics I’m covering and the niche audience I’m targeting. Either way, it’s a nice, round number.
It’s been more than two years since the last time I posted set-up instructions for Vrui and HTC Vive, and a lot has changed in the meantime. While Vrui-5.0 and its major changes are still not out of the kitchen, the current release of Vrui, Vrui-4.6-005, is stable and works very well with the Vive. The recent demise of our CAVE, and our move towards VR headsets until we figure out how to fix it, have caused a lot of progress in Vrui’s set-up and user experience. The rest of this article contains detailed installation and set-up instructions, starting from where my previous step-by-step guide, “An Illustrated Guide to Connecting an HTC Vive VR Headset to Linux Mint 19 (“Tara”),” left off.
If you use a Linux distribution that is not Ubuntu-based, such as my own favorite, Fedora, or another desktop environment such as Gnome Shell or Cinnamon, you will have to make some adjustments throughout the rest of this guide.
This guide also assumes that you have already set up your Vive virtual reality system, including its tracking base stations, and that your Vive headset is connected to your PC via HDMI and USB (I will publish a detailed illustrated guide on that part soon-ish). Continue reading →
Running Vrui-based applications in glorious VR on an HTC Vive head-mounted display requires some initial set-up before Vrui itself can be installed and configured. This step-by-step guide will build upon an already-installed Linux operating system with high-performance graphics card drivers, specifically upon the current (as of 12/17/2018) version 19, code-named “Tara,” of Linux Mint, one of the most popular and user-friendly Linux distributions. This guide picks up right where the previous one in this series, “An Illustrated Guide to Installing Linux Mint 19 (“Tara”),” left off.
If you did not follow that guide, this one assumes that you have a “VR ready” or “gaming” PC with a powerful Nvidia GeForce graphics card, an installation of the 64-bit version of Linux Mint 19 (“Tara”) with the MATE desktop environment, and the recommended proprietary Nvidia graphics card driver. And an HTC Vive VR headset, of course.
Graphics Card Driver Set-up
Using a Vive headset with Vrui requires a change to the Nvidia graphics card driver’s configuration. Nvidia’s driver scans connected display devices for known VR headsets, and hides detected headsets from the desktop environment. This does make sense, as headsets are not standard monitors, and it would be awkward if windows or dialogs were to show up on a headset’s display. That said, here’s one relatively large quibble: headset filtering should happen earlier during the boot sequence, not just when the graphics card driver is loaded. As it is, headsets are still enumerated during boot, meaning that boot screens, BIOS menus, boot menus, etc. often show up on the headset, causing real problems. Anyway, carrying on.
Unfortunately for Vrui, there is currently no way to activate a hidden headset from inside an OpenGL-based VR application. For the time being, this means headset filtering in the driver needs to be disabled. To do so, open a terminal window (click on the terminal icon in the panel along the bottom screen edge, or right-click anywhere on the desktop and select “Open in Terminal” from the pop-up menu), enter exactly the following command into it (also see Figure 1) and press the Enter key (the $ sign indicates the terminal’s input prompt; don’t type it):
The first step towards installing any Vrui-based software, including the Augmented Reality Sandbox, is installing some version of the Linux operating system on a new computer, which might sound like a daunting proposition to those who have never done that kind of thing before, but is actually very straightforward. This guide will be using the current (as of 12/17/2018) version 19, code-named “Tara,” of Linux Mint, one of the most popular and user-friendly Linux distributions.
As this guide is geared towards installing and running Vrui-based 3D graphics applications, it assumes that the computer onto which Linux is to be installed is some type of “gaming” or “VR ready” PC, containing an Nvidia GeForce graphics card. The exact model of graphics card, as well as the exact model of CPU, amount of main memory, and hard drive size are not really important (that said, to run 3D graphics applications effectively, the PC should have a recent CPU, at least 4GB of main memory, and at least 60GB of hard drive space). While AMD/ATI graphics cards are otherwise perfectly serviceable, they have traditionally had inferior Linux driver support, and therefore Vrui and Vrui applications have not been tested on them in quite a while. In other words, we do not recommend them for these purposes.
Before installing Linux, one needs to download an installation image for one’s chosen Linux distribution and flavor, and copy it to an installation medium, like a CD/DVD or USB stick. For this guide, we will be using the 64-bit version of Linux Mint 19 (“Tara”), with the MATE desktop environment. The page in the preceding link offers a 1.9GB disk (“iso”) image via a wide selection of download sites all over the world. Click the link for the site that is located most closely to you, and wait for the download to finish.
If the installation medium created in the previous step is a USB drive, plug it into a USB port on the new computer before turning it on for the first time. If it is a CD/DVD, turn the computer on first, and then insert the medium as quickly as possible.
The next step is to tell the computer to boot from the installation medium instead of from its internal hard drive. There is usually a key that needs to be pressed shortly after powering on the computer; typically either the “Delete” key to enter the computer’s BIOS, or “F8” to enter a boot menu. If this does not work on the first try, and the computer “hangs” or boots into whatever operating system was previously installed on it, don’t wait until it finishes booting — just turn it right back off (or press the reset button if it has one) and try again. We are going to erase any previous operating system anyway, so there’s no harm. However, do wait for about 20-30 seconds between turning the computer off and on again to avoid any danger of electrical damage. If neither the “Delete” nor “F8” keys work, look in the computer’s manual or online for the correct key sequence. Amazingly, finding the correct key to boot from the installation medium is by far the most difficult step of installing Linux Mint.
Once a BIOS screen or boot menu show up, select to boot from the installation medium. From there, it will take a few seconds to boot into a “live” Linux Mint environment, see Figure 1.
Figure 1: Linux Mint’s “live” installation environment.
Computer reviews aren’t my thing, but for this one I had to make an exception. My 3.5 year old laptop, the HP Spectre x360 I had scored as swag at the 2015 Microsoft Build conference, suddenly died a few months ago. I had taken a liking to that thing, so when I had to leave for a conference in early November, and realized I should probably bring a laptop with me, I decided to replace it with the current version of the same model. Fortunately they had one in stock at my neighborhood Best Buy (alas, only the silver one and not the pretty black and gold one), so I was able to pick it right up.
I then had the bad idea to search online for Linux support on the x360 after already having bought it, and was dismayed by what I found. A lot of people reported poor performance, too-hot-to-handle operating temperatures, and very poor battery life. Not having much of a choice at that point, I decided to go ahead anyways and install Fedora 28 on it, the then-current release of my go-to Linux distribution. Long story short: installation was a breeze, everything worked out-of-the-box, performance is great, the laptop runs barely warm, and battery life is awesome (so awesome, in fact, that I initially thought the readings were wrong). In order to provide a counter-narrative to those other reports, this is my experience of installing and running Linux on a 2018 HP Spectre x360. Continue reading →
I wrote an article earlier this year in which I looked closely at the physical display resolution of VR headsets, measured in pixels/degree, and how that resolution changes across the field of view of a headset due to non-linear effects from tangent-space rendering and lens distortion. Unfortunately, back then I only did the analysis for the HTC Vive. In the meantime I got access to an Oculus Rift, and was able to extract the necessary raw data from it — after spending some significant effort, more on that later.
With these new results, it is time for a follow-up post where I re-create the HTC Vive graphs from the previous article with a new method that makes them easier to read, and where I compare display properties between the two main PC-based headsets. Without further ado, here are the goods.
The first two figures, 1 and 2, show display resolution in pixels/°, on one horizontal and one vertical cross-section through the lens center of my Vive’s right display.
Figure 1: Display resolution in pixels/° along a horizontal line through the right display’s lens center of an HTC Vive.
Figure 2: Display resolution in pixels/° along a vertical line through the right display’s lens center of an HTC Vive.
Back in the olden days, in the summer of 1996 to be precise, I was a computer science Master’s student at the University of Karlsruhe, Germany, about to take the oral exam in my specialization area, 3D computer graphics, 3D user interfaces, and geometric modeling. For reasons that are no longer entirely clear to me, I decided then that it would be a good idea to prepare for that exam by developing a 3D rendering engine, a 3D game engine, and a game, all from scratch. What resulted from that effort — which didn’t help my performance in that exam at all, by the way — was “Starglider Pro:”
In the mid to late 80s, one of my favorite games on my beloved Atari ST was the original Starglider, developed by Jez San for Rainbird Software. I finally replaced that ST with a series of PCs in 1993, first running DOS, and later OS/2 Warp, and therefore needed something to scratch that Starglider itch. Continue reading →