How to Track Glowing Balls in 3D

Posted on by
6

Note: I started writing this article in June 2017, because people kept asking me about details of the PS Move tracking algorithm I implemented for the video in Figure 1. But I never finished it because I couldn’t find the time to do all the measurements needed for a thorough error analysis, and also because the derivation of the linear system at the core of the algorithm really needed some explanatory diagrams, and those take a lot of work. So the article stayed on the shelf. I’m finally publishing it today, without error analysis or diagrams, because people are still asking me about details of the algorithm, more than four years after I published the video. 🙂

This one is long overdue. Back in 2015, on September 30th to be precise, I uploaded a video showing preliminary results from a surprisingly robust optical 3D tracking algorithm I had cooked up specifically to track PS Move controllers using a standard webcam (see Figure 1).

Figure 1: A video showing my PS Move tracking algorithm, and my surprised face.

During discussion of that video, I promised to write up the algorithm I used, and to release source code. But as it sometimes happens, I didn’t do either. I was just reminded of that by an email I received from one of the PS Move API developers. So, almost two years late, here is a description of the algorithm. Given that PSVR is now being sold in stores, and that PS Move controllers are more wide-spread than ever, and given that the algorithm is interesting in its own right, it might still be useful. Continue reading →

Visual Simulation of the Resolution of VR Headsets

Posted on by
3

While I was writing up my article on the PlayStation VR headset’s optical properties yesterday, and specifically when I made the example images for the sub-section about sub-pixel layout comparing RGB Stripe and PenTile RGBG displays, it occurred to me that I could use those images to create a rough and simple simulator to visually evaluate the differences between VR headsets that have different resolutions and sub-pixel layouts.

The basic idea is straightforward: Take a test image that has some pixel count, for example WxH=640×360 as the initial low-resolution full-RGB picture seen in Figure 1. If you then blow up that image to fill a monitor that has the same aspect ratio (16:9) and some diagonal size D, the resolution of that image in terms of pixels per degree depends both on D and the viewer’s distance from the monitor Y: the larger the ratio Y/D, the higher is the image’s resolution as seen by the viewer. In detail, the formula for distance Y to achieve a desired resolution R in pixels/° is:

Y = (D / sqrt(W*W + H*H))/(2*tan(1/(2*R)))

where W and H are in pixel units, R is in pixels/°, and D is in some arbitrary length unit (inch, meter, parsec,…). Y will end up being in the same unit as D.

If a viewer then positions one of their eyes at a distance of Y from the center of the monitor and closes the other one, the resolution of the image on the monitor will be R. In other words, if R is the known resolution of some VR headset, the image on the monitor will look the same resolution as that VR headset.

There is one caveat: when looking at a flat monitor, the resolution of the displayed image increases away from the monitor’s center, while in a VR headset, the resolution generally decreases away from the center direction (see this article for reference). Meaning, for a correct evaluation, the viewer has to focus on the area in the center of the monitor. Unfortunately there is no easy way to simulate resolution drop-off using a flat monitor, at least not while also simulating sub-pixel layout.

Simulation Procedure

Here’s what you need to do: Continue reading →

Field of View and Resolution of the PlayStation VR Headset

Posted on by
2

It has been a very long time since I did the original optical measurement of then-current VR headsets. I have owned a PlayStation VR headset (PSVR from now on) for almost a year now, and I finally got around to measuring its optical properties in the same way. I also developed a new camera calibration algorithm (that’s a topic for another post), meaning I am even more confident in my measurements now than I was then.

One approach to measuring the optical properties of a VR headset, which includes measuring its field of view, its resolution in pixels/°, and its lens distortion correction profile, is to take a series of pictures of the headset’s screen(s) through its lenses using a calibrated wide-angle camera. In this context, a calibrated camera is one where each image pixel’s horizontal and vertical angles away from the optical axis are precisely known.

If one then displays a test pattern that lets one identify a particular pixel on the screen, one can measure the viewer-relative angular position of that pixel in the camera image, which is all the information needed to generate the projection matrices and lens distortion correction formulas that are essential to high-quality VR rendering.

Without further ado, here is a series of 7 images taken with the camera lens at increasing distances from the headset’s right lens (Figures 1-7, and yes, I forgot to clean my PSVR’s lens). The camera was carefully positioned and aligned such that it was sliding back along the lens’s optical axis, and looking straight ahead. The first image was captured with an eye relief value of 0mm, meaning that the camera lens was touching the headset’s lens. The rest of the images were captured with increasing eye relief values, or lens-lens distances, of 5mm, 10mm, 15mm, 20mm, 25mm, and 30mm: Continue reading →

KeckCAVES On Mars, pt. Oh-I-lost-count

Posted on by
Reply

Last weekend, we had yet another professional film crew visiting us to shoot video about our involvement in NASA’s still on-going Mars Science Laboratory (MSL, aka Curiosity rover) mission. This time, they were here to film parts of an upcoming 90-minute special about Mars exploration for the National Geographic TV channel. Like last time, the “star” of the show was Dawn Sumner, faculty in the UC Davis Department of Earth and Planetary Sciences, one of the founding members of KeckCAVES, and member of the MSL science team.

Unlike last time, we did not film in the KeckCAVES facility itself (due to the demise of our CAVE), but in the UC Davis ModLab. ModLab is part of an entirely different unit — UC Davis’s Digital Humanities initiative — but we are working closely with them on VR development, they have a nice VR environment consisting of two HTC Vive headsets and a large 4.2m x 2.4m screen with a ceiling-mounted ultra-short throw projector (see Figure 1), their VR hardware is running our VR software, and they were kind enough to let us use their space.

Figure 1: Preparation for filming in UC Davis’s ModLab, showing its 4.2m x 2.4m front-projected screen and ceiling-mounted ultra-short throw projector, and two Lighthouse base stations.

The fundamental idea here was to use several 3D models, created or reconstructed from real data sent back either by satellites orbiting Mars or by the Curiosity rover itself, as backdrops to let Dawn talk about the goals and results of the MSL mission, and her personal involvement in it. Figure 1 shows a backdrop in the real sense of the word, i.e., a 2D picture (a photo taken by Curiosity’s mast camera) with someone standing in front of it, but that was not the idea here (we didn’t end up using that photo). Instead, Dawn was talking while wearing a VR headset and interacting with the 3D models she was talking about, with a secondary view of the virtual world, from the point of view of the film camera, shown on the big screen behind her. More on that later. Continue reading →

New Adventures in Hi-Fi

Posted on by
Reply

I’ve been spending all of my time over the last few weeks completely rewriting Vrui‘s collaboration infrastructure (VCI from now on), from scratch. VCI is, in a nutshell, the built-in remote collaboration / tele-presence component of my VR toolkit. Or, in other words, a networked multi-player framework. The old VCI was the technology underlying videos such as this one:

Figure 1: Collaborative exploration of a 3D CAT scan of a microbial community, between a CAVE and a 3D TV with head-tracked glasses and a tracked controller.

Continue reading →

Hilarity Ensues

Posted on by
3

A funny thing has been happening for the last few days. There is a link farming or SEO or whatever site out there that is currently doing a deep clone of this blog (I’m obviously not going to provide a link here), but unlike all the other times that’s happened, these guys aren’t just straight-up copying, they’re running the text through what looks like a game of Google Translate Telephone (or a simple synonym replacer, but I’m going to pretend it’s the former). I’m guessing the reason is to obfuscate the source and throw off plagiarism detectors, but the result is unintentionally(?) hilarious.

For example, my post title “The Display Resolution of Head-mounted Displays” becomes “The Show Decision of Head-mounted Shows,” and “How Does VR Create the Illusion of Reality” becomes “How Does VR Create the Phantasm of Actuality.”

For a fuller experience, here are two paragraphs from the first article I mentioned above:

“The quick reply is after all that this relies on your mannequin of the headset. However should you occur to have an HTC Vive, view the graphs in footage 1 and a couple of (the opposite headsets behave in the identical method, however the precise numbers differ). These figures present the display decision, in pixels / °, alongside two traces (horizontal and vertical, respectively) that undergo the middle of the suitable lens of my very own Vive. The purple, inexperienced and blue curves present the decision for the purple, inexperienced and blue main colours respectively, this time not on the idea of my very own measurements, however by analyzing the show calibration knowledge measured for every particular person headset on the manufacturing unit after which saved within the firmware.

At this level it’s possible you’ll surprise why these graphs look so unusual, however for that you’ll have to learn the lengthy reply. Earlier than I’m going into that, I need to throw away a single quantity: proper in the course of the suitable lens of my Vive (on pixel 492, 602) the decision for the inexperienced shade channel is 11.42 pixels / °, each horizontal and vertical instructions. When you needed to cite a single decision quantity for headphones, that will be the one I used to be going to, as a result of it’s what you get once you have a look at one thing instantly in entrance of you and much away. As figures 1 and a couple of clearly present, no quantity can inform the entire story.”

Now, having posted this article, I am waiting with bated breath for what kind of hash their cloning bot will make out of it.

Welcome New VR Users!

Posted on by
Reply

Apparently, there were good sales numbers for VR equipment prior to the holiday season, and therefore a host of new VR users are coming in just about now. This meta-post collects a bunch of stuff I’ve written (or presented) in the past that might be of interest to some of those new users. These questions/answers are not hardware-specific, meaning they apply to any current-generation VR system (Oculus Rift, HTC Vive, all the Windows Mixed Reality headsets, PlayStation VR, …), and go beyond basic tech questions such as “how do I plug this in, install drivers, …).

There is one other issue for which I do not have a full article, but it’s quite important for new users: VR sickness (aka motion sickness, simulator sickness, …). Today’s VR headsets, at least the ones doing full head tracking (that means Rift/Vive et al., and not Gear VR, Oculus Go, Google Cardboard, …) should not cause VR sickness per se. These days, it is primarily caused by artificial locomotion in games or applications, as I explain in the second presentation I linked above.

The important message is: do not attempt to fight through VR sickness! If you try to stomach it out, it will only get worse. Stop using VR the moment you feel the first symptoms, take a long break, and then try again if you want to continue with the application/game that made you sick. If you try to power through repeatedly, your body might learn to associate sickness with VR, and that might cause you to get sick even when merely thinking about VR, or smelling the headset, or similar triggers. Just don’t do it.

That’s about it; now go ahead and enjoy your shiny new VR systems!

Want to Know More?

Here are a couple of other, more hardware-specific, topics:

A Milestone

Posted on by
4

Good news, everyone! This here blog, Doc-Ok.org, hit 1,000,000 (that’s one million) total page views early in the morning of Christmas Day, 12/25/2018, just about six years and four months after I started it.

I have no idea whether that’s something to be embarrassed or proud of, given the narrow range of topics I’m covering and the niche audience I’m targeting. Either way, it’s a nice, round number.

Set-up Instructions for Vrui with HTC Vive Head-mounted Display

Posted on by
6

It’s been more than two years since the last time I posted set-up instructions for Vrui and HTC Vive, and a lot has changed in the meantime. While Vrui-5.0 and its major changes are still not out of the kitchen, the current release of Vrui, Vrui-4.6-005, is stable and works very well with the Vive. The recent demise of our CAVE, and our move towards VR headsets until we figure out how to fix it, have caused a lot of progress in Vrui’s set-up and user experience. The rest of this article contains detailed installation and set-up instructions, starting from where my previous step-by-step guide, “An Illustrated Guide to Connecting an HTC Vive VR Headset to Linux Mint 19 (“Tara”),” left off.

If you did not follow that guide and its prerequisite, “An Illustrated Guide to Installing Linux Mint 19 (“Tara”),” this one assumes that you already have:

  • a “gaming” or “VR ready” PC with a powerful Nvidia GeForce graphics card,
  • a full installation of a 64-bit Ubuntu-based Linux operating system, e.g., Ubuntu or Linux Mint, with the MATE desktop environment,
  • proprietary drivers for the Nvidia graphics card installed and working,
  • head-mounted display filtering disabled in the graphics card driver,
  • and a working installation of SteamVR.

If you use a Linux distribution that is not Ubuntu-based, such as my own favorite, Fedora, or another desktop environment such as Gnome Shell or Cinnamon, you will have to make some adjustments throughout the rest of this guide.

This guide also assumes that you have already set up your Vive virtual reality system, including its tracking base stations, and that your Vive headset is connected to your PC via HDMI and USB (I will publish a detailed illustrated guide on that part soon-ish). Continue reading →

An Illustrated Guide to Connecting an HTC Vive VR Headset to Linux Mint 19 (“Tara”)

Posted on by
1

Running Vrui-based applications in glorious VR on an HTC Vive head-mounted display requires some initial set-up before Vrui itself can be installed and configured. This step-by-step guide will build upon an already-installed Linux operating system with high-performance graphics card drivers, specifically upon the current (as of 12/17/2018) version 19, code-named “Tara,” of Linux Mint, one of the most popular and user-friendly Linux distributions. This guide picks up right where the previous one in this series, “An Illustrated Guide to Installing Linux Mint 19 (“Tara”),” left off.

If you did not follow that guide, this one assumes that you have a “VR ready” or “gaming” PC with a powerful Nvidia GeForce graphics card, an installation of the 64-bit version of Linux Mint 19 (“Tara”) with the MATE desktop environment, and the recommended proprietary Nvidia graphics card driver. And an HTC Vive VR headset, of course.

Graphics Card Driver Set-up

Using a Vive headset with Vrui requires a change to the Nvidia graphics card driver’s configuration. Nvidia’s driver scans connected display devices for known VR headsets, and hides detected headsets from the desktop environment. This does make sense, as headsets are not standard monitors, and it would be awkward if windows or dialogs were to show up on a headset’s display. That said, here’s one relatively large quibble: headset filtering should happen earlier during the boot sequence, not just when the graphics card driver is loaded. As it is, headsets are still enumerated during boot, meaning that boot screens, BIOS menus, boot menus, etc. often show up on the headset, causing real problems. Anyway, carrying on.

Unfortunately for Vrui, there is currently no way to activate a hidden headset from inside an OpenGL-based VR application. For the time being, this means headset filtering in the driver needs to be disabled. To do so, open a terminal window (click on the terminal icon in the panel along the bottom screen edge, or right-click anywhere on the desktop and select “Open in Terminal” from the pop-up menu), enter exactly the following command into it (also see Figure 1) and press the Enter key (the $ sign indicates the terminal’s input prompt; don’t type it):

$ sudo xed /usr/share/X11/xorg.conf.d/50-Vive.conf

Figure 1: Creating a configuration file fragment using the xed text editor.

Continue reading →