Author Archives: okreylos

About okreylos

I am a research computer scientist at the University of California, Davis. My research areas are scientific visualization, particularly in immersive ("virtual reality") environments, human/computer interaction in immersive environments, and 3D computer graphics. My primary work is software development, from architecture over design to implementation and coding. I am the primary software developer for the UC Davis W.M. Keck Center for Active Visualization in the Earth Sciences (KeckCAVES). Some of my released packages are Vrui (a VR development toolkit), CollaborationInfrastructure (a tele-collaboration plug-in for Vrui), Kinect (a driver suite to capture 3D video from Microsoft Kinect cameras), LiDAR Viewer (a visualization package for very large 3D point clouds), 3D Visualizer (a system for interactive visual analysis of 3D volumetric data), Nanotech Construction Kit (an interactive molecular design program), and SARndbox (an augmented reality sandbox). I also dabble in VR hardware, in the sense that I take existing custom or commodity hardware components (3D TVs, head-mounted displays, projectors, tracking systems, Wiimotes, Kinect cameras, ...) and build fully integrated immersive environments out of them. This includes a fair share of driver development to support hardware that either doesn't have drivers, or whose vendor-supplied drivers are not up to par.

An Illustrated Guide to Installing Linux Mint 19 (“Tara”)

Posted on by
5

The first step towards installing any Vrui-based software, including the Augmented Reality Sandbox, is installing some version of the Linux operating system on a new computer, which might sound like a daunting proposition to those who have never done that kind of thing before, but is actually very straightforward. This guide will be using the current (as of 12/17/2018) version 19, code-named “Tara,” of Linux Mint, one of the most popular and user-friendly Linux distributions.

Hardware Requirements

As this guide is geared towards installing and running Vrui-based 3D graphics applications, it assumes that the computer onto which Linux is to be installed is some type of “gaming” or “VR ready” PC, containing an Nvidia GeForce graphics card. The exact model of graphics card, as well as the exact model of CPU, amount of main memory, and hard drive size are not really important (that said, to run 3D graphics applications effectively, the PC should have a recent CPU, at least 4GB of main memory, and at least 60GB of hard drive space). While AMD/ATI graphics cards are otherwise perfectly serviceable, they have traditionally had inferior Linux driver support, and therefore Vrui and Vrui applications have not been tested on them in quite a while. In other words, we do not recommend them for these purposes.

Preparation

Before installing Linux, one needs to download an installation image for one’s chosen Linux distribution and flavor, and copy it to an installation medium, like a CD/DVD or USB stick. For this guide, we will be using the 64-bit version of Linux Mint 19 (“Tara”), with the MATE desktop environment. The page in the preceding link offers a 1.9GB disk (“iso”) image via a wide selection of download sites all over the world. Click the link for the site that is located most closely to you, and wait for the download to finish.

The next step is to “burn” the downloaded disk image file to an installation medium. Instructions for several operating systems and media types can be found on the Linux Mint site.

Base System Installation

If the installation medium created in the previous step is a USB drive, plug it into a USB port on the new computer before turning it on for the first time. If it is a CD/DVD, turn the computer on first, and then insert the medium as quickly as possible.

The next step is to tell the computer to boot from the installation medium instead of from its internal hard drive. There is usually a key that needs to be pressed shortly after powering on the computer; typically either the “Delete” key to enter the computer’s BIOS, or “F8” to enter a boot menu. If this does not work on the first try, and the computer “hangs” or boots into whatever operating system was previously installed on it, don’t wait until it finishes booting — just turn it right back off (or press the reset button if it has one) and try again. We are going to erase any previous operating system anyway, so there’s no harm. However, do wait for about 20-30 seconds between turning the computer off and on again to avoid any danger of electrical damage. If neither the “Delete” nor “F8” keys work, look in the computer’s manual or online for the correct key sequence. Amazingly, finding the correct key to boot from the installation medium is by far the most difficult step of installing Linux Mint.

Once a BIOS screen or boot menu show up, select to boot from the installation medium. From there, it will take a few seconds to boot into a “live” Linux Mint environment, see Figure 1.

Figure 1: Linux Mint’s “live” installation environment.

Continue reading →

Running Linux on HP Spectre x360

Posted on by
4

Computer reviews aren’t my thing, but for this one I had to make an exception. My 3.5 year old laptop, the HP Spectre x360 I had scored as swag at the 2015 Microsoft Build conference, suddenly died a few months ago. I had taken a liking to that thing, so when I had to leave for a conference in early November, and realized I should probably bring a laptop with me, I decided to replace it with the current version of the same model. Fortunately they had one in stock at my neighborhood Best Buy (alas, only the silver one and not the pretty black and gold one), so I was able to pick it right up.

I then had the bad idea to search online for Linux support on the x360 after already having bought it, and was dismayed by what I found. A lot of people reported poor performance, too-hot-to-handle operating temperatures, and very poor battery life. Not having much of a choice at that point, I decided to go ahead anyways and install Fedora 28 on it, the then-current release of my go-to Linux distribution. Long story short: installation was a breeze, everything worked out-of-the-box, performance is great, the laptop runs barely warm, and battery life is awesome (so awesome, in fact, that I initially thought the readings were wrong). In order to provide a counter-narrative to those other reports, this is my experience of installing and running Linux on a 2018 HP Spectre x360. Continue reading →

The Display Resolution of Head-mounted Displays, Revisited

Posted on by
22

I wrote an article earlier this year in which I looked closely at the physical display resolution of VR headsets, measured in pixels/degree, and how that resolution changes across the field of view of a headset due to non-linear effects from tangent-space rendering and lens distortion. Unfortunately, back then I only did the analysis for the HTC Vive. In the meantime I got access to an Oculus Rift, and was able to extract the necessary raw data from it — after spending some significant effort, more on that later.

With these new results, it is time for a follow-up post where I re-create the HTC Vive graphs from the previous article with a new method that makes them easier to read, and where I compare display properties between the two main PC-based headsets. Without further ado, here are the goods.

HTC Vive

The first two figures, 1 and 2, show display resolution in pixels/°, on one horizontal and one vertical cross-section through the lens center of my Vive’s right display.

Figure 1: Display resolution in pixels/° along a horizontal line through the right display’s lens center of an HTC Vive.

Figure 2: Display resolution in pixels/° along a vertical line through the right display’s lens center of an HTC Vive.

Continue reading →

A Blast From The Past

Posted on by
3

Back in the olden days, in the summer of 1996 to be precise, I was a computer science Master’s student at the University of Karlsruhe, Germany, about to take the oral exam in my specialization area, 3D computer graphics, 3D user interfaces, and geometric modeling. For reasons that are no longer entirely clear to me, I decided then that it would be a good idea to prepare for that exam by developing a 3D rendering engine, a 3D game engine, and a game, all from scratch. What resulted from that effort — which didn’t help my performance in that exam at all, by the way — was “Starglider Pro:”

In the mid to late 80s, one of my favorite games on my beloved Atari ST was the original Starglider, developed by Jez San for Rainbird Software. I finally replaced that ST with a series of PCs in 1993, first running DOS, and later OS/2 Warp, and therefore needed something to scratch that Starglider itch. Continue reading →

The Display Resolution of Head-mounted Displays

Posted on by
10

What is the real, physical, display resolution of my VR headset?

I have written a long article about the optical properties of (then-)current head-mounted displays, one about projection and distortion in wide-FoV HMDs, and another one about measuring the effective resolution of head-mounted displays, but in neither one of those have I looked into the actual display resolution, in terms of hard pixels, of those headsets. So it’s about time.

The short answer is, of course, that it depends on your model of headset. But if you happen to have an HTC Vive, then have a look at the graphs in Figures 1 and 2 (the other headsets behave in the same way, but the actual numbers differ). Those figures show display resolution, in pixels/°, along two lines (horizontal and vertical, respectively) going through the center of the right lens of my own Vive. The red, green, and blue curves show resolution for the red, green, and blue primary colors, respectively, determined this time not by my own measurements, but by analyzing the display calibration data that is measured for each individual headset at the factory and then stored in its firmware.

Figure 1: Resolution in pixels/° along a horizontal line through my Vive’s right lens center, for each of its 1080 horizontal pixels, for the three primary colors (red, green, and blue).

Figure 2: Resolution in pixels/° along a vertical line through my Vive’s right lens center, for each of its 1200 vertical pixels, for the three primary colors (red, green, and blue).

At this point you might be wondering why those graphs look so strange, but for that you’ll have to read the long answer. Before going into that, I want to throw out a single number: at the exact center of my Vive’s right lens (at pixel 492, 602), the resolution for the green color channel is 11.42 pixels/°, in both the horizontal and vertical directions. If you wanted to quote a single resolution number for a headset, that’s the one I would go with, because it’s what you get when you look at something directly ahead and far away. However, as Figures 1 and 2 clearly show, no single number can tell the whole story.

And now for the long answer. Buckle in, Trigonometry and Calculus ahead. Continue reading →

How Does VR Create the Illusion of Reality?

Posted on by
Reply

I’ve recently written a loose series of articles trying to explain certain technical aspects of virtual reality, such as what the lenses in VR headsets do, or why there is some blurriness, but I haven’t — or at least haven’t in a few years — tackled the big question:

How do all the technical components of VR headsets, e.g., screens, lenses, tracking, etc., actually come together to create realistic-looking virtual environments? Specifically, why do virtual environment in VR look more “real” compared to when viewed via other media, for example panoramic video?

The reason I’m bringing this up again is that the question keeps getting asked, and that it’s really kinda hard to answer. Most attempts to answer it fall back on technical aspects, such as stereoscopy, head tracking, etc., but I find that this approach somewhat misses the point by focusing on individual components, or at least gets mired in technical details that don’t make much sense to those who have to ask the question in the first place.

I prefer to approach the question from the opposite end: not through what VR hardware produces, but instead through how the viewer perceives 3D objects and/or environments, and how either the real world on the one hand, or virtual reality displays on the other, create the appropriate visual input to support that perception.

The downside with that approach is that it doesn’t lend itself to short answers. In fact, last summer, I gave a 25 minute talk about this exact topic at the 2016 VRLA Summer Expo. It may not be news, but I haven’t linked this video from here before, and it’s probably still timely:

Continue reading →

Projection and Distortion in Wide-FoV HMDs

Posted on by
4

There is an on-going, but already highly successful, Kickstarter campaign for a new VR head-mounted display with a wide (200°) field of view (FoV): Pimax 8k. As I have not personally tried this headset — only its little brother, Pimax 4k, at the 2017 SVVR Expo — I cannot discuss and evaluate all the campaign’s promises. Instead, I want to focus on one particular issue that’s causing a bit of confusion and controversy at the moment.

Early reviewers of Pimax 8k prototypes noticed geometric distortion, such as virtual objects not appearing in the correct places and shifting under head movement, and the campaign responded by claiming that these distortions “could be fixed by improved software or algorithms” (paraphrased). The ensuing speculation about the causes of, and potential fixes for, this distortion has mostly been based on wrong assumptions and misunderstandings of how geometric projection for wide-FoV VR headsets is supposed to work. Adding fuel to the fire, the campaign released a frame showing “what is actually rendered to the screen” (see Figure 1), causing further confusion. The problem is that the frame looks obviously distorted, but that this obvious distortion is not what the reviewers were complaining about. On the contrary, this is what a frame rendered to a high-FoV VR headset should look like. At least, if one ignores lenses and lens distortion, which is what I will continue to do for now.

Figure 1: Frame as rendered to one of the Pimax 8k’s screens, according to the Kickstarter campaign. (Probably not 100% true, as this appears to be a frame submitted to SteamVR’s compositor, which subsequently applies lens distortion correction.)

Continue reading →

Measuring the Effective Resolution of Head-mounted Displays

Posted on by
15

Why does everything in my VR headset look so pixelated? It’s supposed to be using a 2160×1200 screen, but my 1080p desktop monitor looks so much sharper!

This is yet another fundamental question about VR that pops up over and over again, and like the others I have addressed previously, it leads to interesting deeper observations. So, why do current-generation head-mounted displays appear so low-resolution?

Here’s the short answer: In VR headsets, the screen is blown up to cover a much larger area of the user’s field of vision than in desktop settings. What counts is not the total number of pixels, and especially not the display’s resolution in pixels per inch, but the resolution of the projected virtual image in pixels per degree, as measured from the viewer’s eyes. A 20″ desktop screen, when viewed from a typical distance of 30″, covers 37° of the viewer’s field of vision, diagonally. The screen (or screens) inside a modern VR headset cover a much larger area. For example, I measured the per-eye field of view of the HTC Vive as around 110°x113° under ideal conditions, or around 130° diagonally (it’s complicated), or three and a half times as much as that of the 20″ desktop monitor. Because a smaller number of pixels (1080×1200 per eye) is spread out over a much larger area, each pixel appears much bigger to the viewer.

Now for the long answer.

Continue reading →

3D Camera Calibration for Mixed-Reality Recording

Posted on by
3

Mixed-reality recording, i.e., capturing a user inside of and interacting with a virtual 3D environment by embedding their real body into that virtual environment, has finally become the accepted method of demonstrating virtual reality applications through standard 2D video footage (see Figure 1 for a mixed-reality recording made in VR’s stone age). The fundamental method behind this recording technique is to create a virtual camera whose intrinsic parameters (focal length, lens distortion, …) and extrinsic parameters (position and orientation in space) exactly match those of the real camera used to film the user; to capture a virtual video stream from that virtual camera; and then to composite the virtual and real streams into a final video.

Figure 1: Ancient mixed-reality recording from inside a CAVE, captured directly on a standard video camera without any post-processing.

Continue reading →

AltspaceVR Shutting Down

Posted on by
7

AltspaceVR, the popular virtual reality social platform, and the eponymous company behind it, will be closing their respective doors on August 3rd. This is surprising, as AltspaceVR has been around since 2013, was well-funded, had a good amount of users given VR’s still-niche status, and had apparently more funding lined up to continue operation and development of their platform (that funding falling through was, according to the announcement linked above, the primary reason for the impending shut-down).

But besides the direct impact on commercial VR as a whole, and the bad omen of a major player closing down, this is also personal to me. Not as a user of AltspaceVR’s service — I have to admit I’ve only tried it for minutes at a time at trade shows or conferences — but as someone who was, albeit tangentially, involved with the company and the people working there.

After having given a presentation at an early SVVR meet-up, I invited SVVR’s founder, Karl Krantz, to visit me at my VR lab at UC Davis. He made the trip a short while later, and brought a few friends, including “Cymatic” Bruce Wooden, Eric Romo, and Gavan Wilhite. I showed them our array of VR hardware, the general VR work we were doing, and specifically our work in VR tele-presence and remote collaboration. According to the people involved, AltspaceVR was founded during the drive back to the Bay Area.

In addition, I co-advised one of AltspaceVR’s developers when he was a PhD student at UC Davis, and I visited them in the summer of 2015 to give a talk about input device and interaction abstraction in multi-platform VR development. During that visit, Eric Romo also gave me my first taste of the newly-released HTC Vive Development Kit (Vive DK1).

For all that, I am sad to see them go under, and I wish everybody who is currently working there all the best for their future endeavors.

Possibly related to this, another piece of news surfaced today: AltspaceVR was named defendant in a patent infringement lawsuit filed by Virtual Immersion Technologies, LLC, regarding this 2002 patent. I do not know whether this filing was a cause in AltspaceVR’s closing, but it is possible that the prospect of a costly court case, or stiff licensing fees, led to some investors getting cold feet.

Either way, this patent deserves closer scrutiny as it is quite broad, and has recently changed ownership from the original inventors to the plaintiff, who has so far been using it exclusively to sue VR companies for infringement. The fact that it specifically claims the use of video to represent performers or users in a shared virtual space might mean that it covers platforms such as our tele-collaboration framework, which would be unfortunate. I have a hunch that this patent, due to its arguably broad applicability, will be the subject of a major legal battle in the near future, and while there is a lot of prior art in multiplayer/multi-user VR, that video component means I cannot dismiss the patent out of hand.