Apparently, the AR Sandbox is still a thing and going strong after ten years, with over 850 registered installations world-wide according to the AR Sandbox World Map. There was a lull in new installations and community activity during the initial COVID-19 lockdowns, but things are picking up again, and with that I am seeing an increasing amount of requests for help arriving in my personal email.
An AR Sandbox.
The old AR Sandbox support forum, which was quite active and significantly reduced my support load, not only by allowing me to answer common questions only once instead of dozens of times, but also by community members directly helping each other, unfortunately went down due to hardware problems a good while ago, and there is currently no avenue of getting it back up.
So I decided to create a new AR Sandbox support forum on this here web site, as a hopefully temporary replacement. I was not able to move over any of the old forum content due to not having access to the original database files, which is a major pity because there was a ton of helpful stuff on there. I am hoping that the new forum will accumulate its own set of helpful stuff quickly, and if/when I migrate the forum to a permanent location, I will be able to move all content because I have full access to this web site’s code and database. So here’s hoping.
This is the first forum on this web site, so I hope that things will work right from the start; if not, we’ll figure out how to fix it. Please be patient.
While I was writing up my article on the PlayStation VR headset’s optical properties yesterday, and specifically when I made the example images for the sub-section about sub-pixel layout comparing RGB Stripe and PenTile RGBG displays, it occurred to me that I could use those images to create a rough and simple simulator to visually evaluate the differences between VR headsets that have different resolutions and sub-pixel layouts.
The basic idea is straightforward: Take a test image that has some pixel count, for example WxH=640×360 as the initial low-resolution full-RGB picture seen in Figure 1. If you then blow up that image to fill a monitor that has the same aspect ratio (16:9) and some diagonal size D, the resolution of that image in terms of pixels per degree depends both on D and the viewer’s distance from the monitor Y: the larger the ratio Y/D, the higher is the image’s resolution as seen by the viewer. In detail, the formula for distance Y to achieve a desired resolution R in pixels/° is:
Y = (D / sqrt(W*W + H*H))/(2*tan(1/(2*R)))
where W and H are in pixel units, R is in pixels/°, and D is in some arbitrary length unit (inch, meter, parsec,…). Y will end up being in the same unit as D.
If a viewer then positions one of their eyes at a distance of Y from the center of the monitor and closes the other one, the resolution of the image on the monitor will be R. In other words, if R is the known resolution of some VR headset, the image on the monitor will look the same resolution as that VR headset.
There is one caveat: when looking at a flat monitor, the resolution of the displayed image increases away from the monitor’s center, while in a VR headset, the resolution generally decreases away from the center direction (see this article for reference). Meaning, for a correct evaluation, the viewer has to focus on the area in the center of the monitor. Unfortunately there is no easy way to simulate resolution drop-off using a flat monitor, at least not while also simulating sub-pixel layout.
It’s been more than two years since the last time I posted set-up instructions for Vrui and HTC Vive, and a lot has changed in the meantime. While Vrui-5.0 and its major changes are still not out of the kitchen, the current release of Vrui, Vrui-4.6-005, is stable and works very well with the Vive. The recent demise of our CAVE, and our move towards VR headsets until we figure out how to fix it, have caused a lot of progress in Vrui’s set-up and user experience. The rest of this article contains detailed installation and set-up instructions, starting from where my previous step-by-step guide, “An Illustrated Guide to Connecting an HTC Vive VR Headset to Linux Mint 19 (“Tara”),” left off.
If you use a Linux distribution that is not Ubuntu-based, such as my own favorite, Fedora, or another desktop environment such as Gnome Shell or Cinnamon, you will have to make some adjustments throughout the rest of this guide.
This guide also assumes that you have already set up your Vive virtual reality system, including its tracking base stations, and that your Vive headset is connected to your PC via HDMI and USB (I will publish a detailed illustrated guide on that part soon-ish). Continue reading →
Running Vrui-based applications in glorious VR on an HTC Vive head-mounted display requires some initial set-up before Vrui itself can be installed and configured. This step-by-step guide will build upon an already-installed Linux operating system with high-performance graphics card drivers, specifically upon the current (as of 12/17/2018) version 19, code-named “Tara,” of Linux Mint, one of the most popular and user-friendly Linux distributions. This guide picks up right where the previous one in this series, “An Illustrated Guide to Installing Linux Mint 19 (“Tara”),” left off.
If you did not follow that guide, this one assumes that you have a “VR ready” or “gaming” PC with a powerful Nvidia GeForce graphics card, an installation of the 64-bit version of Linux Mint 19 (“Tara”) with the MATE desktop environment, and the recommended proprietary Nvidia graphics card driver. And an HTC Vive VR headset, of course.
Graphics Card Driver Set-up
Using a Vive headset with Vrui requires a change to the Nvidia graphics card driver’s configuration. Nvidia’s driver scans connected display devices for known VR headsets, and hides detected headsets from the desktop environment. This does make sense, as headsets are not standard monitors, and it would be awkward if windows or dialogs were to show up on a headset’s display. That said, here’s one relatively large quibble: headset filtering should happen earlier during the boot sequence, not just when the graphics card driver is loaded. As it is, headsets are still enumerated during boot, meaning that boot screens, BIOS menus, boot menus, etc. often show up on the headset, causing real problems. Anyway, carrying on.
Unfortunately for Vrui, there is currently no way to activate a hidden headset from inside an OpenGL-based VR application. For the time being, this means headset filtering in the driver needs to be disabled. To do so, open a terminal window (click on the terminal icon in the panel along the bottom screen edge, or right-click anywhere on the desktop and select “Open in Terminal” from the pop-up menu), enter exactly the following command into it (also see Figure 1) and press the Enter key (the $ sign indicates the terminal’s input prompt; don’t type it):
The first step towards installing any Vrui-based software, including the Augmented Reality Sandbox, is installing some version of the Linux operating system on a new computer, which might sound like a daunting proposition to those who have never done that kind of thing before, but is actually very straightforward. This guide will be using the current (as of 12/17/2018) version 19, code-named “Tara,” of Linux Mint, one of the most popular and user-friendly Linux distributions.
Hardware Requirements
As this guide is geared towards installing and running Vrui-based 3D graphics applications, it assumes that the computer onto which Linux is to be installed is some type of “gaming” or “VR ready” PC, containing an Nvidia GeForce graphics card. The exact model of graphics card, as well as the exact model of CPU, amount of main memory, and hard drive size are not really important (that said, to run 3D graphics applications effectively, the PC should have a recent CPU, at least 4GB of main memory, and at least 60GB of hard drive space). While AMD/ATI graphics cards are otherwise perfectly serviceable, they have traditionally had inferior Linux driver support, and therefore Vrui and Vrui applications have not been tested on them in quite a while. In other words, we do not recommend them for these purposes.
Preparation
Before installing Linux, one needs to download an installation image for one’s chosen Linux distribution and flavor, and copy it to an installation medium, like a CD/DVD or USB stick. For this guide, we will be using the 64-bit version of Linux Mint 19 (“Tara”), with the MATE desktop environment. The page in the preceding link offers a 1.9GB disk (“iso”) image via a wide selection of download sites all over the world. Click the link for the site that is located most closely to you, and wait for the download to finish.
If the installation medium created in the previous step is a USB drive, plug it into a USB port on the new computer before turning it on for the first time. If it is a CD/DVD, turn the computer on first, and then insert the medium as quickly as possible.
The next step is to tell the computer to boot from the installation medium instead of from its internal hard drive. There is usually a key that needs to be pressed shortly after powering on the computer; typically either the “Delete” key to enter the computer’s BIOS, or “F8” to enter a boot menu. If this does not work on the first try, and the computer “hangs” or boots into whatever operating system was previously installed on it, don’t wait until it finishes booting — just turn it right back off (or press the reset button if it has one) and try again. We are going to erase any previous operating system anyway, so there’s no harm. However, do wait for about 20-30 seconds between turning the computer off and on again to avoid any danger of electrical damage. If neither the “Delete” nor “F8” keys work, look in the computer’s manual or online for the correct key sequence. Amazingly, finding the correct key to boot from the installation medium is by far the most difficult step of installing Linux Mint.
Once a BIOS screen or boot menu show up, select to boot from the installation medium. From there, it will take a few seconds to boot into a “live” Linux Mint environment, see Figure 1.
Figure 1: Linux Mint’s “live” installation environment.
Mixed-reality recording, i.e., capturing a user inside of and interacting with a virtual 3D environment by embedding their real body into that virtual environment, has finally become the accepted method of demonstrating virtual reality applications through standard 2D video footage (see Figure 1 for a mixed-reality recording made in VR’s stone age). The fundamental method behind this recording technique is to create a virtual camera whose intrinsic parameters (focal length, lens distortion, …) and extrinsic parameters (position and orientation in space) exactly match those of the real camera used to film the user; to capture a virtual video stream from that virtual camera; and then to composite the virtual and real streams into a final video.
Figure 1: Ancient mixed-reality recording from inside a CAVE, captured directly on a standard video camera without any post-processing.
I know, the Oculus Rift DK2 is obsolete equipment, but nonetheless — there are a lot of them still out there, it’s still a decent VR headset for seated applications, I guess they’re getting cheaper on eBay now, and I put in all the work back then to support it in Vrui, so I might as well describe how to use it. If nothing else, the DK2 is a good way to watch DVD movies, or panoramic mono- or stereoscopic videos, in VR.
Figure 1: Using an Oculus Rift DK2 headset with a pair of Vive controllers — because why not?
Now that I’ve gotten my Oculus Rift DK2 (mostly) working with Vrui under Linux, I’ve encountered the dreaded artifact often referred to as “black smear.” While pixels on OLED screens have very fast switching times — orders of magnitude faster than LCD pixels — they still can’t switch from on to off and back instantaneously. This leads to a problem that’s hardly visible when viewing a normal screen, but very visible in a head-mounted display due to a phenomenon called “vestibulo-ocular reflex.”
Basically, our eyes have built-in image stabilizers: if we move our head, this motion is detected by the vestibular apparatus in the inner ear (our “sense of equilibrium”), and our eyes automatically move the opposite way to keep our gaze fixed on a fixed point in space (interestingly, this even happens with the eyes closed, or in total darkness).
Update: There have been complaints that the post below is an overly complicated and confusing explanation of the IPD measurement process. Maybe that’s so. Therefore, here’s the TL;DR version of how the process works. If you want to know why it works, read on below.
Stand in front of a mirror and hold a ruler up to your nose, such that the measuring edge runs directly underneath both your pupils.
Close your right eye and look directly at your left eye. Move the ruler such that the “0” mark appears directly underneath the center of your left pupil. Try to keep the ruler still for the next step.
Close your left eye and look directly at your right eye. The mark directly underneath the center of your right pupil is your inter-pupillary distance.
Here follows the long version:
I’ve recentlytalked about the importance of calibrating 3D displays, especially head-mounted displays, which have very tight tolerances. An important part of calibration is entering each user’s personal inter-pupillary distance. Even when using the eyeball center as projection focus point (as I describe in the second post linked above), the distance between the eyeballs’ centers is the same as the inter-pupillary distance.
So how do you actually go about determining your IPD? You could go to an optometrist, of course, but it turns out it’s very easy to do it accurately at home. As it so happened, I did go to an optometrist recently (for my annual check-up), and I asked him to measure my IPD as well while he was at it. I was expecting him to pull out some high-end gizmo, but instead he pulled up a ruler. So that got me thinking.
Figure 1: How to precisely measure infinity-converged inter-pupillary distance using only a mirror and a ruler. Focus on the left eye in step one and mark point A; focus on the right eye in step two and mark point B; the distance between points A and B is precisely the infinity-converged inter-pupillary distance (and also the eyeball center distance).
In my detailed how-to guide on installing and configuring Vrui for Oculus Rift and Razer Hydra, I did not talk about installing any actual applications (because I hadn’t released Vrui-3.0-compatible packages yet). Those are out now, so here we go.
Kinect
If you happen to own a Kinect for Xbox (Kinect for Windows won’t work), you might want to install the Kinect 3D Video package early on. It can capture 3D (holographic, not stereoscopic) video from one or more Kinects, and either play it back as freely-manipulable virtual holograms, or it can, after calibration, produce in-system overlays of the real world (or both). If you already have Vrui up and running, installation is trivial.
