No matter what word you use—virtual reality, augmented reality, artificial life, virtual environments—the upcoming family of virtual technology is slowly but surely coming to the general market. New VR companies are popping up. Old companies are finding new uses for VR. Data science is getting a VR makeover. All of this exciting technology represents a cluster of our favorite fields. So how does it all work?
At this point in time, it’s usually obvious when someone is using virtual reality. It’s the almost symbolic headset that gives it away. These headsets are used not just to drown out the users’ real surroundings, but to display the virtual one. In order to achieve this, the headset also needs to understand the users’ movements. For gear like HTC’s Vive or the Oculus Rift, headsets are connected by cable to a data source, namely the computer.
It does this by using the six degrees of freedom to plot a user’s movements. The six degrees are a common tool in robotics and engineering used to describe movement in a three dimensional space. These six values can come from different places. While inertial measuring units or IMUs, like accelerometers and gyroscopes, are useful, they can only track three of the six degrees, and the positioning tends to drift during usage. The Rift is known for their Constellation Tracking System, a positional tracking system which uses an IR LED sensor to paint a room with infrared and LED lights, creating a 3D space. Rather than the headset magically understanding what the user is doing, data on the entire space is used to deduce how the user is moving. This technology, along with Valve’s Lighthouse, are actually open to third party developers, which should help better tracking systems develop faster.
For Good Virtual Reality, the Devil is in the Details
All of this hardware would be nothing without the Internet of Things, which may be one reason VR has taken so long to develop. Those positional tracking systems, like the Constellation Tracking System, are incredibly new, and they wouldn’t be possible without the most modern technology. The tools to create smaller, better, more affordable sensors are growing as part of the entire IoT environment. There’s also another helpful development: as the IoT grows, so does the demand for faster data transfer.
If players thought lag was annoying in an ordinary game, lag in VR would be an instant product-killer. Lag as small as 3 or 4 milliseconds could be enough to drive players nuts and render an expensive product essentially useless. More importantly, this lagging issue is one of the culprits behind “simulator sickness,” another product-killer. Some responses to this problem include prepping video feed beforehand. If the computer can recognize where a player’s head is going to move, it can theoretically create and send that image to avoid lagging behind. Unfortunately, it’s tough to know what a player is about to do, and this also adds to the amount of data needing to be processed. There’s no way to eradicate lag, but, with the right data, VR could reduce it to an acceptable level. For now, a combination of better sensors and combined styles of tracking are still in the works.
Once VR reaches the stage where it’s at least reliable, it’s going to slowly move toward the next level: incredible. For example, when pixel size is displayed in such minutia that the brain really can’t see the difference between screen and reality. Or, when eyes and gaze are properly tracked, user can be fully immersed and active in the virtual world. And what would all this be without full facial tracking? One installment of the horror video game series Silent Hill gained notoriety for psychologically profiling players throughout the game. Even years later, its ability to analyze players and respond to them on a deeper level still leaves fans in awe. If games could replace survey-style questions and easy-to-spot tests with actual facial tracking and understanding, the options and level of immersion would be completely unparalleled.
Virtual Reality Gaming Raises New AI Questions
Artificial Intelligence already plays a huge role in today’s games and trying to bring proper intelligence to VR is going to be a battle in itself. Creating realistic virtual humans, enemies, monsters, and other non-playable characters might be the icing on the cake, but it will take time. Turning a game into a living film, or a less embarrassing live action role playing game, is the next step, and it’s a topic that will be receiving more attention once VR gear is truly marketable.
For competitive players, or ordinary players of competitive games like Starcraft, Dota 2, and other eSport titles, AI is going to have to take on another level. In any came involving player vs. player options, bots aren’t used just in the background, but for practice. They’re stand-ins for real players and other teams. Updates for games and bots are constantly being released, just to let players practice more realistically, honing their skills against human-like bots. It’s such a huge part of gaming that gamers are even begging companies like Valve to let the players and the community design and update their own bots. After Google’s Go-playing AI beat reigning champion Lee Se-Dol 4-1, it’s clear bots are capable of that highly nuanced, even unpredictable, play that makes games fun. However, that is still dependent on AI being adapted for other games, systems and scenarios. One surprising bot is not enough; every platform and game is going to need something a little different. If it’s tough to make virtual worlds, making them act and move in human ways is going to be just as hard.
New hardwares and softwares will soon take virtual reality from the long queues of sweaty tech expos to customers’ homes, but it’s still far from fully matured. Every successful study, inspired startup, and AI victory are important cogs driving the technology forward. Perhaps the most exciting surprise of all is that it’s tough to say what VR will really look like when it finally reaches us at home.
image credit: Knight Center for Journalism
Like this article? Subscribe to our weekly newsletter to never miss out!
