Hit enter after type your search item

    Oculus Insight: how Facebook disconnected VR and opened virtual worlds for everyone


    VR can be tricky – setting up sensors or transmitters, connecting cables and configuring the hardware can take so long, it may not seem worthwhile for a quick game session. It is all too easy for that expensive VR headset to leave dust behind – even if you have invested in a powerful gaming PC to support it.

    With the release of Oculus Go in 2018, that has changed somewhat, with a VR experience that stood entirely on its own, without cables or extra hardware. Simply fasten the headset and get started.

    It was impressive and much cheaper than other VR headsets that were available at the time, but was much more limited than more powerful paired headsets. Most importantly, the motion system only ' orientation ' was, which means that you could turn your head to look around the area, but not move through it.

    That all changed with the release of Oculus Quest in May of this year. The Quest is still completely wireless, but unlike the Go, it offers six degrees of freedom – all without the need for external hardware around your room.

    How is that possible? TechRadar spoke with Anna Kozminski, AR / VR software manager at Facebook Zurich in Switzerland, to learn how Oculus is in control of virtual reality.

    Follow inside out

    First a short explanation: six degrees of freedom means that your body can move in three dimensions along the X, Y and Z axes. Moving up and down along the Y-axis is called "hoisting," moving forward and backward along the X-axis is called "rocking," and moving left and right along the Z-axis is called "swinging." You can also turn to look at another axis. A movement between X and Y is called "pitch," a movement between X and Z is called "yaw," and a movement between Z and Y is called "roll."

    "To make it feel completely natural, we had to follow the movements of the user extremely accurately in real time"

    Anna Kozminski, Facebook Zurich

    If you want to create a truly immersive VR experience, that is essential. "With a VR experience, you ultimately want it to feel just as fluid as real life – you don't want to be distracted because the screen is lagging or jittering," Kozminski explains.

    "To make it feel completely natural, we had to follow the user's movements extremely accurately in real time, so when you move your head and hands, those movements are perfectly reflected in the virtual world."

    Oculus Quest is the first consumer device with full freedom tracking of six degrees, made possible by the Oculus Insight system. This uses a technology called simultaneous location and mapping (SLAM), which uses input from different sensors to fix the location of an object within a constantly updated virtual map. It uses these objects as reference points (just as you would use landmarks to orientate yourself) and anchors them at points in the virtual world.

    (youtube https://www.youtube.com/watch?v=nrj3JE-NHMw)

    These points are observed repeatedly to compensate for drift (with small measurement differences increasing over time, making the map less accurate). The virtual area covers your entire field of vision, so the system must respond as quickly as possible if you can move and any tracking errors or delays are very noticeable.

    "This tracking of six degrees of motion has been done before, but with many external sensors in the room, and that formed a major barrier to entry," said Kozminski. "You cannot share the device, you cannot take it to a friend's house. (Our goal was) to reach its level of immersion, but to make it more practical so that more people can use it."

    Make VR portable

    To achieve that, Oculus had to move the software stack that performs all calculations to the headset itself. That was a serious challenge; how do you place those complex systems on something that is lightweight, portable and powered by a rechargeable battery?

    "There were many challenges in developing the architecture to use the least amount of power and leaving plenty of room on the device for gaming content," she said. "Oculus Insight is a gaming platform, but if there is no content to enjoy it is a matter of. (We had to) break the trend that it can only run on powerful PCs and enable developers create cool games and experiences. "

    "We had to collect thousands and thousands of data sets, with all different lighting and design parameters"

    Anna Kozminski, Facebook Zurich

    Of course a room in your house can be completely different from that of your friend's, so the system had to be able to handle a large number of circumstances. "The algorithm is very sensitive to the environment in which it is used," said Kozminski. “The lighting conditions, the size of the room, the amount of texture on the walls and surfaces – but we cannot predict how people will decorate their living room. Maybe they want dim lighting (for example).

    “We had to collect thousands and thousands of data sets, come up with all the different lighting and device parameters, then replay them on Facebook's servers and simulate the actual use of the headset so that we could calculate metrics. It was a long-standing effort. "

    This room in the Facebook lab in Zurich is one of hundreds used by engineers to test how Oculus Insight performs in real environments

    However, the work was worth the effort and the hardware developers are enormously impressed by what creatives have succeeded in achieving. We have asked Kozminski about her favorites so far.

    "I think there are two," she said. “One is external use in general, and one is an internal project. When it comes to external use, I am a big fan of 360-degree videos, in addition to gaming. When producers take a 360-degree camera ' s to an environment such as a rainforest or refugee camp and tell a message in video content, that is really attractive. You have to be in someone else's shoes to experience it. The video content is really fascinating VR because it feels like you're really there, and it gives you that first-person perspective that is powerful in building empathy.

    "VR can be a lonely experience if you use it yourself, and our goal is to create meaningful connections"

    Anna Kozminski, Facebook Zurich

    “We did a demo on OC5 last year – a proof of concept for future applications of Oculus, with six users in a Western-style shootout game in an arena. We were locating multiple players. Nowadays you can do multiplayer experiences, but there is nothing to indicate that the player is actually standing next to you. In that demo we shared the same card on all devices in that part of the location.

    “VR can be a lonely experience if you use it yourself, and our goal is to create meaningful connections. That demo was a show from where we could go to area-shared experiences in the future – whether it's playing or just hanging out. "

    Break up divides

    Kozminski and her colleagues on Facebook do not rest on their laurels. "Now that we've sent the Quest, our team is writing new features that we send almost every month," she said.

    However, it's not just about gaming – Oculus is a Facebook project, so it's no surprise that the ultimate goal is to connect people. And to do that, the entry threshold must be lowered even further

    "I have a family living in Canada," explains Kosminski, "and with these VR experiences we have multiplayer games, but in the future we want to bring two people who are very far apart into the same experience and make them feel I want to get to a point where I can put a headset on and hang out with my brother in Canada as if we're in the same room – whether in VR or AR.

    “The Oculus Insight technology is what we will use in the future to power AR glasses. We talked about investing in AR and ultimately, when we reach a lighter form factor – something that is more comfortable to wear – more people will use it and we can get more people in this community to gain digital experiences. Go to AR glasses and resolve these geographical differences. "

    • View our complete guide to the best VR headsets
    This div height required for enabling the sticky sidebar