How To Record Passthrough Api Video For Oculus Quest 2

Facebook recently released its experimental Passthrough API for Oculus Quest Passthrough allows developers to create augmented reality apps for this virtual reality headset. Many developers have already started to create such apps, and of course they want to share videos on social media, but a few months ago Facebook blocked all Passthrough video recording options. So how do you record such a video? Very simple… Let me tell you the whole story.

Facebook blocks passthrough

A few months ago I was complaining about Facebook blocking every possible way of recording video from the Quest headset: if you try to record content from the Quest Home environment with passthrough mode enabled, you’ll see black instead of passthrough images when recording through all the Quest features (stills, video recording, broadcasting, passthrough, etc… etc…) and completely black video when capturing screen through low-level and passthroughDB and all programs that use it (SideQuest, Strcpy, Vysor, etc…).

I commented on this decision as rather absurd, because ADB is a special tool with high privileges that has to run with a headset connected to the computer (using a wire or over Wi-Fi), so it cannot be used to carry out any kind of attack on user privacy, and Facebook should allow people to use it to record passthrough apps. I also predicted that when passthrough AR spreads, it would be a disaster because no one would be able to record their own app and share it with the community, making it impossible for every developer to show their results and inspire other developers. passthrough

In fact, when the first passthrough AR app, the Cactus Cowboy AR Passthrough Demo, was released on SideQuest, Upload VR talked about it but could not share any videos. The same thing happened with the first tutorial on Passthrough AR APIs by Dilmer Valetsillos, who had to record a video with the Oculus MRC to demonstrate the result of the first app he created using these APIs. However, I later started to encounter videos made with AR apps built with Quest and was quite puzzled as to how this could have happened. A few weeks ago when trying to video my Passthrough Home, every possible way (I mean really ALL of them) was used, and I couldn’t find any solution… So how were these developers able to record footage from their AR applications so easily? Passthrough

I contacted March Schramm, the developer of Gravity Lab, who had just shared a similar video, and asked him how he did it, what sorcery he used to record this video, because when Facebook blocked passsthrough recording, I had tried every possible solution to make it work. All he replied was that he used scrcpy and that the tool was very good for the purpose.

Scrcpy is an open source tool (using ADB) that is also used by SideQuest to mirror your headset image, and that is how I used to record AR content on Quest before Facebook blocked it, but that method subsequently stopped working after Facebook banned users from recording passthrough. After talking to Mark, I tried it again and made sure I was indeed able to record the AR app on Quest without any problems. The simplest solution, in the end, turned out to be the one that really works… but how is this possible?

Facebook’s selective blocking

After various experiments, I realized that Facebook changed its passthrough recording mechanism a bit after it released the passthrough API. Now the passthrough: passthrough

  • If you try to record your Passthrough Home environment (or other types of passthrough using AR) with Oculus related tools (e.g. take a picture, record video, casting, streaming), you clearly see the virtual elements, but all areas with passthrough vision become a black pass-through
  • If you try to record a Passthrough home environment (or other kind of system ePassthrough using AR) using ADB-related tools (like scrcpy), you just see everything black. You will not even see virtual content, just a black screen
  • If you try to record an application using AR Passthrough APIs via tools related to ADB (e.g. shooting, recording video, casting, streaming), you will see the virtual elements fine, but all areas with transparent backgrounds will be black
  • If you try to record an application using AR Passthrough APIs through tools, usingPassthrough will see directly what is displayed to the user, i.e. virtual elements on top of the real background. This also applies to applications with partial AR content, such as Horizon Workrooms, allowing you to see the keyboard in augmented reality

In short: Facebook has unlocked the ability to record AR apps via ADB, but it still keeps it locked to system menus in AR. This means that all developers can easily record their apps using scrcpy or similar systems!

How do I record an AR app image on Oculus Quest 2?

It has recently been revealed that you can use all ADB-based tools to capture video of an AR app running on your Oculus Quest. This means that you can directly use the “adb screenrecord” command to record a video file of what’s displayed inside the headset, or to stream it directly to your PC. To avoid going deeper into the command line, I’ll just tell you how to stream using SideQuest (which I think everyone has installed), which internally uses scrcpy, which in turn uses ADB (essentially a matryoshka). Thus, just two clicks and everything is ready and you don’t have to fiddle with a bunch of command lines like hackers in movies.

So, to record AR game gameplay on Quest, all you need to do is:

  • Install and set up SideQuest, if you haven’t already.
  • Connect your Oculus Quest device to your computer using a USB cable, but without activating Oculus Link
  • Open SideQuest and wait for it to connect to the Quest device
  • If you want, click on the Wi-fi icon to connect via Wi-fi to the Quest device, so you’ll be unconnected while you’re recording video. I would only recommend doing this if you have a great Wi-fi connection, otherwise the quality of the video you receive may degrade
  • Click on the SideQuest streaming icon in the menu at the top right corner of your screen
  • In the pop-up window select “Quest 2 Crop” and then “Start Stream”
  • At this point, a window should appear on your screen showing the AR content of the headset
  • Record the screen content using your favorite app: I would suggest you use Open Broadcaster Software (OBS) or Xbox Game Bar for Windows 10 (you call it by pressing Win + G)

That’s it… Nice and easy. Of course, it’s inconvenient that you can’t record everything directly from your device like you do in VR apps, but this procedure isn’t that complicated, and you’ll quickly get used to it

Alternative methods

What if the method described above doesn’t work or you want to record the Passthrough Home environment instead? I found two other alternatives: Passthrough

Use your smartphone or webcam to record the contents of the headset. Make sure you set the camera exposure properly to get as few horizontal bars visible as possible. With this method, the resulting video usually has some artifacts (moving horizontal bars, blur, etc…), and since the camera is inside the headset, you can’t actually use it, so it’s hard to show the interaction in AR. This method only comes in handy if you want to make a small GIF about some AR window, for example to make a short video about the Guardian installation scene.

Use a smartphone mounted in front of your device and a computer to mix the smartphone and headset recording to get some semblance of a mixed reality video. The advantage of this method is that you can even record color AR videos with Quest, while the disadvantages are that it’s too cumbersome and doesn’t properly record virtual elements that are black. The idea belongs to Avi Barel, and you can find his excellent tutorial on the subject here.

As far as I know, there are no alternative methods for recording AR on Quest… of course, if you know of another method, let me know and I’ll add it to this list to help other developers!

That concludes this tutorial… I hope it can be useful for all you XR content creators to share your creations with the rest of the world! If you liked it, could you please share it with your social media communities so it can reach other interested developers? Thank you!

Leave a Comment