Dawn + OpenXR

[Mark Sibly]

Hi,

Is it theoretically possible to use dawn with OpenXR? Has anyone successfully done this?

I can't find any 'dawn' extension for OpenXR the way there are 'Direct3D12' and 'Vulkan' extensions, but perhaps it's possible to use these with dawn somehow anyway?

Bye!

Mark

[Corentin Wallez]

Hey Mark,

Someone on the Dawn Matrix chat room had OpenXR working with some tiny local modifications to Dawn but they seem to not have upstreamed it. I asked them again about it.

Using OpenXR with Vulkan is generally not super easy: it happens throught the XR_KHR_vulkan_enable2 and needs to override the initialization of the Vulkan device. The suggestion I had back then was to add additional function pointer members to vulkan::AdapterDiscoveryOptions to force the Vulkan backend to go through the OpenXR entrypoints instead of the Vulkan ones and it seems to have worked for them.

Hopefully we can have a link to the working code so we don't have to redo this from scratch. At some point it might be useful to have a small utility that does this discovery for you so they people don't have to re-figure this out over and over.

D3D12 seems like it could be easier but I haven't investigated in details. There's still the difficulty to import/export images but we have reasonable support for that in Dawn that could be extended.

Hope this helps!

Cheers,

Corentin

--
You received this message because you are subscribed to the Google Groups "Dawn Graphics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dawn-graphic...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dawn-graphics/fc6022bd-3424-4d5c-af45-1ee9b4f447e5n%40googlegroups.com.

[Corentin Wallez]

Here are the attempts. Device creation is working but rendering to the swapchain images isn't yet: https://github.com/Austint30/dawn-cmake/tree/openxr-attempt and https://github.com/Austint30/aurora/tree/openxr-attempt. I pointed out on Matrix that the path to handle the swapchain is very deprecated. You might want to join the Matrix chat (see the README.md) to chat with the developer of these repos.

[Mark Sibly]

> Here are the attempts.

Thanks for that, I will take a look. I've done OpenGL OpenXR before but that's it. I had a quick look at how D3D12 works last night and it doesn't look radically different, and doesn't seem like it'd be too hard to integrate with dawn (ha!), I will give that a shot today.

Does this mean though that webxr + webgpu in the browser is also not possible, at least on Chrome? Isn't this kind of a major shortcoming?

I did find this, but I think it's only a proposal:

https://github.com/immersive-web/WebXR-WebGPU-Binding/blob/main/explainer.md

Bye!

Mark

[Kai Ninomiya]

Indeed WebXR + WebGPU integration hasn't been standardized yet, it hasn't moved past the proposal stage while we're investing all our resources in core WebGPU. It's one of the many things we hope to do later in 2023 after launch.

-Kai (he/they)

--

You received this message because you are subscribed to the Google Groups "Dawn Graphics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dawn-graphic...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dawn-graphics/739df553-c304-4141-86f7-ce95e25d2cc9n%40googlegroups.com.

[Austin]

Hello Mark,

I'm the one who is attempting to add OpenXR functionality to Dawn. I don't have a lot of experience with C++ and game engine development, so this is my first steps in this realm of programming. My current changes only work for native applications using Vulkan at the moment, so no WebXR work has been done at this time. I've implemented it in such a way that Dawn is not directly interfacing with OpenXR, rather I've added a method to optionally pass function pointers through  vulkan::AdapterDiscoveryOptions so OpenXR's version of the Vulkan calls are called instead. Not sure how far this will get me, but it seems to be alright so far.

My current goal is to add very crude VR support for a Metroid Prime reverse engineering project called MetaForce. They recently rewrote their rendering engine to use Dawn Native as its graphics backend. Creating an OpenXR session is working, but swapchain images still need work as a solid gray picture is rendered along with a bunch of Vulkan validation errors (pretty sure that's where the problem is).

I'll try to let you know if I make any significant progress.

Best,

Austin

[Mark Sibly]

I've been hacking away at this for the last few days and have finally gotten both D3D12 and Vulkan going on my Vive2 (Windows only).

I used pretty much the approach suggested by Austin above, which basically just involved adding some openxr config info to AdapterDiscoveryOptions structs, eg:

// DawnNative.h, namespace dawn::native

//

struct DAWN_NATIVE_EXPORT OpenXRConfigBase {
    bool enabled  = false;  // OpenXR off by default
};

// VulkanBackend.h, namespace dawn::native::vulkan

//

struct DAWN_NATIVE_EXPORT OpenXRConfig : dawn::native::OpenXRConfigBase {

    std::function<::VkResult(PFN_vkGetInstanceProcAddr,
                           const VkInstanceCreateInfo*,
                           const VkAllocationCallbacks*,
                           VkInstance*)>
        CreateVkInstance;

    std::function<::VkResult(VkInstance, VkPhysicalDevice*)> GetVkPhysicalDevice;

    std::function<::VkResult(PFN_vkGetInstanceProcAddr,
                           VkPhysicalDevice,
                           const VkDeviceCreateInfo*,
                           const VkAllocationCallbacks*,
                           VkDevice*)>
        CreateVkDevice;
};

(Note: You need to use ::VkResult as dawn::native privately wraps VkResult. They're 2 completely different types, but look the same to the compiler if you only #include the dawn public headers!)

struct DAWN_NATIVE_EXPORT AdapterDiscoveryOptions : public AdapterDiscoveryOptionsBase {
    AdapterDiscoveryOptions();

    bool forceSwiftShader = false;

    OpenXRConfig openXRConfig;
};

So to get an xr compatible 'dawn' adapter you need to go:

auto opts = new dawn::native::vulkan::AdapterDiscoverOptions();

opts->openXRConfig.enabled = true;

opts-> CreateVkInstance = [=](...){...};

opts-> GetVkPhysicalDevice = [=](...){...};

opts->CreateVkDevice = [=](...){...};

instance->DiscoverAdapters(opts);

...etc...

I also had to add a few helpers to VulkanBackend:

DAWN_NATIVE_EXPORT VkPhysicalDevice GetVkPhysicalDevice(WGPUDevice device);
DAWN_NATIVE_EXPORT VkDevice GetVkDevice(WGPUDevice device);
DAWN_NATIVE_EXPORT uint32_t GetGraphicsQueueFamily(WGPUDevice device);
DAWN_NATIVE_EXPORT WGPUTexture CreateWGPUTexture(WGPUDevice device, const WGPUTextureDescriptor* descriptor, VkImage_T* image);

(Note: You needed to use VkImage_T* here as dawn privately wrap the VkImage type.)

Some of these are kind of redundant as they end up returning the same thingscreated by OpenXR in the first place, but this nicely separates wgpu device creation from openxr session creation IMO.

I haven't done a OpenXRConfig for D3D12 yet as D3D12 OpenXR 'just works' for me, ie: I just pass in the (only) d3d12 webgpu device and OpenXR works fine. This wont be the case for everyone though and one needs to be written although it looks like it'll be much simpler (The D3DDevice needs to have a specific 'LUID' which can be passed in via OpenXRConfig). I'm pretty sure OpenGL will also be pretty simple, but have no idea what's involved with metal.

So in my own 'tinyxr' library I've ended up with:

// Creates a (hopefully) XR compatible wgpu device

WGPUDevice createXRCompatibleWGPUDevice(WGPUBackendType);

// Create an XR session from a wgpu device.

XRSession createXRSession(WGPUDevice device);

I've also put together a little 'dawnxr' library that replaces a bunch of OpenXR functions with dawn compatible equivalents, and adds a dawn backend that works the same as the various KHR backends, which makes the above createXRSession much easier to write. I'll upload this to github soon.

Bye!

Mark

[Mark Sibly]

Ok, here's my little interop lib:

https://github.com/blitz-research/dawnxr

Please the required changes to dawn:

https://github.com/blitz-research/dawn/compare/main...openxr

Let me know if you find these useful and want me to keep them up to date.

Bye!

Mark

[Corentin Wallez]

Hey Mark,

It's great that you got OpenXR working! Looking at the changes, these look like easy things to upstream, mind opening a ChangeList (CL) for this?

DAWN_NATIVE_EXPORT VkPhysicalDevice GetVkPhysicalDevice(WGPUDevice device);

DAWN_NATIVE_EXPORT VkDevice GetVkDevice(WGPUDevice device);

DAWN_NATIVE_EXPORT uint32_t GetGraphicsQueueFamily(WGPUDevice device); // Maybe GetDefaultQueueFamilyIndex?

This one is also using a code path that will be deprecated and removed at some point. Instead I suggest adding a new substructure of ExternalImageDescriptorVk that takes a VkImage (instead of FDs or other things like other substructures of ExternalImageDescriptorVk). We could upstream that in another CL too.

DAWN_NATIVE_EXPORT WGPUTexture CreateWGPUTexture(WGPUDevice device, const WGPUTextureDescriptor* descriptor, VkImage_T* image);

Why did you decide to create an OpenXRConfigBase? It seems nothing there is specific to OpenXR so instead, we can just stuff optional function overrides in the vulkan::AdapterDiscoveryOptions directly. Also I suggest overriding the values in VulkanFunctions.h/.cpp instead of doing branches to choose whether to use the overload or not. This could be upstreamed as well.

Cheers,

Corentin

--
You received this message because you are subscribed to the Google Groups "Dawn Graphics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dawn-graphic...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dawn-graphics/b98c6f7d-2c0e-424f-a7e6-3b53f692a199n%40googlegroups.com.