Multi-windowing on iPadOS and visionOS

[Stefan Csomor]

Hi

 

Since iOS 13 on iPads there is the option for multiple-windows – or rather multiple windows grouped together in a so called WindowScene.

https://developer.apple.com/documentation/uikit/uiwindowscene?language=objc

 

So the hierarchy is:

UIScreen -> one or more UIWindowScenes -> one or more UIWindows …

 

Now on visionOS the UIScreen is not available anymore at all, we just can access the UIWindowScenes of our UIApplication.

 

Do we have somewhere in wx a concept that is grouping multiple tlw-windows together or can I start afresh. Any ideas or special thoughts about this ?

 

Thanks,

 

Stefan

 

 

[Vadim Zeitlin]

On Sat, 24 Jun 2023 11:51:12 +0000 Stefan Csomor wrote:

SC> Since iOS 13 on iPads there is the option for multiple-windows – or
SC> rather multiple windows grouped together in a so called WindowScene.
SC> https://developer.apple.com/documentation/uikit/uiwindowscene?language=objc
SC>
SC> So the hierarchy is:
SC> UIScreen -> one or more UIWindowScenes -> one or more UIWindows …
SC>
SC> Now on visionOS the UIScreen is not available anymore at all, we just
SC> can access the UIWindowScenes of our UIApplication.
SC>
SC> Do we have somewhere in wx a concept that is grouping multiple
SC> tlw-windows together or can I start afresh. Any ideas or special
SC> thoughts about this ?

Hi Stefan,

I don't think we have anything like this, wxDisplay is a bit related, but
not really the same thing. AFAICS from the linked documentation,
UIWindowScene has to be declared in Info.plist, so I'm not even sure if we
need to have a corresponding object at wx level, as we could just provide
access to it via wxApp.

OTOH if you think it could be useful to have a corresponding wx class, we
could add one too and provide generic implementation just returning all wx
windows and forwarding the rest of the functions to wxDisplay.

One thing which is not clear to me is whether using UIWindowScene is
optional or mandatory for the new applications? And, if it's the former,
what exactly do you gain from using it? The possibility to have multiple
independent UIs in the same application?

Sorry if I'm totally missing the point here, because I definitely feel
like I am...

Best,
VZ

[Stefan Csomor]

Hi Vadim

SC> Since iOS 13 on iPads there is the option for multiple-windows – or
SC> rather multiple windows grouped together in a so called WindowScene.
SC> https://developer.apple.com/documentation/uikit/uiwindowscene?language=objc
SC>
SC> So the hierarchy is:
SC> UIScreen -> one or more UIWindowScenes -> one or more UIWindows …
SC>
SC> Now on visionOS the UIScreen is not available anymore at all, we just
SC> can access the UIWindowScenes of our UIApplication.
SC>
SC> Do we have somewhere in wx a concept that is grouping multiple
SC> tlw-windows together or can I start afresh. Any ideas or special
SC> thoughts about this ?

Thanks for looking at this

> I don't think we have anything like this, wxDisplay is a bit related, but
> not really the same thing. AFAICS from the linked documentation,
> UIWindowScene has to be declared in Info.plist, so I'm not even sure if we
> need to have a corresponding object at wx level, as we could just provide
> access to it via wxApp.

Yes, it depends on how exposed we want/need these features.

> OTOH if you think it could be useful to have a corresponding wx class, we
> could add one too and provide generic implementation just returning all wx
> windows and forwarding the rest of the functions to wxDisplay.

Yes, I also thought for all other platforms I'd add single WindowScene which basically corresponds to the display, if we expose it at all.

> One thing which is not clear to me is whether using UIWindowScene is
> optional or mandatory for the new applications?

If you want to support multiple windows on iPad you must opt in in the .plist,
on iPhone there is only one UIWindowScene at time, but on visionOS it is mandatory,

> And, if it's the former,
> what exactly do you gain from using it? The possibility to have multiple
> independent UIs in the same application?

Yes, exactly, before one app == one window, now a single app can have multiple UIWindowScenes with multiple UIWindows

https://developer.apple.com/videos/play/wwdc2019/212/
https://developer.apple.com/documentation/uikit/app_and_environment/scenes?language=objc

I also thought about just hiding this entire UIWindowScene situation behind toplevel windows which have no parent. But I'm not sure whether this would be sufficient at all.

Thanks,

Stefan

[Vadim Zeitlin]

On Sat, 24 Jun 2023 16:01:09 +0000 Stefan Csomor wrote:

SC> If you want to support multiple windows on iPad you must opt in in the .plist,
SC> on iPhone there is only one UIWindowScene at time, but on visionOS it is mandatory,

I see, thanks. But now I have the next question: other than changing
Info.plist (BTW, it would be nice to explain how it/what exactly needs to
be done in the "wxiOS" section of docs/doxygen/mainpages/platdetails.h),
what needs to be done at the code level? Can we just create wxTLWs as usual
or does it need to be done in some special way?

SC> I also thought about just hiding this entire UIWindowScene situation
SC> behind toplevel windows which have no parent. But I'm not sure whether
SC> this would be sufficient at all.

We could add a possibility to optionally associate a wxTLW with
wxWindowGroup ("group" seems more clear than "scene" to me) or maybe even a
more general wxUI object if necessary. But if we could allow simply
creating wxTLWs to work, this would be best, I think?

BTW, what is the current state of wxiOS? It looks like you're using it for
actual iPad applications?

VZ

[Stefan Csomor]

Hi Vadim

> SC> If you want to support multiple windows on iPad you must opt in in the .plist,
> SC> on iPhone there is only one UIWindowScene at time, but on visionOS it is mandatory,
>
> I see, thanks. But now I have the next question: other than changing
> Info.plist (BTW, it would be nice to explain how it/what exactly needs to
> be done in the "wxiOS" section of docs/doxygen/mainpages/platdetails.h),
> what needs to be done at the code level? Can we just create wxTLWs as usual
> or does it need to be done in some special way?

I have to test this thoroughly, only did my first experiments today under visionOS and while minimal sample runs now, I have to experiment with multi-windowing on iPadOS as well to make sure I catch all the situations properly, and update to new APIs as good as possible.

> SC> I also thought about just hiding this entire UIWindowScene situation
> SC> behind toplevel windows which have no parent. But I'm not sure whether
> SC> this would be sufficient at all.
>
> We could add a possibility to optionally associate a wxTLW with
> wxWindowGroup ("group" seems more clear than "scene" to me) or maybe even a
> more general wxUI object if necessary. But if we could allow simply
> creating wxTLWs to work, this would be best, I think?
>

Yes, I aim to at least have a 'transparent' good default behaviour

> BTW, what is the current state of wxiOS? It looks like you're using it for
> actual iPad applications?

I used to do trials with a prototype for my own app, but I haven't found the time to advance this further unfortunately. But our commercial card playing app which is based on wxIOS as well was using OpenGL, I ported that one to metal last year - so this was just in time, since OpenGL isn't available on VisionOS anymore ...

Best,

Stefan