Proxy support in wxWebView under Mac

[Vadim Zeitlin]

Hello,

I'd like to add proxy support to wxWebView, i.e. allow setting a specific
proxy to use instead of doing whatever it does by default (which, I think,
is not using any proxy at all under Linux and using the default proxy
configured at the OS level under Windows and Mac).

This was trivial with WebKit GTK and relatively simple with Edge (except
that it has to be set before calling Create() there, but we can live with
this). Unfortunately things are much worse under Mac, where the currently
used WKWebView doesn't support setting proxy at all, or at least not until
the very latest/yet unreleased OS versions.

AFAICS for an application which needs to use wxWebView with a custom proxy
under Mac there are only 2 choices:

1. Resurrect the old UIWebView-based wxWebView version, which doesn't
officially support setting proxy neither but where you could define
a custom URL protocol that would forward the requests via a proxy
(this hack doesn't work with WKWebView which does network stuff out
of process).

2. Use CEF-based wxWebView implementation, i.e. trying to revive this PR:
https://github.com/wxWidgets/wxWidgets/pull/706

Neither seems very appealing, but unless someone knows of another
solution, I'm going to have to choose between these ones. Out of those, it
seems like (1) ought to be simpler, i.e. it looks like it should be
possible to add the old code removed in 4df334cec4 (macOS/iOS: Use
WKWebView for wxWebView Implementation, 2020-11-09) as a new backend, which
could be used by people who really need to use a proxy. But there were, of
course, many changes to wxWebView since then that are only implemented in
the new, WKWebView-based, version, so it's probably not going to be trivial
neither.

OTOH making CEF backend work is probably going to be even worse, as it's
probably even more out of date compared to the other backends. But I don't
really know and would appreciate any insights from people who do, i.e.
mostly Tobias, I guess.

And, of course, if Stefan knows of some magic way to set proxy for
WKWebView, it would be ideal, but after looking at many discussions about
this being impossible, I'm afraid it really is.

Thanks in advance for your thoughts!
VZ

[Tobias T]

You don't have to answer, but I assume you where contracted to implement the proxy support?

I was also recently asked if I could implement this (but declined), probably by the same project (otherwise this would quite a coincidence).

Anyway here my initial considerations:

AFAICS for an application which needs to use wxWebView with a custom proxy
under Mac there are only 2 choices:

1. Resurrect the old UIWebView-based wxWebView version, which doesn't
officially support setting proxy neither but where you could define
a custom URL protocol that would forward the requests via a proxy
(this hack doesn't work with WKWebView which does network stuff out
of process).

2. Use CEF-based wxWebView implementation, i.e. trying to revive this PR:
https://github.com/wxWidgets/wxWidgets/pull/706

Neither seems very appealing, but unless someone knows of another
solution, I'm going to have to choose between these ones. Out of those, it
seems like (1) ought to be simpler, i.e. it looks like it should be
possible to add the old code removed in 4df334cec4 (macOS/iOS: Use
WKWebView for wxWebView Implementation, 2020-11-09) as a new backend, which
could be used by people who really need to use a proxy. But there were, of
course, many changes to wxWebView since then that are only implemented in
the new, WKWebView-based, version, so it's probably not going to be trivial
neither.

I agree with you that option 1 would probably be simpler initially. 

But I think it would be really ugly bringing back the deprecated UIWebView backend.

It's unfortunate enough that the IE backend has to be somewhat maintained when adding new features to wxWebView.

I'm not 100% sure how the custom URL protocol would work, but custom protocols are also supported with WKWebView.

Regardless of using WKWebView or UIWebView one potential problem with using a custom protocol, could be how same origin policy is handled. 

I know that I had a few issues with that. Especially on older macOS versions (10.13 and 10.14 if I recall it correctly). 

E.g. you can't send post requests to custom protocols with these older macOS versions. At least with WKWebView.

Resurrecting the CEF backend implementations on the other hand would probably be more work, but the open ToDo don't seem too daunting.

But obviously CEF and wxWidgets had quite a few changes since the PR. But some things might even become easier because of wxWebView

changes like async script execution support and some of the code I unified when integrating Edge and WKWebView support.

That are just my initial thoughts.

It's obviously your decision how to proceed. But I would much more like to see a CEF backend which could bring web based applications 

complete feature parity across platforms, than resurrecting usage of a deprecated API just for a very rare edge case.

An initial implementation wouldn't even need to provide complete feature parity with the current GTK, macOS and Edge backends at first.

Regards,

Tobias

 

[Vadim Zeitlin]

On Mon, 28 Aug 2023 07:21:23 -0700 (PDT) Tobias T wrote:

TT> You don't have to answer, but I assume you where contracted to implement
TT> the proxy support?

Hi,

It's not really a secret, so yes, I was indeed asked to implement this. I
don't know if I should mention the project, but it's indeed probably the
same people who contacted you. In any case, I thought it would be a
generally useful addition to wx, so I did agree to try to do it. If nothing
else, we're at least going to have support for setting proxy in wxGTK and
wxMSW now.

TT> I agree with you that option 1 would probably be simpler initially.
TT> But I think it would be really ugly bringing back the deprecated UIWebView
TT> backend.

The idea is that it would hopefully be only temporary and could be removed
once WKWebView gains support for setting proxy (which is planned, AFAIU)
and once we don't need to support OS versions with older WKWebView (which
is unfortunately going to take a few years...).

TT> It's unfortunate enough that the IE backend has to be somewhat maintained
TT> when adding new features to wxWebView.

Yes :-(

TT> I'm not 100% sure how the custom URL protocol would work, but custom
TT> protocols are also supported with WKWebView.

From what I read -- but it's just hearsay at this point as I haven't
tested this myself -- using custom protocol did work with UIWebView but
doesn't work with WKWebView because all network operations are not
performed in the process containing it at all but in another, system
process shared by all WKWebViews. So defining custom protocol is still
possible but it's just not used for anything.

TT> Regardless of using WKWebView or UIWebView one potential problem with using
TT> a custom protocol, could be how same origin policy is handled.

This is good (well, bad, but useful :-) to know, thanks.

TT> Resurrecting the CEF backend implementations on the other hand would
TT> probably be more work, but the open ToDo don't seem too daunting.
TT> But obviously CEF and wxWidgets had quite a few changes since the PR. But
TT> some things might even become easier because of wxWebView
TT> changes like async script execution support and some of the code I unified
TT> when integrating Edge and WKWebView support.
TT>
TT> That are just my initial thoughts.
TT>
TT> It's obviously your decision how to proceed.

Well, to be honest, not quite: I don't have enough "free" time to do it on
my own, so it's a decision of the people who would pay for this work. I was
a bit afraid of embarking on CEF work as I know it started back in 2013 and
then you resumed it in 2020 but it still never got merged and so I thought
that perhaps there were some big problems with merging it. But if you think
it is reasonable to do it, I agree that it seems more useful going forward.

TT> But I would much more like to see a CEF backend which could bring web
TT> based applications complete feature parity across platforms, than
TT> resurrecting usage of a deprecated API just for a very rare edge case.

Thanks for your feedback, I'll propose this as the better alternative
then.

VZ

[Tobias T]

Just FYI:

While browsing the WKWebView documentation I saw that, since macOS 14 (and iOS 17),

it seems to be possible to set proxy configuration on WKWebView.

See the documentation here.

From the looks of it it seems to support different proxy types. I would imagine this could also be supported for NSURLSession.