Hello.
My use case is web scraping dynamic content (javascript). I'd like to run PhantomJS as a server and connect to it using Selenium Remote webdriver instances. Every webdriver instance creates a session within PhantomJS process. But even though capabilities contain "proxy" section, it is impossible to set an individual proxy per session.
setProxy is calling QNetworkProxy::setApplicationProxy(proxy) which sets it for the whole application:
Application-wide proxying can be enabled for all subsequent socket connections through the use of the QNetworkProxy::setApplicationProxy() function.
Now, is it possible to tweak PhantomJS in such a way that would open a possibility to set proxy per session? API seems to be already there - desired capabilities passed when instantiating webdriver.
As for the Qt framework, it looks like it is possible:
Proxying can be enabled on a per-socket basis or for all network communication in an application. A newly opened socket can be made to use a proxy by calling its QAbstractSocket::setProxy() function before it is connected.
I believe I could find a time to work on this improvement. For now I need advice and some guidance.
Related issue:
https://groups.google.com/forum/#!searchin/phantomjs/proxy|sort:date/phantomjs/ejDUqz2VYwQ/HvMtG9R3BwAJ
Extra question is about that comment:
Running phantomjs as webserver, will introduce memory leak. And you must monitor it and restart the process if the process hit certain limit.
I will not use phantomjs as webserver just for achieving better performance, as this will lead to a complicated memory management.
What exactly is leaking? Was it fixed by now?
Thanks,
Remy