Web Page triggering phantom callback

[Robert Ferney]

I'd like to be able to use the java-script in the webpage to trigger a callback in the code driving phantom. 

For example, if I'm using Highcharts to render a chart, and I want a trigger a render once Highcharts is done processing. 

[Mike Alaimo]

There is a phantomjs example out there called waitfor.js I believe.  You can modify it to check for the existance of svg tags.  I'm pretty sure if those are in the dom then hc is done or almost done due to animation..

Let me know if you cannot find it.  I have it somewhere.

Mike

[Bryan Bishop]

On Tue, May 15, 2012 at 2:28 PM, Robert Ferney <capn...@gmail.com> wrote:

For example, if I'm using Highcharts to render a chart, and I want a trigger a render once Highcharts is done processing.

Quick outline of how I'd do this:

- create a WebPage object

- load up the page

- onLoadFinished -> inject some javascript into highchart's framework to bind to an event

- when whatever highchart event/function is done, you'll set a variable somewhere on the page

- use setInterval to call a function that checks the state of the page by injecting some javascript and checking for that variable

- eventually your setInterval function will decide it's time to render (and, you should probably disable this setInterval for the duration of the rest of the script)

I'd be interested in hearing any better plans.

- Bryan
http://heybryan.org/
1 512 203 0507

[Mike Alaimo]

Another method, but some may say is silly is to send an ajax request out.  You can check using webpage.onresourceseny and webpage.onresourcerecevied.

If something is sent and not received the page is not yet done.

So your hc is done, triiger ajax to page which print true.  Then the handler onresourcerecevied knows its complete.

If you can have a prior count of total hc on page this can be easy to accomplish.

Webpage.onresourcereceveid = function {
If (r_recevied == test_resource) {
Total_hc--
}
If (total_hc == 0) // page loaded

[Mike Alaimo]

None the less when doing things like this don't make the page load take forver.  It may be nice to die gracefully in the case where hc was not loaded in 3 seconds or something.

Setinterval if no exit or way out is set or even with the onreceived total_hc == 0. There is the potential for the output to stay inside of a phantomjs instance. 

Just saying make sure of a way out.

[James Greene]

I believe Ivan added an "onCallback" mechanism in 1.6 to address this (such that the client can trigger PhantomJS code) for GhostDriver.
~~James

[Ivan De Marino]

Yes, that API is there and working, BUT it's undocumented because we want to do it differently.

For now suites the purposes of GhostDriver.

If you are ok with us breaking compatibility all of a sudden in the future and with little (or no) warning, please go ahead and have fun :)

You can pass to the callback any JS type except things like DOM elements and similia, as they belong  (and work) only within the page context.

In other words, the same kind of logic you have with calls to "evaluate()".

On 24 July 2012 14:23, James Greene <james.m...@gmail.com> wrote:

I believe Ivan added an "onCallback" mechanism in 1.6 to address this (such that the client can trigger PhantomJS code) for GhostDriver.

~~James

--

--

Ivan De Marino

Coder, Technologist, Cook, Italian

blog.ivandemarino.me | www.linkedin.com/in/ivandemarino | twitter.com/detronizator

[James Greene]

Ivan:

In lieu of documentation, can you please give just one quick little example?  I promise not to be upset when the API changes. :)

~~James

[Ivan De Marino]

OK.

But you take the responsibility of this.

At the end of the day, if you read the source code, you can work it out yourself.

But, once again, this will probably change.

From within the page context:

page.evaluate(function() {

   ...

   window.callPhantom(somethingFromThePage);

});

page.onCallback = function(somethingFromThePage) {

   ...

}

naughty boy...

~~James

--
 
 
 

[James Greene]

Haha, thanks Ivan! :-P

~~James

[Ariya Hidayat]

Someone should file a feature request in our issue tracker.

I believe the right way to solve this problem is using Cross-document
messaging. If someone wants to volunteer to investigate that, it will
be appreciated. Note that if postMessage does not work for whatever
reason, then we have to make it working.



--
Ariya Hidayat, http://ariya.ofilabs.com
http://twitter.com/ariyahidayat

[James Greene]

Ariya:
I like the postMessage idea. I assume that means the WebPage object gets a new onPostMessage/onMessagePosted callback hook, then?

Since window.postMessage requires a reference to the "other window", will PhantomJS attach a reference onto the WebPage's window object, like window.phantom (with a postMessage method: window.phantom.postMessage)?

Or would you just have the window postMessage to itself (window.postMessage) but with setting the targetOrigin to a special value like "PhantomJS", perhaps?

[Ivan De Marino]

I agree.

I have shared this with James, with a big warning about us dropping it without any warning.

I can't get around to work on this right now, but if now one does I might do it in the future :)

--

[Ariya Hidayat]

We need to avoid window.phantom for the sake of security sandbox, i.e.
the client page should never realize that it's running within
PhantomJS (as opposed to other normal browsers).

I think the special value for the targetOrigin would be a good solution.

[Ivan De Marino]

Well, the way it is done at the moment can't be of any harm without the explicit intervention of the developer: it works as a callback and, even if the webpage pokes it, it's still up to the dev to register an "harmful callback" to it.

Nothing of this will change with "postMessage": it will be exactly the same security mechanism.

--

[James Greene]

Going with window.postMessage does, however, utilize an existing API and leads to the introduction of PhantomJS-side handlers (page.onPostMessage/page.onMessagePosted) for such, which likely would've desired in future iterations anyway.
~~James

[James Greene]

Ivan/Ariya —

Any updates on this idea?  I've added some notes and a link to this discussion on Issue #224 (which was more basic but about IPC, in general).

~~James

[Ivan De Marino]

Just posted here a surely terrible idea: http://code.google.com/p/phantomjs/issues/detail?id=224#c3

--
 
 
 

[James Greene]

Added my two cents on your idea.

~~James

[Ivan De Marino]

I'm having second thoughts about this:

1) targetOrigin is to apply a "filter" on the targets that can receive this message: using that to point at phantom is "ugly" code wise

2) as long as the phantom interface that we would expose in the page is "safe", I don't see where the issue is

3) in the same way website couldn't detect they are "in phantom" detecting that object, they could just read the user agent: I fail to see the issue with exposing a well crafted/controlled interface to the webpage space

4) of course, as long as we do it in a "controlled manner": no polluting other methods

Ariya, I think you should reconsider this.

ALSO I think in 4.8.2 we got support for "postMessage" in QtWebKit

[James Greene]

Here's the Gist I made for Ivan and I to reference while we were discussing the matter just now:

     https://gist.github.com/3716654 

Disclaimer: As the Gist description states, it shows "injecting the window.postMessage API for phantom (short-term version)".  In other words, using our existing utilities to get the desired API in place before the pending 1.7 release (9 days from now).  This is NOT how I would expect to see it implemented in the long-term!

Anyway, personally, I'm still in favor of using `window.postMessage` with a custom targetOrigin versus exposing non-standard APIs... but I was the one who mentioned that to begin with, so all I'm really saying is that I'm still on board with my original opinion.

Sorry, Ivan.  Happy to continue discussing this API design with you guys, though, so keep the ideas coming....  :)

~~James

[Ivan De Marino]

The problem I have with postMessage respect to what I already implemented is that it's a different, unfitting paradigm.

"window.postMessage" is designed for message passing for windows, filtered by targetOrigin. There is a concept of "window" playing here.

sending stuff to the Phantom scope with postMessage, is like saying that phantom IS a window.

While underneath that is true at the moment (we do run a webpage as our runtime), that might be (hopefully) a separate, more tailored context in the future.

And, if we want to "postMessage" to phantom from within a webpage, we still need to pass a phantom (window) reference into the webpage to do that: using the targetOrigin stinks.

--
 
 
 

[James Greene]

Hmm, I can see your point there.  

I would be OK trying to do it properly like this:

if (window.phantom && window.phantom.postMessage){
    // targetOrigin needs to be "*" unless we give the phantom window a fake origin, e.g. "http://phantom.js"
    window.phantom.postMessage("hi", "*");
}

 

Exposing the "phantom" window to the client may also lead to confusion, though, as users think they can utilize it for other actual "window" purposes beyond just the "postMessage" functionality.  Do we have any plans to expose more direct window functionality than "postMessage"?  I'm guessing not.

Given that consideration, I would also be OK with a one-off exposure like the `window.callPhantom` function that Ivan exposed in PhantomJS 1.6.

Either way, I think there will still be a need in the near future to implement PhantomJS-side callbacks for `postMessage` events sent on the client.

Ariya, others: thoughts?

~~James

[Ariya Hidayat]

> And, if we want to "postMessage" to phantom from within a webpage, we still
> need to pass a phantom (window) reference into the webpage to do that: using
> the targetOrigin stinks.

Can't we even a special magic target for that?

[Ariya Hidayat]

> Can't we even a special magic target for that?

I meant "invent" and not even.

[Ivan De Marino]

The semantic of the API is:

- targetWindow.postMessage(message, targetOrigin)

TargetOrigin is well explained here: https://developer.mozilla.org/en-US/docs/DOM/window.postMessage

Even if we decide that "__phantomjs__" is a special TargetOrigin for us, It would still be "awkward" that we send stuff to "phantom", using the current window as target.

Don't you recon?

I'm open to change my opinion, but I'd still be thinking that the API is "ugly".

--

[Trevor]

In addition to postMessage, I think something like this should be possible:

page.evaluateAsync(function(done) {
    setTimeout(function() {
        done('rat');
    }, 3000);
}, function(vermin) {
    // vermin is 'rat'
});

Has this been considered?

[Ivan De Marino]

That could be built with postMessage/current_callback mechanism.

But that's not a bad idea: please file an issue.

I don't think we can make it by 1.7, but we could definitely think about a clean way to have that.

--
 
 
 

[Trevor]

Issue 778.

[Ariya Hidayat]

> The semantic of the API is:
> - targetWindow.postMessage(message, targetOrigin)
>
> TargetOrigin is well explained here:
> https://developer.mozilla.org/en-US/docs/DOM/window.postMessage
>
> Even if we decide that "__phantomjs__" is a special TargetOrigin for us, It
> would still be "awkward" that we send stuff to "phantom", using the current
> window as target.
> Don't you recon?
>
> I'm open to change my opinion, but I'd still be thinking that the API is
> "ugly".

I still don't see which part of this is "ugly". It may be just
implementation details which need to be flushed out anyway.

The good thing is that this is the standard cross-document (web)
messaging and I think it's beneficial to adopt a model every web
developer would be familiar with.

[Ivan De Marino]

Well, the model "web developer would be familiar with" require a window handle to the phantom "window", to send stuff to it.

That's what it's "ugly" for me.

The only thing we would be fulfilling is the "postMessage" api name. Nothing else really.

--

[James Greene]

Well, I am getting the impression that our opinions on this are too scattered and unproven to push into 1.7 before Saturday with ample testing. Would you all agree?

In the meanwhile....
Ivan: You've mentioned that you think the new version of QtWebKit has the support we need to implement the signals (or override the virtual methods, or whatever) for `Window.postMessage` calls.  Can you refer us to any documentation, discussions, bugs, etc. on its implementation?  I'd like to start looking at whatever info I can since I figure we will want to implement that functionality even if we choose not to utilize it for direct communication back to PhantomJS.

[Ariya Hidayat]

It's always subjective: "special" for someone is "ugly" for someone
else. The trick is to find the specialization so that it feels good
enough.

We likely need to hash it again for 1.8. One criteria will be that the
client page should never know whether it runs under PhantomJS or not.
The current undocumented callback breaches this, any web page can
detect whether 'callPhantom' exists or not and thereby providing a
reliable sniffing mechanism.

Regards,

[Ariya Hidayat]

> Well, I am getting the impression that our opinions on this are too
> scattered and unproven to push into 1.7 before Saturday with ample testing.
> Would you all agree?

Agree.

> In the meanwhile....
> Ivan: You've mentioned that you think the new version of QtWebKit has the
> support we need to implement the signals (or override the virtual methods,
> or whatever) for `Window.postMessage` calls. Can you refer us to any
> documentation, discussions, bugs, etc. on its implementation? I'd like to
> start looking at whatever info I can since I figure we will want to
> implement that functionality even if we choose not to utilize it for direct
> communication back to PhantomJS.

It does not exist right now, so the route must be specially
implemented for PhantomJS.

While we're at that, one "specialization" which I had in mind is the
catch-all onMessageReceived. This does not require us to create a
magical target for PhantomJS because we simply catch all posted
message. Example:

page.onMessageReceived = function (text) { /* do something */ };

page.evaluate(function () {
postMessage('FooBar');
}

It's then a matter of setting up the right covenant, e.g. using
messages with the chosen, agreed prefix, to facilitate the
communication.


Regards,

[James Greene]

I agree that the addition of the `callPhantom` function to the `window` object is indeed a dead giveaway that the page is running in PhantomJS.  However, as Ivan mention, the default user agent (which most people don't change) also indicates that the page is running in PhantomJS.  That said, the user agent can be modified without impacting PhantomJS behavior and capabilities, whereas the `callPhantom` method cannot (and thus, it becomes a reliable sniffing mechanism, as Ariya said).

~~James

[James Greene]

We definitely want to implement a catch-all receiver for `Window.postMessage` calls.  However, your callback function signature is in need of tweaking to one of the following variations:

// Signature #1: Single argument, matches properties of spec
page.onMessageReceived = function(messageEvent) {
    // `data` is not required to be a string, it can be anything that the "structured clone algorithm" can support... so basically JSON)
    var data = messageEvent.data;
    // Caller's origin (protocol + hostname + port)
    var origin = messageEvent.origin;
    // Not so sure we can provide `source`, which is a reference to the calling `Window` object
    var source = messageEvent.source;

    // Do stuff....
};

//Signature #2: Spread/splat the MessageEvent properties into individual arguments
page.onMessageReceived = function(data, origin, source){
    // Do stuff....
};

Which would you prefer?

~~James

[Ariya Hidayat]

Because the goal is capture special agreed message, the form

page.onMessageReceived = function(data, origin){ ... }

is sufficient. Most likely, origin is not needed.

[James Greene]

Right, `origin` would primarily be for the user's sake for filtering and informational purposes, e.g. 'Received data "magicalResult" from http://www.google.com/'.

~~James

[Ivan De Marino]

We likely need to hash it again for 1.8. One criteria will be that the
client page should never know whether it runs under PhantomJS or not.
The current undocumented callback breaches this, any web page can
detect whether 'callPhantom' exists or not and thereby providing a
reliable sniffing mechanism.

Some time ago we had a similar conversation for the FS API.

The very reason for redesigning the way Phantom exposed it's API (our "sandbox" today) was that "website could malevolently use those API to interact with the FS and what not".

That made sense and I agreed that the approach was:

- page run in a vanilla browser js environment

- the API are outside of that scope

Now, this situation is very different.

I don't see what harm is for a Website to know that it's running in a PhantomJS page.

It's not like they could use that knowledge to implement malevolent behaviours. The page is still confined.

The API I propose to expose (window.callPhantom  or whatever we like to call it) it allows the page to pass it anything (not just strings - JSONs) but that's it: it's up to the script developer to do (or ignore) such messages.

Hence, giving control where control is due (the script running the page, not the page itself).

EVERY website that is worth mentioning on the internet does:

- feature detection

- useragent sniffing

- other techniques to detect the browser they are running into

What would be the harm in "letting them know" that it's Phantom rather than Firefox or Chrome or whatever?

It's a genuine question: I really don't see where this concern is coming from.

Ivan

[James Greene]

Any security issues would be a result of how the developer writes his `page.onCallback` handler.  For example, if they made the decision to evaluate whatever data arrived from the client, they would be vulnerable to script injection attacks.

Basic idea would be:

1. Send code that, when evaluated, will either
    (a) grab all the code (potentially easy by exploiting `arguments.callee` and `Function#toString`, if the whole script is wrapped in an IIFE), or
    (b) Print all of the variables and their values in the current and/or global scopes
2. Put all that info into a string
3. Do a page.evaluate that sends that string as POST data on an XHR request to your own domain.
4. Profit!

~~James

[Ariya Hidayat]

> Some time ago we had a similar conversation for the FS API.
> The very reason for redesigning the way Phantom exposed it's API (our
> "sandbox" today) was that "website could malevolently use those API to
> interact with the FS and what not".
> That made sense and I agreed that the approach was:
> - page run in a vanilla browser js environment
> - the API are outside of that scope

The filesystem support was added after WebPage abstraction was
introduced. Regardless whether filesystem access would have been
available or not, the sandboxed web page will be always there.

> Now, this situation is very different.
> I don't see what harm is for a Website to know that it's running in a
> PhantomJS page.

The question should be the other way around. What would be the benefit
of a web page knowing that it's running inside PhantomJS?

> It's not like they could use that knowledge to implement malevolent
> behaviours. The page is still confined.
>
> The API I propose to expose (window.callPhantom or whatever we like to call
> it) it allows the page to pass it anything (not just strings - JSONs) but
> that's it: it's up to the script developer to do (or ignore) such messages.
> Hence, giving control where control is due (the script running the page, not
> the page itself).

I showed another solution/specialization in the other email which gets
the same result without additional exposure.

> EVERY website that is worth mentioning on the internet does:
> - feature detection
> - useragent sniffing
> - other techniques to detect the browser they are running into

There is a difference. Once someday we're technically on par with e.g
Chromium (may happen someday), how would a web page known that it is
executed in a different flavor of Chromium? This is the difference
between "I'm on vacation in Hawaii" vs "You may or may not know that
I'm not at home".

> What would be the harm in "letting them know" that it's Phantom rather than
> Firefox or Chrome or whatever?

Security and choice. It's easier to breach a system when you know as
lot of thing upfront.

It's also about giving a choice, if your PhantomJS script wants to
tell the page that it is in PhantomJS environment, you can go ahead
and sprinkle its window object with gazillions objects. Imagine if a
screen capture service is given a different content by the web page
just because the web page does not like to be treated under PhantomJS.

> It's a genuine question: I really don't see where this concern is coming
> from.

In this wild web world, the principle is always pragmatism. For every
step, it's not about why we can't do it, it's rather why we should do
it. Once you cross it, there is no turning back.

[Ivan De Marino]

> In the meanwhile....
> Ivan: You've mentioned that you think the new version of QtWebKit has the
> support we need to implement the signals (or override the virtual methods,
> or whatever) for `Window.postMessage` calls.  Can you refer us to any
> documentation, discussions, bugs, etc. on its implementation?  I'd like to
> start looking at whatever info I can since I figure we will want to
> implement that functionality even if we choose not to utilize it for direct
> communication back to PhantomJS.

It does not exist right now, so the route must be specially
implemented for PhantomJS.

I don't think so.

I tried to use the "postMessage" method within the current "page" object: it does work.

The only issue is that we don't have a Qt-side support for it to manipulate it to our needs.

Maybe I didn't explain myself clear enough.

 

--
Ariya Hidayat, http://ariya.ofilabs.com
http://twitter.com/ariyahidayat

--

[James Greene]

Ariya, Detro:

Can we continue discussing this?  I created a Gist for it way back when and tagged you two in it but apparently GitHub Gists don't notify the users of mentions (which I didn't realize yet at the time).

   https://gist.github.com/JamesMGreene/3716654

~~James

[James Greene]

Ping!

~~James

--

You received this message because you are subscribed to the Google Groups "phantomjs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phantomjs+...@googlegroups.com.
Visit this group at http://groups.google.com/group/phantomjs?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[James Greene]

Ping!  Can we continue discussing? :)

Sincerely,
    James Greene