Page polls remote url which stops casper execution

[Mike Pearce]

Hi,

I've got a script which logs in, then visits a page. I want to take a screengrab of the page, but the execution pauses on the page because the page has some AJAX which is polling a remote URL to keep some data on the page up to date. How do I tell casper to stop waiting for the poll to finish and just move on to the next step?

Cheers,

Mike

[Julien Escueta]

You want to structure your script like so,

casper.then(function waitForStuff() {

  casper.waitForResource( 'url of the AJAX request' );

});

casper.then(function screenGrab() {

  casper.capture( ... );

});

Another option is to hook onto the resource.received event, documented here, http://casperjs.org/events-filters.html

[Mike Pearce]

Thanks for the reply Julien

My code looks like this:

casper.thenOpen("https://a-url.com/a-page#Stats", function() {

});

It just hangs here, because this page is called http://do-a-poll.com/poll every 5 seconds, so I guess casper thinks it hasn't finished loading. I've tried adding: 

casper.thenOpen("https://a-url.com/a-page#Stats", function() {

        this.on('resource.received', function(resource) {

   casper.capture('test2.png');

})

});

But, whatever I do, this is always the verbose output, it just hangs on "url changed to"

[debug] [phantom] opening url: https://url.com/accounts/***/stuff/9248#Stats, HTTP GET

[debug] [phantom] Navigation requested: url=https://url.com/accounts/***/stuff/9248#Stats, type=Other, lock=true, isMainFrame=true

[debug] [phantom] url changed to "https://url.com/accounts/***/stuff/9248#Stats"

[Martin Myrseth]

I'm having a similar issue with a page which has a long-polling mechanism going in the background. Turning on Casper logging, it stops at the same place: "url changed to ...".

On the server-side I can see that Phantom is continuing to send new requests as the long-polling times out. The page is fully functional in that sense. It's just that Casper won't trigger the next action, as if the document isn't ready loaded or something.

The next action I'm attempting is a waitForSelector, which is completely unresponsive. Supplying a timeout has no effect either.

[Martin Myrseth]

I can verify that it's the onLoadFinished handler which is not properly called. This is probably a Phantom issue entirely? I'm using Phantom v1.8.1 currently. I'll have a check over there.

[Māris Krivtežs]

Does anybody have solved this issue or has any workaround?

Maris

[Fanch]

I have the same problem too, i didn't manage to hack this 'block in step' using event (resource.received).
Can't we ask casper to force the passage to the next step (set the flags to false) when the resource/selector/text wanted appears on the page? And block the next resources (prevent the refreshing loop resources). And manually ask for a refresh if needed.

[Fanch]

Prevent polling so. By the way, slimerJS does the same.

[Fanch]

This ugly hack works though :

casper.wait(7000, function(){
    //wait 7sec, to be sure you open the next url then do what you want
    //ugly wait time, but that works
});

casper.thenOpen("https://a-url.com/a-page#Stats", function() {

    //stuck

});

[Fanch]

Well.... until you don't want to add a new step in the stack...

[Māris Krivtežs]

If I have such test:

var url = 'http://localhost:25173/';

casper.test.begin('Can open companies page', function(test){
  casper.start(url, function(){
    // login here
  }).thenClick('a[href^="/Companies"]', function(){ // <-- here it hangs and function never get's called
    this.echo(this.getCurrentUrl());
    test.assert(url + 'Companies' == this.getCurrentUrl(), "Companies page opened");
  }).run(function(){
    test.done();
  });
});

I'd like to do more steps in the test of course, but can't figure it out how to get past this one.

It would be good to have some filter on resource requests where I could setup some pattern to ignore waiting of some requests.

[Fanch]

Did someone post the issue?

[Māris Krivtežs]

There is an issue at phantom.js:

https://github.com/ariya/phantomjs/issues/12048#issuecomment-39997353

Not sure if casper.js can handle this or it's phantom.js issue.

[Māris Krivtežs]

My colleague solved the issue for my case. I am using SignalR which does long polling and solution is to stop it:

casper.on('resource.received', function(request){

    // Stop SignalR to avoid Casper from hanging

    if((/signalr\/connect/gi).test(request.url)){

        this.evaluate(function(){

            $.connection.hub.stop();

        });