Performing a series of page interactions
Brian Theado
interactions. For instance if you want to visit a login page and fill
out a form and then click on a link at the resulting page and then
perform some other action on the third page, etc.
It still seems a little clumsy, but I'm pretty new to javascript and
maybe there are some easy improvements. Maybe using the coffeescript
syntax would be an improvement (once I've learned coffeescript).
My code includes an 'interact' function which takes a WebPage object
and two function callbacks as input. The first callback contains the
code which will cause the page to reload (clicking a link, submitting
a form, etc.). The second callback contains the code which will be
called when the page load completes. So in order to navigate to a
second page you nest another 'interact' call inside the second
callback of the first 'interact' call. To navigate to a third page,
nest another interact call inside the second interact call, etc.
interact(page, function() {/* open, click, or form submit here
*/}, function() {/* Nested call to interact here */})
Here is an example interaction. It visits the phantomjs google-code
issue list page, clicks on the details link for the last issue on the
page, then displays the date/time of the last comment on the page (or
issue creation if there are no comments):
<code>
phantom.injectJs("interact.js");
page = new WebPage()
interact(page, function() {
// Open the issues list page
this.open("http://code.google.com/p/phantomjs/issues/list");
}, function() {
interact (this, function() {
// Click on the details link for the last issue on the page
// Would be nice to be able to write:
// click (this, '#resultstable > tbody >
tr:nth-last-of-type(1) > td.id > a')
// but I didn't figure out how
this.evaluate(function() {simulateMouseClick('#resultstable >
tbody > tr:nth-last-of-type(1) > td.id > a')})
}, function() {
last_update = this.evaluate(function() {
// Creation date or date of most recent comment
n = document.querySelectorAll('.author > .date,
.issuecomment > .date');
return n.item(n.length-1).getAttribute('title');
});
console.log ("Last updated " + last_update);
phantom.exit();
})
})
</code>
<code name='interact.js'>
var pageNum = 1;
function interact (page,causes_load_callback, onload_callback) {
console.log("Setting onLoadFinished");
page.onLoadFinished = function (status) {
// While developing a script, it is difficult to predict which
interactions
// QtWebkit will consider a page reload, so set a temporary callback to
// notify the script writer that an unexpected page reload took place
page.onLoadFinished = function(status) {
console.log(pageNum + "Unexpected page load: " + status + " - " +
page.evaluate(function() {
return document.title + " - " + document.location.href;
})
);
page.render("/tmp/unexpected-" + pageNum + ".png");
phantom.exit();
};
// For debugging, log the results and render the page to a
temp directory
console.log (status + ":" + pageNum + " Page load complete - " +
page.evaluate(function() {
return document.title + " - " + document.location.href;
})
);
page.render("/tmp/" + pageNum + ".png");
pageNum++;
// Allow elements to be clicked and call the page load callback
page.injectJs("simclick.js");
console.log("about to apply callback");
onload_callback.apply(page);
};
// Call the callback which will run code that will result in page reload
causes_load_callback.apply(page);
console.log("Return from interact");
return;
}
</code>
<code name='simclick.js'>
// code from http://code.google.com/p/phantomjs/issues/detail?id=47
function simulateMouseClick(selector) {
var targets = document.querySelectorAll(selector),
evt = document.createEvent('MouseEvents'),
i, len;
evt.initMouseEvent("click", true, true, window, 0, 0, 0, 0, 0,
false, false, false, false, 0, null);
for ( i = 0, len = targets.length; i < len; ++i ) {
targets[i].dispatchEvent(evt);
}
}
</code>
If I could figure out how to add code to phantomjs to download a file
(http://code.google.com/p/phantomjs/issues/detail?id=52), then most of
my automation needs would be resolved.
Brian
Ariya Hidayat
Looking at the way your code looks like, I believe we shall do
something in order to help you (and others with similar goals) produce
a much more readable and debuggable script.
I was reluctant to go the synchronous way, I even went as far as
removing the semi-blocking sleep() from 1.2. I was under the
impression that using a JavaScript microlibrary which allows writing
sequential statement will be good enough.
However, based on the feedback from other expert in the web testing
tool, I realize maybe this is a mistake from my side. Maybe we need
the synchronous mode after all, consider the interactive testing is
usually procedural and step-by-step. Consider the following
(pseudocode):
open a login page
set the user name
set the wrong password intentionally
click the login button
wait till the page reloads
verify that login is not possible
Having the sync mode built-in, with specified time-out, will allow the
test script to run linearly. Once we have remote debugging capability,
it's supereasy to trace and debug.
Being able to do certain operations both async and sync can lead to
confusion. However, my understanding is people with use only one mode
most of the time. For running unit tests headlessly, you don't really
care about sync mode. For interactive/user-emulation tests, you don'y
really care about async.
What do you guys think?
--
Ariya Hidayat
http://www.linkedin.com/in/ariyahidayat
Ivan De Marino
James Roe
without loosing the asynchrony stuff.
Things like..
Steps(function(err, next) {
do_soemthing()
next(err)
, function () {}
, ...)
When we get our module system up, we could just make it an optional
thing to use, and it could be require'd. :) This is what I think is
optimal.
Ariya Hidayat
whether there is an alternative wrapper for that.
The mysterious question would be, how to make such a procedural
testing easy to write and east to debug? Especially if this is such a
complicated and big test.
For running unit-tests headless, what we have is already more than
enough. For creating smoke tests and user simulations, I just believe
it could be significantly improved.
Regards,
Ariya
Brian Theado
[...]
> Looking at the way your code looks like, I believe we shall do
> something in order to help you (and others with similar goals) produce
> a much more readable and debuggable script.
Sure, I wouldn't object to that. Long-term, that sounds useful.
Brian
Brian Theado
> There are ways to do synchronous steps in an asynchronous environment,
> without loosing the asynchrony stuff.
>
> Things like..
> Steps(function(err, next) {
> do_soemthing()
> next(err)
> , function () {}
> , ...)
Yeah, I think the recursive nesting of my original approach is much
more clumsy looking than it need be. I have made a change to allow
it to be linear. Here is the revised code:
<code>
phantom.injectJs("interact2.js");
page = new WebPage()
var steps = [
function() {
// Open the issues list page
this.open("http://code.google.com/p/phantomjs/issues/list");
},
function() {
// Click on the details link for the last issue on the page
// Would be nice to be able to write:
// click (this, '#resultstable > tbody >
tr:nth-last-of-type(1) > td.id > a')
// but I didn't figure out how
this.evaluate(function() {simulateMouseClick('#resultstable >
tbody > tr:nth-last-of-type(1) > td.id > a')})
},
function() {
last_update = this.evaluate(function() {
// Creation date or date of most recent comment
n = document.querySelectorAll('.author > .date,
.issuecomment > .date');
return n.item(n.length-1).getAttribute('title');
});
console.log ("Last updated " + last_update);
phantom.exit();
}
];
interact(page, steps)
</code>
<code interact2.js>
var pageNum = 1;
function interact (page,callback_list) {
console.log("Setting onLoadFinished");
page.onLoadFinished = function (status) {
// For debugging, log the results and render the page to a
temp directory
console.log (status + ":" + pageNum + " Page load complete - " +
page.evaluate(function() {
return document.title + " - " + document.location.href;
})
);
page.render("/tmp/" + pageNum + ".png");
pageNum++;
// Allow elements to be clicked
page.injectJs("simclick.js");
console.log("about to apply callback");
// Call interact again--pass only the functions that
// haven't been invoked yet
interact(page, callback_list.slice(1));
};
// Call the first callback in the list. It should result in page
reload so the
// above onLoadFinished gets called
callback_list[0].apply(page);
console.log("Return from interact");
return;
}
</code>
<code simclick.js -- unchanged from last email/>
Brian
Peter Lyons
supports 2 styles of control flow: 1. Just a simple linear list using
an array of callbacks and 2. a dynamic linked list type approach where
the next action can be determined on the fly, which should allow
conditionals, loops, skips, etc.
Here's my <interact.coffee>
---------------------------------
pageNum = 1
window.interact = (page, actions, verbose=true) ->
header = (status) ->
return if not verbose
console.log "Page #{pageNum}: #{status}. " + \
page.evaluate ->
document.location.href + " - " + document.title
pageNum++
if Array.isArray actions
#Simple linear sequence of actions
page.onLoadFinished = (status) ->
header status
#Call interact again--pass only the functions that
#haven't been invoked yet
interact page, actions.slice 1
#Call the first callback in the list. It should result in page
#reload so the
#above onLoadFinished gets called
actions[0](page)
else
#Object style interaction. Allows loops, dynamic logic, etc
page.onLoadFinished = (status) ->
header status
actions.next(page, actions)
actions.next(page, actions)
----------------------------------
Here's an example of a multi-page interaction (signing in and backing
up my workflowy.com data) using the linear style array approach:
workflowy_backup_array.coffee
----------------------------
out = (message) ->
console.log message
phantom.injectJs 'interact.coffee'
page = new WebPage()
page.settings.loadImages = false
page.settings.loadPlugins = false
page.onConsoleMessage = (message) ->
if message.indexOf("Unsafe JavaScript") == 0
return
out message
actions = [
(page) -> page.open 'https://workflowy.com'
(page) ->
page.evaluate ->
$('#id_username').val 'pe...@peterlyons.com'
$('#id_password').val 'My super secret password'
$('#login').submit()
(page) -> page.evaluate -> $('#workflowy').exportIt()
(page) ->
page.evaluate ->
console.log $('#exportPopup .textContainer pre').html()
phantom.exit()
]
window.interact page, action
-------------------------------
And here's a version using the object style.
workflowy_backup_obj.coffee
-----------------------------
phantom.injectJs 'interact.coffee'
out = (message) ->
console.log message
page = new WebPage()
page.settings.loadImages = false
page.settings.loadPlugins = false
page.onConsoleMessage = (message) ->
if message.indexOf("Unsafe JavaScript") == 0
return
out message
actions =
open: (page, actions) ->
actions.next = actions.signIn
page.open 'https://workflowy.com'
signIn: (page, actions)->
actions.next = actions.export
page.evaluate ->
$('#id_username').val 'pe...@peterlyons.com'
$('#id_password').val 'My super secret password'
$('#login').submit()
export: (page, actions) ->
actions.next = actions.print
page.evaluate -> $('#workflowy').exportIt()
print: (page) ->
page.evaluate ->
console.log $('#exportPopup .textContainer pre').html()
phantom.exit()
actions.next = actions.open
interact page, actions
----------------------------
Please let me know if you find that useful or can offer other
approaches to this type of use case.
Brian Theado
On Sat, Jul 2, 2011 at 2:39 PM, Peter Lyons <pe...@peterlyons.com> wrote:
> I made my own variation of Brian's interact.js (in coffeescript).
[...]
> Please let me know if you find that useful or can offer other
> approaches to this type of use case.
I like it and have switched the code I was writing to use it. Thanks
for sharing.
I'm new to javascript/dom coding...could you explain why you attached
the function to the window object?
> window.interact = (page, actions, verbose=true) ->
Brian
Peter Lyons
> I like it and have switched the code I was writing to use it. Thanks
> for sharing.
>
> I'm new to javascript/dom coding...could you explain why you attached
> the function to the window object?
>
> > window.interact = (page, actions, verbose=true) ->
all variables with "var" so they are local. If you want to make a
global variable, you must do it explicitly like above.
Mark Riggins
Mark Riggins
Ariya Hidayat
Please star issue 157 to follow the updates.
Bartosz Nitka
https://github.com/niteria/phantomjs/commit/a64d0bcd535b7b54e15b1eb40f7d5d57bb6d14cf
I've found that asynchronous style forces me to write too much boilerplate and is hard to debug.
Also calling asynchronous open multiple times on the same page leads to unpredictable results, sometimes segfaults.
It would be nice to add timeout to the synchronous version, but so far I didn't need it and I think webkit might have
its own timeout.
Another solution might be to return a token from async call that you can wait on, but if it's too general it might be too easy
to do deadlocks.
Bartosz Nitka
Ariya Hidayat
sync. IMO this is better than alternate function name for each
operation. In the context of testing using PhantomJS, I can't imagine
the situation where someone would heavily mix sync and async
operations anyway.
Thoughts?
--
Ariya Hidayat
http://www.google.com/search?q=ariya+hidayat
Bartosz Nitka
compatible with phantomjs.
Async gives the promise of parallelism (or at least non-blocking IO),
but the way things are now you'll have to use
frameworks like Step or flow-js to use it (because if you want to call
phantom.exit
at the end you have to wait for async calls) and it gets messy pretty
quickly (maybe with coffescript it doesn't,
I haven't tried it yet, main problem is lot of anonymous function
noise).
On an unrelated note you can do poor mans tail call optimization with
setTimeout and patch your favorite
framework with it.
There's also another problem with parallelism and phantomjs. With my
use case I wanted
to have different set of cookies/cache for each WebPage object and
currently all WebPage objects
share the same NetworkManager which remembers cookies/cache.
So if you wanted to take a screenshot of each logged in user's page in
parallel there's no way to do it now anyway.
I run everything sequentially so I just implemented clear() which just
replaces NetworkManager with a new one.
Only use case for async in phantomjs I can think of is doing something
in parallel to a set of completely unrelated pages,
but if they're unrelated you can just run multiple phantomjs
processes.
I'd love to hear about other async + phantomjs use cases.
