best recommended way to do multi-step multi-open, sync. Willing to Hire!

52 views
Skip to first unread message

Chris Fortune

unread,
Jul 27, 2016, 9:40:25 PM7/27/16
to phantomjs
Hello,

I need help selecting the right modules / functions / techniques to do a sequential step-by-step navigation through a login and post process.  There are 6 pages that have to be traversed, and two other possible pages that might appear along the way.   I need to maintain cookies (login session) throughout the entire procedure.   Any help much appreciated.  I'm willing to hire a tutor if you want to spend some hours helping me out / debugging, etc.. 

The sequence of events is this:

  1. Page 1:
    1. select a link and click() on it.
  2. Page 2:
    1. select a link and click() on it.
  3. Page 3:
    1. select some page data
    2. do a custom ajax query to another one of our servers, send it the page data, receive the results
    3. fill in form data with results
    4. click submit button
  4. Page 4:
    1. assess what page we got, and switch depending on what html the server gave us
      1. Optionally loop page 3 function a maximum of 4 times if it failed,
      2. Optionally click a checkbox and submit page X
  5. Page 5:
    1. fill out form
    2. post / submit / click()
  6. Page 6:
    1. select the html status message and console.log("Status: " + msg)
    2. phantom.exit()



Bo Zou

unread,
Jul 28, 2016, 12:20:11 AM7/28/16
to phantomjs

Chris Fortune

unread,
Jul 28, 2016, 2:23:55 PM7/28/16
to phantomjs
Hello Bo Zou,

Thanks.  Have you ever used knysa to build such an app?  Is it stable / mature ? 

Is it recommended by other phantomjs programmer?

Are there any other recommendations?  Anyone?

Chris

Bo Zou

unread,
Jul 29, 2016, 11:52:44 AM7/29/16
to phantomjs
Hi Chris,

I wrote knysa.  I used it daily.  There are examples: https://github.com/pahakia/knysa/tree/master/examples.

knysa script is converted into phantomjs script.  The converted script is readable.

It makes web page navigation so much easier especially the call back from inside the browser.  For example, you can wait for the result of an ajax call which is on the browser side.

Give it a try.  There's an intro as well as quite a few real world examples.

Bo

Bo Zou

unread,
Jul 29, 2016, 12:26:19 PM7/29/16
to phantomjs
pseudo code for your application:

kflow.knysa_open(some_url); // example: kflow.knysa_open("https://ottawa.bibliocommons.com/checkedout/index/overdue?mobile=0");
kflow.knysa_click(querySelectorForLinkOne); // example kflow.knysa_click('a[href="/index/checked_out"]');
// the click should result in a new page
// but nowadays, lots of page are just Single Page App with Javascript, so a click does to result in page nav
// in that case, you can use kflow.click(querySelector);
// then use this while loop to check if the page is ready
//     iteration = -1;
//     while (++ iteration < 100 && kflow.exists(querySelector)) {
//         kflow.sleep(100);   // sleep 0.1 second
//     }  // max sleep 100*0.1 = 10 seconds, you can adjust sleep interval and iteration
//     if (iteration == 100) {
//         throw new Error('some page is not loaded');
//     }
numRetries = 0;
while (true) {
   kflow.knysa_click(querySelectorForLink2);
   var retValue = kflow.knysa_evaluate(function(kflowId) {
      // this code will be executed in the browser
      // so you can do what ever browser side javascript
      var someData = ...; // collect your data
      $.ajax({
                dataType: 'json',
                url: "http://some/url",
                data: "some data",
                success: function(e) {
                  console.log("success: " + JSON.stringify(e));
                  window.callPhantom({kflowId : kflowId, status: 'success', data: e});
                  // retValue will be set to whatever passed into callPhantom() above
                },
                failure: function(e) {
                  console.log("failure: " + JSON.stringify(e));
                  window.callPhantom({kflowId : kflowId, status: 'failure', data: e});
                   // retValue will be set to whatever passed into callPhantom() above
                }
            });
   }, kflow.getId());
   if (retValue.status == 'failure') {
      throw new Error("failed ajax: " + JSON.stringify(retValue));
   }
   kflow.fill(querySelectorForYourForm, {key1 : value1, key2 : value2, ...}, false);
   // example: kflow.fill('form.loginForm.left', { 'name' : username, 'user_pin' : password }, false);
   kflow.knysa_click(querySelectorForSubmitButton);  // example: kflow.knysa_click('input[value="Log In"]');
   if (kflow.exists(querSelector)) { // check if certain condition is met, you can also use if (kflow.evalute(function() {...}) == 'some value') {
        break;
   }
   numRetires++;
   if (numRetires >= 4) {
      throw new Error("max repeats reached, still no avail");
   }
}
kflow.knysa_fill(...); // check the checkbox, use kflow.evaluate(function() {...}) if knysa_fill() does not work
kflow.knysa_click(...);  // click the submit button
console.log(kflow.getHTML(querySelectorForCertainPartOfPage));  // example: console.log(kflow.getHTML('a.drawer_button'));
Reply all
Reply to author
Forward
0 new messages