How do I pull links on a webpage?

12 views
Skip to first unread message

rattanapo...@gmail.com

unread,
Nov 7, 2017, 5:27:20 AM11/7/17
to CasperJS
var casper = require('casper').create();
var links;

function getLinks() {
// Scrape the links from top-right nav of the website
    var links = document.querySelectorAll('ul.navigation li a');
    return Array.prototype.map.call(links, function (e) {
        return e.getAttribute('href')
    });
}

// Opens casperjs homepage
casper.start('http://casperjs.org/');

casper.then(function () {
    links = this.evaluate(getLinks);
});

casper.run(function () {
    for(var i in links) {
        console.log(links[i]);
    }
    casper.done();
});

rattanapo...@gmail.com

unread,
Nov 7, 2017, 5:29:17 AM11/7/17
to CasperJS
I will pull the news headlines and all the news in the file .txt

เมื่อ วันอังคารที่ 7 พฤศจิกายน ค.ศ. 2017 17 นาฬิกา 27 นาที 20 วินาที UTC+7, rattanapo...@gmail.com เขียนว่า:

Ken Soh

unread,
Dec 10, 2017, 12:05:40 PM12/10/17
to CasperJS
To get text of the news items, probably you will need to use fetchText method - http://docs.casperjs.org/en/latest/modules/casper.html#fetchtext
but there isn't an easy way to do what you need, depends on the webpage to know how to get that information.
Reply all
Reply to author
Forward
0 new messages