Screen Scrape?

185 views
Skip to first unread message

Darren

unread,
Jan 5, 2011, 8:25:19 AM1/5/11
to phonegap
Hi all,

I was researching screen scraping on android, and thinking about
developing an app a little to bring down specific info from a webpage,
and display it locally on an android phone.

For me, it will scrap a page for specific content, and either parse
the data locally by the app, or use a web based middle-man server to
reformat the data and send it directly to the app itself in a more
uniformed structure, maybe XML.. The app would then display the data
in a widget or with in the app itself. I might scrape to the app
directly as opposed to a proxy.

Anyway, i was thinking would Phonegap be an angle i could go down. It
is possible to scrape webpage content tags, using JS, CSS etc..
without using a framework? Would phonegap allow for this?

Ive have done this on on windows apps, via Python, but im researching
possibilities on android.
I'd be interested knowing what way i could go about this on Android,
if anyone would care to shed some light on this?

Many thanks for your time,
Darren.

Nick McCloud

unread,
Jan 5, 2011, 8:37:48 AM1/5/11
to phonegap
You can do this all in JavaScript therefore you are good to go with
PhoneGap. Be aware that mobile devices don't have the speed or memory
to do lots and lots and lots of string parsing quickly so optimisation
will be key.

Giacomo Balli

unread,
Jan 5, 2011, 9:02:18 AM1/5/11
to phonegap
scraping via ajax is certainly not a problem.
Issue might be volume of data.
Having the raw material in xml or other kind of visually stripped
format helps a lot.

Darren McEntee

unread,
Jan 7, 2011, 8:00:08 AM1/7/11
to Giacomo Balli, phonegap
Thanks for the info people.
I think i will use a middle-man server... and do the scraping donkey work online, before sending it down to the device in a structured format. This will work nicely: http://simplehtmldom.sourceforge.net/

Many thanks,
Darren.




--
You received this message because you are subscribed to the Google
Groups "phonegap" group.
To post to this group, send email to phon...@googlegroups.com
To unsubscribe from this group, send email to
phonegap+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/phonegap?hl=en?hl=en

For more info on PhoneGap or to download the code go to www.phonegap.com



--
Darren McEntee.

Giovanni Battista Lenoci

unread,
Jan 7, 2011, 8:13:55 AM1/7/11
to phon...@googlegroups.com
Il 07/01/2011 14:00, Darren McEntee ha scritto:
> Thanks for the info people.
> I think i will use a middle-man server... and do the scraping donkey
> work online, before sending it down to the device in a structured
> format. This will work nicely: http://simplehtmldom.sourceforge.net/
Didn't know this project... a few month ago I've use this one:

http://code.google.com/p/phpquery/

Don't know what's better, but if you want to compare this 2 project I'm
interested in your tought :-)

bye

Darren McEntee

unread,
Jan 7, 2011, 12:12:28 PM1/7/11
to Giovanni Battista Lenoci, phon...@googlegroups.com
umm interesting, didnt know of phpquery, ill have a look at it and see, thanks :)

Darren.



--
You received this message because you are subscribed to the Google
Groups "phonegap" group.
To post to this group, send email to phon...@googlegroups.com
To unsubscribe from this group, send email to
phonegap+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/phonegap?hl=en?hl=en

For more info on PhoneGap or to download the code go to www.phonegap.com



--
Darren McEntee.

Darren McEntee

unread,
Jan 7, 2011, 8:08:33 PM1/7/11
to Giovanni Battista Lenoci, phon...@googlegroups.com
Ok guys, just a quick one here.
If i scrape my data to a set format, what is the best way to send it to my phonegap app, and display the data on screen?

Just thinking of what possible angles there are..?
Thanks,
Darren.

--
Darren McEntee.
Reply all
Reply to author
Forward
0 new messages