Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

python to enable javascript , tried selinium, ghost, pyQt4 already

40 views
Skip to first unread message

Jaiprakash Singh

unread,
Jan 18, 2014, 6:54:17 AM1/18/14
to
hi,

can you please suggest me some method for study so that i can scrap a site having JavaScript behind it


i have tried selenium, ghost, pyQt4, but it is slow and as a am working with thread it sinks my ram memory very fast.

Denis McMahon

unread,
Jan 18, 2014, 1:05:44 PM1/18/14
to
On Sat, 18 Jan 2014 03:54:17 -0800, Jaiprakash Singh wrote:

> can you please suggest me some method for study so that i can
> scrap a site having JavaScript behind it

Please expand upon the requirement, are you trying to:

a) replace server side javascript with server side python, or
b) replace client side javascript with server side python, or
c) replace client side javascript with client side python, or
d) something else?

(c) is not possible (you can't guarantee that all clients will have
python, or that there will be a mechanism for calling it from your
webpages), (b) doesn't make a lot of sense (you'll be trading cpu in the
client for cpu in the server + network bandwidth and latency).

--
Denis McMahon, denismf...@gmail.com

Chris Angelico

unread,
Jan 18, 2014, 1:13:57 PM1/18/14
to pytho...@python.org
Do you mean "scrape"? You're trying to retrieve the displayed contents
of a web page that uses JavaScript? If so, that's basically impossible
without actually executing the JS code, which means largely
replicating the web browser.

ChrisA

Denis McMahon

unread,
Jan 18, 2014, 4:40:38 PM1/18/14
to
Oh, you think he meant scrape? I thought he was trying to scrap (as in
throw away / replace) an old javascript heavy website with something
using python instead.

--
Denis McMahon, denismf...@gmail.com

Chris Angelico

unread,
Jan 18, 2014, 5:32:36 PM1/18/14
to pytho...@python.org
I thought so too at first, but since we had another recent case of
someone confusing the two words, and since "scrape" would make sense
in this context, I figured it'd be worth asking the question.

ChrisA

Giorgos Tzampanakis

unread,
Jan 19, 2014, 2:58:46 AM1/19/14
to
I have tried selenium in the past and I remember it working reasonably
well. I am afraid you can't get around the slowness since you have to have
a web browser running.

--
Improve at backgammon rapidly through addictive quickfire position quizzes:
http://www.bgtrain.com/
0 new messages