Please let me know how to crawl java script rendering page using scrapy with python

[lee mande]

I am very new to "scrapy", i am scrapping a website and in that i had some anchor tags which consists of href attributes with href="javascript:void(0) functions. When i clicked that javascript function a page is opening from which i need to fetch data.I used Xpath and found href for particular anchor tags but unable to execute that href attribute that contains javascript function. Can anyone tell me how to execute href="javascript:void(0) of anchor tags in scrapy python.My HTML code is



<div class="cont_wrap">
<div class="title_h2">
<div>
<ul class="job_list">
<li>
<div class="cont">
<span style="display:block;vertical-align: bottom;">
<a class="btn_favo" compcd="C10" siteid="AB06" reqstnno="233862" style="display:none;" href="javascript:void(0);">
<a class="title" jobtitle="SSI - Environmental, Health & Safety Supervisor - San Jose" siteid="AB06" compcd="C10" reqstnno="233862" style="display:inline-block;" href="javascript:void(0);">SSI - Environmental, Health & Safety Supervisor - San Jose (233862)</a>
</span>
<span class="type">General Affairs, Regular Full-Time, Travel Required None</span>
<div class="inpo">
</div>
<div class="sub">
</li>

thanks,

Lee

[Zhang Teng-Ji]

Detail url？

I often parse the real data url from javascript code;

Because render javascript code is slow and complex.

在 2014年10月24日星期五UTC+8下午8时17分31秒，john kin写道：

[Aru Sahni]

Scrapy doesn't interpret JavaScript. You'll need to use something like GhostScript or Selenium to interact with page elements.

~A

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at http://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.