How to get a random pic from 500px everyday moment

[李哲]

I want to get a random picture url  from 500px everyday moments ? 

So I run the scrapy shell https://500px.com/ to analysis the page, I found that I can get the url info in the Chrome by using element inspector.

But I can't find the related info in the response.body, for example I search in the source code for the pic url which I get in the element inspector, but I can't find that. Why this happened and how to fix it ?  

[Rolando Espinoza]

The data you are looking for is generated dynamically and the data located in a javascript variable named window.photos.

Also, check my answer on how to figure out where is the content you are looking for: http://stackoverflow.com/questions/34815372/how-to-know-if-web-content-cannot-be-handled-by-scrapy/34822978#34822978 

Best,

Rolando

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscribe@googlegroups.com.
To post to this group, send email to scrapy...@googlegroups.com.
Visit this group at https://groups.google.com/group/scrapy-users.
For more options, visit https://groups.google.com/d/optout.

[李哲]

tnx a lot for your reply I read you post on the stackoverflow and upvoted it. Your answer is very detailed.  The answer give me a direction to explore I will study the selenium tnx :D 

在 2016年8月13日星期六 UTC+8上午5:36:38，Rolando Espinoza La fuente写道：

The data you are looking for is generated dynamically and the data located in a javascript variable named window.photos.

Also, check my answer on how to figure out where is the content you are looking for: http://stackoverflow.com/questions/34815372/how-to-know-if-web-content-cannot-be-handled-by-scrapy/34822978#34822978 

Best,

Rolando

On Fri, Aug 12, 2016 at 4:13 AM, 李哲 <imleg...@gmail.com> wrote:

I want to get a random picture url  from 500px everyday moments ? 

So I run the scrapy shell https://500px.com/ to analysis the page, I found that I can get the url info in the Chrome by using element inspector.

But I can't find the related info in the response.body, for example I search in the source code for the pic url which I get in the element inspector, but I can't find that. Why this happened and how to fix it ?  

--
You received this message because you are subscribed to the Google Groups "scrapy-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users...@googlegroups.com.