Parsehub not clicking all dropdown menu options

[sejan worktrooper]

I was trying to scrape products data from an ecommerce website which has dropdown menu for product variations. But parsehub is not clicking through all the options in dropdown menu. It's only clicking the first option and scraping data for that.

[Andrew11]

It might be best to extract all options’ url to a list, called something like “links” and using Begin new entry/Extract “link” and then putting under everything a For item in links Go to template item.link. If that doesnt work post the address of the pagr here and will take a look.

[sejan worktrooper]

Those options doesn't have any url.
Here is an example.
https://www.prospeedracing.com.au/Kelford-Cams-DLC-Cam-Follower-Single-35mm-KCF-35

[Andrew11]

Ah, I see. Try this:

Template 1:

*add command to Select options and nest inside that command, Extract "numOptions" $selection.length

*beneath that and flush to the left (non-nested), add Loop command for each item in $createArray(numOptions) and nest inside that, one after the other

   -Extract "loopIndex" $index 

   -Select options; nest inside that, an If $selection.index == loopIndex, inside that If, nest:

       ^Begin new entry in "data", inside that, nest

           ^^AJAX click and go to Template 2, where it will do something and when finished, continue from its saved place in Template 1

Let me know if any problems!

[sejan worktrooper]

I've followed your instruction and made an project file with that. It's scraping the variations now but it's repeating same values multiple times and running in infinite loop.

[Andrew11]

Probably the Select options wasn't quite right. Try the attached version.

[sejan worktrooper]

It worked thanks. 

[Andrew11]

:)

[Sauban Abdullahi]

  Hi there! I am a skilled Python developer offering custom automation, Telegram bots, web scraping, and Telegram scraping services tailored to your business needs. My Services Include: Telegram Bots – automated bots for groups, channels, and businesses Web Scraping – structured, accurate data extraction from any website Telegram Scraper – collect users & insights for growth and engagement Web Automation – streamline repetitive tasks and save valuable time Why work with me? Clean, reliable, and scalable code On-time delivery Quick support & revisions 100% satisfaction guaranteed Contact me today and let’s bring your automation ideas to life!Catchy Social Media Ad Automate Your Work – Save Time, Grow Faster!Tired of doing everything manually? I build: Powerful Telegram Bots Smart Web Scrapers Accurate Telegram Scrapers Time-saving Web Automation Tools Boost efficiency, save hours of work, and get results instantly! Let’s make your life easier with Python automation. DM me today to get started https://t.me/Abdullahi_scrapee