Can I stick Amazon "Deliver to" to the USA in Parsehub when I'm located in Tawain ?

[Gary Ou]

Hi All,

I'm located in Taiwan, when I run my project it always set "Deliver to" as Taiwan.

Is there any way I can stick "Deliver to" to the USA in Parsehub ? 

Thank you.

Gary

[Andrew11]

On my Amazon there's a little location marker pic to the right of the main logo at top left where you can enter a different zip code.

[Gary Ou]

Thank you for reply.

Although I changed it, it goes back to Taiwan when I run my project.

1. I entered a ZIP code

2. It goes back to Taiwan

Andrew11 在 2023年5月3日 星期三晚上9:26:10 [UTC+8] 的信中寫道：

[Andrew11]

Yeah, you have to write a scraping step to click on the pic, have the scraper type the right zip code in, and click "OK" at the start of every scrape. The problem's even more pronounced when it's in the cloud run.

[Andrew11]

You might also limit it to 1 worker max... I think sometimes each worker needs to do this clicking/typing separately.

[Gary Ou]

Hi Andrew,

I followed your suggestion to make some steps to change ZIP code and limited workers to 1.

Initially it did work but result was like following the 1st screenshot.

As you can see some of their data were not scraped. (I have around 120 data want to scrape)

Then I input all the urls with blank result into settings\starting value and try again, but it doesn't work. (The result shows only main template is scraped as 2nd screenshot shown below.)

Weird thing is it always works normally in "test run", just failed in "Run" mode.

I've tried to close the software and execute again but same result.

Is there any other suggestions that I can try ? Thank you.

Andrew11 在 2023年5月4日 星期四上午10:03:46 [UTC+8] 的信中寫道：

[Andrew11]

Try putting an Extract pageText ($e.text) right inside the root Select page of your 2nd template, that way you can get a better idea what's going on. Most likely your Selects don't apply to all cases and you might have to open them up a bit to cover the variations.

[Gary Ou]

Now the main problem is I can't run it normally, it always shows following result. (0 product_page). But if I test run it, it works !

I attach 2nd template structure.

Andrew11 在 2023年5月5日 星期五上午9:34:41 [UTC+8] 的信中寫道：

[Andrew11]

Can you export the project, put it in a zip file and attach it here?

[Gary Ou]

Hi Andrew,

I zip it and attached here, thank you.

Andrew11 在 2023年5月5日 星期五中午12:06:34 [UTC+8] 的信中寫道：

[Andrew11]

OK I think I see what's happening. The project opens to amazon.com with a CAPTCHA showing, or small puzzle where you have to type in letters from a picture to prove that you're human. If the run does that too you might be able to fix it with IP rotation in Project Settings. You might also try

https://help.parsehub.com/hc/en-us/articles/115004504048-Solving-Captcha-for-all-Paid-plans-

[Andrew11]

BTW if you're rotating IPs you might need to move the "deliver to" step into the last template, the one that opens on individual pages, as the cookie might get lost when switching IP addresses.

[Gary Ou]

Thank you for the help.

Today I run it again and it works again.

Feel like it seems to be CAPTCHA issue if I run it many times in a short time.

Just I don't understand when it can still work last week, just the data from the link in the middle of my list failed to scrape (for example, 1~50 work, 51~61 fail, 62~80 work)

not all fail from a certain link and after in my list. (for example, 1~60 work and 61~end all fail if it is CAPTCHA issue)

And why CAPTCHA doesn't affect "test run" mode as I never got fail in test run.

That's still OK for me if I can run it every week or every other week.

Thank you again.

Andrew11 在 2023年5月6日 星期六清晨6:30:45 [UTC+8] 的信中寫道：

[Andrew11]

Great! In test run it uses your IP address but in cloud run it uses ParseHub's cluster based in US and Canada and uses a finite number of IP's, so Amazon may have seen one or more of them trying to scrape before.