wget from api.nasdaq fails only on App Engine

[Ananth Chellappa]

Works fine on my home PC. What could be the reason?

wget --user-agent=Safari "https://api.nasdaq.com/api/quote/AAPL/dividends?assetclass=stocks"

hangs on App Engine but returns data immediately on my laptop. 

Do they detect the request is from App Engine and decide not to respond?

wget  "https://finance.yahoo.com" # works fine on App Engine

Thanks!

[George (Cloud Platform Support)]

Do you have a service running in App Engine? How do you run wget in this case from that service?

You should be aware that by default, the App Engine implementation of the HTTPS wrapper attempts to validate the certificate of the host, and rejects requests where the certificate does not match. You may need to disable this behavior, as described in the "Issuing an HTTPS request" paragraph. 

[Ananth Chellappa]

Tried wget with --no-check-certificate and the result is the same - just hangs..

[Ananth Chellappa]

It'll be a python script (flask)

Since it doesn't work (hangs, just like wget) I'm trying wget to see what works and what doesn't..

web is not my core competence. Could you tell me how to format the wget as you suggest? :)

Thanks!

On Monday, October 5, 2020 at 2:58:01 PM UTC-5 George (Cloud Platform Support) wrote:

[Olu]

Hello, 

Thank you again for reporting this issue. I understand your attempt to download data from your App Engine application using wget hangs and probably fails. I understand the command used is wget --user-agent=Safari "https://api.nasdaq.com/api/quote/AAPL/dividends?assetclass=stocks"

Hence, I am assuming the application "https://api.nasdaq.com/api/quote/AAPL/dividends?assetclass=stocks" is running on App Engine. So, basically this is an incoming request to your App Engine application. Please let me know if my understanding is wrong. 

If however, my understanding is right, then there might be a lot more to gather from the logs of your App Engine application. Reviewing the configurations on the App Engine application may also be necessary. I think your best option would be to reach out to a GCP Support Engineer[1]. This way a GCP Support Engineer would be able to review  with you the specific details on your application.

[1]https://cloud.google.com/support-hub

[Ananth Chellappa]

No, api.nasdaq.com is not on AE..

[yananc]

Please try to change the ‘user-agent’, which might have a play in the issue with ‘nasdaq.com’ [1]. If the issue still persists, have you tried to access the data from ‘nasdaq.com’ via other ways, such as writing codes or sending requests instead of using wget? This could possibly help isolate the issue.

Hope it helps!

[1]: https://stackoverflow.com/questions/24728421/why-cant-get-the-web-content-in-www-nasdaq-com-symbol-c-stock-report