Bad gateway

[Guillaume Couture]

Hello!

I have question for you. 

I use PHP to get the data from your API :

json_decode(file_get_contents(sprintf(self::DANDELION_API_URL, urlencode($news['link']), $dandelionConfig['appID'], $dandelionConfig['appKey'])))->text;

I received this error sometimes

failed to open stream: HTTP request failed! HTTP/1.1 502 Bad Gateway\

Even if the website to scrap is reachable.

Can you help me with this?

Guillaume

[Stefano Parmesan]

Hello Guillaume,

This looks weird... I tried dataTXT-NEX with that very same webpage, and sometimes it worked (see link below), sometimes I got the exact same error.

https://www.dropbox.com/s/83v1r5bundy6hlp/Screenshot%202014-06-14%2019.56.26.png

It could be due to some network issues, either on our side, or on Yahoo's, but in the last minutes I always got successful requests, are you still getting errors?

Thanks

-- 

Dott. Stefano Parmesan

Backend Web Developer and Data Lover ~ SpazioDati s.r.l.

Via del Brennero, 52 – 38122 Trento – Italy

[Guillaume Couture]

Now that is fine

I use to scrap yahoo finance website a lot so I suspect it is your API having a problem.

Do you have problem with "batch" calls (a lot of calls in few seconds) I may try to sleep 1 second between each calls..

Have you logs?

[Stefano Parmesan]

We do have logs, but not on requests performed through the "url" parameter (actually I'm not 100% sure on this, but fairly sure, let say... I'll check it later).

If you're submitting many batch requests, it could be that Yahoo is throttling our server, seeing multiple requests coming from the same IP address.

Your idea of "sleeping" between requests may do the trick, please let us know.

Best

[Guillaume Couture]

Thank you for your fast answers

Same results with a sleep 1 sec

Now this is the URL returning and error

https://api.dandelion.eu/datatxt/nex/v1/?url=http%3A%2F%2Ffinance.yahoo.com%2Fnews%2Fpure-industrial-real-estate-trust-120000078.html&$app_id=14.........

Well, I was making the same job using alchemy API and everything was working great .. I've been using it for at least one year using the same code.

Any idea?

[Stefano Parmesan]

I made some tests and checked the logs, I'm even more convinced that Yahoo is refusing connections from our server, which is definitely something we have to fix.

We already know how to tackle this, but I fear you will have to wait a couple of days, unfortunately.

I'm sorry about this, we'll let you know as soon as we have a fix.

Best,

[Guillaume Couture]

Ok, no problem

Thank you :)

[Guillaume Couture]

I continue to encounter the same problem...

Can you help me ?

[Stefano Parmesan]

Hi Guillaume,

Sorry if this took so much, we had many updates under the hood that were delayed, and this has been moved on and on as well.

Today I released a new version of dataTXT with the changes that will hopefully fix this issue, let me know if you encounter anything strange.

Thank you and regards,

-- 

Dott. Stefano Parmesan

Backend Web Developer and Data Lover ~ SpazioDati s.r.l.

Via Adriano Olivetti, 13 – 4th floor

"Le Albere" district – 38122 Trento – Italy