Call to URLFetch failed with application error 5 for url x

[Flav Ible]

Background:

I am using google app engine and am having a weird bug in my site crawler.

I have a backend that will automatically crawl a site every night. This is instigated by a task pushed to a pushQueue due to time limits in php.

Problem:

When I manually run the script that creates the task, the task completes as expected with no errors. However when cron launches the task I get the following error.

Call to URLFetch failed with application error 5 for url x

Code:

function url_get_contents ($Url) {
    global $retry;
    try {
        if (!function_exists('curl_init')){ 
            die('CURL is not installed!');
        }
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $Url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $output = curl_exec($ch);
        curl_close($ch);
        return $output;
    } catch (Exception $e) {
        syslog(LOG_INFO, 'Caught exception: ',  $e->getMessage());
        if($retry > 0){
            $retry -= 1;
            return url_get_contents($Url);
        }
        else{
            return null;
        }
    }
}

Thanks to syslog I can see that the $url is fine which is driving me crazy as it works when the exact same script is launched manually not through cron.

How can I fix this?

Thanks in advance.

[Ryan (Cloud Platform Support)]

I would increase the timeout for the url_fetch. If that does not work I would recommend posting on StackOverflow. You will have a better chance of finding someone who has had a similar issue. If you can prove it's a bug with the platform posting on the PIT with a working sample project would help fix the issue.