Re: urlfetch_stub.py:504] Stripped prohibited headers from URLFetch request: ['Host']

[Mihail Russu]

The message in the title is actually a warning, not an error and shouldn't prevent your code from running (GAE simply does not allow you to set some HTTP headers which is fine most of the time), although the code your are providing seems to be incomplete. You should provide more info/stack trace related to what you're trying to do.

Thanks,

Mihail.

On Wednesday, January 28, 2015 at 12:15:08 AM UTC+2, Emin Yesildere wrote:

Hi;

I try to scrap a website with using beautifulsoup, but this error occurs at gae logs. How can I solve this issue?

I use this script 

import sys

sys.path.append('libs/')

reload(sys); sys.setdefaultencoding('utf-8')

from bs4 import BeautifulSoup

import urllib

from google.appengine.api import urlfetch

from datetime import datetime

import locale

import PyRSS2Gen

locale.setlocale(locale.LC_ALL, '')

import requests

import codecs

def parse(url):

     page.urllib2.urlopen(url)

     soup=BeautifulSoup(page.read())

     for link in soup.find_all('article',{'class':'item-list'}):

              ka= link.find_all('div')[1].get_text()

.....

[Alex Martelli]

Agreed (and I said the same on Stack Overflow, but allowed the post here anyway in case somebody else can fathom the issue).  The code shown importa urllib and urlfetch and never uses either of them, then has a mysterios `page.urllib2...` (apparently it's meant to be `page=urllib2...` -- even though it does import urllib2) -- that innocuous warning is clearly the least of the problems (the first one is that the code shown can't be the one the OP is actually trying to run and getting that innocuous warning from).

Alex