robots.txt 404 errors are not ignored, even though I put it in my settings file

[Sabine Maennel]

Please can anybody help me with 404 errors?

I put this in my settingsfile: 

import re

IGNORABLE_404_URLS = (

    re.compile(r'^/apple-touch-icon.*\.png$'),

    re.compile(r'^/favicon\.ico$'),

    re.compile(r'^/robots\.txt$'),

)    

but I still get 404 errors reported for robots.txt: What am I doing wrong here?
Traceback (most recent call last):

  File "/srv/http/web0263/netteachers_live/lib/django/core/handlers/base.py", line 150, in get_response
    response = callback(request, **param_dict)

  File "/srv/http/web0263/netteachers_live/lib/django/utils/functional.py", line 15, in _curried
    return _curried_func(*(args + moreargs), **dict(kwargs, **morekwargs))

  File "/srv/http/web0263/netteachers_live/lib/django/utils/decorators.py", line 99, in _wrapped_view
    response = view_func(request, *args, **kwargs)

  File "/srv/http/web0263/netteachers_live/lib/django/views/defaults.py", line 30, in page_not_found
    body = template.render(RequestContext(request, {'request_path': request.path}))

  File "/srv/http/web0263/netteachers_live/lib/django/template/context.py", line 169, in __init__
    self.update(processor(request))

  File "/srv/http/web0263/netteachers_live/core/context_processors.py", line 9, in resolvermatch
    match = resolve(request.path)

  File "/srv/http/web0263/netteachers_live/lib/django/core/urlresolvers.py", line 480, in resolve
    return get_resolver(urlconf).resolve(path)

  File "/srv/http/web0263/netteachers_live/lib/django/core/urlresolvers.py", line 352, in resolve
    raise Resolver404({'tried': tried, 'path': new_path})

Resolver404: {u'path': u'robots.txt', u'tried': [[<RegexURLResolver <RegexURLPattern list> (admin:admin) ^admin/>], [<RegexURLResolver <module 'apps.application.urls' from '/srv/http/web0263/netteachers_live/apps/application/urls.py'> (None:application) ^bewerbung/>], [<RegexURLPattern home ^$>], [<RegexURLPattern kontakt ^kontakt$>], [<RegexURLPattern team ^team$>], [<RegexURLPattern impressum ^impressum$>], [<RegexURLPattern geschaeftsmodell ^geschaeftsmodell$>], [<RegexURLPattern geschaeftsmodell_prezi ^geschaeftsmodell/prezi$>], [<RegexURLPattern onlineunterricht ^onlinenterricht$>], [<RegexURLPattern onlineunterricht_prezi ^onlinenterricht/prezi$>], [<RegexURLPattern seminarunterricht ^seminarunterricht$>], [<RegexURLPattern seminarunterricht_prezi ^seminarunterricht/prezi$>], [<RegexURLPattern fragen ^fragen$>], [<RegexURLPattern bewerbung ^bewerbung$>], [<RegexURLPattern steckbrief ^steckbrief$>], [<RegexURLPattern starterkurs ^starterkurs$>]]}


<WSGIRequest
path:/robots.txt,
GET:<QueryDict: {}>,
POST:<QueryDict: {}>,
COOKIES:{},
META:{u'CSRF_COOKIE': u'PwJIAcfj8LQnL0yFygDyqrAvpvVqx5Lb',
 'GATEWAY_INTERFACE': 'CGI/1.1',
 'HTTP_ACCEPT': 'text/plain, text/html',
 'HTTP_ACCEPT_ENCODING': 'gzip,deflate',
 'HTTP_CONNECTION': 'close',
 'HTTP_FROM': 'googlebot(at)googlebot.com',
 'HTTP_HOST': 'netteachers.de',
 'HTTP_USER_AGENT': 'Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)',
 'HTTP_X_FORWARDED_FOR': '66.249.65.33',
 'PATH_INFO': u'/robots.txt',
 'QUERY_STRING': '',
 'REMOTE_ADDR': '10.250.3.3',
 'REMOTE_PORT': '48208',
 'REQUEST_METHOD': 'GET',
 'REQUEST_URI': '/robots.txt',
 'SCRIPT_NAME': u'',
 'SERVER_NAME': 'netteachers.de',
 'SERVER_PORT': '80',
 'SERVER_PROTOCOL': 'HTTP/1.0',
 'wsgi.errors': <mod_wsgi.Log object at 0x7fa7df81df30>,
 'wsgi.file_wrapper': <built-in method file_wrapper of mod_wsgi.Adapter object at 0x7fa7e0679c60>,
 'wsgi.input': <mod_wsgi.Input object at 0x7fa7e0446070>,

[Collin Anderson]

Hi Sabine,

IGNORABLE_404_URLS = (
    re.compile(r'^/apple-touch-icon.*\.png$'),
    re.compile(r'^/favicon\.ico$'),
    re.compile(r'^/robots\.txt$'),
)

remove the slash at the beginning:

IGNORABLE_404_URLS = (
    re.compile(r'^apple-touch-icon.*\.png$'),
    re.compile(r'^favicon\.ico$'),
    re.compile(r'^robots\.txt$'),
)

Collin

[Sabine Maennel]

Thank you very much for your help Collin!