HTTP POST sent from app to Django Server returns 403 Forbidden
etone
I'm trying to sent a HTTP POST from a client application to my Django
app. Django does indeed receive the HTTP POST as I do hit
_HandleRequest(), however it returns a 403 Forbidden, instead of
hitting my handler function. I experimented and sent a HTTP GET from
my client application and in this case I am able to hit my handler
function. I would like to use HTTP POST as I want to upload some data
to my Django app.
What am I doing wrong/missing?
Here is my settings.py in my django app:
try:
from djangoappengine.settings_base import *
has_djangoappengine = True
except ImportError:
has_djangoappengine = False
DEBUG = True
TEMPLATE_DEBUG = DEBUG
import os
INSTALLED_APPS = (
'djangotoolbox',
# 'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
)
if has_djangoappengine:
INSTALLED_APPS = ('djangoappengine',) + INSTALLED_APPS
ADMIN_MEDIA_PREFIX = '/media/admin/'
MEDIA_ROOT = os.path.join(os.path.dirname(__file__), 'media')
TEMPLATE_DIRS = (os.path.join(os.path.dirname(__file__),
'templates'),)
ROOT_URLCONF = 'urls'
Kieran Farr
We're integrating with a third-party API that sends "pings" via simple
POST requests to our server to give us updates re: status video
conversion processes.
We're running Django 1.2.1 with Apache mod_python on Ubuntu 9.04.
I've disabled ALL csrf related middleware for debugging purposes in
settings.py, although we'll want that enabled on production.
I've been attempting to debug with wget. When I hit the listener URL,
it works fine if no post data is sent. However, as soon as I add the --
post-data tag with ANY text, a 403 is always returned.
Even tried @csrf_exempt and @csrf_response_exempt AND adding
csrf_exempt = True to the response object.
This is quite frustrating as we're following all the docs for
disabling CSRF and this is getting in the way of what would be a very
simple task.
Further, debugging 403s is nearly impossible. It'd be very helpful
when in DEBUG mode to reveal who/what/why raised the 403.
Any ideas?
Kieran
raj
exact code block which tries to post the data. Is the login successful
and does the logged-in user have got all necessary permissions
required? Post that part of the code if you can't find out.
Rajeesh.
On Jul 27, 11:20 pm, Kieran Farr <kieran.f...@gmail.com> wrote:
> Further, debugging 403s is nearly impossible. It'd be very helpful
> when in DEBUG mode to reveal who/what/why raised the 403.
>
Kieran Farr
accessible, no login required.
Here's the function in its entirety:
255 @csrf_response_exempt
256 def DistroHeySpreadListener(request):
257 if request.method == 'POST':
258 post = request.POST
259 raw = request.raw_post_data
260 remote_media_id = post.get("video_id", "")
261 # Try to fetch the distros for this
remote_media_id
262 distro_list =
Distro.objects.filter(remote_media_id__exact=remote_media_id)
263
264 # Make sure there's at least one
265 if len(distro_list) < 1:
266 raise Http404
267
268 # For each distro instance, check to see if
there's a response in this api call
269 for distro in distro_list:
270 # Prepare logging
271 log = distro.log
272 log += "Raw data from POST" + str(raw)
273 distro.log = log
274 distro.save()
275
276 # Get distro short name
277 dest_short_name =
distro.credential.destination.short_name
278
279 # From short_name, return either error or
link starting with http://
280 response = post.get('link[%s]' %
dest_short_name, False) or post.get('error[%s]' % dest_short_name,
False)
281
282 if response.startswith('http://'):
283 # If response starts with http
then success, populate as link
284 distro.destination_url = response
285 distro.state = TRANSCODE_SUCCESS
286 log += '\nSUCCESS, url = %s' %
response
287 else:
288 # Else, change state to error and
log error message
289 log += '\nERROR, error = %s' %
response
290 distro.state = TRANSCODE_ERROR
291
292 distro.log = log
293 distro.save()
294
295
296 h = HttpResponse("Thanks, HeySpread! We love
you.", mimetype="text/plain", status=200)
297 h.csrf_exempt = True
298 return h
299 # return HttpResponse("Thanks, HeySpread! We love
you.", mimetype="text/plain", status=200)
300 else:
301 return HttpResponse("Oops, something went wrong.",
mimetype="text/plain", status=200)
Kieran Farr
listener.
The "sender" of this POST request is a third party server that we have
no control over.
However, even when debugging with wget we receive the 403.
Here is the wget command I've been testing with:
$ wget http://www.url.com/subdir/listener/ --post-data "?test=test" -U
"Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-us) AppleWebKit/
533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16"
--2010-07-27 20:06:36-- http://www.url.com/subdir/listener/
Resolving [URL] ... [IP ADDRESS]
Connecting to [URL]|[IP ADDRESS]|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2010-07-27 20:06:37 ERROR 403: Forbidden.
However, if I send without the post-data, here's the response:
$ wget http://www.url.com/subdir/listener/
--2010-07-27 20:31:14-- http://www.url.com/subdir/listener/
Resolving [URL]... [IP ADDRESS]
Connecting to [URL|[IP ADDRESS]|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/plain]
Saving to: `index.html.1'
[ <=> ]
27 --.-K/s in 0s
2010-07-27 20:31:14 (1.84 MB/s) - `index.html.1' saved [27]
Kieran Farr
settings.py if we use Django's built-in auth.
Obviously, we need to still use Django's auth, so we can't just
disable CSRF site-wide like this hack:
http://stackoverflow.com/questions/1650941/django-csrf-framework-cannot-be-disabled-and-is-breaking-my-site
...or this hack...
http://djangosnippets.org/snippets/2069/
Obviously also, we have no control over the third-party server sending
the POST request, so we can't resort to this hack:
http://stackoverflow.com/questions/2405353/having-a-postable-api-and-djangos-csrf-middleware
The decorator @csrf_exempt does not work as described in the docs as
our view always returns a 403 when any content is POSTed.
Very confusing!
Kieran
On Jul 27, 1:32 pm, Kieran Farr <kieran.f...@gmail.com> wrote:
> Raj sorry I misread your question. This initial response is in re: my
> listener.
>
> The "sender" of this POST request is a third party server that we have
> no control over.
>
> However, even when debugging with wget we receive the 403.
>
> Here is the wget command I've been testing with:
>
> 533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16"
>
> --2010-07-27 20:06:36-- http://www.url.com/subdir/listener/
> Resolving [URL] ... [IP ADDRESS]
> Connecting to [URL]|[IP ADDRESS]|:80... connected.
> HTTP request sent, awaiting response... 403 Forbidden
> 2010-07-27 20:06:37 ERROR 403: Forbidden.
>
> However, if I send without the post-data, here's the response:
>
Kieran Farr
My hypothesis:
An HTTP request with POST method sent to a Django view wrapped with
@csrf_exempt will still raise a 403 CSRF error if the wrapped view
raises an Http404 exception.
In this specific case, we raise 404 when there is no valid video_id
passed in the POST request:
265 if len(distro_list) < 1:
266 raise Http404
instead we would have expected the 404 to be raised without CSRF
interfering given the initial view was properly wrapped with
@csrf_exempt. Further contributing, this was rather difficult to
diagnose due to the opaque 403 error message even when DEBUG = True.
Is this expected Django behavior? If not, can we please address this
in a future release?
Is there a better way to provide debugging information for 403 errors
raised from the built-in CSRF methods?
All the best and thanks for a great framework.
Kieran
On Jul 27, 6:54 pm, Kieran Farr <kieran.f...@gmail.com> wrote:
> Further research shows that CSRF is enabled regardless of my
> settings.py if we use Django's built-in auth.
>
> Obviously, we need to still use Django's auth, so we can't just
> ...or this hack...http://djangosnippets.org/snippets/2069/
> Obviously also, we have no control over the third-party server sending
steven314
get to the bottom of what's going on:
http://docs.djangoproject.com/en/dev/ref/contrib/csrf/#rejected-requests
As it says there, a 403 may be due to a lack of the {% csrf_token %}
in the form that is being posted.
Steven.
David De La Harpe Golden
> Finally figured out what's going on.
>
> My hypothesis:
> An HTTP request with POST method sent to a Django view wrapped with
> @csrf_exempt will still raise a 403 CSRF error if the wrapped view
> raises an Http404 exception.
>
Ah, but was your handler404 view csrf_exempt (the default django one
django.views.defaults.page_not_found isn't...) ;-)
http://code.djangoproject.com/browser/django/trunk/django/views/defaults.py#L4
(really, you'd probably better off _returning_ a
django.http.HttpResponseNotFound response than raising a Http404
exception from your exempted view in this case I guess).
IMO there's no reason to leak info even to the extent of "this object
doesn't exist" to scurrilous CSRFers by default.
> Is there a better way to provide debugging information for 403 errors
> raised from the built-in CSRF methods?
>
Note also that django 1.2.1 allows specification of
settings.CSRF_FAILURE_VIEW.
http://docs.djangoproject.com/en/dev/ref/settings/?from=olddocs#csrf-failure-view
You could even make that view immediately throw an exception to be
caught by something else ...I think.
There are quirks, or at least things to be aware of, with the order
things happen in django.core.handlers.base.BaseHandler.get_response()
[1]. Note an exception middleware /doesn't catch/ all exceptions,
some "escape" if they happen early, so e.g. it's best to still specify a
custom handler404/handler500 when using an exception middleware, because
they might still get called.
Even then the only way I've found to completely customize the behaviour
when settings.DEBUG is true (because I wanted ajax technical_response
debug pages to be specially formatted compared to standard whole-page
technical_response) is to subclass the Handler (WSGIHandler) to meddle
with get_response.
I hesitate to call the situation a "bug", because it's all stuff you
probably shouldn't be messing with much, making it too easy
to render your server insecure is probably a django non-goal...
[1]
http://code.djangoproject.com/browser/django/trunk/django/core/handlers/base.py#L66
Benedict Verheyen
> Further research shows that CSRF is enabled regardless of my
> settings.py if we use Django's built-in auth.
>
> Obviously, we need to still use Django's auth, so we can't just
> disable CSRF site-wide like this hack:
> http://stackoverflow.com/questions/1650941/django-csrf-framework-cannot-be-disabled-and-is-breaking-my-site
> ...or this hack...
> http://djangosnippets.org/snippets/2069/
>
> Obviously also, we have no control over the third-party server sending
> the POST request, so we can't resort to this hack:
> http://stackoverflow.com/questions/2405353/having-a-postable-api-and-djangos-csrf-middleware
>
> The decorator @csrf_exempt does not work as described in the docs as
> our view always returns a 403 when any content is POSTed.
>
> Very confusing!
>
> Kieran
<snip>
I'm not entirely sure but i think i've hit the same problem.
I tried to access a view with urllib & urllib that requires a logged
in user with the appropriate rights.
First i need to login in and post the username and password in order to use
a second call to access the protected function.
This is the test code i used to automatically login:
import urllib2,urllib
o = urllib2.build_opener( urllib2.HTTPCookieProcessor() )
urllib2.install_opener(o)
id='Admin'
pw='xyz'
p=urllib.urlencode({"username" : id, "password" : pw})
f=o.open("http://calltracking:8000/accounts/login/", p)
data=f.read()
f.close()
The login view has the decorator @csrf_exempt and login works fine.
Next, i try the actual view that also has @csrf_exempt specified.
I get this back:
>>> f=o.open("http://calltracking:8000/management/statistics/top/user/yearly/", p)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "c:\python26\lib\urllib2.py", line 397, in open
response = meth(req, response)
File "c:\python26\lib\urllib2.py", line 510, in http_response
'http', request, response, code, msg, hdrs)
File "c:\python26\lib\urllib2.py", line 435, in error
return self._call_chain(*args)
File "c:\python26\lib\urllib2.py", line 369, in _call_chain
result = func(*args)
File "c:\python26\lib\urllib2.py", line 518, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 403: FORBIDDEN
I'm not sure how to proceed as it seems that csrf is getting in the way.
Regards,
Benedict
steven314
problem?
Steven.
Kieran Farr
with result code 404.
Benedict, could you post the view for /management/statistics/top/user/
yearly/ that is causing the 403?
Kieran
Benedict Verheyen
> Thanks, David -- you're right on, now I just return an HttpResponse
> with result code 404.
>
> Benedict, could you post the view for /management/statistics/top/user/
> yearly/ that is causing the 403?
>
> Kieran
>
Yes,
here it is.
Depending on wether the user logged in is staff (ICT) or not, i use
a different base toe extend my template from (different menu).
@csrf_exempt
def stats_top_callers_per_year(request):
if ( request.user.is_staff ):
extends_from = "management/base.html"
else: extends_from = "management/public_base.html"
stat_type = "top_callers_per_year"
stats_all = {}
message=_(u"Top 30 callers per year")
class Stats(object):
pass
current_year = datetime.datetime.now().year
archive_year = current_year - 4
years = xrange(archive_year, current_year+1)
for year in years:
stats_year = []
counts = {}
# Initialize count dict
# If we don't do this, we get a KeyError
for user in User.objects.all():
counts[user]=0
# Count the times an initiator has been calling
for i in Call.objects.filter(date_created__year=year):
for _init in i.initiator.all():
counts[_init] += 1
# Sort the dictionary
count_sorted = sorted(counts.items(), lambda x, y: cmp(x[1], y[1]), reverse=True)
# Make the stats ready for the template
for user_stat in count_sorted[0:30]:
if ( user_stat[1] > 0 ):
st = Stats()
st.calls = user_stat[1]
st.user = user_stat[0]
stats_year.append(st)
stats_all[year]=stats_year
stats_sorted = sorted(stats_all.items(), lambda x, y: cmp(x[0], y[0]), reverse=True)
return render_to_response('management/stats.html', {'extends_from': extends_from, 'stats_year': stats_sorted, 'message': message,
'stat': stat_type, 'tab': "top_callers_per_year"}, context_instance=RequestContext(request))
My goal is to be able to automatically login (seems to work) and then call this function so
the page is generated before any user hits the page.
Regards,
Benedict