Avoid Django s3-urls stripping

[Paolo Bertasi]

Hello,

I'm trying to setup a mini django app to allow users to download files from s3. I wrote a simple python function (using boto of course) to fetch the correct url but when i handle the url with django the url is stripped.

I wrote down the exact behaviour here.

Please help :)

[Aaron Cannon]

Try clicking view source in your browser when you view it. Perhaps
the URL needs to be HTML escaped?

Just a guess.

Aaron

> here<http://stackoverflow.com/questions/22531195/avoid-django-s3-urls-stripping>
> .
>
> Please help :)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/e3dba710-5189-4800-b455-09868dcf578b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Paolo Bertasi]

Hi Aaron,

the html is "wrong" as well.

Moreover i tried {% autoescape off %} but with no luck :(

[Aaron Cannon]

Other things I'd probably try to debug this issue would be:

* Hard coding the string into the call to HttpResponse,
* printing the variable that contains the URL to the console.

If the URL was still bad after trying the first item, and the second
item printed the expected result, then you could completely take S3
out of the picture. From there, I would start narrowing it down, to
see if it was a specific character that was causing the problem.

Narrowing things down a bit more might make it easier for folks to
help you figure out your issue.

Luck.

Aaron

On 3/27/14, Paolo Bertasi <paolo...@gmail.com> wrote:
> Hi Aaron,
> the html is "wrong" as well.
>
> Moreover i tried {% autoescape off %} but with no luck :(
>
>
>
>
> Il giorno giovedì 27 marzo 2014 17:24:45 UTC+1, Aaron Cannon ha scritto:
>>
>> Try clicking view source in your browser when you view it. Perhaps
>> the URL needs to be HTML escaped?
>>
>> Just a guess.
>>
>> Aaron
>>

>> On 3/21/14, Paolo Bertasi <paolo...@gmail.com <javascript:>> wrote:
>> > Hello,
>> > I'm trying to setup a mini django app to allow users to download files
>> from
>> >
>> > s3. I wrote a simple python function (using boto of course) to fetch the
>> >
>> > correct url but when i handle the url with django the url is stripped.
>> > I wrote down the exact behaviour
>> > here<
>> http://stackoverflow.com/questions/22531195/avoid-django-s3-urls-stripping>
>>
>>
>> > .
>> >
>> > Please help :)
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> Groups
>> > "Django users" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> an

>> > email to django-users...@googlegroups.com <javascript:>.

>> > To post to this group, send email to

>> > django...@googlegroups.com<javascript:>.

>>
>> > Visit this group at http://groups.google.com/group/django-users.
>> > To view this discussion on the web visit
>> >
>> https://groups.google.com/d/msgid/django-users/e3dba710-5189-4800-b455-09868dcf578b%40googlegroups.com.
>>
>>
>> > For more options, visit https://groups.google.com/d/optout.
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-users/1172243d-6d55-41b4-9cf4-a6bbc6379225%40googlegroups.com.

[Aaron Cannon]

Is the url a bytestring? A quick check of the code for HttpResponse
shows that that class is expecting a byte string. Perhaps the URL
contains unicode characters that aren't being escaped properly?

Aaron

[Paolo Bertasi]

Hi and thanks for your help.

I tried 

* to print url in the console : OK

* to print HttpResponse(url) in the console: OK

* to return the url witout a template in the browser : wrong 

So i am really confused....

I am almost sure that:

1) it is not a s3 problem

2) it is not a HttpResponse problem

3) it is not a templating problem

It seems that django strip the url when i get the url via browser... 

[Aaron Cannon]

I'd like to try to reproduce this locally. If you can send me the URL
(feel free to mangle the signature parameter for security), that will
help. Alternatively, you might cut the URL into pieces, to see if
it's some specific characters that it's choking on, and then just send
me that bit.

Otherwise, if you'd like to create a small example project that
demonstrates this issue, and upload it to github, that would work as
well.

Aaron

On 3/28/14, Paolo Bertasi <paolo...@gmail.com> wrote:
>
> Hi and thanks for your help.
>
> I tried
> * to print url in the console : OK
> * to print HttpResponse(url) in the console: OK
> * to return the url witout a template in the browser : wrong
>
> So i am really confused....
> I am almost sure that:
> 1) it is not a s3 problem
> 2) it is not a HttpResponse problem
> 3) it is not a templating problem
>
> It seems that django strip the url when i get the url via browser...
>
>
>
> Il giorno giovedì 27 marzo 2014 19:34:10 UTC+1, Aaron Cannon ha scritto:
>>
>> Is the url a bytestring? A quick check of the code for HttpResponse
>> shows that that class is expecting a byte string. Perhaps the URL
>> contains unicode characters that aren't being escaped properly?
>>
>> Aaron
>>

>> On 3/27/14, Aaron Cannon <can...@fireantproductions.com <javascript:>>

>> https://groups.google.com/d/msgid/django-users/1172243d-6d55-41b4-9cf4-a6bbc6379225%40googlegroups.com.
>>
>>
>> >> For more options, visit https://groups.google.com/d/optout.
>> >>
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users...@googlegroups.com.
> To post to this group, send email to django...@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-users.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/django-users/d8f379e1-4485-43ea-9a0a-7982ff1d0e35%40googlegroups.com.

[Paolo Bertasi]

Hi Aaron,

this is the url:

http://this-is-the-irish-aeneas-bucket.s3.amazonaws.com/XXXASCIINAMEXXX/5/2014-03-10_08%3A37%3A39.274998-d3421964017734f96198c74d853eb2e7?Signature=2hR%2Fy0iXqimJ5e1umXX2xY1FqjU%3D&Expires=1396003040&AWSAccessKeyId=XXXXXXXXXXXXXXXXXXXXXX

I replaced some byte with X for security reason.

Today, in order to debug, i ve trie:

c = {}

url = aws.get_s3_content(key).replace('/','')

c['s3url'] = url.encode('utf-8')

context = RequestContext(request, c)

return HttpResponse(template.render(context))

and the url is stripped....... incredible!