multiple byte ranges

93 views
Skip to first unread message

Maitane

unread,
Nov 30, 2011, 10:54:50 AM11/30/11
to Paste Users
I was wondering if it would be too complicated to give support for
multi-range. Is there any open project looking for this?

Sergey Schetinin

unread,
Nov 30, 2011, 11:06:06 AM11/30/11
to Maitane, Paste Users
IMO multiple ranges is a part of the spec to be ignored and forgotten.
There's barely a real use case for them, and they are not trivial
enough to support them 'just in case'.

The only real-world use I ever heard of them was for an easy DDOS
attack on Apache: http://httpd.apache.org/security/CVE-2011-3192.txt

On 30 November 2011 17:54, Maitane <zmai...@gmail.com> wrote:
> I was wondering if it would be too complicated to give support for
> multi-range. Is there any open project looking for this?
>

> --
> You received this message because you are subscribed to the Google Groups "Paste Users" group.
> To post to this group, send email to paste...@googlegroups.com.
> To unsubscribe from this group, send email to paste-users...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/paste-users?hl=en.
>

--
http://self.maluke.com/

Sergey Schetinin

unread,
Nov 30, 2011, 11:09:10 AM11/30/11
to Maitane, Paste Users
s/DDOS/DOS

I guess I'll quote from the linked advisory as well:

There are two aspects to this vulnerability. One is new, is Apache specific;
and resolved with this server side fix. The other issue is fundamentally a
protocol design issue dating back to 2007:

http://seclists.org/bugtraq/2007/Jan/83

The contemporary interpretation of the HTTP protocol (currently) requires a
server to return multiple (overlapping) ranges; in the order requested. This
means that one can request a very large range (e.g. from byte 0- to the end)
100's of times in a single request.

Being able to do so is an issue for (probably all) webservers and currently
subject of an IETF discussion to change the protocol:

http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311

--
http://self.maluke.com/

Maitane

unread,
Nov 30, 2011, 11:38:05 AM11/30/11
to Paste Users
Thanks for your answer Sergey,

We wanted to use multi-byte range, due to we might need to read
different parts of the same file, but not the whole file. And we would
like to do it in a single request. As it is not supported, we were
thinking on implementing it for ourselves. Do you think it would
require a lot of modifications?


On Nov 30, 5:09 pm, Sergey Schetinin <mal...@gmail.com> wrote:
> s/DDOS/DOS
>
> I guess I'll quote from the linked advisory as well:
>
> There are two aspects to this vulnerability. One is new, is Apache specific;
> and resolved with this server side fix. The other issue is fundamentally a
> protocol design issue dating back to 2007:
>
>      http://seclists.org/bugtraq/2007/Jan/83
>
> The contemporary interpretation of the HTTP protocol (currently) requires a
> server to return multiple (overlapping) ranges; in the order requested. This
> means that one can request a very large range (e.g. from byte 0- to the end)
> 100's of times in a single request.
>
> Being able to do so is an issue for (probably all) webservers and currently
> subject of an IETF discussion to change the protocol:
>
>      http://trac.tools.ietf.org/wg/httpbis/trac/ticket/311
>
> On 30 November 2011 18:06, Sergey Schetinin <mal...@gmail.com> wrote:
>
>
>
>
>
>
>
>
>
> > IMO multiple ranges is a part of the spec to be ignored and forgotten.
> > There's barely a real use case for them, and they are not trivial
> > enough to support them 'just in case'.
>
> > The only real-world use I ever heard of them was for an easy DDOS
> > attack on Apache:http://httpd.apache.org/security/CVE-2011-3192.txt
>

Sergey Schetinin

unread,
Nov 30, 2011, 1:29:57 PM11/30/11
to Maitane, Paste Users
You would need to change the range header parser to parse and validate
multiple ranges. The other part is generating response. If you want to
do something fancy to get all the necessary ranges (like fetching them
from the database in one query) you'll need to do that. If you want
webob to handle slicing the body / app_iter into the requested ranges,
that could get somewhat ugly too. One would need to handle cases when
the end of the previous range and the start of the next one belong to
the same app_iter chunk etc. Calling the app_iter_range hook to get
multiple ranges would be dangerous for the same reason, there's no
requirement for it to work when called more than once.

Ian Bicking

unread,
Nov 30, 2011, 2:31:43 PM11/30/11
to Maitane, Paste Users
Note that you can always read the range header yourself, and produce any response, setting range headers as you want.  WebOb's range support is handy but you don't have to make any use of it if you don't want.  Or, what I'd probably recommend: read the code, copy and paste it into your own code, and modify as needed ;)

Maitane

unread,
Dec 1, 2011, 4:35:33 AM12/1/11
to Paste Users
Thanks all for your answers :)
Reply all
Reply to author
Forward
0 new messages