XML Parsing (included malformed DTD), 99% CPU DoS Attack

[3arrows]

Hi, Rails Experts!

I think this is important so please read this message.

There is a DoS for Ruby on Rails that is easily exploitable.
The attack involves sending a malformed xml document in
an HTTP request.

There has been an exploitable spec in the Ruby on Rails
which allows:

Anyone on the Internet to...
Send a single HTTP request to...
Any Rails Web application program using...
Rexml parsing with...
A malformed Xml document that...
Causes the Ruby process to go into a 99% CPU heavily-loaded loop.

What happens is that the Xml document (included DTD) is given as:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE foobar [
<!ENTITY x0 "hello">
<!ENTITY x1 "&x0;&x0;">
<!ENTITY x2 "&x1;&x1;">
<!ENTITY x3 "&x2;&x2;">
---- repeat 100 times -----------------
<!ENTITY x100 "&x99;&x99;">
]>
<SOAP-ENV:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema"

xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-
instance">
<SOAP-ENV:Body>
<ns1:echo xmlns:ns1="aaa"><b>&x100;</b>
</ns1:echo>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

And this causes REXML to go into an heavily-loaded xml parsing loop.

Currently, as far as I know, Rails 1.1.x and Rails 1.2.x are affected.
Maybe, all of the Rails based Web application.

And Mongrel, Lighttpd + FastCGI, and WEBrick are vulnerable.

I'm looking for a hot fix release, or a avoidance technique.
(For example, How to disable the DTD for Rails XML Parser.)

Please any one help me!

---
3arrows

[DHH]

> I'm looking for a hot fix release, or a avoidance technique.
> (For example, How to disable the DTD for Rails XML Parser.)

Hi 3arrows,

Thanks for the report. I've immediately tried to contact the REXML
guys. This will be a problem for all Ruby applications that accept XML
from the outside world. So a hotfix should preferably come from them,
so it can benefit everyone in the Ruby community.

But in the mean time, it'd be great if others also looked into ways of
stopping this from the Rails side.

[Daniel N]

Is this a DOM mode only thing?  Is the SAX streaming affected as well?

-Daniel

[Rick Olson]

On Aug 3, 10:55 am, "Daniel N" <has....@gmail.com> wrote:

> On 8/4/07, DHH <david.heineme...@gmail.com> wrote:
>
>
>
> > > I'm looking for a hot fix release, or a avoidance technique.
> > > (For example, How to disable the DTD for Rails XML Parser.)
>
> > Hi 3arrows,
>
> > Thanks for the report. I've immediately tried to contact the REXML
> > guys. This will be a problem for all Ruby applications that accept XML
> > from the outside world. So a hotfix should preferably come from them,
> > so it can benefit everyone in the Ruby community.
>
> > But in the mean time, it'd be great if others also looked into ways of
> > stopping this from the Rails side.

I've done some investigation into what's going on. It's the &x100;
entity causing issues. At the top of the doctype you define

<!ENTITY x0 "hello">

So, &x0; is 'hello'. Then:

<!ENTITY x1 "&x0;&x0;">

&x1; is 'hellohello'

That cycle continues, so &x100; would have to navigate backwards
through all 99 entities and figure out what the value is. It's pretty
big :) This is probably something for REXML or XmlSimple to fix. In
the meantime, someone could create a filter that scrubs doctype
entities before processing them.

Also, don't bother pinging rails apps, you can test it locally like
this:

require 'rubygems'
require 'xmlsimple'
XmlSimple.xml_in 'bad-file.xml'

[Rick Olson]

Lame one-line fix until someone smarter than me comes up with one:

REXML::Document.class_eval { def doctype() nil end }

This hides any doctype so XmlSimple will ignore it. I think it's good
enough until this gets properly patched.

script/plugin install
http://svn.techno-weenie.net/projects/plugins/xml_simple_doctype_fix

[3arrows]

Thanks a lot, Rick and you all!

Rick's code works very well on my test confirming the risk of this
Dos.
It will resolve serious, widespread problems.

(But, I've never checked the adverse affect.)
I'll go on additional tests.

This setting is desirable from the viewpoint of security of the Rails
based Web Application.
As far as I know, Major commercial server applications reject the
outside DTD(Doctypes) by default setting.

I should like to thank you promptly and fully for your cooperation.

[Sean E. Russell]

This is an exponential expansion of entities -- the test XML is
requiring REXML to produce a string that is 5 * 2^100 characters long.

This isn't something for REXML to fix, because it isn't a bug; it is a
side-effect of the proper operation of the entity expansion. You have
two options: either apply the doctype-erasing process that was
suggested by Rick, or have the code that processes the SOAP message
check for DOS attacks.

Cheers,

--- SER

[Sean E. Russell]

Here's a little more information:

There are two solutions: either remove the doctype declaration, or
have the code that interprets the SOAP message check for DOS attacks.
Removing the doctype can be done either prior to parsing:

REXML::Document.class_eval { def doctype() nil end }

(as suggested by Rick) or after parsing:

d = REXML::Document.new( bad_xml )
d << REXML::DocType.new

Unfortunately for both of these cases, the XML spec requires that
undefined entities be reported as errors, so that trying to read the
text will result in an Exception. Therefore, it really is up to the
SOAP toolkit to perform sanity checks on the DocType to see if an
exploit like this is being attempted -- you should NOT blindly delete
the DocType in an attempt to avoid this exploit.

Cheers,

--- SER

[Neil Wilson]

Is this important enough to warrant a message on the 'rubyonrails-
security' group/mailing list? Not everybody monitors this one.

NeilW

[Jeremy Evans]

Also, if you don't need to support XML uploads in your application,
you can disable Rails use of XMLSimple by including the following in
your environment:

ActionController::Base.param_parsers.delete(Mime::XML)

This is untested but should work if I'm reading the source correctly.
If a core member could confirm, that'd be great.

Jeremy

[Michael Koziarski]

> Also, if you don't need to support XML uploads in your application,
> you can disable Rails use of XMLSimple by including the following in
> your environment:
>
> ActionController::Base.param_parsers.delete(Mime::XML)
>
> This is untested but should work if I'm reading the source correctly.
> If a core member could confirm, that'd be great.

That should work fine, the other options is to override
XmlSimple.xml_in to a no-op or raise an exception

--
Cheers

Koz