Limit json payload

[Veit Guna]

Hi.

Currently I'm using jackson together with jersey to implement a REST service. To avoid attackers to perform DoS attacks by sending large JSONs - maybe leading to a OutOfMemoryExceptions, I'm wondering
whether there are any settings to limit the size of the inputstream that jackson will process. It would be great if jackson would then throw some sort of exception if the given limit is exceeded.

Does something exist like that in jackson? I've read about a tomcat/apache settings to limit the request size, but I would like to keep it under application (.war) control instead.

Thanks!

[Tatu Saloranta]

There is no automated support currently. There are multiple levels at which this could work (for example: by raw content length vs number of tokens vs depth of nesting vs number of distinct names), and so question of who and where should add limits is not easy to answer.

Adding simple limits would not be horribly difficult -- I have implemented a set of limits for Woodstox XML parser, for example (see http://coheigea.blogspot.com/2013/06/denial-of-service-attacks-on-apache-cxf.html for more), but there are issues like:

1. How to configure settings -- adding explicit setters is cumbersome, and there isn't a convenient way to add settings other simple on/off features currently

   - Most likely it would have to be a new container object for configuring input limits

2. Should this work for other formats? Do kinds of limits vary between formats

3. Is this best implemented at library level, or outside? For streaming parser or databind

As is, I would suggest implementing your own InputStream or Reader wrapper to enforce limits: it is trivially easy to implement something that updates read counts for 3 read methods, and throw IOException for content that may exceed limits.

Now: if you do not control how input is provided (via Jersey you do not usually provide InputStream or Reader), the way you can still do it is by implementing and registering `InputDecorator` for `JsonFactory` -- it will be given a chance to wrap given input source into appropriate wrapper.

This can be Reader/InputStream you implement that keeps track of amount of input read, and appropriately throw exception.

I hope this helps,

-+ Tatu +-

ps. If you end up implementing something along these lines, a blog article would be great, and probably something many users would find very interesting.

--
You received this message because you are subscribed to the Google Groups "jackson-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
To post to this group, send email to jackso...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Veit Guna]

Hi Tatu.
 
Thanks for the detailed answer!
 
You're right. There are many more things to consider I haven't thought of yet (like the post you mentioned).
 
For now I ended up writing my own ContainerRequestFilter that comes with JAX-RS. It allows to hook in very early of the request processing providing the InputStream.
Since all requests regarding the REST endpoints are going through JAX-RS, it's a central place to apply restrictions. So this applies to JSON content
as well as binary content e.g. when performing multipart uploads etc.
 
Currently I just applied restrictions to the maximum size of the stream that can be handled, as well as max duration of a transfer and minimum throughput.
Of course, more detailed restrictions regarding e.g. the JSON structure could be considered too.
 
Maybe I have to think about that a little longer ;).
 
Thanks!
Veit
 

Gesendet: Montag, 29. Februar 2016 um 02:27 Uhr
Von: "Tatu Saloranta" <ta...@fasterxml.com>
An: jackso...@googlegroups.com
Betreff: Re: [jackson-user] Limit json payload

To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com[jackson-user...@googlegroups.com].
To post to this group, send email to jackso...@googlegroups.com[jackso...@googlegroups.com].
For more options, visit https://groups.google.com/d/optout[https://groups.google.com/d/optout].

 
--
You received this message because you are subscribed to the Google Groups "jackson-user" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com[jackson-user...@googlegroups.com].
To post to this group, send email to jackso...@googlegroups.com[jackso...@googlegroups.com].
For more options, visit https://groups.google.com/d/optout[https://groups.google.com/d/optout].

[Veit Guna]

Hi Tatu.

I'm not familiar with the jackson internal codebase, but here are my ideas from the external perspective.

IMHO checking limits regarding the json structure itself can only be done by jackson I guess. Since it performs
the JSON parsing and binding to java beans by itself (?). I don't know about other supported formats (XML etc.) but
it seems to make sense, that these should also be put under limits when needed. For my personal usecase, JSON would
be sufficient ;).

1) Sounds reasonable to introduce a new configuration container that allows setting limits. E.g. via configure(jsonLimits).
2) I think restrictions should be applicable to different formats. On first sight, there are some common topics like maxNestingDepth, maxTotalElements, maxElementsPerParent etc. I don't know if there are some specials for XML or JSON...
3) It depends where the parsing happens. There I would apply the limits. I think everything that comes after the parsing is too late.

Are there currently any hooks in the jackson API where one can be notified e.g. via listener / visitor when the JSON is parsed/traversed?
This way one could simply implement a counter by oneself without having to put the burden of implementation to jackson itself.

For other formats like XML I guess that jackson simply uses a standard XML parser. So I think one would have to write something
at XML parser level for the limits. One could discuss if jackson would be the right place to configure the XML parser (limits).
Maybe depends whether XML (and other formats) are first class citizens or only the JSON...

I didn't even know that jackson also supports XML as well :).

Thanks
Veit


> Gesendet: Montag, 29. Februar 2016 um 09:34 Uhr
> Von: "Veit Guna" <Veit...@gmx.de>
> An: jackso...@googlegroups.com
> Cc: jackso...@googlegroups.com
> Betreff: Aw: Re: [jackson-user] Limit json payload

> To unsubscribe from this group and stop receiving emails from it, send an email to jackson-user...@googlegroups.com.
> To post to this group, send email to jackso...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

[Tatu Saloranta]

On Wed, Mar 2, 2016 at 6:13 AM, Veit Guna <Veit...@gmx.de> wrote:

Hi Tatu.

I'm not familiar with the jackson internal codebase, but here are my ideas from the external perspective.

IMHO checking limits regarding the json structure itself can only be done by jackson I guess. Since it performs
the JSON parsing and binding to java beans by itself (?). I don't know about other supported formats (XML etc.) but
it seems to make sense, that these should also be put under limits when needed. For my personal usecase, JSON would
be sufficient ;).

1) Sounds reasonable to introduce a new configuration container that allows setting limits. E.g. via configure(jsonLimits).
2) I think restrictions should be applicable to different formats. On first sight, there are some common topics like maxNestingDepth, maxTotalElements, maxElementsPerParent etc. I don't know if there are some specials for XML or JSON...
3) It depends where the parsing happens. There I would apply the limits. I think everything that comes after the parsing is too late.

Are there currently any hooks in the jackson API where one can be notified e.g. via listener / visitor when the JSON is parsed/traversed?
This way one could simply implement a counter by oneself without having to put the burden of implementation to jackson itself.

Not really, although one could perhaps use `JsonParserDelegate` and `JsonGeneratorDelegate` to get a proxy, and override `nextToken()`.

 

For other formats like XML I guess that jackson simply uses a standard XML parser. So I think one would have to write something
at XML parser level for the limits. One could discuss if jackson would be the right place to configure the XML parser (limits).

Correct. Although for specific case of XML, some underlying parsers (Woodstox specifically) do expose limits, and jackson-dataformat-xml does allow caller to configure provider to enforce such limits.

 

Maybe depends whether XML (and other formats) are first class citizens or only the JSON...

True. In practice, each dataformat module would need to either use native facilities (of underlying coder/decoder, if delegating; case for XML, YAML, Avro), or explicitly add support in implementation (like Protobuf, CBOR, Smile, CSV codecs).

 

I didn't even know that jackson also supports XML as well :).

Glad this came up then. :)

Ditto for CSV and others as well.

-+ Tatu +-