Document size limit should be applied to single document not the whole input stream

[Petr Gladkikh]

Hello,

// As a side note, the links in the project README to Slack and the Jira did not work for me.

Link to Slack leads to an empty space, and bug-tracker link says "We can't let you see this page". Probably these should be updated. Google groups link is the only one that I could use.

After recent changes (between 1.26 and 1.33). Loading long streams of documents hits document size limit. It is unusual to have very long or even indefinite input streams. As I can tell Yaml.loadAll method parses documents lazily and they are not accumulated inside of the parser, so limit on the whole input stream size does not make sense technically. Besides, the error message mentions "YAML document" which implies it is per document not per stream limit.

So I believe the limit should be applied to single document and be reset very time a complete document is loaded.

An example of the stack trace where the limit is hit (Snakeyaml v1.33) is below. In my case each document in the input file is only few kilobytes in length (the whole input file is about 12Mb), so there should be no problem reading as many of them as necessary.

Thanks.

-----------------------------
The incoming YAML document exceeds the limit: 3145728 code points.
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:342)
at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:263)
at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingKey.produce(ParserImpl.java:662)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:185)
at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:57)
at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:43)
at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:136)
at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:116)
at org.yaml.snakeyaml.composer.Composer.composeScalarNode(Composer.java:239)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:208)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:357)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:336)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:311)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:212)
at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:357)
at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:336)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:311)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:212)
at org.yaml.snakeyaml.composer.Composer.getNode(Composer.java:134)
at org.yaml.snakeyaml.constructor.BaseConstructor.getData(BaseConstructor.java:168)
at org.yaml.snakeyaml.Yaml$1.next(Yaml.java:499)

[Andrey Somov]

Hi Petr,

sorry, because of a hacker attack I had to close the issue tracker (they created tons of trash already...).

I think your point is valid - I will make an issue and try to fix it.

Cheers,

Andrey

--
You received this message because you are subscribed to the Google Groups "SnakeYAML" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snakeyaml-cor...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/snakeyaml-core/83d00b0c-9d3e-47b4-8c66-4a23bac76fd7n%40googlegroups.com.

[Lee Read]

Hi Andrey,

Did you manage to create an issue for this one?

Much thanks,

Lee

[Andrey Somov]

It was fixed without the ticket.

Feel free to create one - we can check how it works (it should be pre-moderated now)

https://bitbucket.org/snakeyaml/snakeyaml/wiki/Changes

Andrey

To view this discussion on the web visit https://groups.google.com/d/msgid/snakeyaml-core/372f3c2d-0d3f-453c-924c-fbc5a9f73812n%40googlegroups.com.