read-line is limited to 4095 chars

[baptiste...@oscaro.com]

Hello there, 

I was using read-line to read a large line and found it limits its input to 4095 chars. I tested this on both Clojure 1.8.0 and 1.9.0-beta1. 

Steps to reproduce: 

$ lein new foo 
$ cd foo 
$ lein repl 
... 
=> (count (read-line)) 

Then enter a line that’s longer than 4095 characters; I used 8000 a's. 

Expected output: 

8000 

Actual output: 

4095 

No matter the real length of your input, it’ll always be truncated to 4095 chars if it’s longer. 

This is not in the documentation so I assume it’s either an implementation bug or a documentation miss. I wanted to know which one it is before trying to fix it. 

I’ve done some research and it appears to be a limitation of the BufferedReader’s readLine method that uses a buffer of either 4k or 8k depending on the implementation. Should we (1) work around this limit in the code so that read-line does actually read a whole line or (2) just update the documentation to note this limit?

Thanks,

-- Baptiste Fontaine

[Alex Miller]

Please file a jira enhancement. Changes could include updating the doc string, or adding a new arity to read-line that takes a size and passes it through to the buffered reader.

[David Powell]

I'm not experiencing this on Windows... Could it be that your
terminal or tooling is limiting the input to 4096 characters?

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[baptiste...@oscaro.com]

> I'm not experiencing this on Windows...  Could it be that your 
> terminal or tooling is limiting the input to 4096 characters?

Are you able to read more than 8k characters?

[Juraj Martinka]

I'm running on Mac OS X Sierra with Clojure 1.8.

I have to admit that behavior when trying to read a long line in terminal/repl is a bit odd (it may be the limitation of terminal).

However, If I try this via input from the text file it works properly:

    wc -c very-long-line.txt

    8101 very-long-line.txt

    java -cp /Users/jumar/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar clojure.main read-line.clj < very-long-line.txt

    8100

Note: I actually put 8100 chars into the file, so 8101 reported by wc probably includes new line character.

[Matching Socks]

The OP's figure for 4k suggests the "k" must be interpreted as x1024, so the corresponding threshold for 8k should be 8192.

[Juraj Martinka]

It still works for me going beyond 8192:

    wc -c very-long-line.txt

    8263 very-long-line.txt

    java -cp /Users/jumar/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar clojure.main read-line.clj < very-long-line.txt

    8262

[Peter Hull]

On Tuesday, 26 September 2017 12:28:26 UTC+1, baptiste...@oscaro.com wrote:

I was using read-line to read a large line and found it limits its input to 4095 chars. I tested this on both Clojure 1.8.0 and 1.9.0-beta1. 

I tested this on Windows. I found that piping from the CMD.EXE command line into a compiled uberjar was OK (I tested up to 32K and gave up) but pasting into the prompt of a repl was not. (I could get up to approx 8K of ASCII) This was true of lein repl and clojure's own repl.

I got similar on Linux (Fedora LXQT) - 'unlimited' for piping but 4096 for pasting into a repl prompt.

I looked at the source code, can't see why.

Hope that helps someone.

[Tijs Mallaerts]

Running the command "stty -icanon" in the KDE Konsole terminal fixed the 4096 limit for me.

Op donderdag 28 september 2017 13:32:48 UTC+2 schreef Peter Hull: