Plumber persist error

Rishi Mishra

unread,

May 9, 2016, 5:32:37 AM5/9/16

to Druid User

I am using tranquility-kafka.

I get this error a few times. Why would druid be creating file names above the limit?

Exception in thread "plumber_persist_13" java.lang.RuntimeException: java.io.IOException: File name too long

at com.google.common.base.Throwables.propagate(Throwables.java:160)

at io.druid.segment.realtime.plumber.RealtimePlumber.persistHydrant(RealtimePlumber.java:1049)

at io.druid.segment.realtime.plumber.RealtimePlumber$3.doRun(RealtimePlumber.java:445)

at io.druid.common.guava.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:42)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Caused by: java.io.IOException: File name too long

at java.io.UnixFileSystem.createFileExclusively(Native Method)

at java.io.File.createTempFile(File.java:2024)

at java.io.File.createTempFile(File.java:2070)

at io.druid.segment.data.TmpFileIOPeon.makeOutputStream(TmpFileIOPeon.java:55)

at io.druid.segment.data.GenericIndexedWriter.open(GenericIndexedWriter.java:68)

at io.druid.segment.IndexMerger.makeIndexFiles(IndexMerger.java:657)

at io.druid.segment.IndexMerger.merge(IndexMerger.java:421)

slim bouguerra

unread,

May 9, 2016, 10:46:31 AM5/9/16

to druid...@googlegroups.com

Which version of druid and tranquility are you using ?

--
You received this message because you are subscribed to the Google Groups "Druid User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to druid-user+...@googlegroups.com.
To post to this group, send email to druid...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/druid-user/52af4c60-07f6-4de4-a728-db47f9d2bef7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Rishi Mishra

unread,

May 9, 2016, 11:02:05 AM5/9/16

to Druid User

Druid 0.9.0, Tranquility -0.7.4

Fangjin Yang

unread,

May 10, 2016, 8:28:07 PM5/10/16

to Druid User

Hi Rishi, what is your operating system?

Gian Merlino

unread,

May 10, 2016, 9:27:09 PM5/10/16

to druid...@googlegroups.com

I think that code is writing out files named after columns. Is it possible that one of your column names is too long to be a filename? Perhaps you are using dimension discovery (empty dimensions list in your schema) and some of your json objects have very long field names.