Kafka daemon getting stopped!!

[Pradipta Dash]

Hello Folks,

Any idea what can cause my kafka demon to goes down frequently (every few hours). I am getting below error 

java.io.FileNotFoundException: /kafka/replication-offset-checkpoint.tmp (Too many open files) at java.io.FileOutputStream.open0(Native Method) at java.io.FileOutputStream.open(FileOutputStream.java:270) at java.io.FileOutputStream.<init>(FileOutputStream.java:213) at java.io.FileOutputStream.<init>(FileOutputStream.java:162) at kafka.server.OffsetCheckpoint.write(OffsetCheckpoint.scala:49) a

[2017-12-07 14:44:29,976] INFO KafkaConfig values: advertised.host.name = null advertised.listeners = null advertised.port = null authorizer.class.name = auto.create.topics.enable = true auto.leader.rebalance.enable = true background.threads = 10 broker.id = 101 broker.id.generation.enable = true broker.rack = null compression.type = producer connections.max.idle.ms = 600000 control

[2017-12-07 14:44:30,030] INFO starting (kafka.server.KafkaServer)

Much appreciate your help.

Thanks,

Pradipta

[Saïd Bouras]

Hi Pradipta,

Kafka keep a lot of files descriptor open to speed I/O on filesystem, it must be that Kafka outgrow the limit fixe by the system.
Can you check in your hosts the max of open file per process. You can change this maximum number (of open files count per process) via ulimit -n or directly per system with "echo 8000000 > /proc/sys/fs/file-max"

Tell me if that's solve your problem

Best regards

--
You received this message because you are subscribed to the Google Groups "Confluent Platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to confluent-platf...@googlegroups.com.
To post to this group, send email to confluent...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/confluent-platform/556e5119-6520-4e2a-b539-399c8dff3f9f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Saïd BOURAS

Data Engineer

Mobile: 0662988731 

Zenika Paris
10 rue de Milan 75009 Paris
Standard : +33(0)1 45 26 19 15 - Fax : +33(0)1 72 70 45 10