failed to create a child event loop

[prasun sultania]

I am creating about 150 files of around 5MB sizes. Vertx file APIs gives an exception randomly after creating 10-15 files "failed to create a child event loop".

(I am using vertx3 and java8)

Below is my code snippet (After I get a callback then only I call the function again to create the next file. So, file creation is never concurrent):

Vertx.vertx().fileSystem().writeFile(filepath,

       Buffer.buffer(dataList.toString()), result -> {

 if (result.succeeded()) {

   System.out.println("File written");

 } else {

  System.err.println("Oh oh ..." + result.cause());

 }

 lambda.callback();

});

Below is my exception stack trace:

java.lang.IllegalStateException: failed to create a child event loop

    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:68)

    at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:49)

    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:61)

    at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:52)

    at io.vertx.core.impl.VertxImpl.<init>(VertxImpl.java:132)

    at io.vertx.core.impl.VertxImpl.<init>(VertxImpl.java:126)

    at io.vertx.core.impl.VertxImpl.<init>(VertxImpl.java:122)

    at io.vertx.core.impl.VertxFactoryImpl.vertx(VertxFactoryImpl.java:34)

    at io.vertx.core.Vertx.vertx(Vertx.java:78)

    at rc_datawarehouse.CsvToJsonChunkWriter.writeCsvChunkToFiles(CsvToJsonChunkWriter.java:73)

    at rc_datawarehouse.CsvToJsonReadWrite.lambda$2(CsvToJsonReadWrite.java:119)

    at rc_datawarehouse.CsvToJsonReadWrite$$Lambda$16/1470881859.handle(Unknown Source)

    at io.vertx.core.file.impl.AsyncFileImpl.handleData(AsyncFileImpl.java:335)

    at io.vertx.core.file.impl.AsyncFileImpl.lambda$doRead$285(AsyncFileImpl.java:320)

    at io.vertx.core.file.impl.AsyncFileImpl$$Lambda$17/1067800899.handle(Unknown Source)

    at io.vertx.core.file.impl.AsyncFileImpl$2.lambda$done$289(AsyncFileImpl.java:408)

    at io.vertx.core.file.impl.AsyncFileImpl$2$$Lambda$18/1632681662.handle(Unknown Source)

    at io.vertx.core.impl.ContextImpl.lambda$wrapTask$15(ContextImpl.java:314)

    at io.vertx.core.impl.ContextImpl$$Lambda$5/1780132728.run(Unknown Source)

    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)

    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)

    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)

    at java.lang.Thread.run(Unknown Source)

Caused by: io.netty.channel.ChannelException: failed to open a new selector

    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:128)

    at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:120)

    at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:87)

    at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:64)

    ... 22 more

Caused by: java.io.IOException: Unable to establish loopback connection

    at sun.nio.ch.PipeImpl$Initializer.run(Unknown Source)

    at sun.nio.ch.PipeImpl$Initializer.run(Unknown Source)

    at java.security.AccessController.doPrivileged(Native Method)

    at sun.nio.ch.PipeImpl.<init>(Unknown Source)

    at sun.nio.ch.SelectorProviderImpl.openPipe(Unknown Source)

    at java.nio.channels.Pipe.open(Unknown Source)

    at sun.nio.ch.WindowsSelectorImpl.<init>(Unknown Source)

    at sun.nio.ch.WindowsSelectorProvider.openSelector(Unknown Source)

    at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:126)

    ... 25 more

Caused by: java.net.SocketException: No buffer space available (maximum connections reached?): connect

    at sun.nio.ch.Net.connect0(Native Method)

    at sun.nio.ch.Net.connect(Unknown Source)

    at sun.nio.ch.Net.connect(Unknown Source)

    at sun.nio.ch.SocketChannelImpl.connect(Unknown Source)

    at java.nio.channels.SocketChannel.open(Unknown Source)

    at sun.nio.ch.PipeImpl$Initializer$LoopbackConnector.run(Unknown Source)

    ... 34 more

[Jez P]

Vertx.vertx()

Creates a new vertx every time.

Surely you want your function to accept a vertx so you only create one set of event loops, not one set each time you call the function.

[prasun sultania]

Makes sense. I will update my findings :)

Thanks,

Prasun Sultania

http://prasun.io/

    

--
You received this message because you are subscribed to a topic in the Google Groups "vert.x" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vertx/UYabQF1CGhU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vertx+un...@googlegroups.com.
Visit this group at https://groups.google.com/group/vertx.

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/8fb30323-6458-4e50-b1ac-6deac26c8822%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Jez P]

I ran into that a while back (always thought that Vertx.vertx() was a singleton instance, until I abused it and caused a file handle leak). 

I hope it turns out that I'm right and you can get things ticking along nicely again.

[prasun sultania]

Yeah, that worked. Thanks Jez!

(Interesting fact: Original problem occurred on windows machine with normal HDD and 4GB RAM with 10-15 files of 5MB. To reproduce the same problem on Mac with 8GB RAM and SSD it took me about 1500 files of 32MB each. Probably, depends on no. of max threads supported by OS and disk write speed)

Thanks,

Prasun Sultania

http://prasun.io/

    

To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/13149187-5c10-408e-9570-15cb61df83e3%40googlegroups.com.