13/02/02 05:05:49 WARN network.SendingConnection: Error writing in connection to ConnectionManagerId(shd4.quantifind.com,56609) java.io.IOException: Bad address at sun.nio.ch.FileDispatcher.write0(Native Method) at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29) at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:72) at sun.nio.ch.IOUtil.write(IOUtil.java:28) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334) at spark.network.SendingConnection.write(Connection.scala:246) at spark.network.ConnectionManager.run(ConnectionManager.scala:138) at spark.network.ConnectionManager$$anon$4.run(ConnectionManager.scala:73) 13/02/02 05:05:49 INFO storage.MemoryStore: ensureFreeSpace(380777323) called with curMem=50751548973, maxMem=50935627776 13/02/02 05:05:49 INFO storage.MemoryStore: 1 blocks selected for dropping 13/02/02 05:05:49 INFO storage.BlockManager: Dropping block rdd_34_66 from memory 13/02/02 05:05:49 INFO storage.MemoryStore: Block rdd_34_66 of size 298907130 dropped from memory (free 482985933) 13/02/02 05:05:49 INFO storage.MemoryStore: Block rdd_12_228 stored as values to memory (estimated size 363.1 MB, free 97.5 MB) 13/02/02 05:05:49 INFO network.ConnectionManager: Handling connection error on connection to ConnectionManagerId(shd4.quantifind.com,56609) 13/02/02 05:05:49 INFO network.ConnectionManager: Removing SendingConnection to ConnectionManagerId(shd4.quantifind.com,56609) 13/02/02 05:05:49 INFO network.ConnectionManager: Removing SendingConnection to ConnectionManagerId(shd4.quantifind.com,56609) 13/02/02 05:05:49 INFO storage.MemoryStore: ensureFreeSpace(237125032) called with curMem=50833419166, maxMem=50935627776 13/02/02 05:05:49 INFO storage.MemoryStore: 1 blocks selected for dropping 13/02/02 05:05:49 INFO storage.BlockManager: Dropping block rdd_34_46 from memory 13/02/02 05:05:49 INFO storage.MemoryStore: Block rdd_34_46 of size 279686732 dropped from memory (free 381895342) 13/02/02 05:05:49 INFO storage.MemoryStore: Block rdd_12_231 stored as values to memory (estimated size 226.1 MB, free 138.1 MB) 13/02/02 05:05:49 ERROR network.ConnectionManager: Error in select loop java.util.NoSuchElementException: key not found: sun.nio.ch.SelectionKeyImpl@303a169f at scala.collection.MapLike$class.default(MapLike.scala:224) at scala.collection.mutable.HashMap.default(HashMap.scala:43) at scala.collection.MapLike$class.apply(MapLike.scala:135) at spark.network.ConnectionManager$$anon$1.scala$collection$mutable$SynchronizedMap$$super$apply(ConnectionManager.scala:48) at scala.collection.mutable.SynchronizedMap$class.apply(SynchronizedMap.scala:48) at spark.network.ConnectionManager$$anon$1.apply(ConnectionManager.scala:48) at spark.network.ConnectionManager.run(ConnectionManager.scala:96) at spark.network.ConnectionManager$$anon$4.run(ConnectionManager.scala:73)
I haven't been able to find any patterns into when these exceptions occur. Any insights into this would be useful.
Thanks,
Karthik
There were few bugs I Connect ion and Connection Manager wbich were fixed and committed recently ... and is in spark master right now.
Are you seeing same issue with master ?
Regards
Mridul
--
You received this message because you are subscribed to the Google Groups "Spark Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to spark-users...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
13/05/16 18:41:01 INFO storage.BlockManager: maxBytesInFlight: 50331648, minRequest: 10066329 13/05/16 18:41:01 INFO storage.BlockManager: maxBytesInFlight: 50331648, minRequest: 10066329 13/05/16 18:41:01 INFO storage.BlockManager: maxBytesInFlight: 50331648, minRequest: 10066329 13/05/16 18:41:01 INFO storage.BlockManager: Started 166 remote gets in 8 ms 13/05/16 18:41:01 WARN network.SendingConnection: Error finishing connection to achab3/127.0.1.1:37345 java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592) at spark.network.SendingConnection.finishConnect(Connection.scala:221) at spark.network.ConnectionManager.spark$network$ConnectionManager$$run(ConnectionManager.scala:127) at spark.network.ConnectionManager$$anon$4.run(ConnectionManager.scala:70) 13/05/16 18:41:01 INFO network.ConnectionManager: Handling connection error on connection to ConnectionManagerId(achab3,37345) 13/05/16 18:41:01 INFO network.ConnectionManager: Removing SendingConnection to ConnectionManagerId(achab3,37345) 13/05/16 18:41:01 INFO network.SendingConnection: Initiating connection to [achab3/127.0.1.1:35226]