OrientDb 2.1.15 java.net.SocketException: Too many open files when 9+ Databases are created
115 views
Skip to first unread message
pabloa
May 20, 2016, 5:41:30 PM5/20/16
to OrientDB
Hello,
I have a program using pyorientdb that creates several databases (around 10) in 5 minutes approx. It imports a json backup in each database that creates around 300+ clusters.
In some point, when it tries to create a new database, an exception is throw in orientdb log:
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
at java.net.ServerSocket.implAccept(ServerSocket.java:545)
at java.net.ServerSocket.accept(ServerSocket.java:513)
at com.orientechnologies.orient.server.network.OServerNetworkListener.run(OServerNetworkListener.java:194)
Error on client connection
java.net.SocketException: Too many open files
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
at java.net.ServerSocket.implAccept(ServerSocket.java:545)
at java.net.ServerSocket.accept(ServerSocket.java:513)
at com.orientechnologies.orient.server.network.OServerNetworkListener.run(OServerNetworkListener.java:194)
No more databases can be created until server is restarted.
Luca Garulli
May 20, 2016, 5:50:12 PM5/20/16
to OrientDB
Could you check please the number of files opened?
--
---
You received this message because you are subscribed to the Google Groups "OrientDB" group.
To unsubscribe from this group and stop receiving emails from it, send an email to orient-databa...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
pabloa
May 20, 2016, 11:59:01 PM5/20/16
to OrientDB
In this very moment when orientDb is doing nothing (but I was generating like 5 databases using the described method) there are 962 files open.
And:
$ ulimit -Sn
1024
$ ulimit -Hn
4096
So I will increase the limit to something bigger.
Luca Garulli
May 21, 2016, 6:08:27 AM5/21/16
to OrientDB
In order to reduce the number of files generated, you could create the database and then set this command:
alter database minimumclusters 1
In this way OrientDB will avoid to create X clusters per class.
scott molinari
May 22, 2016, 3:56:20 AM5/22/16
to OrientDB
Would that mean pabloa would then need to make a sacrifice on performance?
Scott
Luca Garulli
May 22, 2016, 4:37:24 AM5/22/16
to OrientDB
If you have multiple cores, having multiple clusters means more speed on insertion and with query if you use the PARALLEL keyword.
On 22 May 2016 at 09:56, 'scott molinari' via OrientDB <orient-...@googlegroups.com> wrote:
Would that mean pabloa would then need to make a sacrifice on performance?Scott
--
scott molinari
May 22, 2016, 12:55:42 PM5/22/16
to OrientDB
Ok. Last question. If someone were to use PARALLEL, but had the database set to "minimum clusters 1", maybe like in a sandbox situation, would ODB go about its business despite the illogical query with PARALLEL in it?
Scott
Luca Garulli
May 23, 2016, 11:30:21 AM5/23/16
to OrientDB
When you use PARALLEL, OrientDB checks if parallel processing is applicable. In the case you have 1 core only or 1 cluster only will ignore it.
On 22 May 2016 at 18:55, 'scott molinari' via OrientDB <orient-...@googlegroups.com> wrote:
Ok. Last question. If someone were to use PARALLEL, but had the database set to "minimum clusters 1", maybe like in a sandbox situation, would ODB go about its business despite the illogical query with PARALLEL in it?Scott
--
scott molinari
May 24, 2016, 9:41:12 AM5/24/16
to OrientDB
Roger. So ODB will go about its business and ignore the useless PARALLEL, if the minimum clusters is set to 1. That is good! Thanks!
Scott
pabloa
May 31, 2016, 3:12:20 PM5/31/16
to OrientDB
Hello Luca
alter database minimumclusters 1
That IS NOT an option. Too many open files means user is dealing with LOT of data running in a big server.
The answer is to set ulimit to a higher number.
Pablo
Reply all
Reply to author
Forward
0 new messages