Connection reset when creating an SSL enabled volume
704 views
Skip to first unread message
jeff...@hostdime.com
Jan 4, 2013, 9:58:29 AM1/4/13
to xtre...@googlegroups.com
Hello again,
I have been able to finally get everything I needed working with XtreemFS going in some capacity or another, but my latest testing is with the SSL features. I followed the user guide to set up a very quick version where the server hosting all of the daemons is also the client. The problem is during the mkfs.xtreemfs step, it hangs for a bit before giving an error about the connection being reset:
root@win3:~# mkfs.xtreemfs -d DEBUG --pkcs12-file-path=/etc/xos/xtreemfs/truststore/certs/client.p12 --pkcs12-passphrase='*********' localhost/disk2
Trying to create the volume: localhost/disk2
Using options:
Mode: 777
Access Control Policy: POSIX
Default striping policy: RAID0
Default stripe size (object size): 128
Default stripe width (# OSDs): 1
[ D | 1/ 4 04:13:48.353 | 0x26bfae0 ] Created a new libxtreemfs Client object (version 1.4 (Salty Sticks))
[ I | 1/ 4 04:13:48.353 | 0x26bfae0 ] SSL support activated.
[ I | 1/ 4 04:13:48.353 | 0x26bfae0 ] SSL support using PKCS#12 file /etc/xos/xtreemfs/truststore/certs/client.p12
[ D | 1/ 4 04:13:48.356 | 0x26bfae0 ] tmp file name:/tmp/pmK9igS2 /tmp/ct6RT8Yw
[ D | 1/ 4 04:13:48.356 | 0x26b1260 ] Starting RPC client.
[ D | 1/ 4 04:13:48.356 | 0x26b1260 ] Running in SSL mode.
[ D | 1/ 4 04:13:48.357 | 0x26bfae0 ] Generated client UUID: 5ognN5GI-Dwfj-rBoW-ipQv-Y3X4ExtlHAz7
[ D | 1/ 4 04:13:48.357 | 0x26b1260 ] new connection for localhost:32636
[ D | 1/ 4 04:13:48.357 | 0x26b1260 ] connect timeout is 60 seconds
[ D | 1/ 4 04:13:48.357 | 0x26b1260 ] resolved: localhost
[ D | 1/ 4 04:14:48.361 | 0x26b1260 ] Connection reset, next reconnect in 0 seconds.
[ E | 1/ 4 04:14:48.362 | 0x26b1260 ] operation failed: call_id=1 errno=5 message=connection to 'localhost:32636' timed out
[ E | 1/ 4 04:14:48.362 | 0x26bfae0 ] The client encountered a communication error sending a request to the server: localhost:32636. Error: connection to 'localhost:32636' timed out
Failed to create the volume, error:
connection to 'localhost:32636' timed out
[ D | 1/ 4 04:14:48.362 | 0x26bfae0 ] RPC client stopped.
I should also note this crashes both the MRC and OSD services every time it is run. They become marked as crashed until I restart them.
root@win3:/var/log/xtreemfs# tail -n20 mrc.log osd.log
==> mrc.log <==
[ I | BabuDBImpl | DiskLogger | 23 | Jan 04 04:13:44 ] has been successfully started.
[ I | CheckpointerImpl | ChkptrThr | 22 | Jan 04 04:13:44 ] Thread ChkptrThr started
[ I | BabuDBImpl | ChkptrThr | 22 | Jan 04 04:13:44 ] has been successfully started.
[ I | BabuDBImpl | MRC | 1 | Jan 04 04:13:44 ] BabuDB for Java is running (version 0.5.6)
[ E | HeartbeatThread | MRC | 1 | Jan 04 04:17:29 ] an error occurred while initially contacting the Directory Service: java.io.IOException: Request finally failed after 15 tries.
[ E | MRCRequestDispatcher | MRC | 1 | Jan 04 04:17:29 ] STARTUP FAILED!
[ E | MRCRequestDispatcher | MRC | 1 | Jan 04 04:17:29 ] java.io.IOException: cannot initialize service at XtreemFS DIR: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:271)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
[ E | MRCRequestDispatcher | MRC | 1 | Jan 04 04:17:29 ] root cause: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.dir.DIRClient.syncCall(DIRClient.java:406)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:234)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:228)
... org.xtreemfs.common.HeartbeatThread.registerServices(HeartbeatThread.java:339)
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:151)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
==> osd.log <==
[ I | FleaseStage | FleaseSt | 25 | Jan 04 04:13:51 ] Flease (version 0.2.4 (trunk)) ready
[ I | RPCUDPSocketServer | UDPComStage | 14 | Jan 04 04:13:51 ] UDP socket on port 32640 ready
[ I | FleaseStage | FleaseSt | 25 | Jan 04 04:13:51 ] Thread FleaseSt started
[ I | RPCUDPSocketServer | UDPComStage | 14 | Jan 04 04:13:51 ] Thread UDPComStage started
[ E | HeartbeatThread | OSD | 1 | Jan 04 04:17:36 ] an error occurred while initially contacting the Directory Service: java.io.IOException: Request finally failed after 15 tries.
[ E | OSDRequestDispatcher | OSD | 1 | Jan 04 04:17:36 ] STARTUP FAILED!
[ E | OSDRequestDispatcher | OSD | 1 | Jan 04 04:17:36 ] java.io.IOException: cannot initialize service at XtreemFS DIR: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:271)
... org.xtreemfs.osd.OSDRequestDispatcher.start(OSDRequestDispatcher.java:504)
... org.xtreemfs.osd.OSD.<init>(OSD.java:32)
... org.xtreemfs.osd.OSD.main(OSD.java:102)
[ E | OSDRequestDispatcher | OSD | 1 | Jan 04 04:17:36 ] root cause: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.dir.DIRClient.syncCall(DIRClient.java:406)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:234)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:228)
... org.xtreemfs.common.HeartbeatThread.registerServices(HeartbeatThread.java:339)
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:151)
... org.xtreemfs.osd.OSDRequestDispatcher.start(OSDRequestDispatcher.java:504)
... org.xtreemfs.osd.OSD.<init>(OSD.java:32)
... org.xtreemfs.osd.OSD.main(OSD.java:102)
The instructions were followed nearly verbatim (I used the same internal password throughout) so all service configs have the following lines:
ssl.enabled = true
ssl.service_creds.pw = ********
ssl.service_creds.container = pkcs12
ssl.service_creds = /etc/xos/xtreemfs/truststore/certs/SERVICE.p12
ssl.trusted_certs = /etc/xos/xtreemfs/truststore/certs/trusted.jks
ssl.trusted_certs.pw = *********
ssl.trusted_certs.container = jks
The errors from the best I can see just seem to indicate that the MRC and OSD services have an issue communicating with the DIR service. In trying to connect a number of times, they eventually fail and crash. Has this behavior been observed before? Any ideas on where I should go from here?
Current Setup:
Debian 6 64-bit
2.6.32-5-amd64
I have been able to finally get everything I needed working with XtreemFS going in some capacity or another, but my latest testing is with the SSL features. I followed the user guide to set up a very quick version where the server hosting all of the daemons is also the client. The problem is during the mkfs.xtreemfs step, it hangs for a bit before giving an error about the connection being reset:
root@win3:~# mkfs.xtreemfs -d DEBUG --pkcs12-file-path=/etc/xos/xtreemfs/truststore/certs/client.p12 --pkcs12-passphrase='*********' localhost/disk2
Trying to create the volume: localhost/disk2
Using options:
Mode: 777
Access Control Policy: POSIX
Default striping policy: RAID0
Default stripe size (object size): 128
Default stripe width (# OSDs): 1
[ D | 1/ 4 04:13:48.353 | 0x26bfae0 ] Created a new libxtreemfs Client object (version 1.4 (Salty Sticks))
[ I | 1/ 4 04:13:48.353 | 0x26bfae0 ] SSL support activated.
[ I | 1/ 4 04:13:48.353 | 0x26bfae0 ] SSL support using PKCS#12 file /etc/xos/xtreemfs/truststore/certs/client.p12
[ D | 1/ 4 04:13:48.356 | 0x26bfae0 ] tmp file name:/tmp/pmK9igS2 /tmp/ct6RT8Yw
[ D | 1/ 4 04:13:48.356 | 0x26b1260 ] Starting RPC client.
[ D | 1/ 4 04:13:48.356 | 0x26b1260 ] Running in SSL mode.
[ D | 1/ 4 04:13:48.357 | 0x26bfae0 ] Generated client UUID: 5ognN5GI-Dwfj-rBoW-ipQv-Y3X4ExtlHAz7
[ D | 1/ 4 04:13:48.357 | 0x26b1260 ] new connection for localhost:32636
[ D | 1/ 4 04:13:48.357 | 0x26b1260 ] connect timeout is 60 seconds
[ D | 1/ 4 04:13:48.357 | 0x26b1260 ] resolved: localhost
[ D | 1/ 4 04:14:48.361 | 0x26b1260 ] Connection reset, next reconnect in 0 seconds.
[ E | 1/ 4 04:14:48.362 | 0x26b1260 ] operation failed: call_id=1 errno=5 message=connection to 'localhost:32636' timed out
[ E | 1/ 4 04:14:48.362 | 0x26bfae0 ] The client encountered a communication error sending a request to the server: localhost:32636. Error: connection to 'localhost:32636' timed out
Failed to create the volume, error:
connection to 'localhost:32636' timed out
[ D | 1/ 4 04:14:48.362 | 0x26bfae0 ] RPC client stopped.
I should also note this crashes both the MRC and OSD services every time it is run. They become marked as crashed until I restart them.
root@win3:/var/log/xtreemfs# tail -n20 mrc.log osd.log
==> mrc.log <==
[ I | BabuDBImpl | DiskLogger | 23 | Jan 04 04:13:44 ] has been successfully started.
[ I | CheckpointerImpl | ChkptrThr | 22 | Jan 04 04:13:44 ] Thread ChkptrThr started
[ I | BabuDBImpl | ChkptrThr | 22 | Jan 04 04:13:44 ] has been successfully started.
[ I | BabuDBImpl | MRC | 1 | Jan 04 04:13:44 ] BabuDB for Java is running (version 0.5.6)
[ E | HeartbeatThread | MRC | 1 | Jan 04 04:17:29 ] an error occurred while initially contacting the Directory Service: java.io.IOException: Request finally failed after 15 tries.
[ E | MRCRequestDispatcher | MRC | 1 | Jan 04 04:17:29 ] STARTUP FAILED!
[ E | MRCRequestDispatcher | MRC | 1 | Jan 04 04:17:29 ] java.io.IOException: cannot initialize service at XtreemFS DIR: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:271)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
[ E | MRCRequestDispatcher | MRC | 1 | Jan 04 04:17:29 ] root cause: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.dir.DIRClient.syncCall(DIRClient.java:406)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:234)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:228)
... org.xtreemfs.common.HeartbeatThread.registerServices(HeartbeatThread.java:339)
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:151)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
==> osd.log <==
[ I | FleaseStage | FleaseSt | 25 | Jan 04 04:13:51 ] Flease (version 0.2.4 (trunk)) ready
[ I | RPCUDPSocketServer | UDPComStage | 14 | Jan 04 04:13:51 ] UDP socket on port 32640 ready
[ I | FleaseStage | FleaseSt | 25 | Jan 04 04:13:51 ] Thread FleaseSt started
[ I | RPCUDPSocketServer | UDPComStage | 14 | Jan 04 04:13:51 ] Thread UDPComStage started
[ E | HeartbeatThread | OSD | 1 | Jan 04 04:17:36 ] an error occurred while initially contacting the Directory Service: java.io.IOException: Request finally failed after 15 tries.
[ E | OSDRequestDispatcher | OSD | 1 | Jan 04 04:17:36 ] STARTUP FAILED!
[ E | OSDRequestDispatcher | OSD | 1 | Jan 04 04:17:36 ] java.io.IOException: cannot initialize service at XtreemFS DIR: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:271)
... org.xtreemfs.osd.OSDRequestDispatcher.start(OSDRequestDispatcher.java:504)
... org.xtreemfs.osd.OSD.<init>(OSD.java:32)
... org.xtreemfs.osd.OSD.main(OSD.java:102)
[ E | OSDRequestDispatcher | OSD | 1 | Jan 04 04:17:36 ] root cause: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.dir.DIRClient.syncCall(DIRClient.java:406)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:234)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:228)
... org.xtreemfs.common.HeartbeatThread.registerServices(HeartbeatThread.java:339)
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:151)
... org.xtreemfs.osd.OSDRequestDispatcher.start(OSDRequestDispatcher.java:504)
... org.xtreemfs.osd.OSD.<init>(OSD.java:32)
... org.xtreemfs.osd.OSD.main(OSD.java:102)
The instructions were followed nearly verbatim (I used the same internal password throughout) so all service configs have the following lines:
ssl.enabled = true
ssl.service_creds.pw = ********
ssl.service_creds.container = pkcs12
ssl.service_creds = /etc/xos/xtreemfs/truststore/certs/SERVICE.p12
ssl.trusted_certs = /etc/xos/xtreemfs/truststore/certs/trusted.jks
ssl.trusted_certs.pw = *********
ssl.trusted_certs.container = jks
The errors from the best I can see just seem to indicate that the MRC and OSD services have an issue communicating with the DIR service. In trying to connect a number of times, they eventually fail and crash. Has this behavior been observed before? Any ideas on where I should go from here?
Current Setup:
Debian 6 64-bit
2.6.32-5-amd64
Björn Kolbeck
Jan 7, 2013, 4:25:38 AM1/7/13
to xtre...@googlegroups.com
Hi Jeff,
Looks to me like the directory service is either not running, not listening on the correct port or configured to use plain tcp (not SSL).
Please make sure that the DIR is up by checking the dir log (/var/log/xtreemfs/dir.log) and status webpage (port 30638). I would also recommend to use telnet from the MRC or OSD machine to check that the DIR is reachable, e.g. 'telnet <diraddress> 32638'.
If the DIR is not using SSL for some reason, you should see error messages in the log file about invalid requests. You may have to set 'debug.level = 7' in the dirconfig.properties.
If all of this fails, you can enable java SSL debugging to check for SSL problems. Add "-Djavax.net.debug=ssl" to the JAVA_CALL line in the init.d file.
Björn
jeff...@hostdime.com
Jan 8, 2013, 1:36:51 PM1/8/13
to xtre...@googlegroups.com
Before I run the mkfs command, I definitely have the DIR service up and listening:
root@win3:~# service xtreemfs-dir status
XtreemFS Directory Service (DIR) is running
root@win3:~# telnet localhost 32636
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> Connection closed.
root@win3:~# telnet localhost 32638
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> Connection closed.
I changed the log level to debug and I am not getting a constant exception about the end of stream being reached, but that is all:
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:50 ] java.io.IOException: End of stream has reached.
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:504)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:50 ] closing connection to /127.0.0.1:46928
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:55 ] connect from client at /127.0.0.1:58456
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:55 ] java.io.IOException: End of stream has reached.
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:504)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:55 ] closing connection to /127.0.0.1:58456
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:27:10 ] connect from client at /127.0.0.1:47478
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:27:10 ] java.io.IOException: End of stream has reached.
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:504)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:27:10 ] closing connection to /127.0.0.1:47478
For this simple test case, I am using the same server for everything. The DIR, MRC, and OSD are all on the same machine and I am just trying to create the volume with any success.
Due to some stray rules on our firewall, I changed the http port of the DIR service to 8080, which I can see is being listened on with nestat:
root@win3:~# netstat -plant | grep 8080
tcp6 0 0 :::8080 :::* LISTEN 3678/java
But I am getting a connection reset page when navigating to it in my browser. It works when I do not have SSL enabled. I am not sure if this is relevant since just about everything else indicates the DIR service is listening correctly. By chance, I accidentally left one of my old OSDs running and it was trying to reach this server while not using an SSL, and I got those debug messages you mentioned:
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:23:38 ] javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at sun.security.ssl.EngineInputRecord.bytesInCompletePacket(EngineInputRecord.java:171)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:814)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:727)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:512)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
But I've since disabled that server to test the local setup.
I am still getting a similar error with mkfs:
root@win3:~# mkfs.xtreemfs -d DEBUG --pkcs12-file-path=/etc/xos/xtreemfs/truststore/certs/client.p12 --pkcs12-passphrase='*********' localhost/disk2
Trying to create the volume: localhost/disk2
Using options:
Mode: 777
Access Control Policy: POSIX
Default striping policy: RAID0
Default stripe size (object size): 128
Default stripe width (# OSDs): 1
[ D | 1/ 8 13:33:20.964 | 0x283dae0 ] Created a new libxtreemfs Client object (version 1.4 (Salty Sticks))
[ I | 1/ 8 13:33:20.965 | 0x283dae0 ] SSL support activated.
[ I | 1/ 8 13:33:20.965 | 0x283dae0 ] SSL support using PKCS#12 file /etc/xos/xtreemfs/truststore/certs/client.p12
[ D | 1/ 8 13:33:20.968 | 0x283dae0 ] tmp file name:/tmp/pmTheUtW /tmp/ctEqbXqC
[ D | 1/ 8 13:33:20.968 | 0x282f260 ] Starting RPC client.
[ D | 1/ 8 13:33:20.968 | 0x282f260 ] Running in SSL mode.
[ D | 1/ 8 13:33:20.968 | 0x283dae0 ] Generated client UUID: rj5coAap-UA7S-buw5-eybr-8ETbUmGiIiSa
[ D | 1/ 8 13:33:20.969 | 0x282f260 ] new connection for localhost:32636
[ D | 1/ 8 13:33:20.969 | 0x282f260 ] connect timeout is 60 seconds
[ D | 1/ 8 13:33:20.969 | 0x282f260 ] resolved: localhost
[ D | 1/ 8 13:34:20.974 | 0x282f260 ] Connection reset, next reconnect in 0 seconds.
[ E | 1/ 8 13:34:20.974 | 0x282f260 ] operation failed: call_id=1 errno=5 message=connection to 'localhost:32636' timed out
[ E | 1/ 8 13:34:20.974 | 0x283dae0 ] The client encountered a communication error sending a request to the server: localhost:32636. Error: connection to 'localhost:32636' timed out
Failed to create the volume, error:
connection to 'localhost:32636' timed out
[ D | 1/ 8 13:34:20.974 | 0x283dae0 ] RPC client stopped.
However, despite this error about not being able to reach that port, I can still telnet to it just fine:
root@win3:~# telnet localhost 32636
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> Connection closed.
At this point the MRC and OSD services get marked as crashed.
Thanks for the help.
root@win3:~# service xtreemfs-dir status
XtreemFS Directory Service (DIR) is running
root@win3:~# telnet localhost 32636
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> Connection closed.
root@win3:~# telnet localhost 32638
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> Connection closed.
I changed the log level to debug and I am not getting a constant exception about the end of stream being reached, but that is all:
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:50 ] java.io.IOException: End of stream has reached.
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:504)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:50 ] closing connection to /127.0.0.1:46928
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:55 ] connect from client at /127.0.0.1:58456
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:55 ] java.io.IOException: End of stream has reached.
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:504)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:26:55 ] closing connection to /127.0.0.1:58456
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:27:10 ] connect from client at /127.0.0.1:47478
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:27:10 ] java.io.IOException: End of stream has reached.
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:504)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:27:10 ] closing connection to /127.0.0.1:47478
For this simple test case, I am using the same server for everything. The DIR, MRC, and OSD are all on the same machine and I am just trying to create the volume with any success.
Due to some stray rules on our firewall, I changed the http port of the DIR service to 8080, which I can see is being listened on with nestat:
root@win3:~# netstat -plant | grep 8080
tcp6 0 0 :::8080 :::* LISTEN 3678/java
But I am getting a connection reset page when navigating to it in my browser. It works when I do not have SSL enabled. I am not sure if this is relevant since just about everything else indicates the DIR service is listening correctly. By chance, I accidentally left one of my old OSDs running and it was trying to reach this server while not using an SSL, and I got those debug messages you mentioned:
[ D | RPCNIOSocketServer | PBRPCSrv@32638 | 12 | Jan 08 13:23:38 ] javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at sun.security.ssl.EngineInputRecord.bytesInCompletePacket(EngineInputRecord.java:171)
at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:814)
at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:727)
at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:512)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.readConnection(RPCNIOSocketServer.java:323)
at org.xtreemfs.foundation.pbrpc.server.RPCNIOSocketServer.run(RPCNIOSocketServer.java:272)
But I've since disabled that server to test the local setup.
I am still getting a similar error with mkfs:
root@win3:~# mkfs.xtreemfs -d DEBUG --pkcs12-file-path=/etc/xos/xtreemfs/truststore/certs/client.p12 --pkcs12-passphrase='*********' localhost/disk2
Trying to create the volume: localhost/disk2
Using options:
Mode: 777
Access Control Policy: POSIX
Default striping policy: RAID0
Default stripe size (object size): 128
Default stripe width (# OSDs): 1
[ I | 1/ 8 13:33:20.965 | 0x283dae0 ] SSL support activated.
[ I | 1/ 8 13:33:20.965 | 0x283dae0 ] SSL support using PKCS#12 file /etc/xos/xtreemfs/truststore/certs/client.p12
[ D | 1/ 8 13:33:20.968 | 0x283dae0 ] tmp file name:/tmp/pmTheUtW /tmp/ctEqbXqC
[ D | 1/ 8 13:33:20.968 | 0x282f260 ] Starting RPC client.
[ D | 1/ 8 13:33:20.968 | 0x282f260 ] Running in SSL mode.
[ D | 1/ 8 13:33:20.968 | 0x283dae0 ] Generated client UUID: rj5coAap-UA7S-buw5-eybr-8ETbUmGiIiSa
[ D | 1/ 8 13:33:20.969 | 0x282f260 ] new connection for localhost:32636
[ D | 1/ 8 13:33:20.969 | 0x282f260 ] connect timeout is 60 seconds
[ D | 1/ 8 13:33:20.969 | 0x282f260 ] resolved: localhost
[ D | 1/ 8 13:34:20.974 | 0x282f260 ] Connection reset, next reconnect in 0 seconds.
[ E | 1/ 8 13:34:20.974 | 0x282f260 ] operation failed: call_id=1 errno=5 message=connection to 'localhost:32636' timed out
[ E | 1/ 8 13:34:20.974 | 0x283dae0 ] The client encountered a communication error sending a request to the server: localhost:32636. Error: connection to 'localhost:32636' timed out
Failed to create the volume, error:
connection to 'localhost:32636' timed out
However, despite this error about not being able to reach that port, I can still telnet to it just fine:
root@win3:~# telnet localhost 32636
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
^]
telnet> Connection closed.
At this point the MRC and OSD services get marked as crashed.
Thanks for the help.
Björn Kolbeck
Jan 9, 2013, 4:52:50 AM1/9/13
to xtre...@googlegroups.com
Hi Jeff,
Please set the debug level for all services to 7 and add the "-Djavax.net.debug=ssl" parameter to the java call. This should give us some hint on what the problem is. Looks like the MRC can open a connection to the DIR, but the request is not processed.
I'd suggest to first debug the problem of the MRC not being able to contact the DIR. Java SSL has much better debug messages than openSSL (used by the client and command line tools).
Please make also sure that all servers use real ssl and don't have "gridssl" mode enabled.
Björn
jeff...@hostdime.com
Jan 10, 2013, 12:59:53 PM1/10/13
to xtre...@googlegroups.com
So it seems that the MRC and OSD servers are crashing regardless of whether the volume is made. It seems as though trying to create the volume just speeds up the process or perhaps it's just my imagination. Either way, they've both crashed without me making a volume and with that debug line added to the java call, there is much more verbose output to work with:
***
MRCRequestDispatcher, fatal error: 46: General SSLEngine problem
sun.security.validator.ValidatorException: No trusted certificate found
MRCRequestDispatcher, SEND TLSv1 ALERT: fatal, description = certificate_unknown
MRCRequestDispatcher, WRITE: TLSv1 Alert, length = 2
MRCRequestDispatcher, fatal: engine already closed. Rethrowing javax.net.ssl.SSLHandshakeException: General SSLEngine problem
[ D | RPCNIOSocketClient | MRCRequestDi... | 10 | Jan 10 12:40:03 ] javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1032)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:508)
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1136)
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1108)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:543)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.writeConnection(RPCNIOSocketClient.java:573)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.run(RPCNIOSocketClient.java:265)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1528)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:260)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:252)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1165)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:154)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:610)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:550)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:548)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:969)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doTasks(SSLChannelIO.java:710)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.analyseHandshakeStatus(SSLChannelIO.java:654)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:518)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.readConnection(RPCNIOSocketClient.java:417)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.run(RPCNIOSocketClient.java:262)
Caused by: sun.security.validator.ValidatorException: No trusted certificate found
at sun.security.validator.SimpleValidator.buildTrustedChain(SimpleValidator.java:329)
at sun.security.validator.SimpleValidator.engineValidate(SimpleValidator.java:126)
at sun.security.validator.Validator.validate(Validator.java:235)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:147)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:230)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:270)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1144)
... 11 more
MRCRequestDispatcher, called closeInbound()
MRCRequestDispatcher, fatal: engine already closed. Rethrowing javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
[ D | RPCNIOSocketClient | MRCRequestDi... | 10 | Jan 10 12:40:03 ] closing connection to localhost/127.0.0.1:32638
[ D | DIRClient | TSync Thr | 12 | Jan 10 12:40:03 ] Request failed due to exception: java.io.IOException: sending RPC failed: server closed connection: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
[ D | DIRClient | MRC | 1 | Jan 10 12:40:03 ] Switching to server localhost/127.0.0.1:32638 since the last attempt failed with the error: sending RPC failed: server closed connection: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
[ E | HeartbeatThread | MRC | 1 | Jan 10 12:40:03 ] an error occurred while initially contacting the Directory Service: java.io.IOException: Request finally failed after 15 tries.
[ E | MRCRequestDispatcher | MRC | 1 | Jan 10 12:40:03 ] STARTUP FAILED!
[ E | MRCRequestDispatcher | MRC | 1 | Jan 10 12:40:03 ] java.io.IOException: cannot initialize service at XtreemFS DIR: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:271)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
[ E | MRCRequestDispatcher | MRC | 1 | Jan 10 12:40:03 ] root cause: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.dir.DIRClient.syncCall(DIRClient.java:406)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:234)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:228)
... org.xtreemfs.common.HeartbeatThread.registerServices(HeartbeatThread.java:339)
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:151)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
I see a lot of messages about 'No trusted certificate found' though I wouldn't really understand that since they are all definitely there in the trustedstore just like the user guide had me do.
***
MRCRequestDispatcher, fatal error: 46: General SSLEngine problem
sun.security.validator.ValidatorException: No trusted certificate found
MRCRequestDispatcher, SEND TLSv1 ALERT: fatal, description = certificate_unknown
MRCRequestDispatcher, WRITE: TLSv1 Alert, length = 2
MRCRequestDispatcher, fatal: engine already closed. Rethrowing javax.net.ssl.SSLHandshakeException: General SSLEngine problem
[ D | RPCNIOSocketClient | MRCRequestDi... | 10 | Jan 10 12:40:03 ] javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1032)
at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:508)
at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1136)
at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1108)
at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:543)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.writeConnection(RPCNIOSocketClient.java:573)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.run(RPCNIOSocketClient.java:265)
Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1528)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:260)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:252)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1165)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:154)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:610)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:550)
at sun.security.ssl.Handshaker$1.run(Handshaker.java:548)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:969)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doTasks(SSLChannelIO.java:710)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.analyseHandshakeStatus(SSLChannelIO.java:654)
at org.xtreemfs.foundation.pbrpc.channels.SSLChannelIO.doHandshake(SSLChannelIO.java:518)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.readConnection(RPCNIOSocketClient.java:417)
at org.xtreemfs.foundation.pbrpc.client.RPCNIOSocketClient.run(RPCNIOSocketClient.java:262)
Caused by: sun.security.validator.ValidatorException: No trusted certificate found
at sun.security.validator.SimpleValidator.buildTrustedChain(SimpleValidator.java:329)
at sun.security.validator.SimpleValidator.engineValidate(SimpleValidator.java:126)
at sun.security.validator.Validator.validate(Validator.java:235)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:147)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:230)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:270)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1144)
... 11 more
MRCRequestDispatcher, called closeInbound()
MRCRequestDispatcher, fatal: engine already closed. Rethrowing javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack?
[ D | RPCNIOSocketClient | MRCRequestDi... | 10 | Jan 10 12:40:03 ] closing connection to localhost/127.0.0.1:32638
[ D | DIRClient | TSync Thr | 12 | Jan 10 12:40:03 ] Request failed due to exception: java.io.IOException: sending RPC failed: server closed connection: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
[ D | DIRClient | MRC | 1 | Jan 10 12:40:03 ] Switching to server localhost/127.0.0.1:32638 since the last attempt failed with the error: sending RPC failed: server closed connection: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
[ E | HeartbeatThread | MRC | 1 | Jan 10 12:40:03 ] an error occurred while initially contacting the Directory Service: java.io.IOException: Request finally failed after 15 tries.
[ E | MRCRequestDispatcher | MRC | 1 | Jan 10 12:40:03 ] STARTUP FAILED!
[ E | MRCRequestDispatcher | MRC | 1 | Jan 10 12:40:03 ] java.io.IOException: cannot initialize service at XtreemFS DIR: java.io.IOException: Request finally failed after 15 tries.
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:271)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
... org.xtreemfs.dir.DIRClient.syncCall(DIRClient.java:406)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:234)
... org.xtreemfs.dir.DIRClient.xtreemfs_service_get_by_uuid(DIRClient.java:228)
... org.xtreemfs.common.HeartbeatThread.registerServices(HeartbeatThread.java:339)
... org.xtreemfs.common.HeartbeatThread.initialize(HeartbeatThread.java:151)
... org.xtreemfs.mrc.MRCRequestDispatcher.startup(MRCRequestDispatcher.java:395)
... org.xtreemfs.mrc.MRC.<init>(MRC.java:39)
... org.xtreemfs.mrc.MRC.main(MRC.java:105)
Björn Kolbeck
Jan 10, 2013, 2:12:06 PM1/10/13
to xtre...@googlegroups.com
Looks like the SSL engine can't find a trusted certificate (the root/CA certificate). Make sure that you have imported the CA certificate into trusted.jks and marked it as trusted as described here http://xtreemfs.org/xtfs-guide-1.3.1/xtfs-guide.html#SECTION00524000000000000000
jeff...@hostdime.com
Jan 10, 2013, 2:33:03 PM1/10/13
to xtre...@googlegroups.com
It is indeed there:
root@win3:/etc/xos/xtreemfs/truststore/certs# ll
total 28K
drwxr-xr-x 2 root root 4.0K Jan 3 11:09 .
drwxr-xr-x 3 root root 4.0K Jan 3 09:45 ..
-rw-r--r-- 1 root root 1.7K Jan 3 11:09 client.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:04 dir.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:05 mrc.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:06 osd.p12
-rw-r--r-- 1 root root 722 Jan 3 11:51 trusted.jks
Is there some way I can test that it is valid? Admittedly these errors don't mean much to me, having done little work with Java in the past. I copied that keytool command right out of the guide when I made it. I can't make another with the same alias of 'ca' but if it would be beneficial to troubleshooting I could delete that jks file and do it over again pasting the commands as I go.
root@win3:/etc/xos/xtreemfs/truststore/certs# ll
total 28K
drwxr-xr-x 2 root root 4.0K Jan 3 11:09 .
drwxr-xr-x 3 root root 4.0K Jan 3 09:45 ..
-rw-r--r-- 1 root root 1.7K Jan 3 11:09 client.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:04 dir.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:05 mrc.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:06 osd.p12
-rw-r--r-- 1 root root 722 Jan 3 11:51 trusted.jks
Is there some way I can test that it is valid? Admittedly these errors don't mean much to me, having done little work with Java in the past. I copied that keytool command right out of the guide when I made it. I can't make another with the same alias of 'ca' but if it would be beneficial to troubleshooting I could delete that jks file and do it over again pasting the commands as I go.
Björn Kolbeck
Jan 10, 2013, 3:00:43 PM1/10/13
to xtre...@googlegroups.com
It is indeed there:
root@win3:/etc/xos/xtreemfs/truststore/certs# ll
total 28K
drwxr-xr-x 2 root root 4.0K Jan 3 11:09 .
drwxr-xr-x 3 root root 4.0K Jan 3 09:45 ..
-rw-r--r-- 1 root root 1.7K Jan 3 11:09 client.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:04 dir.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:05 mrc.p12
-rw-r--r-- 1 root root 1.7K Jan 3 10:06 osd.p12
-rw-r--r-- 1 root root 722 Jan 3 11:51 trusted.jks
Is there some way I can test that it is valid? Admittedly these errors don't mean much to me, having done little work with Java in the past. I copied that keytool command right out of the guide when I made it. I can't make another with the same alias of 'ca' but if it would be beneficial to troubleshooting I could delete that jks file and do it over again pasting the commands as I go.
IIRC, keytool asks you if it is a trusted cert when you add it to the keystore. You have to make sure you answer yes. The keytool should also have an option to list all certs in the keystore.
jeff...@hostdime.com
Jan 10, 2013, 3:07:23 PM1/10/13
to xtre...@googlegroups.com
I remember it did ask that, and the guide is very explicit about answering 'YES' to that question so I am sure I did. Would you like me to delete my current trusted.jks file and recreate it? I could copy over the output from what I type. I am sure I created it like the guide says to.
Björn Kolbeck
Jan 11, 2013, 3:47:09 AM1/11/13
to xtre...@googlegroups.com
For testing you could also use the certificates in tests/certs. I created as described in the user guide.
jeff...@hostdime.com
Jan 11, 2013, 10:48:39 AM1/11/13
to xtre...@googlegroups.com
I apologize but I do not understand. Where would I find the test certs? I saw no mention of them in the user guide. I followed the instructions under the "Sample Setup" heading to the end.
Björn Kolbeck
Jan 11, 2013, 11:00:06 AM1/11/13
to xtre...@googlegroups.com
Sorry for the confusion
These are certificates used by the automatic xtreemfs tests. You can find them in the source repository: http://code.google.com/p/xtreemfs/source/browse/#svn%2Ftrunk%2Ftests%2Fcerts
jeff...@hostdime.com
Jan 11, 2013, 4:53:14 PM1/11/13
to xtre...@googlegroups.com
Would I be correct in assuming that the directives in the configuration file should look like the following?
ssl.enabled = true
ssl.service_creds.pw = passphrase
ssl.service_creds.container = pkcs12
ssl.service_creds = /etc/xos/xtreemfs/truststore/certs/DIR.p12
ssl.trusted_certs = /etc/xos/xtreemfs/truststore/certs/trusted.jks
ssl.trusted_certs.pw = jks_passphrase
ssl.trusted_certs.container = jks
Because at this moment when trying to start up any service, I get the following error and the service fails:
[ E | - | main | 1 | Jan 11 16:48:58 ] DIR could not start up due to an exception. Aborted.
[ E | - | main | 1 | Jan 11 16:48:58 ] java.io.IOException: toDerInputStream rejects tag type 10
... sun.security.util.DerValue.toDerInputStream(DerValue.java:844)
... sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:1205)
... java.security.KeyStore.load(KeyStore.java:1201)
... org.xtreemfs.foundation.SSLOptions.createSSLContext(SSLOptions.java:176)
... org.xtreemfs.foundation.SSLOptions.<init>(SSLOptions.java:159)
... org.xtreemfs.dir.DIRRequestDispatcher.<init>(DIRRequestDispatcher.java:163)
... org.xtreemfs.dir.DIR.main(DIR.java:58)
To verify, my new certs directory looks like the following:
root@win3:/etc/xos/xtreemfs# ll truststore/certs
total 1.6M
drwxr-xr-x 2 root root 4.0K Jan 11 16:32 .
drwxr-xr-x 4 root root 4.0K Jan 11 16:27 ..
-rw-r--r-- 1 root root 89K Jan 11 16:31 Client.key
-rw-r--r-- 1 root root 84K Jan 11 16:31 Client.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 Client.pem
-rw-r--r-- 1 root root 88K Jan 11 16:31 Client.req
-rw-r--r-- 1 root root 89K Jan 11 16:31 DIR.key
-rw-r--r-- 1 root root 92K Jan 11 16:31 DIR.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 DIR.pem
-rw-r--r-- 1 root root 88K Jan 11 16:31 DIR.req
-rw-r--r-- 1 root root 89K Jan 11 16:31 MRC.key
-rw-r--r-- 1 root root 93K Jan 11 16:31 MRC.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 MRC.pem
-rw-r--r-- 1 root root 89K Jan 11 16:31 MRC.req
-rw-r--r-- 1 root root 89K Jan 11 16:31 OSD.key
-rw-r--r-- 1 root root 92K Jan 11 16:31 OSD.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 OSD.pem
-rw-r--r-- 1 root root 88K Jan 11 16:31 OSD.req
-rw-r--r-- 1 root root 90K Jan 11 16:31 trusted.jks
ssl.enabled = true
ssl.service_creds.pw = passphrase
ssl.service_creds.container = pkcs12
ssl.service_creds = /etc/xos/xtreemfs/truststore/certs/DIR.p12
ssl.trusted_certs = /etc/xos/xtreemfs/truststore/certs/trusted.jks
ssl.trusted_certs.pw = jks_passphrase
ssl.trusted_certs.container = jks
Because at this moment when trying to start up any service, I get the following error and the service fails:
[ E | - | main | 1 | Jan 11 16:48:58 ] DIR could not start up due to an exception. Aborted.
[ E | - | main | 1 | Jan 11 16:48:58 ] java.io.IOException: toDerInputStream rejects tag type 10
... sun.security.util.DerValue.toDerInputStream(DerValue.java:844)
... sun.security.pkcs12.PKCS12KeyStore.engineLoad(PKCS12KeyStore.java:1205)
... java.security.KeyStore.load(KeyStore.java:1201)
... org.xtreemfs.foundation.SSLOptions.createSSLContext(SSLOptions.java:176)
... org.xtreemfs.foundation.SSLOptions.<init>(SSLOptions.java:159)
... org.xtreemfs.dir.DIRRequestDispatcher.<init>(DIRRequestDispatcher.java:163)
... org.xtreemfs.dir.DIR.main(DIR.java:58)
To verify, my new certs directory looks like the following:
root@win3:/etc/xos/xtreemfs# ll truststore/certs
total 1.6M
drwxr-xr-x 2 root root 4.0K Jan 11 16:32 .
drwxr-xr-x 4 root root 4.0K Jan 11 16:27 ..
-rw-r--r-- 1 root root 89K Jan 11 16:31 Client.key
-rw-r--r-- 1 root root 84K Jan 11 16:31 Client.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 Client.pem
-rw-r--r-- 1 root root 88K Jan 11 16:31 Client.req
-rw-r--r-- 1 root root 89K Jan 11 16:31 DIR.key
-rw-r--r-- 1 root root 92K Jan 11 16:31 DIR.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 DIR.pem
-rw-r--r-- 1 root root 88K Jan 11 16:31 DIR.req
-rw-r--r-- 1 root root 89K Jan 11 16:31 MRC.key
-rw-r--r-- 1 root root 93K Jan 11 16:31 MRC.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 MRC.pem
-rw-r--r-- 1 root root 89K Jan 11 16:31 MRC.req
-rw-r--r-- 1 root root 89K Jan 11 16:31 OSD.key
-rw-r--r-- 1 root root 92K Jan 11 16:31 OSD.p12
-rw-r--r-- 1 root root 89K Jan 11 16:31 OSD.pem
-rw-r--r-- 1 root root 88K Jan 11 16:31 OSD.req
-rw-r--r-- 1 root root 90K Jan 11 16:31 trusted.jks
Reply all
Reply to author
Forward
0 new messages