rpmem_create: out-of-band connection failed: Connection reset by peer

27 views
Skip to first unread message

Wentao Huang

unread,
Nov 11, 2021, 4:15:34 AM11/11/21
to pmem
I got one same errors from three programs ( "basic", "hello", and "manage"),
rpmem_create: out-of-band connection failed: Connection reset by peer

I also try to follow the single machine setup instruction of [Persistent Memory Replication Over Traditional RDMA Part 2: Remote Persistent Memory 101

I guess that something must be wrong with my configuration?

Can anybody help me to fix the problem? 

Thanks.
Wentao

Dorau, Lukasz

unread,
Nov 15, 2021, 7:40:38 AM11/15/21
to Wentao Huang, pmem

Hi Wentao,

 

The librpmem library is not being developed now, because we started developing the new, alternative solution for accessing remote persistent memory - the librpma library:
https://pmem.io/rpma/
https://github.com/pmem/rpma

 

I encourage you to switch to librpma if you can. The examples of librpma are available here:
https://github.com/pmem/rpma/tree/master/examples

and I will be able to help you with them much faster than with librpmem.

 

If you have to use librpmem and you cannot switch to librpma, please let me know, then I will try to help you with this issue.

 

Lukasz

 

--
You received this message because you are subscribed to the Google Groups "pmem" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pmem+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pmem/e14cd1d3-777a-4b0c-83f6-00c556cbd095n%40googlegroups.com.

Wentao Huang

unread,
Nov 15, 2021, 8:17:58 AM11/15/21
to pmem
Hi Lukasz,

Thank you for your reply.
I am aware that librpmem is no longer under maintenance. But the example I followed is actually quite simple (it's a single machine test based on a virtual loopback network and has nothing to do with NVM). I think this example should work, regardless of librpmem maintenance. I guess something must be wrong with my configuration. I ran this example on Ubuntu 20.04 with the kernel 5.4.0-77. Could you please help me identify some potential bugs? Thanks.

Best,
Wentao

Dorau, Lukasz

unread,
Nov 15, 2021, 10:46:21 AM11/15/21
to Wentao Huang, pmem

Yes, that’s right. This example should work and most probably something is wrong with your configuration.

I will check these examples.

 

Łukasz

 

Dorau, Lukasz

unread,
Nov 17, 2021, 4:23:26 AM11/17/21
to Dorau, Lukasz, Wentao Huang, pmem

Hi,

 

First, try to run the examples with RPMEM_LOG_LEVEL set to 10, for example:

 

$ RPMEM_LOG_LEVEL=10 RPMEM_ENABLE_SOCKETS=1 ./hello localhost pool.set

 

then you will see very verbose output.

Here you have a part of such output I got on my machine. I suspect you can see something similar when you try it yourself:

 

<librpmem>: <4> [rpmem.c:208 rpmem_common_init] establishing out-of-band connection
<librpmem>: <4> [rpmem_cmd.c:119 rpmem_cmd_log] executing command 'ssh -T -4 -oBatchMode=yes localhost rpmemd'
<librpmem>: <10> [rpmem_common.c:89 rpmem_xread] recv/read returned 0
<librpmem>: <1> [rpmem_ssh.c:277 rpmem_ssh_open] Host key verification failed.
<librpmem>: <1> [rpmem.c:212 rpmem_common_init] out-of-band connection failed: Connection reset by peer


rpmem_create: out-of-band connection failed: Connection reset by peer

 

As you see, I got the same error message and the root cause is following: "Host key verification failed".

In order to fix it run the following command:

 

$ ssh localhost date
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:/E1PjclFW71/4lVvTwXgLIZs6qmXKA+7ROArEgiIoO0.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Wed 17 Nov 2021 09:27:05 AM CET

 

Now this error will not occur again:

 

$ RPMEM_ENABLE_SOCKETS=1 ./hello localhost pool.set

 

Please try it yourself and let me know if it works for you.

 

Regards,

Lukasz

Wentao Huang

unread,
Nov 17, 2021, 8:27:16 AM11/17/21
to pmem
Hi Lukasz,

It works.  I should have thought about checking the debugging log ...
Many thanks for your nice help.

Best,
Wentao
Reply all
Reply to author
Forward
0 new messages