Anyone using NVDIMM and/or NTB (Non-Transparent Bridge) in production?

291 views
Skip to first unread message

Daniel Worthington-Bodart

unread,
Feb 16, 2017, 4:32:44 PM2/16/17
to mechanical-sympathy
  • Any experience reports? Latency?
  • Are you using memory mapped files or libpmem? Something else.. Java?
  • Anyone done more that 2 machines connected via NTB?
  • Who is your hardware vendor?
  • Anyone managed to get their hands on 3DXPoint?
Many thanks

Dan

Chet L

unread,
Jan 11, 2018, 6:27:28 PM1/11/18
to mechanical-sympathy
Dan,

@NTB: 

We used it more than 12+ years ago but in a mirrored mode. So only 2 nodes for HA failover. PLX were the only guys who had a (sort of)working embedded switch at the time. Supermicro's bridge-in-a-bay offering might have NTB connectivity (not sure though).

Latency(at a very high level): fabric-latency + Remote-DRAM (Rd/Wr)access cost.

fabric-latency : this will be PCIe - depends what you have gen2/gen3 etc but in nsecs.
Remote-DRAM access cost : nsecs.

This is the lowest latency interconnect that we could build. But the servers had to be co-located physically close to each other because of the cable length limitations.

To fanout (>2), it might be better to go via an external PCIe switch.

Linux's NTB support was merged only 3-4 years ago. You should be able to find sample code too in the linux kernel repo. You can use the sample code to create a channel between two peers and start sending/receiving IO.


Chetan
Reply all
Reply to author
Forward
0 new messages