CXL.mem and type3 devices

30 views
Skip to first unread message

Anton Gavriliuk

unread,
Sep 29, 2022, 1:25:58 PMSep 29
to pmem
Hi all

Many CXL.mem/type3 devices are coming such as AsteraLabs (Leo E-Series).

https://www.asteralabs.com/cxl-memory-connectivity/unlocking-cloud-server-performance-with-cxl/

I have a question - If the server uses local DRAM DDR5, could CXL DRAM expanders use DDR4 or even DDR3 ??

I asked AsteraLabs, but no response...

Anton

Eduardo Berrocal

unread,
Sep 29, 2022, 1:34:14 PMSep 29
to pmem
Hi Anton,

As I understand it, yes. As long as the expanders support the CXL.mem protocol, you can plug in slower DRAM than the one used by the CPU (and actually, that is kind of the point if you want to do memory tiering).

Cheers,

Anton Gavriliuk

unread,
Sep 29, 2022, 1:59:30 PMSep 29
to Eduardo Berrocal, pmem
Hi Eduardo

> As I understand it, yes. As long as the expanders support the CXL.mem protocol, you can plug in slower DRAM than the one used by the CPU (and actually, that is kind of the point if you want to do memory tiering).

I hope so.  It would be ideal for memory tiering and price-performance.

Here is my current Optane (AEP) based tiered memory setup - 

[root@memverge src]# numactl -H
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27
node 0 size: 386616 MB
node 0 free: 337067 MB
node 1 cpus: 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55
node 1 size: 387063 MB
node 1 free: 337252 MB
node 2 cpus:
node 2 size: 3033088 MB
node 2 free: 3033088 MB
node 3 cpus:
node 3 size: 3033088 MB
node 3 free: 3033088 MB
node distances:
node   0   1   2   3
  0:  10  21  17  28
  1:  21  10  28  17
  2:  17  28  10  28
  3:  28  17  28  10
[root@memverge src]#

And using Redis I compared DDR4 and Optane performance

[root@memverge src]# numactl --cpunodebind=0 --membind=0 ./redis-server /home/anton/redis-7.0.4/redis.conf --daemonize yes
[root@memverge src]# ./redis-cli
127.0.0.1:6379> DEBUG POPULATE 70000000 PHPREDIS_SESSION
OK
(41.51s)
127.0.0.1:6379>

[root@memverge src]# numactl --cpunodebind=0 --membind=2 ./redis-server /home/anton/redis-7.0.4/redis.conf --daemonize yes
[root@memverge src]# ./redis-cli
127.0.0.1:6379> DEBUG POPULATE 70000000 PHPREDIS_SESSION
OK
(74.69s)
127.0.0.1:6379>

Anton

чт, 29 сент. 2022 г. в 20:34, Eduardo Berrocal <edube...@gmail.com>:
--
You received this message because you are subscribed to the Google Groups "pmem" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pmem+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pmem/afb09a8e-0c5f-4430-a352-776e2e3450d7n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages