ESOS SAN memory and SSD

[Willard Griselda]

Hi,

I am rebuilding my IB SRP SAN, it doesn't involve CePH or ZFS by the way. I have 2 questions:

1, How does ESOS take advantage from ECC RAM? Does ESOS take advantage from large size of RAM?

2, Can I use SSDs to speed up read and write performance to HDDs?

Thank you in advance

[Chris Kostecki]

Unless something has changed, ESOS uses LVM. I don't think it would know if you are using ECC but it is recommended because the benefits of ECC are at a hadrware level not a software level. There are reasons why having more ram is better in ESOS but it has been a while so I do not remember.
Unless there is a feature change, LVM has no way of using SSDs as a read/write cache. If you are looking for that feature there are some HBAs that support it, like LSI/Broadcom and the CacheCade feature.

--
You received this message because you are subscribed to the Google Groups "esos-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to esos-users+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/esos-users/52c0c118-0679-4c79-bdd5-762d048b83d8n%40googlegroups.com.

[Chris Kostecki]

Looks like LVM does support SSD cache, this may be not be in the TUI and would have to be a manual setup

[Wilard]

This is what I got from Gemini ai, how many percent are real?

Rebuilding an InfiniBand (IB) SAN with the Enterprise Storage OS (ESOS) is a solid choice for low-latency performance. Since you are moving away from Ceph and ZFS, you are likely looking at a more "traditional" block storage approach using SCST (Generic SCSI Target Subsystem).

Here is how ESOS handles your hardware resources:

1. ESOS and RAM: ECC vs. Capacity

In a storage environment, RAM isn't just for running the OS; it’s the primary landing zone for data.

Does ESOS take advantage of ECC?

Yes, but indirectly. ESOS itself (the Linux kernel and the SCST mid-layer) doesn't have "ECC-specific" features, but it relies entirely on the hardware's ability to ensure data integrity.

Data Integrity: Without ZFS’s checksumming, you are relying on the hardware to ensure bits don't flip. In a SAN environment, a single bit flip in RAM can result in "silent data corruption" being written to your HDDs.

Stability: Storage targets are high-interrupt environments. ECC helps prevent kernel panics that could lead to an ungraceful shutdown of your entire fabric.

Does it benefit from Large RAM?

Absolutely. Since you aren't using ZFS (and its ARC cache), ESOS uses the standard Linux Page Cache.

Read Caching: Any RAM not used by the OS is used to cache frequently accessed blocks. If your active "hot" data set fits in RAM, your IB SRP performance will be limited only by your network throughput, not your disks.

Write Buffering: ESOS can use RAM to buffer incoming writes (if "write-back" is enabled). However, be cautious: without a battery-backed RAID controller or an NVDIMM, a power failure could lose data sitting in that RAM buffer.

2. Using SSDs to Speed Up HDDs

Since you aren't using ZFS (which uses L2ARC/SLOG), you need a Linux-based caching layer. ESOS supports several methods to use SSDs as a "front-end" for your HDDs.

Recommended Technologies in ESOS:

TechnologyDescriptionUse CasebcacheThe most common Linux kernel caching shim. It hides the HDD behind an SSD.Best for general-purpose read/write acceleration.dm-cachePart of the LVM (Logical Volume Manager) stack.Best if you are already using LVM to manage your block devices.OpenCAS(Cache Acceleration Software) High-performance caching originally by Intel.Best for high-throughput environments where you need granular control over cache metadata.

How to implement it:

Read Cache (Write-Through): Data is written to both the SSD and HDD simultaneously. This speeds up reads but doesn't accelerate writes. It is the safest method.

Write Cache (Write-Back): Data is written to the SSD first and flushed to the HDD later. This provides massive write speed boosts, but requires a reliable SSD (ideally PLP/Power Loss Protection) to prevent data loss during a crash.

Pro-Tip: If you are using SRP (SCSI RDMA Protocol), your bottleneck will almost always be the latency of the storage media. Using bcache with a pair of mirrored NVMe or SATA SSDs as the "caching" tier for your HDD RAID group is the most effective way to keep your IB pipes full.

How many initiators (clients) are you planning to connect to this SAN?

[Chris Kostecki]

Yeah, that sounds right to me

To view this discussion visit https://groups.google.com/d/msgid/esos-users/BE793F1E-53B2-4827-A2AF-6F1BA0D08A24%40gmail.com.