I recently started building a 3.0 Pod as a prototype project for work. It's been very interesting and the more I get into it, the more questions I have. I'm hoping the community here can help me with some of the questions.
First thing I want to share is about the PSU. The PSU listed by BackBlaze and ProtoCase seems to be very hard to source, perhaps due to all the Pod builders out there? We ended up ordering the 850W version of the PSU made by Zippy, however we did not use it. As we started looking into how to wire the entire system up, we realized that we'd be better off with a fully modular PSU. We went for the Seasonic X-850, which was by the way, cheaper than the Zippy and 80Plus Gold rated efficiency. The nice thing about the Seasonic is that the cables are fully modular AND you can buy the 18-pin + 10-pin connectors to build custom harness. This is what we did and basically built power wiring harness that is to exact length and specification so we didn't have any excess wires to deal with. The results were great and a very clean build. It also made me realize I wish there were more "holes" or areas to zip-tie wires in the chassis to keep them neatly tucked away. We were really happy with this decision, although it took some time to build the harness ourselves. I'm happy to share the details if anyone else is attempting the same.
Another thing we are doing is regarding boot drive. The 3.0 Pod now has holes for 1x 3.5" drive or 2x 2.5" (laptop or SSD) drives for the boot drive location. We actually are not using this for boot drive and instead we added 2 CF-SATA adapter that sit in the empty PCI slots. We use 16GB or 32GB Compact Flash cards in these slots for the boot drive. In Linux, we mirror the two CF cards for some level of redundancy. This makes it very very simple matter to pull the CF cards out to swap boot drives. Right now, we have a couple of sets of CF cards with a variety of OSes, CentOS5/6, OpenFiler, etc. and it takes 5 seconds to swap the CF cards and reboot.
Now, that means that the holes for the boot drive isn't being used, so that gave us the idea of putting a pair of SSDs in that location for use as additional high IOPS storage. We added two 256GB Samsung 840PRO SSDs. They are currently connected to the motherboard's SATA controller, which is SATA-II only and not really fast enough for the SSDs.
So, the last part really got us thinking about alternative options for SATA connectivity which leads to some of my questions:
1. Especially for the SSDs, we realize we need some SATA-3 controllers that can really handle the bandwidth. For example SATA-3 controllers on a 1x lane PCI-E 2.0 wouldn't work. This can be done via add-on PCI-E card or onboard SATA-3 on the motherboard. So, this got us thinking, what are some good alternative SATA-3 controllers we can use that would have a faster PCI-E connection (at least 2x lane or 4x lane PCI-E 2.0 or above) or a motherboard with SATA-3 implemented correctly (connected to the PCI-E bus on the motherboard in a way that can handle full SATA-3 specs)?? Such a controller could also open up the pipe to the backplanes. Also, such a controller, if connected to the backplanes, must be able to support the port multipliers too. Anyone have any suggestions on this?
2. We plan to make the storage available via the network. The current X9SCL-F mobo has 2x 1Gbps connections, which even if fully utilized will only get us about 120MB/s on each 1Gbps link. We're exploring the idea of finding a mobo with 10Gbps/copper links and with some network stack tuning and jumbo frames we're hoping can give us 1GB/s speeds. Of course, this points us to #1 again, since we would need the 45x HDD to be able to provide that level of bandwidth or this is all pointless. Anyway, has anyone tried any other motherboards with 10Gbps connections?
3. One thing I didn't mention yet, the backblaze hardware specs for 12V fans that have no speed control. So they run at full speed all the time. We experimented with some PWM fans for the 3 rear fans that blow over the motherboard. This seems to be ok. But it got us thinking about using PWM fans for the 3 front fans... but we are not sure if that would be sufficient cooling since the PWM would be coming from the motherboard (but powered directly from PSU). Any thoughts on this topic?
TIA for any tips/suggestions.
-James