Have you aggregated 4 x 10Gb NICs?

[bits45]

I was wondering if anyone has created an aggregate with four 10 GBit/s
XFP Ethernet cards on a T5220 before.

I've done two without issue. We must get off our T6320 Sun Blade file
server since those 10 Gb NICs do not like jumbo frames in any way
shape or form (with the load we're giving it). Oracle confirmed in a
3 week open service request that the NIC's buffers in that Intel model
cannot handle my load (during writes - nicstat shows very high
Saturation (SAT) levels - upwards of 11,000 at times).

But, as it turns out all the other Intel 10Gb NICs I have for "real"
servers do well in my T5220s. I wish I had some more X4240 or Sparc
VIIs to do this with, but i don't... YET.

I'm going to load the T5220 with the following, running as an NFS/
Samba file server for 22 TB ZFS Pool off a 6540 RAID array, supporting
very large files 2 GB to 10 GB to 50 Windows workstations.

Server specs will hopefully be:

Solaris 10 Update 9 (CPU_Patches April 2011)
9K Jumbo Frames -> Cisco Nexus 7010 Switch

32 GBytes of RAM (ready for ZFS caching)
2 Mirrored ZFS root disks
2 mirrors LOG (ZIL) SSD disks @ 32 GB - for writes.
4 striped CACHE SSDs, @ 32GB disks - for reads (That should get me
around 168 GByte for ZFS caching with RAM and disks, minus OS
overhead)
Two 8Gbit FC cards into director-class Brocades -> 6540 RAID

FOUR (4) 10Gbit/s aggregated XFP Ethernet cards (I think P/N:
501-7283)

Hope this fixes our performance issues until Oracle can finally send
us our 7410 Unified Storage Trays (slow pokes - 5 months is long
enough, after cashing the check).

I'll let ya'll know if this works. :-)

[Cydrome Leader]

why only 32GB of memory?

[bits45]

On May 12, 3:22 pm, Cydrome Leader <prese...@MUNGEpanix.com> wrote:

Beggers can't be choosers. All I have at the time. :-)

[Cydrome Leader]

bits45 <chris....@gmail.com> wrote:

> On May 12, 3:22?pm, Cydrome Leader <prese...@MUNGEpanix.com> wrote:
>> bits45 <chris.sca...@gmail.com> wrote:
>> > I was wondering if anyone has created an aggregate with four 10 GBit/s
>> > XFP Ethernet cards on a T5220 before.
>>

>> > I've done two without issue. ?We must get off our T6320 Sun Blade file

>> > server since those 10 Gb NICs do not like jumbo frames in any way

>> > shape or form (with the load we're giving it). ?Oracle confirmed in a

>> > 3 week open service request that the NIC's buffers in that Intel model
>> > cannot handle my load (during writes - nicstat shows very high
>> > Saturation (SAT) levels - upwards of 11,000 at times).
>>
>> > But, as it turns out all the other Intel 10Gb NICs I have for "real"

>> > servers do well in my T5220s. ?I wish I had some more X4240 or Sparc

>> > VIIs to do this with, but i don't... YET.
>>
>> > I'm going to load the T5220 with the following, running as an NFS/
>> > Samba file server for 22 TB ZFS Pool off a 6540 RAID array, supporting
>> > very large files 2 GB to 10 GB to 50 Windows workstations.
>>
>> > Server specs will hopefully be:
>>
>> > Solaris 10 Update 9 (CPU_Patches April 2011)
>> > 9K Jumbo Frames -> Cisco Nexus 7010 Switch
>>
>> > 32 GBytes of RAM (ready for ZFS caching)
>> > 2 Mirrored ZFS root disks
>> > 2 mirrors LOG (ZIL) SSD disks @ 32 GB - for writes.
>> > 4 striped CACHE SSDs, @ 32GB disks - for reads (That should get me
>> > around 168 GByte for ZFS caching with RAM and disks, minus OS
>> > overhead)
>> > Two 8Gbit FC cards into director-class Brocades -> 6540 RAID
>>
>> > FOUR (4) 10Gbit/s aggregated XFP Ethernet cards (I think P/N:
>> > 501-7283)
>>
>> > Hope this fixes our performance issues until Oracle can finally send
>> > us our 7410 Unified Storage Trays (slow pokes - 5 months is long
>> > enough, after cashing the check).
>>

>> > I'll let ya'll know if this works. ?:-)

>>
>> why only 32GB of memory?
>
> Beggers can't be choosers. All I have at the time. :-)

I need to have a word with whoever bought you all that other stuff, then
got cheap on the memory. the price of the stuff you listed is way up
there.