SAS HBA LSI 9200-8e supported under 8.2?
Denny Schierz
I'm new to BSD and I want to know, if the SAS HBA LSI 9200-8e is
supported under FreeBSD 8.2.
We installed FreeBSD on SATA disks, but I can't see any devices through
the SAS HBA, so maybe I have to load the driver first.
cu denny
_______________________________________________
freebsd...@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stabl...@freebsd.org"
Rumen Telbizov
LSI 9200-8e is based on the SAS2008 chip and the driver for that chip (mps)
has recently been merged
from current into stable but before 8.2-RELEASE. Therefore if you're using
8.2-RELEASE iso image to
install FreeBSD on a SAS2008 system - you won't see any disks whatsoever.
Unfortunately I can't find an 8.2-STABLE snapshot on the
ftp.freebsd.orgeither which would have
contained the mps driver. I hope someone who's capable of doing this can
build one.
Now in the meantime what I suggest is that you do the following (that's what
I do):
1. Install 8.2-RELEASE on a system that uses another, supported, controller.
2. Upgrade it to latest STABLE and recompile world and kernel. Make sure you
have device mps in the kernel configuration.
3. Download http://mfsbsd.vx.sk/iso/mfsbsd-8.2-amd64.iso which is a small
ram live system.
Still it doesn't have mps. I wrote an email to Martin Matuska requesting a
build against stable but he didn't reply.
4. Replace the kernel in the mfsbsd with the kernel that you've built. Copy
zfs.ko and opensolaris.ko too
5. Boot mfsbsd from PXE (easiest) and install the system as per
http://wiki.freebsd.org/RootOnZFS#head-378d1d7d308d71637a3466a4f98f845aaeaa116a
if you're planning on using ZFS root.
Improvisation in the above process is strongly suggested ;)
I hope this helps a bit.
Cheers,
Rumen Telbizov
--
Rumen Telbizov
http://telbizov.com
Denny Schierz
Am Sonntag, den 20.03.2011, 11:09 -0700 schrieb Rumen Telbizov:
> Denny,
>
> LSI 9200-8e is based on the SAS2008 chip and the driver for that chip (mps)
> has recently been merged
> from current into stable but before 8.2-RELEASE. Therefore if you're using
> 8.2-RELEASE iso image to
> install FreeBSD on a SAS2008 system - you won't see any disks whatsoever.
> Unfortunately I can't find an 8.2-STABLE snapshot on the
> ftp.freebsd.orgeither which would have
> contained the mps driver. I hope someone who's capable of doing this can
> build one.
thanks a lot for this way, but we don't need to install BSD on the SAS
disks :-) For the system itself we have to SATA disk (Raid1 gmirror or
raidz, if it isn't to complicated) and the SAS we need only for ISCSI
with ZFS.
So maybe we need only to recompile the kernel or modules to get it
running.
cu denny
Dmitry Morozovsky
DS> > LSI 9200-8e is based on the SAS2008 chip and the driver for that chip (mps)
DS> > has recently been merged
DS> > from current into stable but before 8.2-RELEASE. Therefore if you're using
DS> > 8.2-RELEASE iso image to
DS> > install FreeBSD on a SAS2008 system - you won't see any disks whatsoever.
DS> > Unfortunately I can't find an 8.2-STABLE snapshot on the
DS> > ftp.freebsd.orgeither which would have
DS> > contained the mps driver. I hope someone who's capable of doing this can
DS> > build one.
DS>
DS> thanks a lot for this way, but we don't need to install BSD on the SAS
DS> disks :-) For the system itself we have to SATA disk (Raid1 gmirror or
DS> raidz, if it isn't to complicated) and the SAS we need only for ISCSI
DS> with ZFS.
DS>
DS> So maybe we need only to recompile the kernel or modules to get it
DS> running.
FWIW, (and you can find info about it in mailing list archives) I just built
and about to out it into day-to-day use 8.2-stable system, working as
big-just-in-case archive with sources from Mar1 based on SuperMicro case/mobo
with LSI SAS2008 + LSI expander + 24 SATA RE4 disk bays.
For now (and array is only half filled with disks) I'm very glad to have
one-thread 500 MBps+ read stream from raidz2 array ;)
Booting from mps (while I had to set up gmirror, as only 12 disks are exported
to BIOS, hence very large raidz's are not allowed to boot from) was not a
problem either.
--
Sincerely,
D.Marck [DM5020, MCK-RIPE, DM3-RIPN]
[ FreeBSD committer: ma...@FreeBSD.org ]
------------------------------------------------------------------------
*** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- ma...@rinet.ru ***
------------------------------------------------------------------------
Rumen Telbizov
> Booting from mps (while I had to set up gmirror, as only 12 disks are
> exported
> to BIOS, hence very large raidz's are not allowed to boot from) was not a
> problem either.
What I did into my 48 disk machine was - create a separate 'zroot' pool
comprising of 20G gpt partitions carved of off 8 of those 48 disks
in mirror (yeah paranoia!!!) and boot of them. It works nice since I don't
waste too much space since it's a partition and gives me a lot of
redundancy.
What I have is simply the first vdev consisting of partitions that are 20G
smaller
compared to the rest of the vdevs. All those were organized in raidz2 = 6 x
8disks.
Anyway ... something to consider in the cases when you don't have or you
don't
want to have a dedicated bunch of disks for the operating system.
Cheers
--
Rumen Telbizov
http://telbizov.com
Denny Schierz
Am Dienstag, den 22.03.2011, 02:10 +0300 schrieb Dmitry Morozovsky:
> and about to out it into day-to-day use 8.2-stable system, working as
> big-just-in-case archive with sources from Mar1 based on SuperMicro
> case/mobo
> with LSI SAS2008 + LSI expander + 24 SATA RE4 disk bays.
exactly the same to us: LSI SAS Expander switch, with 4 x LSI JBOD630J
with 12 x 2TB SAS disks.
The internal SATA disks are only for BSD, nothing more :-)
So i try to get it working now.
cu denny
Denny Schierz
I've got FreeBSD running:
FreeBSD foobar.domain 8.2-STABLE FreeBSD 8.2-STABLE
with our LSI 9200 SAS :-)
thanks to Rumen, Jeremy and you :-)
zpool with two raidz:
root@iscsihead-m:/[-bash]# zpool create bigPool raidz da0 da1 da2 da3
da4 raidz da6 da7 da8 da9 da10 spare da5 da11
root@iscsihead-m:/[-bash]# zpool status
pool: bigPool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
bigPool ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
da8 ONLINE 0 0 0
da9 ONLINE 0 0 0
da10 ONLINE 0 0 0
spares
da5 AVAIL
da11 AVAIL
errors: No known data errors
root@iscsihead-m:/[-bash]# zfs create bigPool/foobar
root@iscsihead-m:/[-bash]# cd /bigPool/foobar/
root@iscsihead-m:/bigPool/foobar[-bash]# dd if=/dev/zero of=bla bs=1m
count=2048
2048+0 records in
2048+0 records out
2147483648 bytes transferred in 1.923756 secs (1116297239 bytes/sec)
round about 800 - 1064 MB/sec
Raidz2:
root@iscsihead-m:/[-bash]# zpool create bigPool raidz2 da0 da1 da2 da3
da4 da5 raidz2 da6 da7 da8 da9 da10 da11
root@iscsihead-m:/[-bash]# zpool status
pool: bigPool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
bigPool ONLINE 0 0 0
raidz2 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1 ONLINE 0 0 0
da2 ONLINE 0 0 0
da3 ONLINE 0 0 0
da4 ONLINE 0 0 0
da5 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
da8 ONLINE 0 0 0
da9 ONLINE 0 0 0
da10 ONLINE 0 0 0
da11 ONLINE 0 0 0
errors: No known data errors
root@iscsihead-m:/[-bash]# zfs create bigPool/foobar
root@iscsihead-m:/[-bash]# cd /bigPool/foobar/
root@iscsihead-m:/bigPool/foobar[-bash]# dd if=/dev/zero of=bla bs=1m
count=2048
2048+0 records in
2048+0 records out
2147483648 bytes transferred in 2.716165 secs (790630759 bytes/sec)
root@iscsihead-m:/bigPool/foobar[-bash]# dd if=/dev/zero of=bla bs=1m
count=4096
4096+0 records in
4096+0 records out
4294967296 bytes transferred in 5.329954 secs (805816968 bytes/sec)
Round about 750 - 770MB/sec :-)
So, how complicated is load balancing with two Gb Network cards? :-) Ok,
that should be a new thread;-)
cu denny
Freddie Cash
> So, how complicated is load balancing with two Gb Network cards? :-) Ok,
> that should be a new thread;-)
Simple as pie. Read through lagg(4) to see how it's done from the
command-line using ifconfig(8).
Then you put the settings into /etc/rc.conf like so:
cloned_interfaces="lagg0"
ifconfig_em0="up"
ifconfig_em1="up"
ifconfig_lagg0="laggproto round-robin laggport em0 laggport em1 inet
192.168.0.1/24"
Change laggproto to suit your needs.
--
Freddie Cash
fjw...@gmail.com
Rumen Telbizov
I am happy to hear you made it work.
On Fri, Mar 25, 2011 at 7:38 AM, Freddie Cash <fjw...@gmail.com> wrote:
> On Fri, Mar 25, 2011 at 5:28 AM, Denny Schierz <linu...@4lin.net> wrote:
> > So, how complicated is load balancing with two Gb Network cards? :-) Ok,
> > that should be a new thread;-)
>
> Simple as pie. Read through lagg(4) to see how it's done from the
> command-line using ifconfig(8).
>
> ifconfig_lagg0="laggproto round-robin laggport em0 laggport em1 inet
> 192.168.0.1/24"
>
That's actually an interesting problem (although it really belongs to a
different thread).
Freddie, have you tried this with an HP Procurve (say 2910al) switch.
I did and a dual gigabit connection between two machines made the switch
cpu utilization to jump from around 1% to 30%. I guess it might be HP
specific problem due to the enormous mac address bounce between the two
ports. I am curious to know if anybody else has experienced similar
problems?
It's a really important problem to solve since having zpool's that can
achieve
a gigabyte a second transfers makes little sense when you are limited by a
gigabit (120MB/s) network connection to the rest of the world.
Alternative to round-robin is only maybe LACP, but it doesn't really cut it
when
it comes to transfers between two nodes only due to the hashing that LACP
uses. Or maybe 10GbE anyone?
Cheers,
--
Rumen Telbizov
http://telbizov.com
Freddie Cash
> On Fri, Mar 25, 2011 at 7:38 AM, Freddie Cash <fjw...@gmail.com> wrote:
>> On Fri, Mar 25, 2011 at 5:28 AM, Denny Schierz <linu...@4lin.net> wrote:
>> > So, how complicated is load balancing with two Gb Network cards? :-) Ok,
>> > that should be a new thread;-)
>>
>> Simple as pie. Read through lagg(4) to see how it's done from the
>> command-line using ifconfig(8).
>>
>> ifconfig_lagg0="laggproto round-robin laggport em0 laggport em1 inet
>> 192.168.0.1/24"
>
> That's actually an interesting problem (although it really belongs to a
> different thread).
> Freddie, have you tried this with an HP Procurve (say 2910al) switch.
> I did and a dual gigabit connection between two machines made the switch
> cpu utilization to jump from around 1% to 30%. I guess it might be HP
> specific problem due to the enormous mac address bounce between the two
> ports. I am curious to know if anybody else has experienced similar
> problems?
Yes, using LACP trunking on the switch and lagg(4). I'll have to
double-check the exact switch models but I believe one is a 2910 and
the other is a 28-something.
I've also done it using load-balance and fail-over using unmanaged
switch ports. Never used round-robin, though.
--
Freddie Cash
fjw...@gmail.com