Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

HP/UX to Linux - raw space setup question

152 views
Skip to first unread message

Kennedy, Randy

unread,
Nov 9, 2010, 5:29:49 PM11/9/10
to inform...@iiug.org
We are moving from 9.4 on HP/UX 11.11 (physical/hard drives) to 11.1 on
RHEL 5 (VMWare/SAN) and have a couple of questions about the raw storage
setup.

In HP/UX, the Unix system administrator (no longer here) put the hard
drives in a volume group. From that volume group, I would use SAM to
create raw logical volumes that created both character and block
devices. I then used symbolic links to the character devices and used
those as my chunks for the dbspaces.

For RHEL, the Linux system administrator (not familiar with raw devices)
has been given a chunk of SAN space and we are not sure how to get it
from there to the character devices I need to setup up my symbolic
links. The SAN space can be seen in /dev as a block device already, but
my reading leads me to believe I need to use raw to do something next.
Also, everything for RHEL seems to point to using block devices and not
the character devices we currently use, so not sure what I should do
next.

Anyone with RHEL using raw space that could provide a quick step by step
process to follow would be appreciated. We don't mind having to
research how to do those steps or use the necessary tools, just not sure
what the steps are to create and size the raw devices.

HP/UX setup (subset of all devices/links for illustration):
Devices created by using SAM to create logical volumes within
the volume group:
brw-rw-rw- 1 informix informix 64, 65559 Jun 30 2006 1gbdbs1
brw-rw-rw- 1 informix informix 64, 65560 Jun 30 2006 1gbdbs2
brw-rw-rw- 1 informix informix 64, 65562 Oct 30 2007 1gbdbs3
brw-rw-rw- 1 informix informix 64, 65563 Apr 16 2008 2gbdbs
crw-rw-rw- 1 informix informix 64, 65559 Jun 30 2006 r1gbdbs1
crw-rw-rw- 1 informix informix 64, 65560 Jun 30 2006 r1gbdbs2
crw-rw-rw- 1 informix informix 64, 65562 Oct 30 2007 r1gbdbs3
crw-rw-rw- 1 informix informix 64, 65563 Apr 16 2008 r2gbdbs

Symbolic links created to use as the chunks for the dbspaces:
lrwxrwxrwx 1 informix informix 18 Jun 30 2006 1gbdbs1 ->
/dev/vg01/r1gbdbs1
lrwxrwxrwx 1 informix informix 18 Jun 30 2006 1gbdbs2 ->
/dev/vg01/r1gbdbs2
lrwxrwxrwx 1 informix informix 18 Oct 30 2007 1gbdbs3 ->
/dev/vg01/r1gbdbs3
lrwxrwxrwx 1 informix informix 17 Apr 16 2008 2gbdbs ->
/dev/vg01/r2gbdbs


Linux info:
/dev folder
brw-r----- 1 root disk 8, 0 Oct 26 07:59 sda (big SAN chunk)
brw-r----- 1 root disk 8, 1 Oct 26 07:59 sda1 (partitioned sda
space for file system)
brw-r----- 1 root disk 8, 2 Oct 26 07:59 sda2 (partitioned sda
space for file system)
brw-r----- 1 root disk 8, 3 Oct 26 07:59 sda3 (partitioned sda
space for swap file)

parted info
Model: VMware Virtual disk (scsi) (from the SAN)
Disk /dev/sda: 69.8GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number Start End Size Type File system Flags
1 32.3kB 9434MB 9434MB primary ext3 boot
2 9434MB 14.7GB 5240MB primary ext3
3 14.7GB 16.8GB 2097MB primary linux-swap


TIA,
Randy

Doug Lawry

unread,
Nov 9, 2010, 7:11:48 PM11/9/10
to
Hi Randy.

IDS 11 supports IO_DIRECT, meaning it will use a block device as
efficiently as a character device, which is fortunate as the latter is
deprecated. Make sure KAIO is being used. There is a pseudo character
device in Linux which you can bind to the block device, but our
findings are that there is no benefit and it's actually slower on some
hardware. If you have a large device presented by the SAN, either
break it into logical volumes or create multiple chunks at offsets
within it. Don't use huge chunks or it will slow down checkpoints.
Usual advice regarding functional separation of dbspaces apply.
Continue to use symbolic links to the devices.

Regards,
Doug Lawry

Fernando Nunes

unread,
Nov 10, 2010, 6:45:57 AM11/10/10
to inform...@iiug.org
On Tue, Nov 9, 2010 at 10:29 PM, Kennedy, Randy <RKen...@scottsdaleaz.gov> wrote:
We are moving from 9.4 on HP/UX 11.11 (physical/hard drives) to 11.1 on
RHEL 5 (VMWare/SAN) and have a couple of questions about the raw storage
setup.

Please verify why you cannot go to 11.5 or even 11.7.
11.1 has support but it won't receive any new features. If you're having a lot of work to move, you might as well move to a more recent and feature rich version.
I suppose you may be limited by some third party certification, but it would be a good idea to make some pressure...

As for the raw devices, as somebody already mentioned, you could use standard filesystem (no journaling) files with DIRECT_IO.
It isn't as efficient as raw devices....
Another option would be to use block devices (devices created with lvm (Linux Volume Manager?). I'm almost sure I found a site where this was explained, but I couldn't find it. The closest I get was this:

http://www.informix-zone.com/node/38

But this article still talks about the "raw" command. Currently it's deprecated and it is not needed. You can use the block devices directly AFAIK.

Regards.



--
Fernando Nunes
Portugal

http://informix-technology.blogspot.com
My email works... but I don't check it frequently...

Art Kagel

unread,
Nov 10, 2010, 7:20:30 AM11/10/10
to Fernando Nunes, inform...@iiug.org
The developer of the Linux RAW module still maintains it and it is a viable option.  However, following Linus Torvald's lead, Red Hat has deprecated RAW (as was already mentioned).  Linus never wanted RAW devices included in Linux and has always opposed it on philosophical bases.  This is from an email Linus sent on the subject in 1996 (before the RAW module was released): "The main reason there are no raw devices is that I personally think that raw devices are a stupid idea ...there are old-fashioned databases that think they can do a better job of it than the kernel does. They are usually wrong".  Linus just doesn't get it and now he is on a campaign to eradicate RAW from all Linux releases.  So far, AFAIK, he has only been successful in convincing RH to drop them, but you can get the module separately and include it yourself if you want.

On using other options:
  • Block devices go through the cache unless DIRECT_IO is enabled.  I do not know if DIRECT_IO works with block devices on Linux.  They will not work together on AIX for example.
  • Filesystem files -
    • With DIRECT_IO enabled and KAIO configured correctly (so you have to tune the Linux KAIO resources) filesystems are about 5-8% slower than RAW on Linux.  Without DIRECT_IO filessystem chunks are about 30-35% slower than RAW.
    • You should NEVER use ANY journaled filesystems for database chunks.  That means no EXT3 or EXT4 filesystems on Linux, no ZFS on Linux or Solaris, only EXT2 filesystems should be used on Linux for database chunks.  AIX's JFS2 (and the equivalent open source JFS filesystem) is less problematic since it only journals meta-data and not data pages.  Fully journaling filesystems do not overwrite data pages but instead write a new page to replace it logically in the file and unlink the original version releasing it to freespace.  This results in files that are massively fragmented internally and, since databases rewrite the same pages many times a day, results in a serious increase in IO and disk bandwidth requirements.  Also, journaled filesystem meta-data writes are NOT affected by O_SYNC or O_DIRECT, so they are still cached even though the database is forcing a data flush - yet another chance for file corruption. - see my Journaled Filesystem Rant in the July entry on my blog (www.informix-myview.blogspot.com) for more details.
    • BTW, Linus says that EXT4 was "designed by idiots" and it is not safe (it writes out the meta-data indicating the new location of a modified page BEFORE it actually writes out the new version of the data resulting in a hole in the file if the system crashes).  I don't know if the problems have been fixed recently, but caviat emptor.  EXT3 with write-back enabled has similar safety problems (EXT4's write-back mode is the default and cannot be disabled - that's another main problem).

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)
Blog: http://informix-myview.blogspot.com/

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.



_______________________________________________
Informix-list mailing list
Inform...@iiug.org
http://www.iiug.org/mailman/listinfo/informix-list


Fernando Nunes

unread,
Nov 10, 2010, 7:33:04 AM11/10/10
to inform...@iiug.org
On Wed, Nov 10, 2010 at 12:20 PM, Art Kagel <art....@gmail.com> wrote:
The developer of the Linux RAW module still maintains it and it is a viable option.  However, following Linus Torvald's lead, Red Hat has deprecated RAW (as was already mentioned).  Linus never wanted RAW devices included in Linux and has always opposed it on philosophical bases.  This is from an email Linus sent on the subject in 1996 (before the RAW module was released): "The main reason there are no raw devices is that I personally think that raw devices are a stupid idea ...there are old-fashioned databases that think they can do a better job of it than the kernel does. They are usually wrong".  Linus just doesn't get it and now he is on a campaign to eradicate RAW from all Linux releases.  So far, AFAIK, he has only been successful in convincing RH to drop them, but you can get the module separately and include it yourself if you want.

On using other options:
  • Block devices go through the cache unless DIRECT_IO is enabled.  I do not know if DIRECT_IO works with block devices on Linux.  They will not work together on AIX for example.
  • Filesystem files -
    • With DIRECT_IO enabled and KAIO configured correctly (so you have to tune the Linux KAIO resources) filesystems are about 5-8% slower than RAW on Linux.  Without DIRECT_IO filessystem chunks are about 30-35% slower than RAW.
    • You should NEVER use ANY journaled filesystems for database chunks.  That means no EXT3 or EXT4 filesystems on Linux, no ZFS on Linux or Solaris, only EXT2 filesystems should be used on Linux for database chunks.  AIX's JFS2 (and the equivalent open source JFS filesystem) is less problematic since it only journals meta-data and not data pages.  Fully journaling filesystems do not overwrite data pages but instead write a new page to replace it logically in the file and unlink the original version releasing it to freespace.  This results in files that are massively fragmented internally and, since databases rewrite the same pages many times a day, results in a serious increase in IO and disk bandwidth requirements.  Also, journaled filesystem meta-data writes are NOT affected by O_SYNC or O_DIRECT, so they are still cached even though the database is forcing a data flush - yet another chance for file corruption. - see my Journaled Filesystem Rant in the July entry on my blog (www.informix-myview.blogspot.com) for more details.
    • BTW, Linus says that EXT4 was "designed by idiots" and it is not safe (it writes out the meta-data indicating the new location of a modified page BEFORE it actually writes out the new version of the data resulting in a hole in the file if the system crashes).  I don't know if the problems have been fixed recently, but caviat emptor.  EXT3 with write-back enabled has similar safety problems (EXT4's write-back mode is the default and cannot be disabled - that's another main problem).

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)
Blog: http://informix-myview.blogspot.com/

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.



Informix supports KAIO on block devices on Linux since v10:

http://www.ibm.com/developerworks/data/library/techarticle/dm-0503szabo/

So, block devices may be a good compromise between ease of use, supportability and performance.
File system is the best for usability, but still has some performance impact.
"Pure" raw devices aren't properly supported (whether Linus is right or not - and I believe he's not -) and are harder to setup.

Richard Kofler

unread,
Nov 10, 2010, 2:27:59 PM11/10/10
to
Hi all,

just let me add one additional hint:
We use a few smallish instances on SuSeE and those use files sitting on separate
filesystems (ext2) as chunks, and DIRECT_IO. When we mount the ext2 filesystems
using -noatime mount option, then we do 33% less I/O seen from the OS perspective.
I was not able to find out why the difference is a third of the I/O, as these
instances tend to be read bound like 12:1 but it is reproduceable on SuSE 11.1,
11.2 and 11.3 in combination with IDS 11.50.FC7

dic_k


Art Kagel schrieb:


> The developer of the Linux RAW module still maintains it and it is a viable
> option. However, following Linus Torvald's lead, Red Hat has deprecated RAW
> (as was already mentioned). Linus never wanted RAW devices included in
> Linux and has always opposed it on philosophical bases. This is from an
> email Linus sent on the subject in 1996 (before the RAW module was
> released): "The main reason there are no raw devices is that I personally
> think that raw devices are a stupid idea ...there are old-fashioned
> databases that think they can do a better job of it than the kernel does.
> They are usually wrong". Linus just doesn't get it and now he is on a
> campaign to eradicate RAW from all Linux releases. So far, AFAIK, he has
> only been successful in convincing RH to drop them, but you can get the
> module separately and include it yourself if you want.
>
> On using other options:
>

> - Block devices go through the cache unless DIRECT_IO is enabled. I do


> not know if DIRECT_IO works with block devices on Linux. They will not work
> together on AIX for example.

> - Filesystem files -


> - With DIRECT_IO enabled and KAIO configured correctly (so you have to
> tune the Linux KAIO resources) filesystems are about 5-8% slower
> than RAW on
> Linux. Without DIRECT_IO filessystem chunks are about 30-35% slower than
> RAW.

> - You should NEVER use ANY journaled filesystems for database chunks.


> That means no EXT3 or EXT4 filesystems on Linux, no ZFS on Linux
> or Solaris,
> only EXT2 filesystems should be used on Linux for database chunks. AIX's
> JFS2 (and the equivalent open source JFS filesystem) is less problematic
> since it only journals meta-data and not data pages. Fully journaling
> filesystems do not overwrite data pages but instead write a new page to
> replace it logically in the file and unlink the original version
> releasing
> it to freespace. This results in files that are massively fragmented
> internally and, since databases rewrite the same pages many times a day,
> results in a serious increase in IO and disk bandwidth
> requirements. Also,
> journaled filesystem meta-data writes are NOT affected by O_SYNC or
> O_DIRECT, so they are still cached even though the database is forcing a
> data flush - yet another chance for file corruption. - see my Journaled
> Filesystem Rant in the July entry on my blog (
> www.informix-myview.blogspot.com) for more details.

> - BTW, Linus says that EXT4 was "designed by idiots" and it is not


--
Richard Kofler
SOLID STATE EDV
Dienstleistungen GmbH
Vienna/Austria/Europe

Art Kagel

unread,
Nov 10, 2010, 2:47:45 PM11/10/10
to Richard Kofler, inform...@iiug.org
YES!   I just did a project on AIX and setting noaccess in the filesystem mount was a very big performance improvement.


Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)
Blog: http://informix-myview.blogspot.com/

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.



Superboer

unread,
Nov 11, 2010, 2:45:36 AM11/11/10
to
Yet another couple of really good reasons why to gofor char mode raw.

Superboer


On 10 nov, 20:47, Art Kagel <art.ka...@gmail.com> wrote:
> YES!   I just did a project on AIX and setting noaccess in the filesystem
> mount was a very big performance improvement.
>
> Art
>
> Art S. Kagel
> Advanced DataTools (www.advancedatatools.com)
> IIUG Board of Directors (a...@iiug.org)
> Blog:http://informix-myview.blogspot.com/
>
> Disclaimer: Please keep in mind that my own opinions are my own opinions and
> do not reflect on my employer, Advanced DataTools, the IIUG, nor any other
> organization with which I am associated either explicitly, implicitly, or by
> inference.  Neither do those opinions reflect those of other individuals
> affiliated with any entity with which I am affiliated nor those of the
> entities themselves.
>

> > > On Wed, Nov 10, 2010 at 6:45 AM, Fernando Nunes <domusonl...@gmail.com


> > >wrote:
>
> > >> On Tue, Nov 9, 2010 at 10:29 PM, Kennedy, Randy <

> > RKenn...@scottsdaleaz.gov

> > >> Informix-l...@iiug.org


> > >>http://www.iiug.org/mailman/listinfo/informix-list
>
> > --
> > Richard Kofler
> > SOLID STATE EDV
> > Dienstleistungen GmbH
> > Vienna/Austria/Europe
> > _______________________________________________
> > Informix-list mailing list

> > Informix-l...@iiug.org
> >http://www.iiug.org/mailman/listinfo/informix-list

0 new messages