st_blksize

365 views
Skip to first unread message

Zeyd M. Ben-Halim

unread,
Jan 6, 1992, 12:42:31 AM1/6/92
to
What is the purpose of the field st_blksize in the stat structure used for
stat and fstat calls? Is it fixed for a given system? Is it fixed for all
types of files (regular, pipes, sockets, etc).
How do you compensate for it in system which don't have it?
Thanx,
zeyd

David J. MacKenzie

unread,
Jan 6, 1992, 3:07:55 AM1/6/92
to

Here's an old posting that should help.

From g...@auspex.auspex.com Sun Jul 15 23:01:44 1990
From: g...@auspex.auspex.com (Guy Harris)
Newsgroups: comp.unix.wizards
Subject: Re: what are st_blksize and st_blocks exactly?
Date: 15 Jul 90 00:29:53 GMT
Distribution: na
Organization: Auspex Systems, Santa Clara

>The stat(2) documentation for SunOS 4.0.3 says:
>
> long st_blksize; /* preferred blocksize for file system I/O*/
> long st_blocks; /* actual number of blocks allocated */

And the 4.3-tahoe documentation - those two fields originally appeared
in 4.2BSD (well, maybe 4.1c or something if you want to be *really*
picky) - says:

long st_blksize; /* optimal blocksize for file system i/o ops */
long st_blocks; /* actual number of blocks allocated */

which isn't any better.

>It doesn't mention what units st_blocks is measured in, but those
>units don't look like they're the value of st_blksize.
>How are these two fields related, if at all?
>Is st_blocks always measured in 512 byte units, or is there a value
>somewhere that indicates the block size for it?

The answer is "st_blocks is measured in 512-byte units, at least on
systems with a DEV_BSIZE of 512 - this includes all Suns, and VAXes, and
probably most other systems - or running System V Release 4."

The System V Release 4 documentation could either be considered worse
than, or better than, the BSD/S5 documentation in this regard.

Better, because it does say what the units are.

Worse, because it says it in both the comment on the "st_blocks" field
in the list of fields *and* in the description of the field, and the two
descriptions disagree with one another!

The correct description is almost certainly the one in the description
of the field:

st_blocks
The total number of physical blocks of size 512 bytes
actually allocated on disk.

because that's what it is in 4.[23]BSD, SunOS, and probably all the
systems that picked it up from 4.[23]BSD. The description in the
comment in the S5R4 manual page says it's in units of "st_blksize",
which makes no sense - "st_blksize" is the optimal size for I/O, and in
UFS (4.[23]BSD) file systems is typically larger (by factors of 8 or
more) than the allocation quantum for disk blocks in the file system.

As I remember, the AT&T folk originally thought that "st_blocks" was
supposed to be in units of "st_blksize", but were convinced otherwise;
the comments in the code may not have been fixed in time to get the fix
into the man page, but the actual description of the field was fixed in
time.

The Third Edition of the SVID has the same confusion; I hope the SVVS
makes sure that the number of 512-byte blocks is returned (even if the
disk has 1024-byte sectors; just return twice the number of those
sectors).

Still, it's better than the S5R4 Migration Guide claiming that "To allow
binary compatibility with Release 2 programs, the Relase 4.0 'stat'
structure is identical to the Release 3.2 'stat' structure. BSD
programs requiring the two additional fields need to be relinked with a
compatibility library."

That statement contradicts the SVID (which lists the two fields,
"st_blocks" and "st_blksize"), the S5R4 documentation (which lists those
two fields), and the experience of people who did 4.2BSD (4.2BSD changed
the system call number for "stat", and had an option to let the old
system call number return the old "stat" structure, for binary
compatibility with 4.1BSD, which used the V7 structure that
S5-prior-to-R4 used).

--
David J. MacKenzie <d...@eng.umd.edu> <d...@ai.mit.edu>

Lars Henrik Mathiesen

unread,
Jan 7, 1992, 10:52:34 AM1/7/92
to
d...@eng.umd.edu (David J. MacKenzie) writes:
>The answer is "st_blocks is measured in 512-byte units, at least on
>systems with a DEV_BSIZE of 512 - this includes all Suns, and VAXes, and
>probably most other systems - or running System V Release 4."

HP-UX measures st_blocks in 1K-byte units, and it sets DEV_BSIZE to 1K
as well, so perhaps this should be rephrased to say that the unit is
DEV_BSIZE, at least when that number is 512. Does SVR4 guarantee a
unit of 512 bytes, even with DEV_BSIZE different or absent?

>The correct description is almost certainly the one in the description
>of the field:

> st_blocks
> The total number of physical blocks of size 512 bytes
> actually allocated on disk.

>because that's what it is in 4.[23]BSD, SunOS, and probably all the
>systems that picked it up from 4.[23]BSD.

As indicated, HP-UX is a counterexample. The implementation seems to
be derived from BSD 4.2-ish, but the interface is heavily skewed
towards SYSV. The manual pages stat(2) and stat(5) do not admit the
existence of the st_blocks and st_blocksize fields (but <sys/stat.h>
has them). The different units cause some trouble in porting --- even
the HP-supplied /etc/quot in 8.0 gets them wrong!

What's worse, HP-UX does not do any mapping of the corresponding field
in NFS file attribute structures. (Since Sun invented NFS, I assume
that Sun's units should be used over-the-wire.) This is compounded by
its SYSV-inspired propensity to output sizes as numbers of 512 byte
blocks: For the same file, 'ls -s' will disagree by a factor of four
between HP-UX and SunOS, while moving a file between OSes will change
the reported numbers by a factor of two:

units | ls -s runs on
reported | SunOS HP-UX
---------+-------------------
file SunOS | 1024 256
resides |
on HP-UX | 2048 512

People could deal with one of these differences, but both at the same
time is quite confusing. (Before we got HP-UX, we could provide our
users with a homogenous environment with few visible OS differences.
Now we are just trying to get to where the staff understand what's
going on.)

--
Lars Mathiesen, DIKU, U of Copenhagen, Denmark [uunet!]mcsun!diku!thorinn
Warning: This article may contain unmarked humour. tho...@diku.dk

Chip Rosenthal

unread,
Jan 7, 1992, 10:20:26 PM1/7/92
to
In article <1992Jan7.1...@odin.diku.dk>

tho...@diku.dk (Lars Henrik Mathiesen) writes:
>HP-UX measures st_blocks in 1K-byte units

The pseudo-unices seem to have *ahem* novel ways of handling st_blocks.
AIX also does not use the straight 512 number -- it does exactly the
opposite of HP-UX (512 for local filesystems, 1024 for remote filesystems).

I have a `du' reimplementation which will be sent to comp.sources.misc
one of these days. (It's a successor to `enh-du' published many moons
ago.) I solved this problem by asking the installer to select one of the
following definitions iff the system has `st_blocks'.

#define STAT_BLKSIZE(fsp,sbufp) 512 /* Sun, S5R4 */
#define STAT_BLKSIZE(fsp,sbufp) ((fsp)->remote ? 1024 : 512) /* AIX */
#define STAT_BLKSIZE(fsp,sbufp) ((fsp)->remote ? 512 : 1024) /* HPUX */

The `fsp' is a structure with information on the filesystem; the
`remote' flag is TRUE for remote mounts.

The disk usage for an entry would then be:

/*
* "sbufp" is a (struct stat *) for the filesystem entry.
* "fsp" has info on filesystem containing this entry.
*/
bytes_used = sbufp->st_blocks * STAT_BLKSIZE(fsp, sbufp)

--
Chip Rosenthal 512-482-8260 | Percentage of Americans who think
Unicom Systems Development | oatmeal is made of wheat: 48%
<ch...@chinacat.Unicom.COM> | -Harper's Index

Andy Feibus

unread,
Jan 8, 1992, 6:41:41 PM1/8/92
to
tho...@diku.dk (Lars Henrik Mathiesen) writes:
> As indicated, HP-UX is a counterexample. The implementation seems to
> be derived from BSD 4.2-ish, but the interface is heavily skewed
> towards SYSV. The manual pages stat(2) and stat(5) do not admit the
> existence of the st_blocks and st_blocksize fields (but <sys/stat.h>
> has them). The different units cause some trouble in porting --- even
> the HP-supplied /etc/quot in 8.0 gets them wrong!
That's because it's not really part of stat(2), but statfs(2).
Check there. Also, check out the POSIX.1 standard, which doesn't list
these fields for stat.

Onward!
Andy Feibus.
a...@amfent.gwinnett.com

Guy Harris

unread,
Jan 12, 1992, 5:12:27 PM1/12/92
to
>> As indicated, HP-UX is a counterexample. The implementation seems to
>> be derived from BSD 4.2-ish, but the interface is heavily skewed
>> towards SYSV. The manual pages stat(2) and stat(5) do not admit the
>> existence of the st_blocks and st_blocksize fields (but <sys/stat.h>
>> has them). The different units cause some trouble in porting --- even
>> the HP-supplied /etc/quot in 8.0 gets them wrong!
>That's because it's not really part of stat(2), but statfs(2).
>Check there.

To what does "it" refer? The units of "st_blocks"?

HP may have made the mistake of requiring you to do a "statfs()" to find
out the units of "st_blocks", but other OS suppliers, such as Berkeley
and AT&T, didn't make the same mistake.

>Also, check out the POSIX.1 standard, which doesn't list
>these fields for stat.

The POSIX.1 standard doesn't list a *lot* of stuff. Have you foregone
the use of any calls to establish network connections, for example, just
because they're not in POSIX.1?

POSIX.1 doesn't say you *can't* put those fields into the "stat"
structure on your machine - which is rather good, given that SVR4 and
various other systems that *do* have those fields are claiming
compliance to POSIX.1 - it just says that if you want to write an
application portable to *all* POSIX.1-compliant systems, you can't use
any fields not described there.

A number of applications were not intended to be portable to *all*
POSIX.1-compliant systems; some may be intended to be ported only to
those systems with "st_blocks" in the "stat" structure....

Guy Harris

unread,
Jan 12, 1992, 5:06:13 PM1/12/92
to
>Does SVR4 guarantee a unit of 512 bytes, even with DEV_BSIZE different
>or absent?

That was answered in the posting of mine cited by the previous poster,
but I'll answer it again:

The SVID says, in the "stat(BA_OS)" section, that "st_blocks" is "the


total number of physical blocks of size 512 bytes actually allocated on
disk".

However, the comment in the list of SVID-mandated members of the
"stat" structure says it's the "number of st_blksize blocks allocated
for this object".

The explanation is probably that they originally thought it should be
the number of "st_blksize blocks allocated for this object", but were
later made to realize that using "st_blksize" as the units of
"st_blocks" is a Very Bad Idea, and fixed up the SVID and, hopefully,
the code not to do so, but didn't get around to fixing the comment in
<sys/stat.h>, and that comment got stuck into the SVID.

If so, then yes, SVR4 guarantees a unit of 512 bytes, unless the person
who ported SVR4 really badly screwed up.

>As indicated, HP-UX is a counterexample.

Yup. I suspect they won't run HP-UX through the SVR4 version of the
SVVS - as far as I know, they don't claim SVR4-compatibility - so even
if that version of the SVVS *does* happen to make sure the implementor
didn't screw up the units of "st_blocks", it won't help HP-UX users.

Now, maybe OSF did the same thing Berkeley and AT&T did, and said "it's
in units of 512 bytes, period" for OSF/1, in which case HP's OS Will Be
Made Sane if, as, and when they adopt OSF/1, unless OSF lets them
continue to be insane (which I sincerely hope they do *not* do; Berkeley
uses 512, AT&T uses 512, everybody else should either get with the program
or get the world to adopt some portable way to find out what units *are*
used).

Lars Henrik Mathiesen

unread,
Jan 8, 1992, 2:01:13 PM1/8/92
to
ch...@chinacat.unicom.com (Chip Rosenthal) writes:

>In article <1992Jan7.1...@odin.diku.dk>
> tho...@diku.dk (Lars Henrik Mathiesen) writes:
>>HP-UX measures st_blocks in 1K-byte units

>The pseudo-unices seem to have *ahem* novel ways of handling st_blocks.
>AIX also does not use the straight 512 number -- it does exactly the
>opposite of HP-UX (512 for local filesystems, 1024 for remote filesystems).

What is ``remote?'' If you mean ``on a Sun machine via NFS,'' AIX is
being weird (possibly trying to make ls -s give the same result as on
a Sun machine). If it is ``on a HP-UX machine via NFS,'' AIX is doing
exactly the same as SunOS would do.

SunOS and MORE/bsd send the number of 512-byte blocks in response to
an NFS getattr RPC, while HP-UX sends the number of 1k-byte blocks.
There is no (protocol-internal) way to see which units was used, and
all the systems mentioned just assume that numbers they receive are in
the same units as they send themselves. SVR4 is probably the same as
SunOS.

>I have a `du' reimplementation which will be sent to comp.sources.misc
>one of these days. (It's a successor to `enh-du' published many moons
>ago.) I solved this problem by asking the installer to select one of the
>following definitions iff the system has `st_blocks'.

>#define STAT_BLKSIZE(fsp,sbufp) 512 /* Sun, S5R4 */
>#define STAT_BLKSIZE(fsp,sbufp) ((fsp)->remote ? 1024 : 512) /* AIX */
>#define STAT_BLKSIZE(fsp,sbufp) ((fsp)->remote ? 512 : 1024) /* HPUX */

Sorry, that's not the solution in a heterogenous environment (or even
a pure HP-UX one). On the systems we have here, STAT_BLKSIZE should
_only_ depend on the OS under which the file resides, and it could
usefully be defined as

#define STAT_BLKSIZE(fsp,sbufp) ((fsp)->ishpux ? 1024 : 512)

(AIX may very well spoil this). The problem is how to detect HP-UX
systems (or others that use large units). The following should work
for UFS (whether local or across NFS):

* Find a directory. (Can't have holes, because even empty
dirblocks have a non-zero size field.)

* Do a stat and a statfs on it. Round up st_size to a multiple
of f_bsize (the fragment size) and divide by st_blocks.

* If st_size was less than ND_ADDR full blocks (usually 48 or
96 KB) the ratio should be exactly the st_blocks unit.
Larger files are rounded to full blocks, so the ratio may be
up to 7% too small --- round up to a power of two.

(A sample program follows my signature.) Then be paranoid and store
the block size in the fs description instead of a flag. Somebody out
therewill be using 2K over the wire.

Somebody please try this on AIX (as server, client and both) and tell
us what goes wrong. (If AIX just scales incoming and/or outgoing block
counts, it might even work.)

--
Lars Mathiesen, DIKU, U of Copenhagen, Denmark [uunet!]mcsun!diku!thorinn
Warning: This article may contain unmarked humour. tho...@diku.dk

#include <stdio.h>
#include <sys/types.h>
#include <sys/vfs.h>
#include <sys/stat.h>

main(argc, argv)
char **argv;
{
struct statfs sf;
struct stat st;
int round, ratio, unit;

if (statfs(argv[1], &sf) < 0 || stat(argv[1], &st) < 0)
perror(argv[1]), exit(1);
if (st.st_blocks <= 0)
fprintf(stderr, "%s: No block count\n", argv[1]), exit(1);
round = (st.st_size + sf.f_bsize - 1) & ~(sf.f_bsize - 1);
ratio = round/st.st_blocks;
for (unit = 1; unit < ratio; unit <<= 1)
;
printf("%s bsize %d size %d blocks %d\n\tround %d ratio %d unit %d\n",
argv[1], sf.f_bsize, st.st_size, st.st_blocks, round, ratio, unit);
exit(0);
return;
}

Andy Feibus

unread,
Jan 14, 1992, 9:36:07 PM1/14/92
to
> >That's because it's not really part of stat(2), but statfs(2).
> >Check there.
> HP may have made the mistake of requiring you to do a "statfs()" to find
> out the units of "st_blocks", but other OS suppliers, such as Berkeley
> and AT&T, didn't make the same mistake.
The point I was *trying* to make was that if you check out statfs on
HP-UX systems, you'll find what you're looking for. You might even
find it there for most systems (BSD-based systems are the only ones I'm
not sure about).

Andy.
a...@amfent.gwinnett.com

Reply all
Reply to author
Forward
0 new messages