File size and volume size.

[Shiva]

I'd like to embark on a journey to find the size of all the files present in a sub volume. I'd like to build a macro that would do that. It would be pretty simple, as it would involve just getting the file info for those files and using set many to get the EOF, but I would like to convert the answer to our real life values. Like MB, GB etc.

So what's the math for that?

And also when I do a FUP VOLS, I get the following information about all the disks in the system.

Volume, (M), TOTAL, FREE, % FREE, Free Extents (count), Free Extents (biggest)

What does (M) mean in the above? And what does the free extents mean? I have seen that in the cup info <file-name>, detail. But I don't know much about the relation between extents, pages, max extents etc.. I have started reading the Enscribe programmer's guide a few days ago, and I still couldn't find any explanation for that (crossed some 40 pages now, of the 200 pages book). Possibly the FUP manual will have that information?

For some clarity can somebody explain in brief what they mean? Just like cylinders and blocks in mainframes?

I'd also like to calculate the total size and free size of the volumes in our real life units... MB, GB, etc.

As always, thanks for your inputs. :)

[Doug Miller]

Shiva <subrama...@gmail.com> wrote in
news:b78886b5-c19d-454a...@googlegroups.com:

> I'd like to embark on a journey to find the size of all the
> files present in a sub volume. I'd like to build a macro that
> would do that. It would be pretty simple, as it would involve
> just getting the file info for those files and using set many to
> get the EOF, but I would like to convert the answer to our real
> life values. Like MB, GB etc.

EOF won't really help you. That just tells you where the EOF pointer is in the file, but gives
no indication of how much space the file is actually taking up on disk, other than as a lower
limit.

>
> So what's the math for that?

The space in bytes which a file takes up is given by
(primary extent size + secondary extent size * (extents allocated - 1)) * 2048

Divide by 1024 to get KB
Divide by 1024 again to get MB
Divide by 1024 a third time to get GB

>
> And also when I do a FUP VOLS, I get the following information
> about all the disks in the system.
>
> Volume, (M), TOTAL, FREE, % FREE, Free Extents (count), Free
> Extents (biggest)
>
> What does (M) mean in the above?

Whether the volume listed is the mirror volume, or not: Y means it's the mirror drive, blank
means the primary drive.

>And what does the free extents mean?

The "count" is the number of areas of free (unallocated) space on that disk, and "biggest"
means (you've probably guessed this already) the size of the largest chunk of free space
on that disk.

>I have seen that in the cup info <file-name>, detail. But
> I don't know much about the relation between extents, pages, max
> extents etc.. I have started reading the Enscribe programmer's
> guide a few days ago, and I still couldn't find any explanation
> for that (crossed some 40 pages now, of the 200 pages book).
> Possibly the FUP manual will have that information?

Every Guardian file has one primary extent, and 0 to (maxextents - 1) secondary extents,
which are allocated as needed to provide space for the data stored in the file. Primary and
secondary extents need not be the same size, although they frequently are. Extent sizes are
specified in pages; one page is 2048 bytes.

[Tone]

Actually, it indicates if the volume is currently mirrored or not. You
can get the same output with DSAP *,SHORT

>
>> And what does the free extents mean?
>
> The "count" is the number of areas of free (unallocated) space on that disk, and "biggest"
> means (you've probably guessed this already) the size of the largest chunk of free space
> on that disk.

DSAP $VOL,FREESPACE(,TERAFORM) will provide more detail. DSAP is
documented in the Guardian Disk and Tape Utilities Manual.

[Doug Miller]

Tone <tss...@iinet.net.au> wrote in news:VYSdnTQ4cp8D9A_JnZ2dnUU7-
KWd...@westnet.com.au:

> On 12/18/2014 09:33 AM, Doug Miller wrote:
>> Shiva <subrama...@gmail.com> wrote in

>>> What does (M) mean in the above?
>>
>> Whether the volume listed is the mirror volume, or not: Y means it's the mirror drive, blank
>> means the primary drive.
>
> Actually, it indicates if the volume is currently mirrored or not.

Right you are. Thanks for the correction.

[Shiva]

Hi Doug,

Thanks again for your informative answer! So the following is the size in bytes.

(primary extent size + secondary extent size * (extents allocated - 1)) * 2048

Where do I get the extents allocated value from?

And in the below,

> Volume, (M), TOTAL, FREE, % FREE, Free Extents (count), Free
> Extents (biggest)

The value under TOTAL is in bytes or like EOF which needs to be converted to bytes. Sorry for a flurry of questions! Pardon my ignorance.

And just a clarification, so (M) in the above list means if a disk had Y against this column, that would mean that the disk is currently mirrored. And blank means not mirrored. But what's mirrored. This, I believe, stems from HP NS fundamental ideology of having duplicates for everything. As the name suggests, this disk with 'Y' is mirrored somewhere else and those with blanks are not mirrored at all? But that can't be, can it?

Or is it like the ones with Y are mirror images and blanks are primary disks, that's what you said first, but tone said that it indicates if the volume is currently mirrored. Just a bit confused. May be just too much fuzz over nothing, but better to know correctly than to have false information. :)

[Doug Miller]

Shiva <subrama...@gmail.com> wrote in
news:fe649657-c955-4eb4...@googlegroups.com:

> Hi Doug,
>
> Thanks again for your informative answer! So the following is
> the size in bytes.
>
> (primary extent size + secondary extent size * (extents
> allocated - 1)) * 2048
>
> Where do I get the extents allocated value from?

#FILEINFO /EXTENTSALLOCATED/

>
> And in the below,
>
>> Volume, (M), TOTAL, FREE, % FREE, Free Extents (count), Free
>> Extents (biggest)
>
> The value under TOTAL is in bytes or like EOF which needs to be
> converted to bytes. Sorry for a flurry of questions! Pardon my
> ignorance.

The value under TOTAL is megabytes.

>
> And just a clarification, so (M) in the above list means if a
> disk had Y against this column, that would mean that the disk is
> currently mirrored. And blank means not mirrored. But what's
> mirrored.

Mirroring = RAID-1

> This, I believe, stems from HP NS fundamental ideology
> of having duplicates for everything. As the name suggests, this
> disk with 'Y' is mirrored somewhere else and those with blanks
> are not mirrored at all? But that can't be, can it?

Sure it can -- it's up the the sysadmin whether to configure volumes as mirrored or
unmirrored. There's no requirement that a volume be mirrored.

>
> Or is it like the ones with Y are mirror images and blanks are
> primary disks, that's what you said first, but tone said that it
> indicates if the volume is currently mirrored.

He's right, I was wrong. I've been away from this stuff for too long.

[Shiva]

Thanks, Doug. Sorry for any trouble! :) You've been a world of help lately!

[masterpu...@gmail.com]

On Thursday, December 18, 2014 at 9:44:02 PM UTC+1, Shiva wrote:
> Thanks, Doug. Sorry for any trouble! :) You've been a world of help lately!

Hello Shiva/Doug,

I understand that this is rather old post but found it while looking for some info.
Can you help me out to see if there is any such direct FUP command to to list all file over certain %.

[Randall]

You could run DSAP to generate output using the DETAIL option and perhaps SIZE OVER 0, pipe the output to awk or python in OSS, convert the lines to variables and do the math using a script.

gtacl -c 'dsap $*,detail,teraform,size over 0' | awk your-script

your-script has to parse lines like the following... stripping stuff on the right you can ignore. Use a regex to match likes you want:

<ignore>
Volume $AUDIT is logical device 187
<ignore>
Name/ID Filename Type Code Pages Pages
SUPER.SUPER
(255,255)E1222H71.A0CINFO K 963 16 14
E1222H71.A7CINFO 180 160 37

Match the volume line, and save the disk.
Match the file lines, save the name, and prepend the disk. Capture the pages to two awk/python variables and do the math (1.0-used/total)*100.0 making sure it's done in double or float. print the line as needed.

Good luck.

[Keith Dick]

If you aren't familiar with Unix tools, but are familiar with either TACL, ENFORM, or SQL, you can solve the problem with either of them.

DSAP can be told to save the information about each of the files in a disk file (see the WORKFILE option). The DDL description of the records of that file is available on your system in $SYSTEM.SYSTEM.DSAPDDL. I believe a fairly simple ENFORM report can select and list the names of the files that are over a certain percent used.

Although I have not tried it, I believe you could convert the DSAP workfile to a SQL/MP table using the SQLCI CONVERT command. I don't know whether the DDL for the workfile is suitable for being converted to a SQL/MP table. Once you have a SQL/MP table, you can use normal SQLCI reporting command to select and output the information for the tables over a certain percent full.

If you don't want to use ENFORM or SQL to create your report, and if the output of DSAP is not too large to hold in a TACL variable, you could use OUTV to capture the output of DSAP and process it in a manner similar to how Randall describes. You could use an #EXTRACT or #EXTRACTV loop to get each line from the TACL variable, one at a time. I think I would not try to parse the lines using #SETMANY, but rather use the approach of selecting the range of columns from each line that contains the filename, total pages, and unused pages by using #CHARGET or #CHARGETV. You would have to discard lines that are not file detail lines, then a simple calculation to determine whether the file is one you want to report or not would do the trick. To figure out whether a line is a file detail line, check that its length is greater than the last column of the unused pages field and that the total pages and unused pages contain a number. #CHARCOUNT can be used to get the length of the va
riable holding the current line. I think the easiest way to test that a field is a number would be to create a small TACL ROUTINE that uses #ARGUMENT to attempt to parse a number from its argument, and return the number if one is found or blank if not. Invoke that TACL ROUTINE passing the expansion of the variable into which you put the columns containing the total pages, and again passing the expansion of the variable for the unused pages.

If you want to be able to display the file owner in your output, you'd also have to recognize the lines that contain the userid and save it. I think that could be done just by saving the line if its length is greater than 1 and less than 20.

If the output of DSAP is too long to fit into a TACL variable, I think a workaround would be to send the DSAP output to a file, then use #REQUESTER READ to read the contents of the file one line at a time. The loop using #REQUESTER READ would take the place of the loop using #EXTRACT or #EXTRACTV.

[wbreidbach]

The main question is, what information do you need?
If you just need the space the file occupies on the disk, Randall and Keith already provided the necessary information.
But from my experience the information usually needed is "how much of the maximum filesize is really used?".
The only chance to get that information with standard tools is using
FUP INFO <filename>, STAT[,partonly]
FUP will read the whole file or partition and give you the information, how many blocks in the file are completely free and how mauch slack the data and index blocks contain. Depending on the size of the file this can take some time. FUP does not give information about fragmentation.
For deeper analysis I know there has been a tool called Tandem Reload Analyzer (extra cost!), I do not know if it still exists.