[Hardhats] Mumps globals size tools?
30 views
Skip to first unread message
kdt...@gmail.com
Apr 15, 2010, 8:06:57 PM4/15/10
to Hardhats
Hey All,
The hard drive holding my VistA instance suddenly filled up. I don't
think this is due to an excess of valid data. Instead, I think it is
an adherent process that started doing something over and over,
filling up data logs etc etc, and making the mumps.dat file suddenly
grow huge. I ran into this another time with one of our custom
scanning routines trying to work on a stuck source file over and
over, leading to tens of thousands of empty notes, and a big file.
So are there tools that one can use to tell the size of the various
globals? Something like a tree-size program that one might use on a
filesystem to find out where the huge files are hiding?
I need to know which globals have gotten huge, and then ideally be
able to drill down to sub nodes etc.
Any such tools already in existence?
Thanks
Kevin
--
http://groups.google.com/group/Hardhats
To unsubscribe, send email to Hardhats+u...@googlegroups.com
The hard drive holding my VistA instance suddenly filled up. I don't
think this is due to an excess of valid data. Instead, I think it is
an adherent process that started doing something over and over,
filling up data logs etc etc, and making the mumps.dat file suddenly
grow huge. I ran into this another time with one of our custom
scanning routines trying to work on a stuck source file over and
over, leading to tens of thousands of empty notes, and a big file.
So are there tools that one can use to tell the size of the various
globals? Something like a tree-size program that one might use on a
filesystem to find out where the huge files are hiding?
I need to know which globals have gotten huge, and then ideally be
able to drill down to sub nodes etc.
Any such tools already in existence?
Thanks
Kevin
--
http://groups.google.com/group/Hardhats
To unsubscribe, send email to Hardhats+u...@googlegroups.com
K.S. Bhaskar
Apr 15, 2010, 10:37:05 PM4/15/10
to hard...@googlegroups.com
mupip integ -full gives you a detailed global by global report. So, if
you want to do all regions: mupip integ -full -region "*" (note that *
has to be quoted to escape it from shell file expansion).
There are no specific tools at the node level - you could probably do an
extract and look at sizes.
Regards
-- Bhaskar
GT.M - Rock solid. Lightning fast. Secure. No compromises.
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_____________
you want to do all regions: mupip integ -full -region "*" (note that *
has to be quoted to escape it from shell file expansion).
There are no specific tools at the node level - you could probably do an
extract and look at sizes.
Regards
-- Bhaskar
GT.M - Rock solid. Lightning fast. Secure. No compromises.
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_____________
kdt...@gmail.com
Apr 17, 2010, 8:42:17 AM4/17/10
to Hardhats
Chris,
Thank you very much for putting this utility together. I appreciate
it.
It just does the total size of the global, not the sizes of subnodes.
I thought about adding this as a feature, but then realized that all
WP fields are stored with a trailing ",0)" node, so such a utility
would report totals for every line, and it would likely be a bloody
mess on output. So I left it the way you made it. Thanks again.
Bhaskar,
I ran mupip -integ. It is also helpful. Of course, it is never as
easy for me as your simple command shows. I don't export the gtm
variables during startup of a shell. So I have to write a script that
sets up the gtm environment before running gtm. And I had to do the
same thing for the mupip. And then it complained about other
processes using the file, so I had to get all the stray processes
properly shut down. But in the end, it did work. Thanks!
It turned out the problem was not a problem with extra data in
globals, but rather accumulated journal files that filled up the
disk. I am going to post another thread about this.
Thanks.
Kevin
> --http://groups.google.com/group/Hardhats
> _GSIZE.m
> < 1KViewDownload
Thank you very much for putting this utility together. I appreciate
it.
It just does the total size of the global, not the sizes of subnodes.
I thought about adding this as a feature, but then realized that all
WP fields are stored with a trailing ",0)" node, so such a utility
would report totals for every line, and it would likely be a bloody
mess on output. So I left it the way you made it. Thanks again.
Bhaskar,
I ran mupip -integ. It is also helpful. Of course, it is never as
easy for me as your simple command shows. I don't export the gtm
variables during startup of a shell. So I have to write a script that
sets up the gtm environment before running gtm. And I had to do the
same thing for the mupip. And then it complained about other
processes using the file, so I had to get all the stray processes
properly shut down. But in the end, it did work. Thanks!
It turned out the problem was not a problem with extra data in
globals, but rather accumulated journal files that filled up the
disk. I am going to post another thread about this.
Thanks.
Kevin
> _GSIZE.m
> < 1KViewDownload
K.S. Bhaskar
Apr 17, 2010, 10:01:38 AM4/17/10
to hard...@googlegroups.com
GT.M - Rock solid. Lightning fast. Secure. No compromises.
[KSB] <...snip...>
> I ran mupip -integ. It is also helpful. Of course, it is never as
> easy for me as your simple command shows. I don't export the gtm
> variables during startup of a shell. So I have to write a script that
> sets up the gtm environment before running gtm. And I had to do the
> same thing for the mupip. And then it complained about other
> processes using the file, so I had to get all the stray processes
> properly shut down. But in the end, it did work. Thanks!
does not. In releases through V5.3-004A, mupip integ would freeze
updates to the database. In V5.4-000/-000A, mupip integ can operate even
as application processes continue operation.
If you go into your normal VistA/GT.M direct mode, and launch mupip, you
should have the environment already setup, e.g., something like:
GTM>ZSY "mupip integ -full -region ""*"""
Regards
-- Bhaskar
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_____________
r...@rcresearch.us
Apr 17, 2010, 10:47:41 AM4/17/10
to hard...@googlegroups.com
Kevin;
The trick is to recognize the root that one is using and then comparing
the results of the $QUERY and making sure that the new reference still
contains that root reference. As soon as it doesn't then you are done.
The trick is to recognize the root that one is using and then comparing
the results of the $QUERY and making sure that the new reference still
contains that root reference. As soon as it doesn't then you are done.
r...@rcresearch.us
Apr 17, 2010, 12:32:28 PM4/17/10
to hard...@googlegroups.com
Kevin;
This falls into the realm of knowing what one wants on the first go
around. Obviously, this is seldom known on the first iteration. Here
is the next iteration.
GTM>W $$SUB^%GSIZE("^DD(2,.01)")
96;6112
GTM>W $$SUB^%GSIZE("^DD(2,.03)")
81;5128
GTM>W $$SUB^%GSIZE("^DD(2)")
8909;517863
GTM>
where the first piece is the node count, and the second is the number of
characters used.
This falls into the realm of knowing what one wants on the first go
around. Obviously, this is seldom known on the first iteration. Here
is the next iteration.
GTM>W $$SUB^%GSIZE("^DD(2,.01)")
96;6112
GTM>W $$SUB^%GSIZE("^DD(2,.03)")
81;5128
GTM>W $$SUB^%GSIZE("^DD(2)")
8909;517863
GTM>
where the first piece is the node count, and the second is the number of
characters used.
kdt...@gmail.com
Apr 17, 2010, 3:17:17 PM4/17/10
to Hardhats
Excellent. I'll take a look at this and keep it in my bag of handy
tools. :-)
Kevin
tools. :-)
Kevin
kdt...@gmail.com
Apr 17, 2010, 3:18:18 PM4/17/10
to Hardhats
On Apr 17, 10:01 am, "K.S. Bhaskar" <ks.bhas...@fisglobal.com> wrote:
...
> If you go into your normal VistA/GT.M direct mode, and launch mupip, you
> should have the environment already setup, e.g., something like:
>
> GTM>ZSY "mupip integ -full -region ""*"""
>
If I run this from direct mode GT.M, then would mupip integ have stand
> should have the environment already setup, e.g., something like:
>
> GTM>ZSY "mupip integ -full -region ""*"""
>
along access?
Kevin
K.S. Bhaskar
Apr 17, 2010, 5:37:07 PM4/17/10
to hard...@googlegroups.com
-region doesn't require stand-alone access; -file does.
Regards
-- Bhaskar
GT.M - Rock solid. Lightning fast. Secure. No compromises.
Regards
-- Bhaskar
GT.M - Rock solid. Lightning fast. Secure. No compromises.
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_____________
The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank you.
_____________
r...@rcresearch.us
Apr 18, 2010, 12:01:21 AM4/18/10
to hard...@googlegroups.com
Kevin;
Here are some refinements to the tool;
GTM>W $$SUB^%GSIZE("^DIC")
544592;28173336
GTM>W $$SUB^%GSIZE("^DIC(2)")
37;1929
GTM>W $$SUB^%GSIZE("^DIC(4)")
320;10219
GTM>W $$SUB^%GSIZE("^DIC(19)")
149242;7497206
GTM>W $$SUB^%GSIZE("^DIC(19,")
149242;7497206
GTM>W $$SUB^%GSIZE("^DIC(19")
149242;7497206
GTM>s X=19
GTM>W $$SUB^%GSIZE("^DIC(X")
149242;7497206
GTM>
As you can see, the input has been liberalized for easier use.
Here are some refinements to the tool;
GTM>W $$SUB^%GSIZE("^DIC")
544592;28173336
GTM>W $$SUB^%GSIZE("^DIC(2)")
37;1929
GTM>W $$SUB^%GSIZE("^DIC(4)")
320;10219
GTM>W $$SUB^%GSIZE("^DIC(19)")
149242;7497206
GTM>W $$SUB^%GSIZE("^DIC(19,")
149242;7497206
GTM>W $$SUB^%GSIZE("^DIC(19")
149242;7497206
GTM>s X=19
GTM>W $$SUB^%GSIZE("^DIC(X")
149242;7497206
GTM>
As you can see, the input has been liberalized for easier use.
kdt...@gmail.com
Apr 18, 2010, 6:25:23 PM4/18/10
to Hardhats
Richard,
Why not have the output something like this:
544592 nodes;28173336 bytes
I don't like programs that I can't figure out when I come back to them
cold after a year or so. So I like to be verbose with requests for
input, and in describing output.
Kevin
Why not have the output something like this:
544592 nodes;28173336 bytes
I don't like programs that I can't figure out when I come back to them
cold after a year or so. So I like to be verbose with requests for
input, and in describing output.
Kevin
Steven McPhelan
Apr 18, 2010, 9:24:32 PM4/18/10
to hard...@googlegroups.com
Gee Kevin, I thought only old farts like me prefer that method because of our sharp and accurate memory. A few times a year later I would have to review the code to remember why I wrote it. For this same reason, especially for IT utilities, I prefer lots of on screen help because we all know how developers like to read documentation.--
Steve
A wise and frugal government, which shall leave men free to regulate their own pursuits of industry and improvement, and shall not take from the mouth of labor and bread it has earned - this is the sum of good government. - Thomas Jefferson
Steve
A wise and frugal government, which shall leave men free to regulate their own pursuits of industry and improvement, and shall not take from the mouth of labor and bread it has earned - this is the sum of good government. - Thomas Jefferson
John McCormack
Apr 18, 2010, 10:20:40 PM4/18/10
to hard...@googlegroups.com
If you look closely the program is really returning random $H values
with the delimiter changed.
--
"If you can read this, thank a teacher;
... if you are reading it in English, thank a soldier."
kdt...@gmail.com
Apr 18, 2010, 10:57:12 PM4/18/10
to Hardhats
LOL to both of you... :-)
Kevin
On Apr 18, 10:20 pm, John McCormack <johnm...@ix.netcom.com> wrote:
> If you look closely the program is really returning random $H values
> with the delimiter changed.
>
> On 4/18/2010 8:24 PM, Steven McPhelan wrote:
>
>
>
> > Gee Kevin, I thought only old farts like me prefer that method because
> > of our sharp and accurate memory. A few times a year later I would
> > have to review the code to remember why I wrote it. For this same
> > reason, especially for IT utilities, I prefer lots of on screen help
> > because we all know how developers like to read documentation.
>
> > On Sun, Apr 18, 2010 at 6:25 PM, kdt...@gmail.com
> > <mailto:kdt...@gmail.com> <kdt...@gmail.com <mailto:kdt...@gmail.com>>
> > <mailto:Hardhats%2Bunsu...@googlegroups.com>
Kevin
On Apr 18, 10:20 pm, John McCormack <johnm...@ix.netcom.com> wrote:
> If you look closely the program is really returning random $H values
> with the delimiter changed.
>
> On 4/18/2010 8:24 PM, Steven McPhelan wrote:
>
>
>
> > Gee Kevin, I thought only old farts like me prefer that method because
> > of our sharp and accurate memory. A few times a year later I would
> > have to review the code to remember why I wrote it. For this same
> > reason, especially for IT utilities, I prefer lots of on screen help
> > because we all know how developers like to read documentation.
>
> > On Sun, Apr 18, 2010 at 6:25 PM, kdt...@gmail.com
> > wrote:
>
> > Richard,
>
> > Why not have the output something like this:
> > 544592 nodes;28173336 bytes
>
> > I don't like programs that I can't figure out when I come back to them
> > cold after a year or so. So I like to be verbose with requests for
> > input, and in describing output.
>
> > Kevin
>
> > On Apr 18, 12:01 am, r...@rcresearch.us
>
> > Richard,
>
> > Why not have the output something like this:
> > 544592 nodes;28173336 bytes
>
> > I don't like programs that I can't figure out when I come back to them
> > cold after a year or so. So I like to be verbose with requests for
> > input, and in describing output.
>
> > Kevin
>
> > On Apr 18, 12:01 am, r...@rcresearch.us
>
> > > >>> > --
> > > >>> >http://groups.google.com/group/Hardhats
> > > >>> > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
> > <mailto:Hardhats%2Bunsu...@googlegroups.com>
> > > >>> > --
> > > >>> >http://groups.google.com/group/Hardhats
> > > >>> > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
>
> > > >>> --http://groups.google.com/group/Hardhats
> > > >>> To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
> > <mailto:Hardhats%2Bunsu...@googlegroups.com>
> > > >>> --http://groups.google.com/group/Hardhats
> > > >>> To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
>
> > > >>> _GSIZE.m
> > > >>> < 1KViewDownload
>
> > > >> --
> > > >>http://groups.google.com/group/Hardhats
> > > >> To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
> > <mailto:Hardhats%2Bunsu...@googlegroups.com>
> > > >>> _GSIZE.m
> > > >>> < 1KViewDownload
>
> > > >> --
> > > >>http://groups.google.com/group/Hardhats
> > > >> To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
>
> > > > --
> > > >http://groups.google.com/group/Hardhats
> > > > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
> > <mailto:Hardhats%2Bunsu...@googlegroups.com>
> > > > --
> > > >http://groups.google.com/group/Hardhats
> > > > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
>
> > > --http://groups.google.com/group/Hardhats
> > > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
> > <mailto:Hardhats%2Bunsu...@googlegroups.com>
> > > --http://groups.google.com/group/Hardhats
> > > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
>
> > > _GSIZE.m
> > > < 1KViewDownload
>
> > --
> > http://groups.google.com/group/Hardhats
> > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
> > <mailto:Hardhats%2Bunsu...@googlegroups.com>
> > > _GSIZE.m
> > > < 1KViewDownload
>
> > --
> > http://groups.google.com/group/Hardhats
> > To unsubscribe, send email to
> > Hardhats+u...@googlegroups.com
>
> > --
> > Steve
> > A wise and frugal government, which shall leave men free to regulate
> > their own pursuits of industry and improvement, and shall not take
> > from the mouth of labor and bread it has earned - this is the sum of
> > good government. - Thomas Jefferson
> > --
> >http://groups.google.com/group/Hardhats
> > To unsubscribe, send email to Hardhats+u...@googlegroups.com
>
> --
> "If you can read this, thank a teacher;
> ... if you are reading it in English, thank a soldier."
>
> > --
> > Steve
> > A wise and frugal government, which shall leave men free to regulate
> > their own pursuits of industry and improvement, and shall not take
> > from the mouth of labor and bread it has earned - this is the sum of
> > good government. - Thomas Jefferson
> > --
> >http://groups.google.com/group/Hardhats
> > To unsubscribe, send email to Hardhats+u...@googlegroups.com
>
> --
> "If you can read this, thank a teacher;
> ... if you are reading it in English, thank a soldier."
>
> --http://groups.google.com/group/Hardhats
> To unsubscribe, send email to Hardhats+u...@googlegroups.com
> To unsubscribe, send email to Hardhats+u...@googlegroups.com
Nancy Anthracite
Apr 18, 2010, 11:20:54 PM4/18/10
to hard...@googlegroups.com
John has an uncanny ability to figure out when these sorts of things happen.
I still chuckle to think back when he was working with us on the CCR-CCD
project and recognized that some entries we were trying so hard to figure out
were merely random poundings on a keyboard used to fill the field in the VA
demo. You should have heard what we had been cooking up to explain what it
all meant.
I still LOL about it when I tell people about it all of these many months
later! :-)
Nancy Anthracite
I still chuckle to think back when he was working with us on the CCR-CCD
project and recognized that some entries we were trying so hard to figure out
were merely random poundings on a keyboard used to fill the field in the VA
demo. You should have heard what we had been cooking up to explain what it
all meant.
I still LOL about it when I tell people about it all of these many months
later! :-)
r...@rcresearch.us
Apr 19, 2010, 8:52:25 AM4/19/10
to hard...@googlegroups.com
Look, guys,
1) the code was basically written in about 20 minutes before bed.
2) It was designed to solve an expressed need.
3) It was written to be used interactively and also be called as a function
4) Any calling routine just wants the data and not the descriptive baggage
5) I will be extensively documenting the code internally
6) and there may be a flaw in the estimation of the overhead of 4
characters/node (depending upon the vendor),
7) The precision was always presented as little more than a WAG (Wild
Assed Guess) estimate.
8) The code is free and given freely to one and all and there seems to be
some old adage about gift horses and looking them in the mouth. The
beauty of a gift horse is that you can always make glue and some form of
stew or fertilizer if it doesn't come through as expected.
Too many critics and not enough chefs, Best wishes and cheers; Chris
1) the code was basically written in about 20 minutes before bed.
2) It was designed to solve an expressed need.
3) It was written to be used interactively and also be called as a function
4) Any calling routine just wants the data and not the descriptive baggage
5) I will be extensively documenting the code internally
6) and there may be a flaw in the estimation of the overhead of 4
characters/node (depending upon the vendor),
7) The precision was always presented as little more than a WAG (Wild
Assed Guess) estimate.
8) The code is free and given freely to one and all and there seems to be
some old adage about gift horses and looking them in the mouth. The
beauty of a gift horse is that you can always make glue and some form of
stew or fertilizer if it doesn't come through as expected.
Too many critics and not enough chefs, Best wishes and cheers; Chris
r...@rcresearch.us
Apr 19, 2010, 9:01:20 AM4/19/10
to hard...@googlegroups.com
One additional point. Any program without requirements can never be
wrong, only surprising.
wrong, only surprising.
Nancy Anthracite
Apr 19, 2010, 10:59:19 AM4/19/10
to hard...@googlegroups.com, r...@rcresearch.us
Chris, we have all been there and done that. Nobody is judging you. Have a
good laugh about it, please!
good laugh about it, please!
kdt...@gmail.com
Apr 19, 2010, 6:51:17 PM4/19/10
to Hardhats
Chris,
You wrote this in response to my request, and I appreciate the work
you did. It is a good utility and deserves to be appreciated.
Thanks again
Kevin
> ...
>
> read more »
You wrote this in response to my request, and I appreciate the work
you did. It is a good utility and deserves to be appreciated.
Thanks again
Kevin
>
> read more »
chuck...@gmx.com
Apr 21, 2010, 9:01:48 AM4/21/10
to hard...@googlegroups.com
Chris? Reminds me of the old adage about a miserable life: still preferable to the alternative, n'es pas? ;-)
Reply all
Reply to author
Forward
0 new messages