Examples of LMT.pm usage?

[davour]

Have anybody tried to use the LMT.pm module for something? I'd love to
see some examples, or documentation, since I'm not very good at
figuring that out just from reading the source to the module.

/andreas

[Andrew Uselton]

Prod me again in the next day or two if I haven't said more (I have to run right now). If you'd like to see examples of what a Perl interface to the LMT DB can do take a look at:

      http://portal.nersc.gov/project/pma/

Cheers,
Andrew

--
You received this message because you are subscribed to the Google Groups "lmt-discuss" group.
To post to this group, send email to lmt-d...@googlegroups.com.
To unsubscribe from this group, send email to lmt-discuss...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/lmt-discuss?hl=en.

[Andrew Uselton]

So, my use of LMT.pm is rudimentary. With it I can query the DB from <start> to <end> where those are two time stamps. Be sure to look at the place in LMT.pm where the MySQL connection is established, it may need some configuring before you can use it.

First, you need a connection and your Perl script needs to know about the LMT.pm module:

use lib '/usr/share/lmt/lib/perl';
use LMT;
# details about creating a Perl "object" elided
$self->{lmt} = LMT->new();
# You may want to parse /usr/share/lmt/etc/lmtrc to get the filesystem name
$self->{lmt}->connect($self->{filesystem});

If I query the OST_DATA table (which is the only one I really mess with), then I get a set of rows from the table. It is useful to know beforehand what the OST names are so:

my $ost_query = "SELECT * FROM OST_INFO";
my $ost_sth = $self->{lmt}->execQuery($ost_query);
$self->{num_osts} = 0;
while( my $ost_ref = $ost_sth->fetchrow_hashref() )
{
    my $ost_name = $ost_ref->{"OST_NAME"};
    $self->{num_osts}++;
...
}

Now you can get the actual data. $start and $end are LMT  "TIMESTAMP" values, which are recorded as text in the format, eg. "2011-02-14 08:00:00":

my $query = $self->{lmt}->generateGetDataWindowQuery(
"OST_DATA", $start, $end,
("TS_ID", "TIMESTAMP", "OST_ID", "READ_BYTES", "WRITE_BYTES", "PCT_CPU", "KBYTES_FREE", "KBYTES_USED", "INODES_FREE", "INODES_USED"));
my $query_result = $self->{lmt}->execQuery($query);
while( my $row = $query_result->fetchrow_hashref() )
{
# the row data is a hash, this is just to exhibit it
  $row_hash = {
    ost_id         => $row->{"OST_ID"},
    timestamp   => $row->{"TIMESTAMP"},
    ts_id       => $row->{"TS_ID"},
    read_bytes  => $row->{"READ_BYTES"},
    write_bytes => $row->{"WRITE_BYTES"},
    pct_cpu     => $row->{"PCT_CPU"},
    kbytes_free => $row->{"KBYTES_FREE"},
    kbytes_used => $row->{"KBYTES_USED"},
    inodes_free => $row->{"INODES_FREE"},
    inodes_used => $row->{"INODES_USED"}
    };
# Now go do something useful with the data
}

For a given OST_ID and TIMESTAMP you will have two counters: READ_BYTES and WRITE_BYTES. Since they are counters they are always increasing. You can calculate a data rate for the OST_ID by taking the difference in the counter from one TIMESTAMP to the next. You can sum up those differences across all OST_IDs and get an aggregate data rate for the entire file system for that interval. Keep in mind that the counters are reset at the time a server is rebooted, otherwise the difference should never be negative. Also keep in mind that the values are sent to the DB via UDP, which means there can be data loss. You may have incomplete data for some interval, and how you handle that will affect what your differential and aggregate values look like. Have fun!
Cheers,
Andrew

[davour]

Thanks. That was some food for thought.

It seems like LMT.pm is a very thin layer indeed. I'm not very good at
perl, and had some hopes LMP.pm could be used by any bozo. It seems
like some more skill is needed.

The stuff you have on that web page looks really nice. Is there any
possibility you would be allowed to share that setup, or would it be
to hard to retrofit to another site anyway?

/andreas

[Andrew Uselton]

Jim's been after me for a long time to push the scripts back into the LMT project. My worry is that it is sight-specific and people would need a lot of help getting it localized. Andrew disappears into black hole. Nevertheless, it needs doing. I recently started a PHP/Javascript project (http://portal.nersc.gov/project/pma/yams/yams.php) that provides a much nicer user experience. The LMT interaction there isn't quite working right now, but maybe we can build on that rather than the Perl stuff, which is admittedly hack-cough-snort.
Cheers,
Andrew

/andreas