[MRTG] Perl Script to Fetch a Web Page and Extract Data.

[Jamie Bekkers]

Hello.

I would like to create a MRTG graph of our local weather. I am not a
programmer so I was hoping that someone out there could give me a hand.

I want to have a script (or whatever) that will go out to a Web Page and use
text from a web page as values in a MRTG graph.

Thanks
Jamie.

---[Jamie Bekkers]--------------------------
Bekkers IT Ph: +61 8 93705611
j...@bekkers.com.au Fx: +61 8 92726625
http://www.bekkers.com.au ICQ # 2866084

--
* To unsubscribe from the mrtg mailing list, send a message with the
subject: unsubscribe to mrtg-r...@list.ee.ethz.ch
* The mailing list archive is at http://www.ee.ethz.ch/~slist/mrtg

[Starkweather, Mike]

Jamie,

Here's something I threw together last night. It works on Windows 95 and NT.
You might have to make some changes if you are running on Unix.

Four files in this message.

weather.bat - batch file to run data gather and MRTG
weather.plx - data gather perl script - get web page and extract data
weather.cfg - MRTG cfg file to display perth temperature
type.plx - perl script to print file into MRTG

MRTG does not like the decimals in the temperature and truncates them.

Put all four files in the same directory. Adjust the cfg file as
appropriate for your system.

----weather.bat----
rem graph temperature in perth
perl weather.plx perth 3
perl d:\mrtg\mrtg-2.5.3\mrtg weather.cfg

----weather.plx----
#!d:/perl/perl
# Extract weather data from web page
# Usage: perl weather.plx vname var
# where lname = location name (e.g. Perth)
# and var = number of the weather variable to extract

require LWP::UserAgent;
$ua = new LWP::UserAgent;

# if you use a firewall, enter the address here and uncomment
#$ua->proxy('http', 'http://1.1.1.1/'); # set proxy for external sites

$url = 'http://www.bom.gov.au/products/IDO30W01.shtml';
$fname = 'weather.out'; # where to put the web page output
$request = new HTTP::Request('GET', $url) || die "HTTP request error:
$!\n";;
$request->header('Accept' => 'text/html');
$response = $ua->request($request,"$fname");
# check the outcome
if ($response->is_success) {
} else {
print STDERR "Error: " . $response->status_line . "\n";
print STDERR "$url\n";
}

# get input parameters
die "Usage: perl weather.plx lname var\n" unless ( $#ARGV == 1 );
$lname = $ARGV[0];
$var = $ARGV[1];
@vars = qw( date time temp dewpoint winddir windspeedk windspeedm pressure
relhum );
$vname = $vars[$var-1];
print "lname $lname, var $var, vname $vname\n";

# open files
open SRC, "<$fname" || die "Unable to open $fname for input: $!\n";
open DST, ">weather\.txt" || die "Unable to open weathr\.txt for output:
$!\n";

$state = 0; # initial state
while ( <SRC> ) {
chomp;
# print "state $state line $_\n";
if ( $state == 0 )

if ( /<TD ALIGN.*>$lname/i )

$state = 1; # found name
$i = 0;
}
}
elsif ( $state == 1 )

$i++;
if ( $i == $var ) {
$state = 2; # found desired variable
/<TD ALIGN=center>(.*)/i; # match data pattern
$out = $1; # extract data variable
}
}
}
# write output in MRTG format
print DST "$out\n$out\n0\n$lname $vname\n";
print "$out\n$out\n0\n$lname $vname\n";

# close files
close SRC;
close DST;

----weather.cfg----
WorkDir: d:\InetPub\wwwroot\MRTG\

#---------------------------------------------------------------
Target[perthtemp]: `perl type.plx weather.txt`
Title[perthtemp]: Temperature (C) in Perth
Directory[perthtemp]: fun
Options[perthtemp]: growright,gauge,nopercent
Background[perthtemp]: #a0a0a0a
MaxBytes[perthtemp]: 50
AbsMax[perthtemp]: 100
Unscaled[perthtemp]: dwyn
YLegend[perthtemp]: Degrees C
XSize[perthtemp]: 500
YSize[perthtemp]: 100
ShortLegend[perthtemp]: (C)
Legend1[perthtemp]: Temperature
Legend2[perthtemp]:
Legend3[perthtemp]: Max Temperature
Legend4[perthtemp]:
LegendI[perthtemp]: &nbsp deg &nbsp
LegendO[perthtemp]:
PageTop[perthtemp]: <H1>Temperature (C) in Perth</H1>
#-----This is the End

----type.plx----
while (<>) {
print $_;
}

Enjoy...Mike Starkweather

> -----Original Message-----
> From: Jamie Bekkers [SMTP:J...@bekkers.com.au]
> Sent: Tuesday, November 17, 1998 6:39 PM
> To: Starkweather, Mike
> Subject: RE: [MRTG] Perl Script to Fetch a Web Page and Extract Data.
>
> http://www.bom.gov.au/products/IDO30W01.shtml
>
> -----Original Message-----
> From: Starkweather, Mike [mailto:mike.sta...@anheuser-busch.com]
> Sent: Wednesday, 18 November 1998 7:13
> To: 'Jamie Bekkers'
> Subject: RE: [MRTG] Perl Script to Fetch a Web Page and Extract Data.
>
>
> Jamie,
>
> Can you point me to the web page?
>
> Mike Starkweather