Howto strip html from a $calar with HTML::Parser

[Alan C]

Hi,

I saw the doc for HTML::Parser. I looked at its hstrip program in the eg
folder but dunno how to strip a scalar instead of a file. I can strip a
file. But I want to strip a scalar. Any help appreciated.

As my code is currently, when ran, it prints text to STDOUT and the $source
scalar variable holds html

I do not need to print to STDOUT

Is the print @ line in the sub doing this?

I need to strip the html from $source

What am I doing wrong whereby the html does not get stripped from $source ?
--

So that I can then uncomment my __END__

So that the latter part of my program will then run.

The latter part of my program works correctly; it finds a unique text string
then prints it along with a specified number of subsequent lines. (This is
what I want) but in order for this latter part to work, the html needs to be
stripped from $source

Current code is next:

#!/usr/bin/perl
use strict;
use warnings;

use LWP::UserAgent;

my $ua = LWP::UserAgent->new();

my $url = '
http://www.wrh.noaa.gov/total_forecast/getprod.php?wfo=sto&sid=STO&pil=ZFP';
# my $url = 'http://www.slackware.com/changelog/current.php?cpu=i386';
my $response = $ua->get( $url );

$response->is_success
or $response->code == 404 # ignore 404
or die "Can't get '$url': ", $response->status_line;

# print $response->content;

# __END__

my $source = $response->content;
$source =~ s/\n+/\n/g;

# Strip html markup ####################
use HTML::Parser ();

sub text_handler { # Ordinary text
print @_;

}

my $p = HTML::Parser->new(api_version => 3);
$p->handler( text => \&text_handler, "dtext");

$p->parse($source);
$p->eof;

print "~source~\n";
print $source;
__END__

# print $source;
my @raw = split(/\n/, $source);
# print scalar( @raw );
my $line;
my $stop_at;
foreach ( @raw ) {
$line++;
if ( /CAZ017/ ) {
$stop_at = $line + 40;
}
next unless $stop_at;
print $_, "\n" if $line <= $stop_at;
}
# end ------------------------------------------------------

Ultimately, I want to print the CA (California) zone 17 forecast (southern
Sacramento valley forecast). Probably will knock that $line + 40 down to $line
+ 15 or thereabouts. But, I need to strip the html from $source first.
Thanks.

--
Alan.

[Mumia W.]

On 08/03/2007 03:30 AM, Alan C wrote:
> [...]

> I do not need to print to STDOUT
>
> Is the print @ line in the sub doing this?

> [...]

Yes.

Why not append the text to $string instead of printing the text?

[Alan C]

Thanks for the replies. I've added them to my searchable data.

My forgetfullness worsens as I get older.

I'd totally forgotten that I'd once done this with HTML::Strip

And I began reinventing the wheel the other day. Lynx too, forgot about
that (Slackware 12.0 right now).

Anyways, here's a more featured weather prog. that I'd written a while back
but had forgotten about the other day.

It's for weather. Prints both the current conditions and the current
forecast. Makes use of the clweather app at

http://freshmeat.net/projects/clweather/

Does some (I think) interesting things using the DateTime module. Also uses
HTML::Strip Uses LWP Uses Text::Autoformat
--

#!/usr/bin/perl -w
use strict;
use DateTime;
# Alan_C with help from others on portions herein

# locate station 4 letter identifiers
# http://www.nws.noaa.gov/tg/siteloc.shtml
# at next url, scroll to bottom, finds text_only_forecast
#
http://www.wrh.noaa.gov/total_forecast/index.php?wfo=sto&zone=caz017&county=cac067
# ftp://weather.noaa.gov/data/observations/state_roundup/ca/caz017.txt
# ftp://weather.noaa.gov/data/observations/metar/decoded
# ftp://weather.noaa.gov/data/observations/metar/stations
# http://weather.noaa.gov/cgi-bin/mgetmetar.pl?cccc=KSAC

# Sac area by zone hazard/warnings
#
http://www.wrh.noaa.gov/warnings.php?wfo=sto&zone=CAZ017&pil=XXXHWOSTO&productType=HAZARDOUS%20WEATHER%20OUTLOOK

# Pismo, San Luis forecast
#
http://www.wrh.noaa.gov/total_forecast/text_only.php?wfo=lox&zone=caz034&fire=&county=cac079&dgtl=1&lat=35.14278&lon=-120.64028
my $url_4cast = '
http://www.wrh.noaa.gov/total_forecast/text_only.php?wfo=sto&zone=caz017&fire=&county=cac067&dgtl=&lat=38.51000&lon=-121.4900
';

print <<STUF;

clweather -s KSAC (default if none/nothing specified/entered)
clweather -s XXXX (see stn list, enter 4 letter code)
KSMF Sac Metro || KMHR Mather || KSAC Sac executive
KOXR || KSBP San Luis Obispo, Pismo Beach
1 weather SAC (Slack expect script)
STUF
print "\nenter (nothing) OR a 4 letter stn code OR a 1: ";
chomp(my $wthr_choice = <STDIN>);
print "\n";

unless ($wthr_choice) {
$wthr_choice = 'KSAC';
}

if ($wthr_choice =~ /[A-Z]{4}|[A-Z]{2}\d\d/) {

# system("clweather -s $wthr_choice");
my @report = `clweather -s $wthr_choice`;

foreach ( @report ) {
last if /^ob/i;
print unless /\/ \d{4}\.\d{2}.\d{2} \d{4} UTC/;
if (/\/ \d{4}\.\d{2}.\d{2} \d{4} UTC/) {
my $utc_line = $_;
$utc_line =~ s/^.+\/ \d{4}\.//;

$utc_line =~ s/(\d{2})\.(\d{2}) (\d{2})(\d{2}) UTC/$1 $2 $3 $4/;
chomp($utc_line);
# (substitution) (no match nothing happens) if string begins with 1 instead
of 0
$utc_line =~ s/^0//;
my @parsed_utc_fields = split(/ /,$utc_line);

my $source = DateTime->new(year => 2006, month => $parsed_utc_fields[0],
day => $parsed_utc_fields[1],
hour => $parsed_utc_fields[2], minute =>
$parsed_utc_fields[3],
time_zone => 'UTC');
my $result = $source->clone()
->set_time_zone( 'America/Los_Angeles' );
# print $source->strftime("%F %r %Z"), "\n",
# $result->strftime("%F %r %Z"), " <- Sacto, CA <- ***** |\n";

print $result->strftime("%F %r %Z"), " \/ ", $_;

# print "$utc_line\n";
# print "yes matched\n";
# print "\n@parsed_utc_fields\n";
}
}
#print "@report";

print <<STUFF;
[ summer UTC - 7 || EDT -2 ] [ winter UTC - 8 || EDT -3 ]

STUFF
$wthr_choice = 9;
# print `lynx -dump $url_4cast`;

use LWP::UserAgent;
#: bin/lwp_dwnld download get www lwp slack changelog

my $ua = LWP::UserAgent->new();

# my $url = 'http://0daycheck.eastgame.net/0day/archives/20060901.html';

# my $url = 'http://www.slackware.com/changelog/current.php?cpu=i386';

my $response = $ua->get( $url_4cast );

$response->is_success
or $response->code == 404 # ignore 404

or die "Can't get '$url_4cast': ", $response->status_line;

# print $response->content;
my $raw_html = $response->content;
# $raw_html =~ s/\n+/\n/g;
$raw_html =~ s/\<br\>/\n/gi;
# print $raw_html;

use HTML::Strip;

my $hs = HTML::Strip->new();

my $clean_text = $hs->parse( $raw_html );
$hs->eof;
# $clean_text =~ s/\n+/\n/g;
# print $clean_text;

my $line_quantity = 24; # desired number of lines
my @lines = split(/\n/, $clean_text);
# print scalar( @lines );
my $curr_line = '0';
my ($stop_at, $post);
foreach ( @lines ) {
$curr_line++;
# print "$curr_line $_\n" if /ChangeLog for Intel/;
if ( /^Area Forecast For/i ) {
$stop_at = $curr_line + $line_quantity;
}
next unless $stop_at;
s/(^.{2})/\.$1/ unless /^Area Forecast For|^Issued/i;
$post .= "$_\n" if $curr_line <= $stop_at;
}
# print $post;

use Text::Autoformat qw(autoformat);

my $post_formatted;
$post_formatted = autoformat($post, {left=>0, right=>72, all=>1});
$post_formatted =~ s/\n+/\n/g;
print $post_formatted, "\n";

}

if ($wthr_choice == 1) {
# system("weather SAC");
#my @weather = `weather SAC`;
#foreach my $line (@weather) {
#<STDIN> = `weather SAC`;
#while (<STDIN>) {
my $pid;
my $readme;
$pid = open $readme, "-|", "weather", "SAC" or die "cant fork: $!\n";
print "pid is: $pid\n";
while (<$readme>) {
#print if $line =~ /Weather Conditions at/i .. /=/;
#print "$line" if $line =~ /Weather Conditions at/i .. /=/;
print if /Weather Conditions at/i .. /SAC exec/;
#print if /Weather Conditions at/i .. /^\=$/;
}
#print "\n\n";
#print @weather;
}
# end -------------------------------------------------------

--
Alan.