Richard Kettlewell wrote:-
Obviously the greatest [productivety eg.] improvements come from optimising
at the highest level. So to use your <blogtoNTTP> I'd want a mental picture
of how it works. We all want to avoid starting a journey which doesn't
succeed in reaching a worthwhile destination. But since it seems that we're
both on the same page, I'll try it - for a further optimisation of my inet
usage.
Perhaps you can understand my post:
Newsgroups: comp.lang.forth
Subject: Re (2): existing code to build a 'sed-like'?
I'm investigating another quirk which may interest you:
as you know, many webPublishers give you each page with the same 90% duplicated.
So [with eg. `mc`] we see that the repeated-garbage, of the lynx viewed <dump>
ends at byte 12345, and then in future we do:
<curl fetch from 12344 URL >> AccumulatingFile>
I've just tested an amazing minute `html2txt` which comes from mulinux: the
90's linux on a 1M7 fd0. I think lynx, links, elinks ..etc would crash if
the *html is mangled, with <unmatched brackets>.
The whole idea is like they had/have <advert supressors> for TV viewing.
You'd `curl` the good-text after the repeat-garbage and read it through
a `html2txt`. mulinux uses `nc` to fetch every thing. It's apparently
even more low-level than `curl`. Perhaps it's also possible to abort the
tail-part of repeated-garbage?
So you'd have a script with the 3 args, to do:
`fetchHttp head tail URL`
and then as you regularly visit certain sites you have a tuned script for each,
and then as the sites changed their format from time to time you'd easily,
retune the scripts accordingly.
Of course the trimmed down text, is no good to get your next cycle of links
from your <lynx dump>.
What I'd really like to have is a method/script which would <extract/append the
URLs to a file> while I'm reading my `lynx -dump URL` and see interesting
'links'. So while you're reading the text at the top and you see that
<link 129> looks interesting to fetch later,
" 129. <URL129>"
can automagically append the <URL129> to the file-of-URLs which will be
`lynx -dumped` next time on-line.
Thanks,
== Chris Glur.