Loop over all new articles of a feed

48 views
Skip to first unread message

rap...@raphaelfournier.net

unread,
Sep 13, 2014, 6:55:38 PM9/13/14
to newsb...@googlegroups.com
First, I would like to send congratulations to the main developer of newsbeuter, it's an excellent piece of software!

I subscribed to several tumblr blogs, where most of the posts are images (mostly of food), which newsbeuter is not mainly aimed at handling (like every CLI-based soft). I have found a macro to collect the links of the images into a file, which I can then process (filter non images, "wget" all the images, then watch them). For reference:
macro o set external-url-viewer "dehtml | grep 'http.* ' | sed 's/^.*: //' | sed 's/ .*$//' >> /home/me/RSS/img/imglinks.txt" ; show-urls ; toggle-article-read

But, I have to ";o" on each new post to gather the links. I would like something more automatic, where I could type a few keystrokes and it would handle all new posts of a feed, until there are no new unread posts. I could not find anything while browsing the documentation. I briefly looked into cache.db, saw the .schema/.tables, tried a few queries but could not get how tags are handled, nor how I could iterate over rss_item lines and grouping them by feed/status.

If anyone has an idea, thanks for your help!

rfs

yitz

unread,
Sep 13, 2014, 7:13:19 PM9/13/14
to newsb...@googlegroups.com
sqlite3 cache.db <<< "SELECT content FROM rss_item WHERE feedurl LIKE '%tumblr%' AND unread = 1;" | dehtml | grep -ow 'http.*' ...
You can also get links using elinks --dump and pulling the links from the end.

For more interactive help, try to catch me in the IRC channel on FreeNode.

Raphaël Fournier-S'niehotta

unread,
Sep 15, 2014, 4:52:15 PM9/15/14
to newsb...@googlegroups.com
Thanks for quick and efficient reply, it works.

To improve my knowledge of newsbeuter, would it be possible to act on feed
according to their tags?

I chose the mailing list because it allows asynchronous answers and my question
was not urgent, but I'll definitely connect to the channel soon to say hi and
ask my next questions!

Le 13/09/14 16:13, yitz a écrit:
> --
> You received this message because you are subscribed to a topic in the Google Groups "newsbeuter" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/newsbeuter/p4yTS1FwGi0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to newsbeuter+...@googlegroups.com.
> To post to this group, send email to newsb...@googlegroups.com.
> Visit this group at http://groups.google.com/group/newsbeuter.
> For more options, visit https://groups.google.com/d/optout.

--
Raphaël Fournier-S'niehotta
http://raphael.fournier-sniehotta.fr/apropos

yitz

unread,
Sep 15, 2014, 4:59:23 PM9/15/14
to newsb...@googlegroups.com
I don't think the tag gets added to the DB. You could do something like:
awk '{for (i=2; i<=NF; i++) if ($i = "tag) { print $0 ; break; }' | while read url ; do sqlite3 cache.db <<< "SELECT stuff FROM rss_item WHERE feed_url = \"$url\";"

I prefer the channel because requirements, solutions and requests can be refined and tweaked.
Reply all
Reply to author
Forward
0 new messages