Nextdb And An RSS Reader

[Eric Dorman]

Hey guys,

Does anyone know if we can use Nextdb to store rss feeds and have them
then fetch the feeds using the xml http request object?

Thanks & God Bless,
Eric

[geoffrey hendrey]

XML http request doesn't work across domains. You can either setup a proxy on your end, or use our JSAPI, since our JSAPI was designed to work across domains.

-geoff

--
http:nextdb.net - RESTful Relational Database
http://www.nextdb.net/wiki/en/REST

[Eric Dorman]

Hey Geoff,

Do you think it is possible to create a rss aggregator in just
javascript and ajax?

If xml http request object cannot do what I want then how would I use
XML to fetch the rss feeds from their domains?

I know you said some stuff about your JSAPI,but can you elaborate on
how I could use it to fetch feeds or domians?

Thanks & God Bless,
Eric

On Nov 1, 9:00 pm, geoffrey hendrey <geoff.hend...@gmail.com> wrote:
> XML http request doesn't work across domains. You can either setup a proxy
> on your end, or use our JSAPI, since our JSAPI was designed to work across
> domains.
>
> -geoff
>

[geoffrey hendrey]

JQuery's ajax request probably works across domains. You can grab arbitrary XML files from various domains using that. I didn't mean that nextdb's js api would be used to grab other XML feeds off the web. I thought you wanted to use XML http req to access nextdb. I misunderstood.

-g

[Eric Dorman]

Hey Geoff,

Sorry about that man. I should have worded it differently.

I am just trying find a way to build a rss aggregator and use nextdb
to store the feed links that use to aggregate.

I just am stuck because I am trying to build my rss aggregator using
only javascript and no libraries.

Thanks for that info on Jquery though.

Thanks & God Bless,
Eric

[ibec]

Hi Eric,

I've build an RSS aggregator with NextDB earlier this year for a
client in the finance world that needed a site with all the latest
news on "operational risk management": http://bit.ly/riskfriendssite

One NextDB table contains all the feed links (including the the latest
offset signature per feed) and another table contains all the feed
items. A background process periodically syncs all the feeds and uses
the Google feed reader service for this:

http://ajax.googleapis.com/ajax/services/feed/load?v=1.0&q={feed}
http://code.google.com/apis/ajaxfeeds/documentation/#fonje

And the frontend process is just a NextDB query that serves the sites'
content.

If you need any further info or code snippets, please let me know.

-Ivo

[Eric Dorman]

Hey ibec,

Thanks for that great information.

I am trying to build a sports rss feed aggregator.

I would really like some code snippets as well.

Can you give me some more info please on how you managed to fetch the
feeds data and retrieve the results?


I appreciate all your help ibec.

Thanks & God Bless,
Eric

[Eric Dorman]

Hey Ibec,

So you store the feeds you want to fetch in a Nextdb Table and then
program Nextdb to sync with the Google Feed Reader Service?

Is that what you do?

If you can give me some code snippets of how you did it I would really
appreciate it?

Thanks & God Bless,
Eric

[ibec]

Eric,

First, define two tables; one that contains the feeds and one that
contains the items. Here are my table specs: http://skitch.com/focusfriend/nf5yc/feedcentral

Like mentioned before, there are two processes (in my case running on
the server, all in JavaScript); one that periodically walks thru the
feeds and adds the new items (every 5 minutes), and one that serves
and generates the site when a user visits it. The main process is the
one that keeps the feed-items in sync and has the following steps:

[1] Retrieve feeds to be parsed (with a NextDB query on the "feeds"
table)

[2] Retrieve the individual feeds
http://ajax.googleapis.com/ajax/services/feed/load?v=1.0&q={feed}

[3] Insert new feed items

[4] Update feed signature

There are two important gotchas in this process; one is the
"signature" or offset of each feed during each sync interval so you
insert only the new items since the last sync and the other is the
feed item's timestamp for making a proper timeline and age indication.
You just can't rely on the timestamps or publish date in the feed
itself, so I use the parse timestamp in both UTC and epoch. And as a
signature I use the feed items' link of the youngest / top most feed
item, considering that this is unique for a given feed.

Because I run both processes on the server, I don't have to deal with
cross-domain issues. And everything is in well-structured JSON (both
the feed payload and NextDB queries), which makes live easy.

Enjoy! Ivo

[Eric Dorman]

Hey Ibec,

Thanks for all of that great information.

I don't mean to be a pain,but can add some code to this topic so I can
go by something?

I don't mean to sound like I need code posted all the time in a topic

I just am a visual person and I would appreciate it if you could add
some code to this topic.

Thanks & God Bless,
Eric

[ibec]

I hope this is visual enough for you: http://gist.github.com/225957

It's the complete source code of the background sync process. It uses
my Jaxer NextDB class and YQL is used for the bit.ly API requests to
shortlink the feed item links (which may not be necessary in your
case).

Enjoy! Ivo

[Eric Dorman]

Hey Ibec,

I like your source code,but it I am changing my idea to a tweetmeme
for sports.

Didn't you have a twitter project that you built for someone?

I am having trouble getting the Twitter API to work with fetching a
users status.

Thanks & God Bless,
Eric

[Geoffrey Hendrey]

Hi Eric,

Please raise your twitter issues on a Twitter forum. For general Ajax
issues you might try a jquery forum. This forum is for issues tha
relate to nextdb.

Sent from my iPhone