Gmail Calendar Documents Reader Web more »
Recently Visited Groups | Help | Sign in
Google Groups Home
Importing external data [Python]
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  4 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
benji  
View profile  
 More options Oct 27 2009, 12:35 pm
From: benji <bco...@gmail.com>
Date: Tue, 27 Oct 2009 09:35:23 -0700 (PDT)
Local: Tues, Oct 27 2009 12:35 pm
Subject: Importing external data [Python]
Hi,

I am brand new to GAE and Python and have very limited programming
skills thus far, so this is a cry for help after spending the last few
days trawling google for a solution :(

I want to import, parse and embed an RSS feed from a Google
spreedsheet. I have done this before in a PHP script that simply acts
as a proxy to parse RSS into HTML and used a Server Side Include to
embed the output of the PHP script - nice and easy and SEO friendly.

So far I currently only have a very simple static set up on GAE as
follows:

handlers:

- url: /

  static_files: assets/index.html

  upload: assets/index.html

- url: /

  static_dir: assets

I thought I may have 3 different potential solutions,

1) somehow point a url at an external domain that can handle PHP -
thought it was worth a try in the YAML file - wishful thinking!!

ie.

- url:/PageNeedingRSSProxyData

static_dir: http://otherdomain.com/PageWithRSSProxyData

2) Grabbing the output of the external PHP file with URL fetch API and
caching it, then including the file in the HTM using django templates
- i looked at this and got very lost.

3) Grabing the RSS feed directly and parsing it with Python and
embedding it in a web page - again got very lost.

Can anyone point me in the right direction? or am i too far out of my
depth?

Any help on this would be awesome and very much appreciated.

Cheers

Ben


    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
ryan baldwin  
View profile  
 More options Oct 28 2009, 10:27 am
From: ryan baldwin <ryanbald...@gmail.com>
Date: Wed, 28 Oct 2009 08:27:20 -0600
Local: Wed, Oct 28 2009 10:27 am
Subject: Re: [google-appengine] Importing external data [Python]

Point 3 seems like the proper choice (along with some caching). Where are
you getting lost? In how to parse with python, how to cache, or how to
render in a template (or all of the above?)

- ryan.


    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
benji  
View profile  
 More options Oct 28 2009, 10:47 am
From: benji <bco...@gmail.com>
Date: Wed, 28 Oct 2009 07:47:12 -0700 (PDT)
Local: Wed, Oct 28 2009 10:47 am
Subject: Re: Importing external data [Python]
Thanks Ryan,

I thought point 3 seems like the proper direction to go in, thanks.

I can handle putting together a simple django template I would say -
seems straight forward enough. However I'm stuck from there on in!
i.e. the python based importing of the RSS feed, cache, parsing to
html elements and output into the template...

I think URL Fetch seems to be the way to go from what I can work out
form the documentation as this has in built caching facility - right?

I got this functionality working with JSON using JS code provided by
google code examples - but the requirement is to embed the data for
SEO purposes and of course implementing a cache for
unobtrusiveness....

where do I start?

Thanks again for the help, really very much appreciated!

Cheers

Ben

On Oct 28, 2:27 pm, ryan baldwin <ryanbald...@gmail.com> wrote:


    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
ryan baldwin  
View profile  
 More options Oct 28 2009, 10:59 am
From: ryan baldwin <ryanbald...@gmail.com>
Date: Wed, 28 Oct 2009 08:59:20 -0600
Local: Wed, Oct 28 2009 10:59 am
Subject: Re: [google-appengine] Re: Importing external data [Python]

Yea, URLFetch has some builtin caching, however that will only cache the
response of the URLFetch, it won't cache all the parsing/processing of the
RSS feed (which would be a better thing for you to cache in Memcache).  The
idea of Memcache is fairly simple: Look for the item in memcache by some key
that you know about, if the item isn't in memcache then put the item in
memcache (in your case, urlfetch the rss, parse it, and throw it in
memcache) using the same key you tried to get it with.  The Memcache pattern
article can be found here:
http://code.google.com/appengine/docs/python/memcache/usingmemcache.h...

As far as parsing/processing the RSS feed... there's a million ways to skin
a dead cat. Personally I would be inclined to write an xslt transformation
that does the RSS to HTML for me, but if you're a beginner then xslt is
probably not a good option.  There's an opensource RSS feed parser for
python lib that you can use - http://www.feedparser.org/ - although I
haven't used it. A quick looks as though it converts the feed into a
dictionary.  You could then pass this dict to whatever django template
you're using.

Those are some high level ideas for you. Hopefully it's enough to get you
started.

Good luck!

- ryan.


    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »

Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2010 Google