Any suggestions for migrating an application from Google Apps Script to Google App Engine?

386 views
Skip to first unread message

Joshua Mulloy

unread,
Aug 3, 2011, 12:33:35 PM8/3/11
to google-a...@googlegroups.com
Hello,

I have been creating an application in Google Apps Script but have run into a wall with Session.getUser() disabled for us Education Edition users.  I would still prefer not to serve the application from our own servers, but that is a known system and I am running out of time to develop a solution.  I am looking into rewriting the app in GAE and was hoping for tips and recommendations.  

I like Python better but I wonder if I might be able to salvage some of the code from GAS if I use Java instead.  Is there an easy way to migrate the spreadsheet data to the datastore or should I just use csv?

I am nervous about the quota system.  Storage size is easier to predict but bandwidth and CPU has me concerned.  Are there any examples of applications and the resources they used?

I have setup a GAE account as myself, should I create it under the role-based account I was writing the GAS instead?

Thank you for any guidance you can provide!

Robert Kluin

unread,
Aug 4, 2011, 1:26:16 AM8/4/11
to google-a...@googlegroups.com
Hi Joshua,
Google Apps Script is just running JavaScript on the client isn't
it? Unless you're going to try doing something clever with Rhino (or
some other similar solution) I'm not sure Java will offer you any more
than Python.

Have you looked at the gdata-apis
(http://code.google.com/apis/gdata/)? Once your app is running on GAE
you can access spreadsheets stored in Google Docs using the gdata
libs. Of course, depending on your app, it may be easier to directly
use the datastore instead of a spreadsheet.

It is very hard to say anything about what your app may cost to run
on GAE. Also don't forget the pricing will soon be changing, no one
yet knows how it will change things (other than becoming more
expensive).


Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/dbBcc3ZF4HoJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

Rajkumar Radhakrishnan

unread,
Aug 4, 2011, 2:39:07 AM8/4/11
to google-a...@googlegroups.com, jmu...@sau4.org
Hi Joshua,

I am trying to make it easier to get a database app up and running on Google App Engine without having to write code. 


If iFreeTools Creator meets your requirements, you also have the option to go in for a single-tenant deployment (iFreeTools code will be deployed into your Google App Engine app instance). Apart from enabling you to use the free quota, you can further purchase additional resources directly from Google.

It is true that storage is really very affordable and you will have to look out for CPU usage. iFreeTools can automatically cache views if they are going to take more CPU (enabled in our free plan, disabled by default in single-tenant deployment). The single-tenant deployment of iFreeTools offers free-trial which will enable you to get an idea of the resource usage and billing, before you can commit on taking this path.

Thanks & Regards,
R.Rajkumar

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/dbBcc3ZF4HoJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.



--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Build online database applications, over Google App Engine.
iFreeTools Creator - http://creator.ifreetools.com


Gary Frederick

unread,
Aug 4, 2011, 7:50:19 AM8/4/11
to google-a...@googlegroups.com
Howdy Joshua,

I read your question and was thinking about it when the announcement that Google is open sourcing App Inventor came out
and got tied up :-)

Google Apps Script (GAS) uses JavaScript running on the server, Docs or Sites or ...

I think Robert's suggestion is what I would do. I did a simple test of the Licensing API for Google Chrome Web Store that did the authentication in Python (and the Java one for fun) and then the app did various HTML5 things, not even GAS. It was not complex, but may give you an idea.

and there are various choices to get at data. I am currently planning on testing GAS with some of the data in Fusion Tables. I do not know if that makes the authentication any easier but it is yet another choice rather than migrate the data.

and JavaScript looks pretty much like Java or Python to me. Go with which you are more comfortable with if that is what you do.

and let us know what you end up doing.

Gary


Joshua Mulloy

unread,
Aug 4, 2011, 2:57:48 PM8/4/11
to Google App Engine
Hi Rajkumar,

Thank you for this information. I checked out ifreetools and I think
I like the concept but I am still a bit fuzzy on your business model
and some of your terminology. In one of your blogs you mentioned you
were working on a video introduction. Has that been setup? It might
help me understand how it works and whether it would work for me.

On Aug 4, 2:39 am, Rajkumar Radhakrishnan <r.rajku...@gmail.com>
wrote:

Rajkumar Radhakrishnan

unread,
Aug 4, 2011, 11:57:56 PM8/4/11
to google-a...@googlegroups.com
Hi Joshua,

Glad that you like the concept. Since the user-interface is being upgraded with new features, I did not want to end up with an out-dated video. I will have to do a quick-one for use in the short-term, I guess. Will let you know once this is up.

Thanks & Regards,
Raj

Rajkumar Radhakrishnan

unread,
Aug 5, 2011, 8:30:36 AM8/5/11
to google-a...@googlegroups.com
Hi Joshua,

Here it is..

And the first screen-cast (no audio, but should be easy to follow)..
Creating custom online database apps over Google App Engine, using iFreeTools Creator 

For other specifics you can check out the upcoming user guide..

and our blogs, where specific features are detailed..

You can also get in touch with me directly for any further assistance in using the app.

Thanks & Regards,
Raj

Joshua Mulloy

unread,
Aug 5, 2011, 8:35:26 AM8/5/11
to Google App Engine
Hi Robert and Gary,

Thank you for your input! I have been frantically trying to decide
between Python and Java. I have done most of my work with PHP/SQL/
JavaScript. I have just been getting into Ruby on Rails when I
started wanting to reduce the number of servers we maintain.
Switching the school over to Google Apps was the first step. I wear a
lot of hats responsible for pretty much the whole stack (networking,
database, programming, design, server admin, training, etc.). I like
the idea of sleeping at night not worrying if a raid on one of my ten
year old servers is about to fail and bring down a mission critical
app. When I discovered GAS I started a project to migrate an old
FileMaker application that was used to track professional development
for educator renewals. It was an old, difficult to maintain, poorly
written application that required licensed client installs and a
dedicated server to keep running. I suspect this app will not be much
of a load if I can get it up and running on GAE.

I find the Java syntax more familiar but I haven't programmed much in
either language so I think I am going the python route. I have wanted
to have a good reason to learn python for a few years. The code feels
kind of naked, no comforting semi-colons at the end of lines or braces
wrapping my if elses ... but it looks fast. Right now, I need fast
and I like the brevity and clean style of code examples. I think at
least for this app I will just stick with the webapp framework and use
the django templates. Part of the reason I decided to go with python
is the inclusion of django which should decrease development time like
ruby on rails. I am not comfortable with django/gae integration until
I have time to understand what works and what doesn't. I liked the
Python/webapp/django template setup from the "Introduction", from what
I have read it seems mature and well integrated, which means
dependable to me.

Thank you for passing on the gdata-apis link I looked over it last
night. I think using the datastore is a wise choice. There were many
limitations trying to use Google Spreadsheet as a datastore, it will
be nice to be free of that. Is there a datastore browser similar to
the mysql-query-browser? Is there a way to link directly to the
datastore from an ETL tool?

I should have the Google App Engine book by Dan Sanderson in my
mailbox today which will hopefully help me with some of my newbie
questions.

Thank you!

Joshua Mulloy

unread,
Aug 5, 2011, 9:03:06 AM8/5/11
to Google App Engine
Hi Raj,

I watched the video and it really helped. For this project, I think I
am going to just code it directly but I will keep ifreetools in mind
for future projects. I am curious if there is a way to have
ifreetools extract dropdown list data from the datastore rather than
just a comma separated list in the setup.

On Aug 5, 8:30 am, Rajkumar Radhakrishnan <r.rajku...@gmail.com>
wrote:
> Hi Joshua,
>
> Here it is..http://www.youtube.com/user/ifreetools
>
> And the first screen-cast (no audio, but should be easy to follow)..
> *Creating custom online database apps over Google App Engine, using
> iFreeTools Creator *http://www.youtube.com/watch?v=WT1EPEZ7OCc
>
> For other specifics you can check out the upcoming user guide..http://help.creator.ifreetools.com/
>
> and our blogs, where specific features are detailed..http://blogs.ifreetools.com/
>
> You can also get in touch with me directly for any further assistance in
> using the app.
>
> Thanks & Regards,
> Raj
>
> On Fri, Aug 5, 2011 at 9:27 AM, Rajkumar Radhakrishnan <r.rajku...@gmail.com
>
>
>
>
>
>
>
>
>
> > wrote:
> > Hi Joshua,
>
> > Glad that you like the concept. Since the user-interface is being upgraded
> > with new features, I did not want to end up with an out-dated video. I will
> > have to do a quick-one for use in the short-term, I guess. Will let you know
> > once this is up.
>
> > Thanks & Regards,
> > Raj
>
> > iFreeTools Creator -http://creator.ifreetools.com

Gary Frederick

unread,
Aug 5, 2011, 9:03:48 AM8/5/11
to google-a...@googlegroups.com
Joshua,

Sounds to me like you are making good choices.

I do not have a good opinion on Java or Python, Python may... be a bit easier at first.

I got into Java way before GAE and Python and it's web app stuff. I have done more Python with GAE
and feel that some day I will want to switch to Java cause it may be 'bigger'???

Good luck.

I am just starting to play with GAS. I like.

Gary

Rajkumar Radhakrishnan

unread,
Aug 5, 2011, 9:25:19 AM8/5/11
to google-a...@googlegroups.com
Hi Joshua,

Yes, you can have reference (look up fields), which can have a drop down (if there are limited values, else a pop-up) based on another record-type in the database. You can have a look here..
http://help.creator.ifreetools.com/p/defining-attributes-or-fields.html#Adding Lookup (Reference) field

Even if you are not going to use iFreeTools for this project, it is fine. Hope to win you over some other time.

By the way, Python is what is being for my app.. it was the only option available when I got started with GAE, so I had to pick it up (after over 8 years in Java world), but now have no plans to go back to Java. I just love the Python.

There are some other aspects which you will have to consider. One.. the start up time for Java apps can be a bit slower, so you may have pay for some warm instances ready to serve requests. Another quick point is Python on GAE currently does not have multi-threading and the proposed pricing changes (at least, in my opinion) gives an unfair advantage to Java apps. The currently suggested remedy is to make use of half-priced instances. That would be okay if Java instances were to process at most 2 threads concurrently, which I believe will not be the case. So, I am somewhat uneasy to have made the Python choice post the new pricing proposals, but hope things will be set right before it actually comes into effect. 

Thanks & Regards,
Raj

Robert Kluin

unread,
Aug 5, 2011, 3:25:53 PM8/5/11
to google-a...@googlegroups.com
Hey Joshua,
Sounds like you've thought it through pretty well. All I have to
say is: Python, Python!

Seriously though, I came from a C with some C++ and C# background.
The lack of braces and semicolons was hard to accept at first. What I
can tell you for sure is that when I write C-type language code now
all I can think is "what is with the unnecessary markup?!" It is
crazy nice to work in, once you get past a few nuances.

Not to throw our more flame-bait, but I also agree with your choice
to skip django. Use webapp, or webapp2
(http://code.google.com/p/webapp-improved/). Personally I'd also
suggest finding another template language too, but that is personal
preference.

I'm not sure if you've got some specific tool in mind, but you can
create what ever API you might need for interacting with your data.
There is also the bulkloader tool to upload / fetch data.


Robert

Joshua Mulloy

unread,
Aug 8, 2011, 11:27:21 AM8/8/11
to Google App Engine
Thank you Gary,

I feel I have a pretty good bearing now thanks to help here. While it
was intimidating at fist, I think it is awesome that there are so many
choices available.

Now that I have skimmed the Programming Google App Engine book I can
answer one of my own questions. There is a query-browser called
Datastore Viewer in the App Engine Account.

I am still not certain if I can "connect" like I can with a MySQL
database, or whether I can extract and load directly to the datastore.

Mani Doraisamy

unread,
Aug 8, 2011, 11:45:43 AM8/8/11
to Google App Engine
Joshua,
You might want to take a look at http://www.orangescape.com, if you
are interested in visual basic kind of application development on app
engine.

thanks,
mani

Joshua Mulloy

unread,
Aug 10, 2011, 4:11:57 PM8/10/11
to google-a...@googlegroups.com
I definitely agree with you on the python vs c markup.

I just came back to post on my progress and read your post.  After reading your post, I am not sure I made the right decision (actually using more of the Django framework).  You sound very opposed to Django.  If you are willing to share, I would be very interested in your reasons.  I found Django-nonrel and two things made me decide to use the model framework.

First, I really liked the Model syntax.  I think it was easier for me to understand because they provided the related SQL code in their book http://www.djangobook.com/en/2.0/chapter05/.  I was very frustrated by the lack of comparison examples in Programming Google App Engine.  For example, instead of just mentioning (page 197) "This technique is similar to how you'd use 'join tables' in an SQL database." an actual mapped example would probably have made that entire chapter immediately mentally digestible.

Second, I like that I can take the code and plop it somewhere else if necessary.  I got burned from an unexpected limitation in GAS.  If I run into another road block with GAE, I can take my code and run it on my server here with minor adjustment to the code.

I have basically setup the data model for my app and the Django admin interface already has at least as much functionality as the old app.   In looking at it, I could probably just put that up and give out one login and volunteers could start entering data (basically what they had before but needed a client).  Then I could start "adding" features like: users inputting their own data instead of volunteers copying it from a paper form.  There is still a ton I need to do and better understand.  But now, I at least have an interface I can show to the PD committee and my boss and that is a big relief.  I now have a bit more time to plan and consider alternatives, time I didn't think I would have this morning.

Robert Kluin

unread,
Aug 11, 2011, 12:41:06 AM8/11/11
to google-a...@googlegroups.com
Hi Joshua,
Responses inline.

On Wed, Aug 10, 2011 at 16:11, Joshua Mulloy <jmu...@sau4.org> wrote:
> I definitely agree with you on the python vs c markup.
> I just came back to post on my progress and read your post.  After reading
> your post, I am not sure I made the right decision (actually using more of
> the Django framework).  You sound very opposed to Django.  If you are
> willing to share, I would be very interested in your reasons.  I found

I'm not necessarily personally opposed to it, but it carries overhead
which does not always play well on app engine. It is also very heavy
and I find it rather monolithic. I personally prefer lighter, smaller
frameworks.


> Django-nonrel and two things made me decide to use the model framework.
> First, I really liked the Model syntax.  I think it was easier for me to
> understand because they provided the related SQL code in their
> book http://www.djangobook.com/en/2.0/chapter05/.  I was very frustrated by
> the lack of comparison examples in Programming Google App Engine.  For
> example, instead of just mentioning (page 197) "This technique is similar to
> how you'd use 'join tables' in an SQL database." an actual mapped example
> would probably have made that entire chapter immediately mentally
> digestible.

The problem with App Engine is that you'll get into trouble by
thinking in terms of SQL. It is not SQL, and your *really* should try
to not think in SQL-like terms. Instead think simple data structures
and denormalize the hell out of stuff, precisely the opposite of
typical RDMS thinking.

> Second, I like that I can take the code and plop it somewhere else if
> necessary.  I got burned from an unexpected limitation in GAS.  If I run
> into another road block with GAE, I can take my code and run it on my server
> here with minor adjustment to the code.

Yeah, that is a bummer. Luckily most of your code will probably be
pretty portable no matter what you choose, worst case you might need
to implement a thin datastore abstraction layer. Or you could use
TyphoonAE!


> I have basically setup the data model for my app and the Django admin
> interface already has at least as much functionality as the old app.   In
> looking at it, I could probably just put that up and give out one login and
> volunteers could start entering data (basically what they had before but
> needed a client).  Then I could start "adding" features like:
> users inputting their own data instead of volunteers copying it from a paper
> form.  There is still a ton I need to do and better understand.  But now, I
> at least have an interface I can show to the PD committee and my boss and
> that is a big relief.  I now have a bit more time to plan and consider
> alternatives, time I didn't think I would have this morning.

Honestly, I can't argue with results.

Nice to hear you chose Python. :)


Robert

>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/IyyYM6sSoHcJ.

Joshua Mulloy

unread,
Aug 11, 2011, 11:06:36 AM8/11/11
to Google App Engine
Hey Robert,

Thank you for taking the time for your thoughtful responses. The
overhead of Django does concern me and as I get more comfortable and
have more time for reflection I will definitely re-evaluate. Just
getting it up and running was a pretty rough road and I think I hit
every pothole on the way. I will also try much harder to denormalize
my thinking, I remember studying it a bit a few years ago, something
about stars, snowflakes and cubes. In desperate development mode, I
tend to stick with data structures programming design that I know. A
man's got to know his limitations.

I was under the impression that denormalization was just a technique
to improve read times. My applications generally serve a small
audience so read performance has not surfaced in the user experience;
however, I recognize this is an area I need to grow in understanding.
I hate bloated overhead and using modern processor speeds and
bandwidth increases as a crutch to hide inefficient code practices.
I would love to pass on some of my applications to larger schools or
even begin designing applications for larger audiences. Even if I
stick with Django as a framework for this project I will be looking at
ways to reduce the overhead and increase efficiency.

I will read up on TyphoonAE and what it takes to implement a datastore
abstraction. If I can still retain decent portability, then greater
code efficiency and is reason enough for me to put more effort into
understanding webapp2. Thank you again for your guidance!

On Aug 11, 12:41 am, Robert Kluin <robert.kl...@gmail.com> wrote:
> Hi Joshua,
>   Responses inline.
>

Joshua Mulloy

unread,
Aug 11, 2011, 11:20:10 AM8/11/11
to Google App Engine
Hey Robert,

You had asked if I had a particular ETL tool in mind. I use Pentaho
Data Integration which used to be known as KETTLE for moving data
around. I connects to all sorts of databases and makes transformation
of data a snap. I haven't figured out how to connect it to a Google
App Datastore instance but I will keep looking and post back if I find
a way.

On Aug 11, 12:41 am, Robert Kluin <robert.kl...@gmail.com> wrote:
> Hi Joshua,
>   Responses inline.
>

Robert Kluin

unread,
Aug 12, 2011, 1:10:20 AM8/12/11
to google-a...@googlegroups.com
Hey Joshua,
I'm not at all familiar with that tool, but i'd guess unless you can
find some type of adapter someone else has written you'll probably
need to write something. Maybe Pentaho can use REST services (or some
other service) which would make your life easier. ;)


Robert

Joshua Mulloy

unread,
Aug 15, 2011, 1:55:13 PM8/15/11
to Google App Engine
I do not see anything, I will probably just stick with manual CSV or
XML transfers. I was thinking of using it to load all the current
data and later to pull down data for report generation, but I am sure
I can figure out alternatives. If a real need arises I can setup some
sort of automated procedure to generate and transfer the CSV or XML
files.
Reply all
Reply to author
Forward
0 new messages