online webapp for fem

[Ondrej Certik]

Hi,

I'd like to brainstorm some ideas about webapps.

we currently use Sage notebook for allowing online access to hermes
and this is a good start, because it allows us show almost anything,
because it's essentially just a Python console, and it can show any
images and soon videos too.

One slight problem is that we have to maintain the server, and it
already happened, that for some reason it stopped working and we had
to call someone at UNR to restart it. No doubt this will happen again
sooner or later.

We can (and we will) try to make sure it doesn't happen often and have
some backups servers, that we can just plug in in case something
happens, but I still think if we can leverage the Google app engine
for the main app, it will be much easier for us in terms of the
maintaining. However, just rewriting the Sage notebook to run on the
appengine will introduce some delays, because the main computation
engine cannot run on the app engine (only pure python code and there
are limitations like that every code must finish in 30s etc), e.g. we
would still have to maintain the actuall worker (engine) with our
code, but at least we don't have to maintain the frontend. I don't
know if this is worthy.

Nevertheless, I played with appengine some more and I wrote this app:

http://repos.sympy.org/

it's a showcase of what a webapp can do and it all runs on the
appengine, e.g. google takes care of everything. It starts with
putting there some repository on github, e.g. my hermes2d repository:

http://repos.sympy.org/hooks/repos/agZzeW1weTJyEQsSClJlcG9zaXRvcnkYmwsM/

it then uses appengine task queue and GitHub web API to poll the
repository to retrieve a list of all contributors, finds their email
addresses and uses gravatar.com to get a picture. hermes2d repo is
boring in this sense, but see some other repo:

http://repos.sympy.org/hooks/repos/agZzeW1weTJyEQsSClJlcG9zaXRvcnkYmRMM/

anyway, the offline computation is cool (but 30s limit still
applies!), as google gives you about 10000 tasks a day (for free, if
you need more, you need to pay) and one task is essentially just a
request to some URL in the webapp. So with those emails it works in a
way that I create a task for each email and then google polls my URL,
which triggres some Python code, in which I ask GitHub for the email
and update the database -- if I am successful, google deletes the task
(as done), if some exception gets raised (e.g. I return anything else
besides HTTP 200 OK return code), google will try to get the same task
done again, until it succeeds.

So this could be used for example in this way:

* user setups some computational problem and hits solve
* webapp creates a task for this, google polls some URL, my code gets
triggered, which polls *our* server with femhub, and it starts doing
the calculation (if our server is down, then this task will fail and
google will retry later)
* our server finishes calculation and polls the webapp, uploads the
result (webapp stores it in the database on the google app engine)
* webapp shows the result to the user

One advantage of this approach would be that the webapp would queue
all solving tasks etc and if we have some problem with our servers,
all we have to do is to restart them, we don't have to maintain any
database, nothing, google takes care of everything, we take care of
the actual computation.

I am just throwing ideas here. Maybe Pavel or anyone else has some
nice vision, or something, what could be done.

Ondrej

P.S. examples of the web hooks (e.g. what goes between the github and
the webapp) are here:

http://repos.sympy.org/hooks/repos/agZzeW1weTJyEQsSClJlcG9zaXRvcnkY3AoM/

Go to Updates, and see the "JSON payload" table.

[Pavel Solin]

Ondrej,
  maybe I am lacking some information but I do not
see any real benefit of adding Google to the workflow
(1) problem definition in our web notebook
(2) calculation at our computer at UNR
(3) visualisation in our web notebook
I see that we cannot do queing, but do you think
that the user would queue his tasks and be willing to
wait if our server is down?

Pavel

--
Pavel Solin
University of Nevada, Reno
http://hpfem.math.unr.edu/people/pavel/
Hermes project: http://hpfem.org/
FEMhub project: http://femhub.org/

[Ondrej Certik]

On Sun, Jul 19, 2009 at 12:09 AM, Pavel Solin<so...@unr.edu> wrote:
> Ondrej,
>   maybe I am lacking some information but I do not
> see any real benefit of adding Google to the workflow
> (1) problem definition in our web notebook
> (2) calculation at our computer at UNR
> (3) visualisation in our web notebook
> I see that we cannot do queing, but do you think
> that the user would queue his tasks and be willing to
> wait if our server is down?

No, probably not. It was just an idea, no problem if nothing comes out of it.

Ondrej

[Sameer Regmi]

On Sat, Jul 18, 2009 at 10:15 PM, Ondrej Certik <ond...@certik.cz> wrote:

We can (and we will) try to make sure it doesn't happen often and have
some backups servers, that we can just plug in in case something

happens,[...]

 

Liked the idea of backup servers. How easy it is to setup on a backup server? I wanted to test it on my server (shared) if you give me instructions to do so.

Sameer

[Sameer Regmi]

On Sat, Jul 18, 2009 at 10:15 PM, Ondrej Certik <ond...@certik.cz> wrote:

We can (and we will) try to make sure it doesn't happen often and have
some backups servers, that we can just plug in in case something

happens,[...]

[Ondrej Certik]

Just install the source or binary femhub distribution, run notebook()
and forward it to apache/lighttpd. More instructions are here:

http://code.google.com/p/spdproject/wiki/SPDServer

(e.g. for added security, you should setup users etc.)

See also here how to setup it into a virtualbox:

http://wiki.sagemath.org/SageVirtualBox

Ondrej

[Ondrej Certik]

Any progress?

Ondrej

[Sameer]

Not any. Because my shared hosting provider did not allow full shell
support. I could not create users as described in
http://code.google.com/p/spdproject/wiki/SPDServer

On Jul 27, 5:05 pm, Ondrej Certik <ond...@certik.cz> wrote:
> On Tue, Jul 21, 2009 at 1:48 PM, Ondrej Certik<ond...@certik.cz> wrote:

[Ondrej Certik]

On Mon, Jul 27, 2009 at 10:16 PM, Sameer<reg...@gmail.com> wrote:
>
> Not any. Because my shared hosting provider did not allow full shell
> support. I could not create users as described in
> http://code.google.com/p/spdproject/wiki/SPDServer

I use linode.com, that works well.

I suggest you set it up on you laptop first and if it works, we can
set it up on our servers.

Ondrej

[Sameer Regmi]

Ok I want to set it up on laptop. So what should I do for this? Use virtualbox, or any other method?

Thanks

[Ondrej Certik]

It depends on your aim. I think you wrote above that you want to set
up backup servers. For this it's a good idea to use a VirtualBox.

If you just want to run the notebook, it's enough to type "notebook()"
in femhub.

Ondrej

[Pavel Solin]

Sameer,
  thanks for working on this!
Pavel