two questions regarding backends (configuring them and addressing them)

[Rishi Arora]

1. Configuration: When I add my first backend for the first time, and introduce the backends.yaml file for the first time, is it important that the associated version in the app.yaml file be the currently active version?

2.  Addressing: I know of three ways of addressing backends:

a) an HTTP Get/Put request to https://[backend name].[appID].appspot.com/[path to any RequestHandler defined in app.yaml] (as long as the request is from an admin when the backend is private)

b) a taskqueue.add() with URL = [path to any RequestHandler defined in app.yaml], and target=https://[backend name].[appID].appspot.com

c) a crob job defined in cron.yaml, with a fully qualitified url=https://[backend name].[appID].appspot.com/[path to any RequestHandler defined in app.yaml]

Am I correct about these alternatives?  I'm assuming only a single backend instance (specified in backends.yaml), so I have ommitted the [instance] from any URLs above.  Also, as long as my app is using one of the above 3 alternatives, am I assured that my backend won't get started by any other means?  I don't want my backend running in-advertently, because it costs money, and the new pricing scheme is forcing me (in a good way) to treat these resources as precious :)

Thanks in advance.

-Rishi

[Rishi Arora]

Turns out, in order to create and configure a backend, its not sufficient to just add it in the backends.yaml file and deploy the app.  I had to run this appcfg command to create the backend:

appcfg.py backends <location of app.yaml> update <backend name in backends.yaml>

[Steve Sherrie]

Also, you can do...

appcfg.py update --backends your_app

Steve

[Rishi Arora]

Thanks.  Yes that worked for me.  It just seems that backend documentation isn't very intuitive.  I couldn't tell from any of the online docs that simply adding a backend to backends.yaml won't configure a backend for you.  It won't show up in the backends section of the admin console until you execute appcfg.py update backends.

Another non-intuitive behavior - probably only for dynamic backends.  The admin console has a start/stop button.  But obviously, if you hit start, the dynamic backend won't actually start.  That's clear from the documentation.  But, if you hit "stop", it almost looks like that the backend is made inaccessbile.  It won't process requests, and it won't automatically restart on the next request.  You have to hit the "start" button first.  I think for dynamic backends that button should read "enable/disable".

Now the next thing I'm trying to get to work is sending a backend a request from a task queue.  I'm using the default task queue to do this, and simply doing this doesn't work (a front-end instance processes the taskqueue request):

taskqueue.add(url=[relative path to request handler], method='POST', target=[backend name from app.yaml])

Modifying the above statement to specify the entire URL for the backend in the "target" parameter does not help either - the taskqueue request is still processed by a front-end instance.  I also tried just getting rid of the "target" parameter, and specifying the entire url in the "url" parameter i.e. url='https://backend.appid.appspot.com/path/to/requesthandler'.  And even that did not help.  I fear forcing backends to process cron entries defined with a full URL like above won't do the trick either.

So is there no way to force a backend to process a request?

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/gvpp7TaOlykJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Steve Sherrie]

I'm able to direct tasks to the backends using the target arg as in your example. And the backend state is 'start' when you're trying this?

Are you directing toward a specific backend instance (N.backend_name) or just to backend_name? I am doing the latter.


Steve

[Rishi Arora]

I came across this (http://code.google.com/appengine/docs/python/taskqueue/overview-push.html#Push_Queues_and_Backends).  Which suggests that I should be able to use a specific backend instance.  Not only was the backend started (really, enabled), but it was up and running too, from serving up a request recently.  In fact, I'm enqueueing the taskqueue request from the backend itself (the backend logs prove it too).  I tried your method too (using just the backend name, and no instance prefix), and that didn't work either.  The only difference is that I'm not passing any "params" parameter in the taskqueue.add() call.  The request is just meant to be a trigger and does not carry any data.  Perhaps the app engine scheduler thinks that my backend is busy (cause it was processing the current request which enqueued the new taskqueue request), and that causes it to send the request to a front-end instance.  Maybe I should send the taskqueue rquest with a "countdown" set to a comfortable 10 seconds or something.

Here's the snippet from the link above:

Push tasks typically must finish execution within 10 minutes. If you have push tasks that require more time or computing resources to process, you can use App Engine Backends to process these tasks outside of the normal limits of App Engine applications. Backends are addressable. When you create the push task, all you need to do is address it to the URL of the backend you wish to process it. The following code sample demonstrates how to create a push task addressed to an instance 1 of a backend named backend1:

from google.appengine.api import taskqueue

...

    def post(self):

        key = self.request.get('key')

        # Add the task to the default queue.

        taskqueue.add(url='/path/to/my/worker/', params={'key': key}, target='1.backend1')

[Rishi Arora]

Found my problem.  Looks like I must execute that appcfg.py update backends command each time I make a code change.  It seems obvious now, but of course, it never occurred to me!  I'm all good now, even with countdown set to 0, and tasqueue target set to either instance.backend_name or just backend_name.

It also might seem silly that I'm making my backend queue up requests for itself (in an attempt to break up requests), when in fact it could process the whole big request inline in one shot.  But its actually common code that might also get executed by a front-end in some circumstances.  I should look into a run-time condition that I can use to determine whether I'm in the context of a front-end or a backend, and queue up requests to myself only for front-end instances.  Do you know if there's a way to do that?

Thanks for your help!

Regards

-Rishi

[Steve Sherrie]

Oh yeah, I guessed the --backends flag on appcfg.py update was intended as a replacement for running a separate backends update each time.

Steve

[Jay Meydad]

Rishi and all,

I followed what's in the thread but things are still not working for me. Perhaps someone could assist / clarify exactly how things are defined on your end. 

In my app, all user-facing requests should be handled by the front-end code & the backend, named 'worker', should handle a cron job and a task (performed by a push queue). However, referencing the backend by its name ('worker') in cron.yaml, queue.yaml and when enqueuing a task, does not make things work (see code below). In the admin console, I see that tasks are added to the queue but they are not processed. The log has the following error:

2011-09-26 06:59:36.283 /_ah/start 302 505ms 420cpu_ms 0kb instance=0

Process terminated because it failed to respond to the start request with an HTTP status code of 200-299 or 404.

Questions:

1. How did you end up referencing the backend? full path? [instance.backend_name]? [backend_name]?

2. When deploying the code, did you have to update the app's code as well as the backends? adding '--backends' to the Launcher does not seem to work. I can not see it in the launcher's documentation as well.

3. Any idea why I am getting this error? I don't use the 'start' directive.

4. Any idea why the task queue is not processing the queued tasks?

Here are some code/config snippets:

* backends.yaml looks like:

backends:

- name: worker

  class: B2

  instances: 1

  options: dynamic

* cron.yaml :

 

cron:

- description: hourly data fetch

  url: /admin/datafetch

  schedule: every 1 hours

  target:worker

* queue.yaml :

queue:

- name: relateditems

  target:worker

  

* and in the python code I used target='worker' when enqueuing a task:

taskqueue.add (url='/admin/fetchdata', params='kw=apple', target='worker')  

[Ugorji Nwoke]

Not sure if this helps, but yaml requires a space after the : in a mapping. ie. use target: worker, not target:worker

Yes, Yaml can be picky. The parser used here may or may not be lenient, so it's safer to be compliant.

[Jay Meydad]

Thanks for the suggestion. I fixed the yaml files however the tasks are still queued and not being processed.

To clarify, the /_ah/start error occurs only if I force the task to run from the Task Queues viewer. 

[Will]

I use cron job to drive my backend. As soon as I put an empty handler
to /_ah/start, it started to work.

HIH,

Will

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/MJngtxIzjf4J.

[Jay Meydad]

What do you mean an empty handler? Can you provide a sample please?