silently dying proxy-state

[Michael Frericks]

* Situation
1. I create an InProcessCluster
2. I create a foreign-proxy this way
#+BEGIN_SRC
(rama/foreign-proxy path pstate {:pkey p-key :callback-fn call-back-fn})
#+END_SRC
3. Changes to to data in pstate  at `path` are reported by the callback-fn
4. But after some time no further calls to callback-fn are made

Questions
1. Is this a limitation due to the use of InProcessCluster?

[Nathan Marz]

There's no limitations with proxy states with IPC.  Are you seeing logs from the cluster? You need to have a log4j2.properties file set up to see them (like how rama-demo-gallery is set up https://github.com/redplanetlabs/rama-demo-gallery/tree/master/src/test/resources).

When something goes wrong with proxy states, you should see a number of logs indicating what's going on. Besides listening for reactive callbacks, they also poll every 30s to make sure they're still connected to the leader and to inform the leader they're still listening. Anything going wrong in either the reactive or polling codepaths is logged.

--
You received this message because you are subscribed to the Google Groups "rama-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rama-user+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/rama-user/d221414b-730f-46a4-a94d-0d51f987ccc6n%40googlegroups.com.

[Michael Frericks]

Hi,

after activating the logs with the log4j2.properties, i see the following log message

#+BEGIN_SRC
19:16:37.712 WARN  [Worker bb1ad573 00000000@2005 DTT 2 0-task-group-exec-0] ject.pstate.topology - :rpl.rama.distributed.pobject.pstate.topology/expiring-proxy Expiring proxy$ state {:uuid 850fdca7-1934-4c81-9275-40c54d43984d}
#+END_SRC

None of the https://redplanetlabs.com/docs/~/all-configs.html#_foreign_configs seem to fit here

Thanks as always for your prompt response and support

Best regards

Michael

[Nathan Marz]

That means the worker did not receive a heartbeat from the proxy state within the timeout, which is 90 seconds. I wonder if your callback function on the proxy is blocking for an excessive period of time. Everything for the proxy runs on the same thread, so if your callback function is blocking it will prevent heartbeats from being sent. It's best for heartbeat functions to be non-blocking, and if they have to block to send that work to another thread.

To view this discussion visit https://groups.google.com/d/msgid/rama-user/69befba0-2140-4d5e-bf1c-2aac2a727e5an%40googlegroups.com.

[Michael Frericks]

Hi Nathan,

just to confirm.

Delegating the data processing to be done in the callback function to another thread solved the problem

Thanks