Developer documentation/examples

[Cybertizzen]

Hi,

I've checked out the latest source code from GIT, and so far I'm very
impressed with the Reef platform, based on the documentation and
technical design. However, I wish there would be more examples and
tutorials apart from the ones found in the client-api docs.

More specifically, what I am trying to understand, is how to simulate
a network of event producers and consumers communicating through the
reef middleware.

Is there any of the examples that covers this?

Regards,
Pål

[Cybertizzen]

I've managed to run the intergration sub-project against a local Reef
installation, but I'm having trouble running it against a remote
server, running Reef.

I suspect there is some configuration setting that needs to be changed
on the remote installation in order to expose the amqp queue to the
outside world.

Any tips regarding this?

Cheers,
Pål

[Sam Hendley]

Look at:

https://github.com/gec/reef/blob/master/integration/src/test/java/org/totalgrid/reef/integration/helpers/ReefConnectionTestBase.java#L93

For the java project we are not loading the same org.totalgrid.reef.amqp.cfg config file as we use for the scala parts of the project. This means it is necessary to set the host manually using system properties.

 

Assuming qpid is configured on the standard port you should be able to add the following argument:

-Dreef_node_ip=[your remote host ip]

In fact that java project doesn't even use our standard naming conventions for the reed servers ip and port. If you run the other set of integration tests (in api-request) you'll need to set the other parameter:

-Dorg.totalgrid.reef.amqp.host=[your remote host ip]

This is a shortcoming we have noticed and have actually been working in the past week or so. Since SSL support was added it has become even more of an issue, another 5 parameters are needed in that case. We are working to expose and document the utility classes needed to load the configuration from a number of sources to the java clients.

Hope that helps, let us know if you have any other issues. Did you have any problems with the installation we should know about?

Sam Hendley

[Sam Hendley]

Can you be more specific about the "event producers and consumers" you are interested in. What sorts of data are you hoping to distribute?

We exercise alot of the system in the main services code and may be able to point you to some "production" examples with more details.

Sam

[Pål Evensen]

Hi Sam, and thank you for the quick response.

Specifically, I'm trying to evaluate Reef in the context of an
automated meter reading system for a power company.

The producers would then be the meters themselves, transmitting data
in DLMS or a similar application layer protocol, and the consumers
would be services subscribing to these readings.
Basically, I'm trying to set up a simulation of this scenario, using Reef.

Pål

[Pål Evensen]

Thanks for the reply. I did actually change the parameter reef_node_ip
in ReefConnectionTestBase.java (I forgot to mention that in my
original post), but I cannot get a connection to the remote broker.

I've done a tcptraceroute on port 5672 against this, and loaded the
config samples/integration/config.xml on the remote machine, where the
traceroute tells me that the connection is denied at the remote host.
This is why I suspect that there is some change in configuration
needed on the remote site. There is no firewall running on that
machine.

[Kaplan, Michael A]

[Sam Hendley]

Hi Pål, that is surprising, it sounds like it should be working. I would check that QPid on the remote machine is bound to the correct interface. I didn't think it defaulted to 127.0.0.1 but that would explain the issue. Check netstat and make sure qpid is bound to all interfaces (0.0.0.0).

$ sudo netstat -atpn | grep qpid

tcp        0      0 0.0.0.0:5672            0.0.0.0:*               LISTEN      1038/qpidd

I would also double check you don't have a firewall running. I was positive that a server I was configuring had no firewall installed and it can be tricky to determine one is blocking your traffic.

In regards to your application, what we would normally suggest is that you create/port/interface a ProtocolAdapter for the DLMS protocol which publishes the interesting measurements onto reef and then use the standard measurement subscriptions mechanisms to attach your consumer applications. This could be simulated by creating a model that somewhat represents your system in the xml format and then letting the default simulators create some random walk data for you. Then you can hook up your prototype applications and see if reef is the right sort of solution for you. If you like what your seeing we can help you make some custom simulators/protocol adapters.

Good luck and let us know how it's going and what you think of the system.

Thanks

Sam

[Pål Evensen]

Turns out that qpid isn't running on the remote machine.

In karaf.log, the following lines is repeating:

09 Aug 2011 10:48:34,418 ERROR [Thread-48]
totalgrid.reef.measproc.ProcessorEntryPoint$$anon$2
(ApplicationEnroller.scala: 103) Error getting auth token. Request
failed with status: RESPONSE_TIMEOUT msg:
09 Aug 2011 10:48:34,418 ERROR [Thread-43]
totalgrid.reef.entry.FepActivator$$anon$3 (ApplicationEnroller.scala:
103) Error getting auth token. Request failed with status:
RESPONSE_TIMEOUT msg:
Aug 9, 2011 10:48:34 AM org.apache.karaf.main.SimpleFileLock lock
INFO: locking
Aug 9, 2011 10:48:35 AM org.apache.karaf.main.SimpleFileLock lock
INFO: locking
Aug 9, 2011 10:48:35 AM org.apache.karaf.main.SimpleFileLock lock
INFO: locking
Aug 9, 2011 10:48:36 AM org.apache.karaf.main.SimpleFileLock lock
INFO: locking
09 Aug 2011 10:48:36,420 INFO [Thread-48]
totalgrid.reef.app.SystemProperty$ (SystemProperty.scala: 28)
reef.user not defined, defaulting to: system
09 Aug 2011 10:48:36,421 INFO [Thread-48]
totalgrid.reef.app.SystemProperty$ (SystemProperty.scala: 28)
reef.user.password not defined, defaulting to: -system-
09 Aug 2011 10:48:36,422 INFO [Thread-48]
totalgrid.reef.app.SystemProperty$ (SystemProperty.scala: 28)
reef.user not defined, defaulting to: system
09 Aug 2011 10:48:36,422 INFO [Thread-48]
totalgrid.reef.app.SystemProperty$ (SystemProperty.scala: 28)
reef.user.password not defined, defaulting to: -system-

I used the sprinkle install for Ubuntu. Funny thing it worked on my
local Ubuntu.

Regards,
Pål

On Mon, Aug 8, 2011 at 4:52 PM, Sam Hendley

[Pål Evensen]

I've managed to get both the Java and C++ version of Qpid running on
the remote host, but whichever version I run, I get the following
error when trying to connect with user 'core' and password 'core' from
the clients in the integration project:

org.totalgrid.reef.japi.ResponseTimeoutException: Failed to get auth
token for user: core -

On the remote host, I've loaded the integration config and activated the node.

How does the Qpid broker and Reef connect regarding authentication? I
was under the impression that Qpid looks up the authentication tokens
in the postgresdatabase provided. Is this wrong?

Regards,
Pål