Issues Receiving C-Class Messages

[Robert Commons]

Hi All, 

I am having issues receiving C-Class messages on the public data feed since the ActiveMQ switch yesterday. I am using Gozirra/Stomp to receive the messages, I have updated the port number from 61618 to 61628. It successfully connects to 61628 but then no messages are received. Am I missing something? 

The application works if I use port 61638 (the newly added ActiveMQ classic instance). 

Any thoughts would be welcomed!

Thanks!

Rob

[Peter Hicks (Poggs)]

Hi Rob

> On 24 Oct 2023, at 11:40, Robert Commons <robert....@cogitare.biz> wrote:
>
> I am having issues receiving C-Class messages on the public data feed since the ActiveMQ switch yesterday. I am using Gozirra/Stomp to receive the messages, I have updated the port number from 61618 to 61628. It successfully connects to 61628 but then no messages are received. Am I missing something?
>
> The application works if I use port 61638 (the newly added ActiveMQ classic instance).

You don’t need to update the port - 61618 is now the Artemis port, and 61638 is what used to be 61618.

Does it work if you use port 61618?


Peter

[Robert Commons]

Hi Peter, 

No, I get the same issue using port 61618, it connects, but then no messages are received. 

Cheers

Rob

[Peter Hicks (Poggs)]

Hi Rob

> On 24 Oct 2023, at 11:49, Robert Commons <robert....@cogitare.biz> wrote:
>
> No, I get the same issue using port 61618, it connects, but then no messages are received.

Can you paste some sample code here that will reproduce the problem? I’m suspecting you might be specifying the topic you want to subscribe to in a way that only ActiveMQ understands, or you might be encountering an error but not handling it in your code.


Peter

[Robert Commons]

Hi Peter, 

Below is the code we are using to poll the messages. It executes without exception. The Client object is part of Gozirra's Stomp implementation, which I am thinking might be the issue if this is incompatible with the new ActiveMQ version. 

Thanks

Rob

String serverHost = sysParamConfigurationService.findByParamName(TD_NR_PUBLIC_FEED_SERVER_HOST);
String serverPort = sysParamConfigurationService.findByParamName(TD_NR_PUBLIC_FEED_SERVER_PORT);
Client client = new Client(serverHost,
Integer.parseInt(serverPort),
sysParamConfigurationService.findByParamName(TD_NR_PUBLIC_FEED_USERNAME),
sysParamConfigurationService.findByParamName(TD_NR_PUBLIC_FEED_PASSWORD), 60000, 60000);
logger.info("| Connected to {} :: {}", serverHost, serverPort);
String tdFeedTopics = sysParamConfigurationService.findByParamName(TD_NR_PUBLIC_FEED_TOPIC);
String[] topicNames = tdFeedTopics.split(",");
for (String topicName : topicNames) {
client.subscribe(topicName, (Listener) this);
}
logger.info(" **** CONNECTED SUCCESSFULLY && WAITING FOR TD MESSAGES");

[eta]

Are you subscribing to /topic/TD_ALL_SIG_AREA instead of just TD_ALL_SIG_AREA? My application was doing the former, and didn't work on the new platform until I changed it to the latter.

~eta

--

You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/b33c047e-e21f-47b5-b74c-76116f4ba165n%40googlegroups.com.

[Peter Hicks (Poggs)]

Hi Rob

On 24 Oct 2023, at 12:05, Robert Commons <robert....@cogitare.biz> wrote:

Below is the code we are using to poll the messages. It executes without exception. The Client object is part of Gozirra's Stomp implementation, which I am thinking might be the issue if this is incompatible with the new ActiveMQ version. 

Gozirra is so old and unmaintained that I can’t even find it on Maven Central.

Check out https://github.com/openraildata/openwire-camel-client-java for a more robust client based on Apache Camel and, if you can, don’t both with Stomp if you’re using Java (unless you absolutely must) because native OpenWire support is available.

Peter

[Robert Commons]

Hi, 

Yes, that's the problem! Thank you, much appreciated!

Rob

[Robert Commons]

Hi Peter, 

Yes I've been aware Gozirra is outdated and have been meaning to update it for a while. Thanks for the suggestion of an alternative, I will certainly investigate that and perhaps replace Stomp altogether. 

Thanks for your help!

Rob

[andrew l]

I too am seeing something similar. I'm using the go stomp library https://github.com/go-stomp/stomp which, since the ActiveMQ switchover waits forever after receiving the 3rd batch of 32 messages from the /topic/TD_ALL_SIG_AREA topic.

If I point my application at port 61638 (the old ActiveMQ classic server) then all works ok - messages come flying in. Switch back to 61618 and it hangs after receiving the 3rd batch of messages.

I'll do some more digging, but interested to see if others have had a similar experience.

Thanks

Andrew

[Evelyn Snow]

Hi Andrew

Solution in this thread was, I think, to use TD_ALL_SIG_AREA instead of
/topic/TD_ALL_SIG_AREA on the new broker

Evelyn

[Andrew Larcombe]

Many thanks for the reply. I've tried both

/TD_ALL_SIG_AREA and

TD_ALL_SIG_AREA both of which give me 'not authorized' errors (NB  /topic/TD_ALL_SIG_AREA gives me no such error)

User a*****@a********* is not authorized to read from: queue://TD_ALL_SIG_AREA

I'll dig a bit deeper tonight.

Thanks,

Andrew

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/ZToW6oq/9lbuMPUo%40evelyn.moe.

--

Andrew Larcombe
and...@andrewl.net

[andrew l]

Scrub that, I was using the broker on 61638.

Using 61618 (ie AMQ Artemis) and subscribing to topic TD_ALL_SIG_AREA just gives me the first 3 batches of 32 messages, then hangs. Same as using /topic/TD_ALL_SIG_AREA :/

Thanks,

Andrew

[Robert Commons]

Hi Andrew, 

I think I am having the exact same issue. I am using the Gozirra's Stomp implementation and my application now receives messages after making the changes to the topic name suggested above. However, it then freezes as it stops receiving messages after about a minute. 

I haven't had time to look into a solution yet, but I was planning on replacing Stomp with OpenWire as Peter suggested above. I'm hoping this will solve the problem. 

I'll let you know if I have any success. Let me know if you discover anything further!

Cheers!

Rob

[Evelyn Snow]

Hi Rob,

I've done some experiments, I set up a STOMP 1.2 client with heartbeating off and acknowledgements set to auto,
connected to 61618, then subscribed to /topic/TRAIN_MVT_ALL_TOC. After a minute, I get an ERROR frame, and the
broker hangs up. Here's the error frame in question:

ERROR
message:AMQ229014\c Did not receive data from /nnn.nnn.nnn.nnn\c55388 within the 60000ms connection TTL. The connection will now be closed.
^@

Artemis' TTL feature is meant to prevent dead clients holding connections open while doing nothing themselves,
but if you've got no heartbeating and no acknowledgements, you're sending nothing, which means that after
60 seconds, without fail, you will be disconnected.

Gozirra only supports a subset of STOMP 1.0, but heartbeating was first specified in 1.1. Despite being
specified in 1.0, it also doesn't support acknowledgements either, something the original author describes as
"gratuitous bloat" in the readme.

The code is enough of a mess that I haven't worked out whether or how it deals with disconnects, it might
or might not tell you if you set up an error listener, but you haven't done so.

In the short term, I'd suggest that one possible untested fix is modifying Gozirra to unilaterally send
heartbeats. All versions of STOMP allow any number of linebreaks between messages, part of the basis for
the feature in later versions, so what you'd need to do is send a linebreak (\r\n or \n are both legal)
to the broker every so often, within that TTL window.

In the medium (or even short) term, I'd suggest ditching the library. I've written STOMP client libraries that
can cope with the rather more demanding Darwin push port, I have an extremely low opinion on Gozirra. I don't
think it's possible to paper over the cracks and build quality software on top of this.

(I'm starting to think a wiki list of recommendations and anti-recommendations for STOMP clients might be an idea)

Evelyn

[Robert Commons]

Hi Evelyn, 

That's great, thanks very much for that, that's really useful info. 

I agree Gozirra isn't fit for purpose, the only reason we are still using it is it's legacy code that's only used in our test environments. These environments are now using the fallback port 61638 temporarily and then I will find some time in the coming weeks to replace this library altogether. 

I also agree a list of recommended libraries on the wiki would be useful.

Thanks again for your help, it's much appreciated!

Rob

[Andrew Larcombe]

Found it - bit of a wierd one. I had my stomp client configured to send ACK(nowledgment) messages. Whilst this works with the old ActiveMQ Classic instance (now on port 61638), it seemed to cause ActiveMQ Artemis (now on port 61618) to break after sending 96 messages. It never disconnected or timed out, it just seemed to...stop. Changing my client so that it doesn't send ACKs fixes the issue.

Thanks to all,

Andrew

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/bb54b22c-94e5-45a0-8e5d-3c594abea57cn%40googlegroups.com.

--

Andrew Larcombe
and...@andrewl.net

[Matthew Burdett]

Also seeing a similar thing here, stomp php client cuts out after 60 seconds TTL. Never saw any problems with the old active classic.

I am using a 20 second heartbeat in the following way:

 $con = new Stomp($server, $user, $password, array('client-id' => 'my_stomp',  'heart-beat'=>'0,20000'));

and then using $con to subscribe to topics etc.

Whether or not I include /topic/ or not doesn't matter, I still face the same problem. It's like heartbeats are not being recognised.

I am acking the messages like so in each frame:

 $con->ack($msg);

I can receive messages fine but I am getting TSR and RTPPM so sometimes the code fails in between the RTPPM messages each minute.

Has anyone got any suggestions? Not like I need either of the above and sucking up TD messages might be a short term fix but I've never had a problem in the past. 61638 works fine. I'm using port 61628 now.

Thanks 

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CAB_f8B3gSSU2HjZrcAw4dF7njs5UyEr34JoiBcsVNNriNMu6JA%40mail.gmail.com.

[Andrew Larcombe]

Try _removing_ the ack - that's what fixed it for me.

Thanks,

Andrew

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CA%2B2-wYF5UW%2BO_2vfZJyEuZH-YT8KGE%2B5Tn5OGvHEvr%3Dx0GUjNw%40mail.gmail.com.

--

Andrew Larcombe
and...@andrewl.net

[Jack Brewer]

I don’t speak PHP but activemq Artemis requires a heartbeat with a min time of 500ms, so setting 0,20000, Artemis will default to 20000,20000 but your client might not be handling this.

Jack

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CAB_f8B0e%3DgMriHWopHfQDEL3psKmYQKftsdwwWqo6eo7tyk0hg%40mail.gmail.com.

[Phil Wieland]

What version of the STOMP protocol are people running?  My home-brewed STOMP client failed with the new server because I hadn't specified the version in the CONNECT message, and the new server made a different assumption, I think.  This resulted in a connection failure after about a minute.  A quick tweak and it worked fine.

Can you see the actual messages going up and down, to work out what exactly is happening?

Phil

[Jack Brewer]

Stomp 1.0 doesn’t handle heartbeats, but activemq Artemis requires heartbeats. So if you connect with 1.0, the connection will fail unless your code handles the heartbeats. So best to specify. Personally I use 1.2 and have had no issues.

Jack

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/2a497ece-0b5a-4ead-a672-a1d9a917ce5bn%40googlegroups.com.

[Evelyn Snow]

Hi Matthew,

1. Your heartbeat header is the wrong way around, you're offering no heartbeats
client to server, but requesting heartbeats every 20000ms from the broker. It
should be 20000,0 instead of 0,20000.

2. Does the client in question actually support heartbeating? It's not a normal
design choice (albeit it's one I made in Barytherium) to check the headers and
do this transparently.

3. Is this the library described at https://www.php.net/manual/en/class.stomp.php?
Does it support 1.1 or 1.2? It looks worryingly 1.0-shaped at a glance

Evelyn

> CAB_f8B3gSSU2HjZrcAw4dF7njs5UyEr34JoiBcsVNNriNMu6JA%40mail.gmail.com.

>
> --
> You received this message because you are subscribed to the Google Groups "A
> gathering place for the Open Rail Data community" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to openraildata-t...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/
> openraildata-talk/

> CA%2B2-wYF5UW%2BO_2vfZJyEuZH-YT8KGE%2B5Tn5OGvHEvr%3Dx0GUjNw%40mail.gmail.com.

[Evelyn Snow]

Hi Jack,

This is not the case, Artemis correctly interprets a cx of 0 as an offer of no
heartbeats client-to-server, as specified.

Evelyn

> CAB_f8B3gSSU2HjZrcAw4dF7njs5UyEr34JoiBcsVNNriNMu6JA%40mail.gmail.com
> .

>
> --
> You received this message because you are subscribed to the Google
> Groups "A gathering place for the Open Rail Data community" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openraildata-t...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/
> msgid/openraildata-talk/

> CA%2B2-wYF5UW%2BO_2vfZJyEuZH-YT8KGE%2B5Tn5OGvHEvr%3Dx0GUjNw%40mail.gmail.com
> .

>
>
>
> --
> Andrew Larcombe
> and...@andrewl.net
>
> --
> You received this message because you are subscribed to the Google Groups
> "A gathering place for the Open Rail Data community" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openraildata-t...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid
> /openraildata-talk/

> CAB_f8B0e%3DgMriHWopHfQDEL3psKmYQKftsdwwWqo6eo7tyk0hg%40mail.gmail.com.

>
> --
> You received this message because you are subscribed to the Google Groups "A
> gathering place for the Open Rail Data community" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to openraildata-t...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/
> openraildata-talk/

> CAHjWcZ7htRjzuk3%3DF-vvidZiTNScuexor_OeMg7dkJE2EySeWg%40mail.gmail.com.

[Jack Brewer]

Hi Evelyn,

That’s good to know, thank you!

Jack

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/ZTvQw045LiOkEvTK%40evelyn.moe.

[Evelyn Snow]

Hi Matthew,

On 2 and 3, if it's indeed that STOMP library, it supports only 1.0, which
means no heartbeating.

Evelyn

> To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/ZTvKf%2BHca6f4rJDn%40evelyn.moe.

[Matthew Burdett]

Hi Both

I'm using the php stomp library from pecl. Initially I used 1.0 and I'm sure that it didn't support heartbeats so after trying a few githubs I later switched to the newest version from PHP as it worked best for me.  

Unfortunately switching the heartbeats around still results in the 60sec problem will look at my configs.

Kind regards 

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/ZTvZhwGSb4pLw0du%40evelyn.moe.

[Andrew Larcombe]

Did you try removing the acks? 

Regards

Andrew

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CA%2B2-wYGeB3nOyMVeqxr6ReiL7XrjE7MgVhwc30MEYPy7VwJgzw%40mail.gmail.com.

[Matthew Burdett]

Also getting the timeout with acks removed.

Seeing as I'm only using this for the TSR stuff, I'm just going to suck up TD  messages at 0559 on a Friday morning and kill it once the stuff is received. Don't have the patience to fix this sort of thing these days lol 

Kind regards 

To view this discussion on the web, visit https://groups.google.com/d/msgid/openraildata-talk/CAB_f8B03OnZy3RfPHjB369BiS7Mp%2B5F8-BFPiiq6Aeobt3a4rA%40mail.gmail.com.