Help: train consist feed

[Anthony Ashurst]

Hi

Im new to the RDM feeds and kafka in general but Ive been trying to connect to the train allocation and consist feed ive got all my credentials and ‘think’ im doing it right but to no avail can i get anything from the feed and know theres something missing which i cant put my finger on

Im using this code to try to connect

from kafka import KafkaConsumer

import json

import certifi

# =====================

# CONFIG

# =====================

BROKER = "pkc-z3p1v0.europe-west2.gcp.confluent.cloud:9092"

USERNAME = "”

PASSWORD = "”

TOPIC = "prod-1033-Passenger-Train-Allocation-and-Consist-1_0"

GROUP_ID = "allocation-test-reset-1"

AUTO_OFFSET = "earliest"

# =====================

# CONNECT

# =====================

consumer = KafkaConsumer(

    bootstrap_servers=BROKER,

    security_protocol="SASL_SSL",

    sasl_mechanism="PLAIN",

    sasl_plain_username=USERNAME,

    sasl_plain_password=PASSWORD,

    ssl_cafile=certifi.where(),

    group_id=GROUP_ID,

    auto_offset_reset=AUTO_OFFSET,

    enable_auto_commit=True,

    value_deserializer=lambda m: m.decode("utf-8")

)

consumer.subscribe([TOPIC])

print("Subscription object:", consumer.subscription())

print("Topics available:", consumer.topics())

print(f"Listening to topic: {TOPIC} ...")

consumer.poll(timeout_ms=1000)

parts = consumer.partitions_for_topic(TOPIC)

print("Partitions result:", parts)

# =====================

# LISTEN

# =====================

try:

    for message in consumer:

        print("---- New Message ----")

        print(json.dumps(message.value, indent=2))

except KeyboardInterrupt:

    print("Stopping Listener...")

finally:

    consumer.close()

Hopefully someone can help

Thanks

[Peter Hicks]

Hello

On Friday, 27 March 2026 at 19:31, 'Anthony Ashurst' via A gathering place for the Open Rail Data community <openrail...@googlegroups.com> wrote:

Im new to the RDM feeds and kafka in general but Ive been trying to connect to the train allocation and consist feed ive got all my credentials and ‘think’ im doing it right but to no avail can i get anything from the feed and know theres something missing which i cant put my finger on

As a very first step, I'd put some debugging in your code to see if you're actually connecting successfully to Kafka, and if so, where it's failing.

Peter

[Anthony Ashurst]

Ive been doing some debugging and keep getting that im not connecting to the broker/bootstrap

Ive been using this -  print("Broker reachable:", consumer.bootstrap_connected()) and it keeps coming back as (Broker reachable: False), im not sure if its to do with the security certificate or not

Tony

[Matthew Burdett]

I could be wrong but it doesn't look like you've got the consumer group ID in there?

You've got "allocation-test-reset-1", but there are 3 fields in my RDM access credentials;

Consumer username, consumer password and consumer group, all of which are long alphanumeric strings. 

Are you using all 3?

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/1068204978.1400070.1774639752224%40mail.yahoo.com.

[Anthony Ashurst]

Yes im using all 3 of them ive changed the group_id back to the alpha numeric string

Im using group_id for the consumer group credentials if thats right, plus im making it through to the server now but getting errors (SASL Authentication error: Authenication failed

Sorry about the vague replies as its the first time using this everything else with the NR data feeds i can do no problem but for some reason i cant get my head round this

Thanks

[Peter Hicks]

Hello

On Saturday, 28 March 2026 at 11:58, 'Anthony Ashurst' via A gathering place for the Open Rail Data community <openrail...@googlegroups.com> wrote:

Yes im using all 3 of them ive changed the group_id back to the alpha numeric string

Im using group_id for the consumer group credentials if thats right, plus im making it through to the server now but getting errors (SASL Authentication error: Authenication failed

Sorry about the vague replies as its the first time using this everything else with the NR data feeds i can do no problem but for some reason i cant get my head round this

Try using some known-good code, such as at https://github.com/openraildata/kafka-client-rdm-darwin.  Bear in mind that the stream of consist messages is not continuous - if you connect and are expecting messages to flow straight away, wait 5-10 minutes.  They're extracted from a back-end system in batch.

Peter

[Anthony Ashurst]

Ive used the code from github that Peter sent and added all my credentials but still getting the same error as before

%3|1774702043.971|FAIL|rdkafka#consumer-1| [thrd:sasl_ssl://pkc-z3p1v0.europe-west2.gcp.confluent.cloud:9092/boo]: sasl_ssl://pkc-z3p1v0.europe-west2.gcp.confluent.cloud:9092/bootstrap: SASL authentication error: Authentication failed (after 5072ms in state AUTH_REQ, 1 identical error(s) suppressed)

but inbetween the errors its printing (waiting....) so at least its connecting now, ive double checked my credentials and they are correct

Tony

[Peter Hicks]

On Saturday, 28 March 2026 at 12:51, 'Anthony Ashurst' via A gathering place for the Open Rail Data community <openrail...@googlegroups.com> wrote:

Ive used the code from github that Peter sent and added all my credentials but still getting the same error as before

%3|1774702043.971|FAIL|rdkafka#consumer-1| [thrd:sasl_ssl://pkc-z3p1v0.europe-west2.gcp.confluent.cloud:9092/boo]: sasl_ssl://pkc-z3p1v0.europe-west2.gcp.confluent.cloud:9092/bootstrap: SASL authentication error: Authentication failed (after 5072ms in state AUTH_REQ, 1 identical error(s) suppressed)

but inbetween the errors its printing (waiting....) so at least its connecting now, ive double checked my credentials and they are correct

In that case, it sounds like it could be a problem with you credentials - raise a ticket with the RDM Support team.

Peter

[Anthony Ashurst]

Ive put a ticket in about it

Thank you

[Mikey Whiston]

I have minimal experience with the Train Consist feed, but have you tried removing the SSL CA certificate parameter from the KafkaConsumer?
For instance, removing the line that states
>  ssl_cafile=certifi.where()

Just a shot in the dark based on my experience with the Darwin Push Port Kafka feed.

--
You received this message because you are subscribed to the Google Groups "A gathering place for the Open Rail Data community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openraildata-t...@googlegroups.com.

To view this discussion, visit https://groups.google.com/d/msgid/openraildata-talk/ad0f9203-264e-44b4-b066-236f6775f56fn%40googlegroups.com.

[Anthony Ashurst]

Mikey yes ive tried that to and still get the same error as posted before

Tony