[Discourse.ros.org] [Next Generation ROS] Maximum number of nodes in ROS2

[Secretary Birds]

SecretaryBirds October 6

I've been playing around with launching lots of nodes in the ROS2 alpha7 infrastructure. I'm facing a hard limit of 10 maximum nodes. After snooping around in the opensplice docs and google searching the issue, I think I'm running out of Participant IDs. After reading up on the Alpha8 release notes, the Opensplice portion of this issue isn't as relevant, but maybe FastRTPS will suffer from the same issue? Is there a design methodology for supporting more than 10 nodes in a subsystem that skirts the Participant Id issue? Or is there a way to increase the available Participant Ids to allow more than 10? Has anyone else encountered this issue, or have recommendations? Thanks.

---

Visit Topic or reply to this email to respond.

To unsubscribe from these emails, click here.

[William Woodall]

wjwwood October 6

It should be possible to have many more than 10 nodes. I'd assume this is a bug in our code. I'll try to reproduce it when I get a chance.

[Secretary Birds]

SecretaryBirds October 6

The error I get is on the DDS layer. It is: rtps_init: failed to find a free participant index for domain 0 For your reference, I looked at this Vortex documentation: http://download.prismtech.com/docs/Vortex/html/ospl/DeploymentGuide/guide.html#configuration specifically these sections appeared to me to be the cause of the failure: 12.7.3.6. MaxAutoParticipantIndex This element specifies the maximum DDSI participant index selected by this instance of the DDSI service if the Discovery/ParticipantIndex is “auto”. Full path: //OpenSplice/DDSI2Service/Discovery/MaxAutoParticipantIndex Format: integer Default value: 9 Occurrences min-max: 0-1 12.7.3.7. ParticipantIndex This element specifies the DDSI participant index used by this instance of the DDSI service for discovery purposes. Only one such participant id is used, independent of the number of actual DomainParticipants on the node. It is either: auto: which will attempt to automatically determine an available participant index (see also Discovery/MaxAutoParticipantIndex), or a non-negative integer less than 120, or none:, which causes it to use arbitrary port numbers for unicast sockets which entirely removes the constraints on the participant index but makes unicast discovery impossible. The default is auto. The participant index is part of the port number calculation and if predictable port numbers are needed and fixing the participant index has no adverse effects, it is recommended that the second be option be used. Full path: //OpenSplice/DDSI2Service/Discovery/ParticipantIndex Format: string Default value: auto Occurrences min-max: 0-1 The crazy thing is that if I try to add these fields into the ros_ospl.xml file, DDS can't recognize them and fails.

[William Woodall]

wjwwood October 6

I was aware of the 120 participants per machine per domain limit, I'm not sure why it's starting out at 9. Again I'll try to reproduce it and try changing the configs when I have time.

[Secretary Birds]

SecretaryBirds October 13

SecretaryBirds:

none:, which causes it to use arbitrary port numbers for unicast sockets which entirely removes the constraints on the participant index but makes unicast discovery impossible.

After playing with this some more, changing the ParticipantIndex tag in the ros_ospl.xml file actually works, and setting it to 'none' allows 10+ nodes to launch successfully. Of course, you loose unicast discovery, but this is at least a temporary workaround.

[William Woodall]

wjwwood October 17

Cool, sounds like a workaround for now. I probably won't pursue it further since we're going to be moving away from OpenSplice, but if you think it makes sense to change this open a pr. Or you could add a wiki page which describes how to change this setting.