RSVP not blocking sender when message is addressed to a group member

37 views
Skip to first unread message

Larry Bunch

unread,
Sep 9, 2020, 5:07:42 PM9/9/20
to jgroups-dev
I'm using RSVP in version 5.0.2 to send messages. When messages are sent to the entire group (e.g. destination Address is null) the RSVP blocks the sender until the receivers' ack is returned as expected. However, when a destination address is set for the message, the sender is not blocked. Is this expected?

Sample Code 
import org.jgroups.*;
import org.jgroups.util.MessageBatch;

import java.io.InputStream;
import java.io.OutputStream;

public class RsvpTest
{
public static void main(String[] args)
{
try
{
String group = "test-group";
JChannel channel = new JChannel("udp.xml");
channel.setReceiver(new Receiver()
{
@Override
public void receive(Message msg)
{
System.out.println("Received: " + msg);
}
});
channel.connect(group);
Thread.sleep(3000);
for (int i = 0; i < 10; i++)
{
Message message = new ObjectMessage(null, "entire group blocks");
message.setFlag(Message.Flag.RSVP);
long sendTime = System.currentTimeMillis();
channel.send(message);
System.out.println("Group message time to send was " + (System.currentTimeMillis() - sendTime));
Thread.sleep(1000);
}
for (int i = 0; i < 10; i++)
{
View view = channel.getView();
for (Address address : view.getMembers())
{
if (address.equals(channel.getAddress()))
{
continue;
}
Message message = new ObjectMessage(address, "peer does not block");
message.setFlag(Message.Flag.RSVP);
long sendTime = System.currentTimeMillis();
channel.send(message);
System.out.println("Peer " + address + " message time to send was " + (System.currentTimeMillis() - sendTime));
Thread.sleep(1000);
}
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}

Sample UDP+RSVP Configuration
<config xmlns="urn:org:jgroups"
        xsi:schemaLocation="urn:org:jgroups http://www.jgroups.org/schema/jgroups-5.0.xsd"
        version="5.0.2">
    <UDP
            bundler_capacity="1"
            mcast_port="${jgroups.udp.mcast_port:45588}"
            thread_pool.min_threads="0"
            thread_pool.max_threads="200"
            thread_pool.keep_alive_time="30000"
    />
    <PING />
    <MERGE3 max_interval="30000"
            min_interval="10000"/>
    <FD_SOCK/>
    <FD_ALL3/>
    <VERIFY_SUSPECT timeout="5000"  />
    <BARRIER />
    <pbcast.NAKACK2 max_rebroadcast_timeout="5000" xmit_interval="2000"/>
    <UNICAST3 xmit_interval="2000" />
    <pbcast.STABLE desired_avg_gossip="50000"
                   max_bytes="4M"/>
    <pbcast.GMS view_ack_collection_timeout="4000" print_local_addr="true" join_timeout="5000"/>
    <UFC max_credits="10M"
         min_threshold="0.4"/>
    <MFC max_credits="10M"
         min_threshold="0.4"/>
    <FRAG2 frag_size="60K"  />

    <RSVP ack_on_delivery="true" timeout="6000" throw_exception_on_timeout="true"/>

    <pbcast.STATE_TRANSFER />
</config>

Sample Output (note that the test network adds 1 second latency to all comms)
Sep 09, 2020 8:43:22 PM org.jgroups.JChannel setAddress
INFO: local_addr: a7f95eea-41b9-7ffa-6771-2dd478997920, name: uas-24-2967
Sep 09, 2020 8:43:22 PM org.jgroups.protocols.UDP setBufferSize
WARNING: JGRP000015: the receive buffer of socket MulticastSocket was set to 5MB, but the OS only allocated 212.99KB
Sep 09, 2020 8:43:22 PM org.jgroups.protocols.UDP setBufferSize
WARNING: JGRP000015: the receive buffer of socket MulticastSocket was set to 5MB, but the OS only allocated 212.99KB

-------------------------------------------------------------------
GMS: address=uas-24-2967, cluster=test-group, physical address=192.168.14.24:35481
-------------------------------------------------------------------
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 31
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 3
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2013
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2012
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2012
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2011
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2012
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2012
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2015
Received: [uas-24-2967 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Group message time to send was 2015
Peer uas-23-6919 message time to send was 0
Peer uas-23-6919 message time to send was 0
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Peer uas-23-6919 message time to send was 0
Peer uas-23-6919 message time to send was 0
Peer uas-23-6919 message time to send was 0
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Peer uas-23-6919 message time to send was 0
Peer uas-23-6919 message time to send was 0
Peer uas-23-6919 message time to send was 0
Received: [uas-23-6919 to <all>, 25 bytes, flags=RSVP], obj: entire group blocks
Peer uas-23-6919 message time to send was 1
Peer uas-23-6919 message time to send was 0
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block
Received: [uas-23-6919 to uas-24-2967, 25 bytes, flags=RSVP], obj: peer does not block

Larry Bunch

unread,
Sep 10, 2020, 5:24:52 PM9/10/20
to jgroups-dev
On linux one can use tc (traffic control) to add network latency to reproduce this issue. tc is typically bundled with the OS and doesn't require installation. Here's a sample command, just replace eth0 with your network interface:
   sudo tc qdisc add dev eth0 root netem delay 1000ms

To remove the latency effect:
   tc qdisc del dev eth0 root

Bela Ban

unread,
Sep 23, 2020, 8:05:15 AM9/23/20
to jgrou...@googlegroups.com
You can also use DELAY to be OS-agnostic. Just back from a PTO, taking a
look at your previous email tomorrow. Cheers,


On 10.09.20 11:24 PM, Larry Bunch wrote:
> On linux one can use *tc* (traffic control) to add network latency to
> reproduce this issue. tc is typically bundled with the OS and doesn't
> require installation. Here's a sample command, just replace
> /eth0/ with your network interface:
> sudo tc qdisc add dev eth0 root netem delay 1000ms
>
> To remove the latency effect:
> tc qdisc del dev eth0 root
>
> On Wednesday, September 9, 2020 at 4:07:42 PM UTC-5 Larry Bunch wrote:
>
> I'm using RSVP in version 5.0.2 to send messages. When messages
> are sent to the entire group (e.g. destination Address is
> null) the RSVP blocks the sender until the receivers' ack is
> returned as expected. However, when a destination address is set
> for the message, the sender is not blocked. Is this expected?
>
> _*Sample Code*_
> *_Sample UDP+RSVP Configuration_*
> *_Sample Output (note that the test network adds 1 second latency
> to all comms)_*
> Sep 09, 2020 8:43:22 PM org.jgroups.JChannel setAddress
> INFO: local_addr: a7f95eea-41b9-7ffa-6771-2dd478997920, name:
> uas-24-2967
> Sep 09, 2020 8:43:22 PM org.jgroups.protocols.UDP setBufferSize
> WARNING: JGRP000015: the receive buffer of socket MulticastSocket
> was set to 5MB, but the OS only allocated 212.99KB
> Sep 09, 2020 8:43:22 PM org.jgroups.protocols.UDP setBufferSize
> WARNING: JGRP000015: the receive buffer of socket MulticastSocket
> was set to 5MB, but the OS only allocated 212.99KB
>
> -------------------------------------------------------------------
> GMS: address=uas-24-2967, cluster=test-group, physical
> address=192.168.14.24:35481 <http://192.168.14.24:35481>
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jgroups-dev/2d0f838d-cddc-460a-b842-2c0cced7fa29n%40googlegroups.com
> <https://groups.google.com/d/msgid/jgroups-dev/2d0f838d-cddc-460a-b842-2c0cced7fa29n%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Bela Ban, JGroups lead (http://www.jgroups.org)

Bela Ban

unread,
Sep 24, 2020, 5:31:06 AM9/24/20
to jgrou...@googlegroups.com
Hi Larry

simple reason unicasts didn't block: when UNICAST3 is found in the
stack, RSVP does *not* block, see [1].

I modified your program slightly (see below), to make it execute faster.

If you comment UNICAST3 in your config, things will work as you expected.

However, RSVP doesn't make any sense for unicasts if you have a UNICAST
protocol in your config, so the code is correct.
Cheers,


[1]
https://github.com/belaban/JGroups/blob/master/src/org/jgroups/protocols/RSVP.java#L88

below is the modified program:

package org.jgroups.tests;

import org.jgroups.*;
import org.jgroups.util.Util;

public class RsvpTest {
protected final JChannel[] channels=new JChannel[3];
protected JChannel channel;


protected void start() throws Exception {
for(int i=0; i < channels.length; i++)
channels[i]=new
JChannel("/Users/bela/rsvp.xml").name(String.valueOf((char)('A'+i))).connect("test-group");
Util.waitUntilAllChannelsHaveSameView(10000, 500, channels);
channel=channels[0];

for(JChannel ch: channels) {
ch.setReceiver(new Receiver() {
public void receive(Message msg) {
System.out.printf("%s: received %s\n",
ch.getAddress(), msg.getObject());
Util.sleep(200);
}
});
}

for(int i=0; i < 5; i++) {
Message message=new ObjectMessage(null, "entire group
blocks").setFlag(Message.Flag.RSVP);
long sendTime=System.currentTimeMillis();
channel.send(message);
System.out.println("Group message time to send was " +
(System.currentTimeMillis() - sendTime));
}
for(int i=0; i < 5; i++) {
View view=channel.getView();
for(Address address : view.getMembers()) {
if(address.equals(channel.getAddress())) {
continue;
}
Message message=new ObjectMessage(address, "peer does
not block").setFlag(Message.Flag.RSVP);
long sendTime=System.currentTimeMillis();
channel.send(message);
System.out.println("Peer " + address + " message time
to send was " + (System.currentTimeMillis() - sendTime));
}
}
Util.close(channels);
}


public static void main(String[] args) throws Exception {
new RsvpTest().start();
}
}


On 09.09.20 23:07, Larry Bunch wrote:
> I'm using RSVP in version 5.0.2 to send messages. When messages are sent
> to the entire group (e.g. destination Address is null) the RSVP blocks
> the sender until the receivers' ack is returned as expected. However,
> when a destination address is set for the message, the sender is not
> blocked. Is this expected?
>
> _*Sample Code*_
> *_Sample UDP+RSVP Configuration_*
> *_Sample Output (note that the test network adds 1 second latency to all
> comms)_*
> --
> You received this message because you are subscribed to the Google
> Groups "jgroups-dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to jgroups-dev...@googlegroups.com
> <mailto:jgroups-dev...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jgroups-dev/c66137ea-7d24-4f87-90ff-f1a53501b0afn%40googlegroups.com
> <https://groups.google.com/d/msgid/jgroups-dev/c66137ea-7d24-4f87-90ff-f1a53501b0afn%40googlegroups.com?utm_medium=email&utm_source=footer>.

--
Bela Ban | http://www.jgroups.org

Reply all
Reply to author
Forward
0 new messages