Performance check of Yami4 Broker

[hiro...@gmail.com]

Hello,

          I am a college student who's evaluating message brokers for asynchronous communication. I found Yami4 quite catering to my requirements.But would like to check Yami4 on certain parameters. I went through the performance code  of the test application for request response pattern of messaging. But I would like to check the same for pub- sub pattern in   a brokered model. 

Parameters for testing involve:

1.Throughput

2. Latency

3. Memory Footprint

4. Persistence

5 .Scalability

6. Failover mechanism

7. Communication patterns support

8. Security

9. Routing algorithm used and its algorithmic complexity w.r.t Time and space. I would like to find statistics on the same by performing demonstration on yami4.

Some more queries involve:

1. I tried enabling tcps for security but the test applications crashed . 

2.Even the queue mechanism in yami4broker.exe does not store the messages when the subscriber is down and does not update the previous  messages.

         Kindly  look into such queries and any help in the same  is highly appreciated.

         Best,

      Shirley Holmes   

     

[Maciej Sobczak]

Hi Shirley,

Thanks for asking these questions, but you have to be aware that such extensive evaluation of the broker is a substantial effort, especially if we take into account that with the flexibility of pub-sub communication, there can be several ways (with their own tradeoffs) to achieve similar patterns.

It is not meaningful to simply rewrite the request/response benchmark for the broker and measure the interesting parameters. The reason for this is that the broker does not require any responses and with communication going in one direction only there are opportunities to do some things differently. Still, I will try to give you some pointers.

 

Parameters for testing involve:

1.Throughput

Throughput of the broker alone can be evaluated with a single pub-sub pair or with multiple clients (either on pub and sub sides, which allows arbitrary M:N scenarios). You can try to restructure the basic broker tests from the YAMI4 package to do this.

 

2. Latency

This will mostly depend on your network topology, but is easier if your machines are synchronized.

 

3. Memory Footprint

YAMI4 was designed with embedded systems in mind and a single agent (which is an internal component of the broker as well) has very small memory footprint. The actual memory usage will therefore depend on how many messages you are willing to keep in its queues, which is a configurable parameter. That is, it can be whatever you allow it to be.

 

4. Persistence

Currently message persistence is not supported by the broker.

 

5 .Scalability

This can be measured in many ways, depending on what is your target scalability axis. Do you want to connect multiple publishers and a single subscriber? Or the other way round? Or maybe the pub/sub pairs are logically independent? Or maybe you mean how the performance depends on average message size?

 

6. Failover mechanism

Some ideas where failover can be achieved are described in the YAMI4 book: http://inspirel.com/yami4/book/7-2-2.html

 

7. Communication patterns support

The broker allows to achieve virtually unlimited number of different communication patterns! :-) Which ones are you interested in?

 

8. Security

Which aspect of it do you need? Apart from secure data transfer (which can be achieved with the SSL transport or with tunneling), many useful patterns (like authorization to perform some remote operation) can be achieved at the application level.

 

9. Routing algorithm used and its algorithmic complexity w.r.t Time and space.

The broker uses straightforward linear search for a matching tag. This means that both the number of subscriptions as well as the total length of tag (individually and when used in combination for multi-tag pattern matching) have an impact on the complexity of routing.

 

Some more queries involve:

1. I tried enabling tcps for security but the test applications crashed . 

Did you manage to enable tcps with the basic request/response example?

Regards,

-- 

Maciej Sobczak * http://www.inspirel.com

[Shirley Holmes]

     Thank you so much Maciej for your immense support.

1. Considering the throughput, I am interested in a single pub sub pair.I m trying to restructure the code of the broker and the library as well .It would beo f great help if you can lend some support in this context. I have sought after a mechanism wherein publishing time to the broker and  publishing time to the subscriber is to be found. I am unable to find the latter one.

4. For scalability , I am interested in all the four scenarios.

8. I tried the security mechanism for the request response tests ,but still the application crashed. The  screenshot of the same is provided.

9. Can any other approach be applied for routing to decrease the linear search time?

   Additional issue that I am facing is everytime I try and run the broker test applications , the queue overflows for different message counts for same message . How is that to be interpreted?

     Kindly reply.

     Best,

Shirley Holmes

--
You received this message because you are subscribed to a topic in the Google Groups "yami4" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/yami4/90Qq1uDfgJQ/unsubscribe.
To unsubscribe from this group and all its topics, send an email to yami4+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Shirley Holmes]

Sorry for inconvenience regarding the error output that was not attached. 

     Kindly refer the same.

     Best,

Shirley Holmes

[Maciej Sobczak]

 

1. Considering the throughput, I am interested in a single pub sub pair.I m trying to restructure the code of the broker and the library as well .It would beo f great help if you can lend some support in this context. I have sought after a mechanism wherein publishing time to the broker and  publishing time to the subscriber is to be found. I am unable to find the latter one.

You can expect both times to be equal and equivalent to the sending of a single message between client and server in a peer-to-peer scenario. The mechanics of message sending are the same in all these situations.
 

4. For scalability , I am interested in all the four scenarios.

What problems did you find when implementing appropriate benchmarks?
 

8. I tried the security mechanism for the request response tests ,but still the application crashed. The  screenshot of the same is provided.

The error report that you have shown in other post indicates that the function call just before line 108 was not successful. I guess it is because you have not provided the necessary key and certificate files (ssl_server.key and ssl_server.crt), which are expected in the same directory.
 

9. Can any other approach be applied for routing to decrease the linear search time?

Of course, but why do you think there is a need to do so? The biggest performance problem in distributed systems is related to the transmission (including message serialization, frame management, and so on). The cost of sending a single message is so high that searching a matching tag in an array is negligible and in those extremely rare cases where routing scalability would be a problem (like you have created millions of subscriptions), rethinking the tag and subscription policy would be more effective than any optimizations in table searches.
 

   Additional issue that I am facing is everytime I try and run the broker test applications , the queue overflows for different message counts for same message . How is that to be interpreted?

Apparently your test environment is not repeatable in terms of timings and available processing power. Which is to be expected if you use Windows on a Dell computer (as I can guess from the report that you have attached in your other post).

Best regards,

[Shirley Holmes]

Thank you for your reply,  Maciej.
1. From throughout context in brokered mechanism,  the time taken to send message upto the broker and the time taken to send the message from broker upto the end subscriber seems different because of queue overflow.  This might be because of inequality between number of incoming messages and outgoing messages due to queue overflow.I would like to find the same.

9. With respect to routing , I agree with your answer. But I was trying to find efficiency of Yami4 from all the perspectives and routing was one of the parameters.

[Maciej Sobczak]

1. From throughout context in brokered mechanism,  the time taken to send message upto the broker and the time taken to send the message from broker upto the end subscriber seems different because of queue overflow.  This might be because of inequality between number of incoming messages and outgoing messages due to queue overflow.I would like to find the same.

Assuming that your machines are synchronized, it might be possible to get meaningful measurements by sending timestamps and appending them at each stage. In other words, let the publisher send its own timestamp, then let the broker append its own timestamp when the message is received, another one when it is posted (which happens right after routing) and finally check the arrival time at the subscriber. This way, you will be able to measure each stage of the channel, including the time spent waiting in each queue. Note however that this will measure the latency from the point of view of a single message and will tell you nothing about the throughput, which considers multiple messages over a period of time. These are different performance parameters and should be measured differently.

Note also that the broker does not interpret the data and it operates on binary blocks instead of repackaging and packaging data using the parameters format. This means that timestamps can be appended by simply adding few bytes to the payload, not as adding new field to the parameters object.

Regards,

[Shirley Holmes]

A great suggestion, sir.
1. But  I am getting abandoned subscription message in the logger... Can you specify for what error condition such a message is displayed?  Is it possible to find queue statistics during message transfer?? How to go about?
2.Regarding security, I did generate certificate and private key. I renamed both to server_ssl.cert and server_ssl.key and placed both in the calculator directory in core.But still the issue persists.  How is that to be resolved?
3. For demonstrating failover mechanism,  do I need to prepare my own application or the applications in broker service and cpp cater to the mechanism?
   
Best, 
Shirley Holmes

[Shirley Holmes]

Hello Sir,

              I have certain other questions in my mind.Do answer if possible.

1. Memory footprint

I have found the statistics while running the exe in task manager. But  how do I calculate the space complexity of yami4 in terms of BigOh , Theta or Omega? 

2 .Scalability

 How many pub sub pairs can  be active at a time in  yami4  without affecting the throughput of the messages. Is this configurable or fixed? where is it initialized?I have found the max subscriptions field in .cfg file but what about the number of  publishers ?

3. Routing algorithm 

 How do I calculate time complexity of routing algorithm in terms of bigoh , theta or omega? 

4. What is the use of server side selectors ?

5.How do I calculate number of bytes for a tag: message pair?

Best,

Shirley Holmes

[Maciej Sobczak]

              I have certain other questions in my mind.Do answer if possible.

1. Memory footprint

I have found the statistics while running the exe in task manager. But  how do I calculate the space complexity of yami4 in terms of BigOh , Theta or Omega? 

Why do you try to complicate your reasoning so much? The "space complexity" is linear in that if you put twice as many messages into outgoing queue, then they will take twice as much memory. What's important, *all* messaging systems have this property (unless they are broken), in which case it is not even meaningful to compare them based on such parameters. So why do you need to use these concepts?

2 .Scalability

 How many pub sub pairs can  be active at a time in  yami4  without affecting the throughput of the messages.

What do you mean by "affecting the throughput"? Note that what matters most is not the number of pub/sub pairs, but the number of distinct networking targets, as they are mapped to separate physical connections. If you have multiple clients operating on the same target (for example, many publishers in the same process), then they will not be different than a single client sending the same amount of traffic.
 

Is this configurable or fixed? where is it initialized?I have found the max subscriptions field in .cfg file but what about the number of  publishers ?

The number of publishers is irrelevant (see above), but be careful about the number of network connections, as this is what will first hit system limits in a huge system.
 

3. Routing algorithm 

 How do I calculate time complexity of routing algorithm in terms of bigoh , theta or omega? 

We have already established that it is linear with respect to the number of subscriptions. There is nothing to calculate. Why do you need such parameters?
 

4. What is the use of server side selectors ?

Where did you get this term?
 

5.How do I calculate number of bytes for a tag: message pair?

You can calculate it based on the protocol description in the documentation, or you can measure it using techniques described here:

http://www.inspirel.com/articles/YAMI4_Tip_8.html

[Maciej Sobczak]

A great suggestion, sir.
1. But  I am getting abandoned subscription message in the logger... Can you specify for what error condition such a message is displayed?

The possible message states are described here:

http://www.inspirel.com/yami4/book/5-1.html
 

  Is it possible to find queue statistics during message transfer?? How to go about?

You can use the get_pending_outgoing_bytes function for this.

2.Regarding security, I did generate certificate and private key. I renamed both to server_ssl.cert and server_ssl.key and placed both in the calculator directory in core.But still the issue persists.  How is that to be resolved?

Note that it should be ssl_server.crt, not server_ssl.cert and ssl_server.key, not server_ssl.key.
Can you print the value of the "res" variable just before the failing assert?
 

3. For demonstrating failover mechanism,  do I need to prepare my own application or the applications in broker service and cpp cater to the mechanism?

The failover can be demonstrated with any existing program, as this feature relies on target name resolution.

[Shirley Holmes]

Okay sir,
Regarding security,  I had provided ssl_server.key and ssl_server.cert,  there was this typing error in the mail.  Yes, I'll print the res value and see.
But I would like to know some other fact: Does yami handle messages differently in Windows 8, because my application crashes when receiving data from broker to subscriber.

Kindly reply.
Best, 
Shirley Holmes
              

[Maciej Sobczak]

Hello,

But I would like to know some other fact: Does yami handle messages differently in Windows 8, because my application crashes when receiving data from broker to subscriber.

No, YAMI4 is supposed to be portable and except for a small system-dependent layer, the code is the same for all target platforms. Windows 8 should be handled as previous versions, as only basic socket-related functions are used.

By "my application" do you mean your own code, or the example programs from the distribution package?

[Shirley Holmes]

Yes,  By my application  I mean  my own code. 
Regarding the security problem,  I addressed the other day,  u had suggested of printing the res value,  I did do that and it gives unexpected value as the result value,  what could be the issue,  because I did generate the key and certificate using openssl,  renamed them as ssl_server.cert and ssl_server.key,  placed both in the calculator example's directory,  still the example crashes. Kindly revert back.
Best,
Shirley Holmes

[Maciej Sobczak]

Yes,  By my application  I mean  my own code. 

Can you try to minimize your application to the point where the problem disappears? Is it possible to isolate the source of the problem?

 

Regarding the security problem,  I addressed the other day,  u had suggested of printing the res value,  I did do that and it gives unexpected value as the result value,  what could be the issue,  because I did generate the key and certificate using openssl,  renamed them as ssl_server.cert and ssl_server.key,  placed both in the calculator example's directory,  still the example crashes.

Can you write what command was used, exactly, to generate these files?

[Shirley Holmes]

OK, I'll look into that.
Regarding ssl_server.cert and ssl_server.key generation,  I referred the below link,  and used those in calculator directory.
http://blog.didierstevens.com/2015/03/30/howto-make-your-own-cert-with-openssl-on-windows/
One more thing that I am uncertain is,  yami's current library contains only server side authentication functionality,  right??
Best, 
Shirley Holmes

[Maciej Sobczak]

Regarding ssl_server.cert and ssl_server.key generation,  I referred the below link,  and used those in calculator directory.
http://blog.didierstevens.com/2015/03/30/howto-make-your-own-cert-with-openssl-on-windows/

Please try with this command instead:

$ openssl req -x509 -newkey rsa:2048 -keyout ssl_server.key -out ssl_server.crt -days 365 -nodes
[...]

You will have to answer some questions about the new certificate, but at the end you will get the files that can be directly used by the example calculator server program:

$ ./server_ssl tcps://your_server_machine:12345

and then at the client site:

$ ./client tcps://your_server_machine:12345 100 5
sum        = 105
difference = 95
product    = 500
ratio      = 20
$

One more thing that I am uncertain is,  yami's current library contains only server side authentication functionality,  right??

Yes.

Regards,

[Shirley Holmes]

Thank you, Maciej and apologies for late reply. I was on a vacation,  so couldn't reply. 
I will certainly try your approach for enabling security in yami 4.
Best, 
Shirley Holmes

[Shirley Holmes]

Thank you, Maciej. Security functionality got enabled for the client and server_ssl programs in the calculator example. But I am in trouble with another thing. I tried the same security for the broker code using similar approach as of calculator example, but I keep getting 'cannot set up listener' with res value = 'bad protocol'. I made necessary changes in the config file as 'tcps' and even in the 'listener' function of Broker configuration file ,still the issue persists.

Kindly explain.

Best,

Shirley Holmes

[Maciej Sobczak]

I tried the same security for the broker code using similar approach as of calculator example, but I keep getting 'cannot set up listener' with res value = 'bad protocol'. I made necessary changes in the config file as 'tcps' and even in the 'listener' function of Broker configuration file ,still the issue persists.

I think that you have still omitted some important changes. Note that the broker is not, out-of-the-box, prepared for SSL operation in the sense that it does not refer to the certificate files. You need to take these steps to enable SSL in the broker:

- compile the core library with SSL enabled (see Makefile for the correct set of options),
- modify the broker code, where the agent is initialized (broker-messaging.cpp), to set up the certificate files in agent options, as in the example src/core/examples/calculator/server_ssl.cpp,
- modify broker's Makefile to link with the SSL library,
- compile the broker,
- modify the yami4broker.cfg file to use the "tcps" listener name

If you do only the last step, you will get the "bad protocol" error, as the SSL operation will not be enabled at all.

[Shirley Holmes]

Thank you,  Maciej. I'll definitely look into what I might have missed.
But, would like an answer as to why did you choose the certificate method for authentication for a middleware designed for control and monitoring systems.  I mean such systems have very rapid influx and outflux of data and are always in connected mode,  there is very rare use of session concept since they get disconnected very rarely. Hence in a scenario where the certificate is found to be revoked, the client won't connect to the server and by that time,  there is a chance of huge data loss affecting performance. My basic concern is regarding the security approach for control and monitoring systems. Why not go for Kerberos authentication rather ??

Kindly reply.
Best,
Shirley Holmes

[Maciej Sobczak]

But, would like an answer as to why did you choose the certificate method for authentication for a middleware designed for control and monitoring systems

It was not part of the library from the beginning. It is true that YAMI4 was designed with control and monitoring (and in particular with embedded systems) in mind, but it was used in a variety of other system domains as well, including financial trading. This is where the requirement for SSL came from and that was added to the library as an optional feature, disabled by default (that is, by default YAMI4 has no such dependency).

Why not go for Kerberos authentication rather ??

Because it was not required in any of the systems where YAMI4 was used.

[Shirley Holmes]

Yes, Maciej. SSL got enabled at the broker end too. The issue was from my side. Thank you for your prompt reply.
Your reply for usage of ssl certificates in yami4 broker is also satisfactory. But what is your take on enabling certificate authentication for control and monitoring systems?  Is this approach suitable enough or any other approach should be looked upon?
Best, 
Shirley Holmes

[Maciej Sobczak]

But what is your take on enabling certificate authentication for control and monitoring systems?  Is this approach suitable enough or any other approach should be looked upon?

As usual, it will depend on what you are trying to achieve and what are your constraints. In particular - what is the established technical culture in the target environment, in the company, in the team, etc. If you decide that you need crypto services, your tech staff already knows how to install and maintain the SSL infrastructure (which likely) and the final user accepts the level of security that it provides (which is also likely), then SSL is a valid solution. But you might also attempt some completely different path, depending on circumstances.

[Shirley Holmes]

Thank you, Maciej.I am trying with SSL for security , I have associated ​certificates to the client code(yami broker's test clients-publisher and subscriber) ,the applications do build successfully , I can also see the server and client certificates on wireshark but the published data reach the broker very late each time for different message count(for eg. 1200th message in one instance).This is seen when I set certificate like case1 as in attachment. I have also seen a very abnormal behaviour as the publisher tries to ssl_connect every time it sends a message to broker ;on the contrary to  the standard mechanism(ssl mechanism) in which connection is established once, and after  ssl handshake,  messages are sent in an encrypted form. 

2. If I try using case 2: sending and receiving are fine but the wireshark shows no client certificates even though the certificates have been set.

These problems occur when I try to set client certificates using

SSL_CTX_set_verify(ssl_ctx_, SSL_VERIFY_PEER, verify_callback);

Everything works fine when only server certificates are set.

A document is attached for output reference for the above mentioned two cases.If you could please look into these issues and reply.

Best,

Shirley Holmes

[Shirley Holmes]

Hello,  Maciej, in my previous mail,  I had listed some security related issues w.r.t Openssl and YAMI4. Such issues have arisen only when certificates were associated with publisher and subscriber,  the usual client -server applications work fine. Could you please look into the issue. Awaiting reply.
Best,
Shirley Holmes

[Maciej Sobczak]

Hi,

Hello,  Maciej, in my previous mail,  I had listed some security related issues w.r.t Openssl and YAMI4.

The issues you have described seem to be related to how the OpenSSL works and have no source in the YAMI4 library itself - the communication logic is exactly the same with or without encryption, and the YAMI4 library does not influence how OpenSSL handles its protocols. You might want to check these issues on groups devoted to SSL.

[Shirley Holmes]

Thank you, Maciej. I'll take a look at the openssl forum for the security related solution.
I have been trying a new feature to make it work with yami4 middleware. I am trying to incorporate websockets support to yami4 for real time monitoring for Web clients and for this,  I would like broker. exe to listen on multiple ports. I have developed a websocket server but am facing difficulty in integrating it with yami. How do I make the broker listen on multiple ports in a single threaded manner. Will this require modifying  YAMI's methods or could it be done independently.  I found out an approach wherein if I get the listen socket I'd of the broker,  I would use a select on both the broker's  listen socket I'd and my websocket server's listen socket I'd to find out for connection events on respective ports. But YAMI's API doesn’t expose any method to return socket I'd for an agent object.
Please suggest some way out. Awaiting response.
Best,
Shirley Holmes

[Maciej Sobczak]

How do I make the broker listen on multiple ports in a single threaded manner.

It is possible to have multiple listeners in a single YAMI4 agent (and they are all managed by a single event loop). The broker service would need to be modified slightly for this, as currently it allows only a single listener target to be defined in the configuration file, but there are no obstacles at the agent level.

Still, even if there are many listeners in the single agent, it will not be possible to use multiple protocols, as the agent will apply the same logic to all sockets that it manages. There is currently no way to merge sockets from different protocol domains in a single event loop and the only way to achieve this effect is to create a gateway service that will translate between two worlds. Whether such a gateway is created as a separate program or is embedded inside the existing broker is another question.

Another possibility is to treat a new protocol as a wrapper for the YAMI4 protocol. This is how SSL is handled and if websockets are intended to be used as wrappers for normal YAMI4 communication, this strategy might be used here as well. But it will definitely require extension of the current YAMI4 code base.

Regards,

[Shirley Holmes]

Hello Maciej, 
                        I tried with websockets and built a wrapper, and it's working fine now.  1.    But I am still stuck with the security issue. As we know,  YAMI4 had been used in some financial trading applications and that required security support  and hence ssl support was added.  But my question is security comes into picture only in an environment where the server is in public domain and there is some web client that tries to connect to the server.  But as of now, YAMI4 is basically a TCP server,  and the server is used in a closed environment which nullifies the security concern.
So what is the use of the feature??
2.  The certificate and key generation is a manual process and only one during handshake ssl authentication is done and certificate verification is done. In control and monitoring systems,  the clients are in ever connected state to the server,  and once verification of certificates has been done,  and after some days if the certificate expires,  the issue would not be known till the client disconnects and then reconnects which will lead to Handshake and then verification will fail. This is not the normal condition because due to certificate invalidity,  attack chances are high,  How can this be fixed?
3. Does this certificate validity checking and installation of new ones a manual process or can be automated?  What is the general mechanism?
Please answer.
Best,
Shirley Holmes

[Maciej Sobczak]

But as of now, YAMI4 is basically a TCP server,  and the server is used in a closed environment which nullifies the security concern.
So what is the use of the feature??

If you do not need SSL, do not use it.

 

In control and monitoring systems,  the clients are in ever connected state to the server,  and once verification of certificates has been done,  and after some days if the certificate expires,

My understanding is that the SSL layer can handle this - in particular, it can inject additional read/write operations into the stream whenever it needs to and handling of expiring sessions is one possible reason for this. But I would also consider it to be a quality of implementation issue, to be checked with implementer.

 

How can this be fixed?

You can always introduce periodic disconnections at the YAMI4 level, to force clients to re-connect (and re-validate their credentials).

 

3. Does this certificate validity checking and installation of new ones a manual process or can be automated?  What is the general mechanism?

Certificate management is a system administration topic that is outside of the YAMI4 domain - you can do whatever is recommended in your particular work environment.

[Shirley Holmes]

OK... Thank you,  Maciej.
The websockets implementation works fine for a private network.  But I would like to check the websocket implementation for a server listening over a public ip. I tried associating my machine's public ip to listener in yami4broker.cfg file as tcp ://public ip :port,  but the broker. exe shows cannot setup listener as the message.
Why is such an issue and how can that be resolved??

Please answer.
Best,
Shirley Holmes

[Maciej Sobczak]

I tried associating my machine's public ip to listener in yami4broker.cfg file as tcp ://public ip :port,

Sorry, I see some spaces there in this target name. Are you really using spaces or is it just a typo in the post?

 

but the broker. exe shows cannot setup listener as the message.

Sorry, I do not understand this question. As what message?

Is there any error message that is printed by the broker when it starts?

[Shirley Holmes]

Yes.That's a typo in post. The problem I was facing got resolved. I was trying to make the yami4broker listen on tcp:// public ip:port so as to allow web clients to connect to it. But understood that's not possible because the socket implementation is for internal network and the to achieve web aspect the router does the Network Address Translation from public to private ip.
Best,
Shirley Holmes

[Shirley Holmes]

Hello,  Maciej. I was trying with the tcp keepalive feature of yami4 and hence set it to true at the library level, but do not see any effect at the publisher -subscriber applications.
My goal is: publisher client should be notified as soon as the subscriber client closes connection so that the publisher doesn't keep sending packets to prevent network from flooding.  For such a use case,  how can be tcp keepalive used?
Or do I need to write an event notification callback at the broker and the client applications? 
What is the methodology? Please suggest.
Best,
Shirley Holmes

[Maciej Sobczak]

On Sunday, June 19, 2016 at 6:11:35 PM UTC+2, Shirley Holmes wrote:

Hello,  Maciej. I was trying with the tcp keepalive feature of yami4 and hence set it to true at the library level, but do not see any effect at the publisher -subscriber applications.
My goal is: publisher client should be notified as soon as the subscriber client closes connection so that the publisher doesn't keep sending packets to prevent network from flooding.  For such a use case,  how can be tcp keepalive used?

It is not possible to flood the network by sending packets to the receiver that has closed its connection - the TCP connection logic will prevent this.

The tcp_keepalive setting can make it faster for either side to discover that the connection was closed.

The publisher is intended to "publish" data whenever some data is ready. It does not matter whether there is any subscriber ready to receive the data. If the receivers are connected, they will get the message, but if they are not connected (or if they are not consuming data), then messages will not be sent over the network at all.

 

Or do I need to write an event notification callback at the broker and the client applications? 

Event callback can be useful to automatically re-subscribe when the publisher is started and for other session-level accounting.

[Shirley Holmes]

OK Maciej.
1. Is it that the keepalive won't be helpful for my requirement?
2. I have a need regarding security in the broker. I need to pass an I'd to the init method of channel_group.cpp like ssl certificate and ssl key files are sent,  can this be done without changing the channel_group. cpp ?
Best,
Shirley Holmes

[Maciej Sobczak]

1. Is it that the keepalive won't be helpful for my requirement?

From what you have described, I think not.
 

2. I have a need regarding security in the broker. I need to pass an I'd to the init method of channel_group.cpp

I'm sorry, but I do not understand this. What exactly do you need to pass?
Note that SSL certificate and key files are set up by configuration options, which are passed to the agent's initialization function.

[Shirley Holmes]

2. Yes,  I need to set a new variable using configuration options the way ssl certificate and key files are set.  But for a new variable to be set,  I need to declare it in the option_names. h file,  which will be an intrusion into YAMI's code. Is there any alternative to this?
Best,
Shirley Holmes

[Maciej Sobczak]

2. Yes,  I need to set a new variable using configuration options

Why? How do you intend to use this new value? Where do you want to read it?

Configuration options are used by the existing code within the agent internals. Your new variable will be ignored unless there is a code specifically written to use it.

[Shirley Holmes]

I want ssl enabled server to keep track of the connecting clients by using client identities that I have set it at my application level so that the server executes the authentication logic for my identified clients (trusted)  and untrusted ones.The requirement is my system does not want the server to terminate every client that fails authentication.  I would like to have a different behavior for each client based on their identities.
I would want to read the variable in the init method of channel_group.cpp so as to use my callback function verify_callback  differently for each client based on their identities.
SSL_CTX_set_verify (ssl _ctx,  SSL_VERIFY_PEER, verify_callback )
Best,
Shirley Holmes

[Maciej Sobczak]

I would want to read the variable in the init method of channel_group.cpp so as to use my callback function verify_callback  differently for each client based on their identities.
SSL_CTX_set_verify (ssl _ctx,  SSL_VERIFY_PEER, verify_callback )

Now I see - you will need to modify the channel_group.cpp file to introduce this functionality.

Regards,

[Shirley Holmes]

Hello Maciej,
I am trying to run the subscription example of the cpp library. The reason I do this is I understand the example shows the implementation of brokerless architecture where there is no central broker and the publisher itself acts as the server. The point where I am stuck is I need to implement the secure version of the example and am unable to do so.  If I create a yami cpp agent and pass a parameter object (the ssl certificate,  ssl key embedded) in both the publisher and subscriber and no more modifications in the example. I run as app.exe tcps://ip:port ...  I get the message :value given or received is wrong.  This is because,  there is no ssl related processing in the options.cpp of cpp library.
I tried with even open connection (pub address, parameters object),  this still doesn't work.  I get io error. This open connection is for broker related application I guess. 
I even saw the calculator example of the core library.  But that is a request response scenario.
How could I implement secure brokerless example for publish subscribe model ?
This is basically throughput and latency measurement of the brokerless solution. Hence a small pseudo code would be really helpful.
Please reply soon.
Best,
Shirley Holmes

[Shirley Holmes]

Hello Maciej,
I tried with different solutions for the problem stated earlier but every time I receive a Handshake failure from publisher (server) and only initial client Hello happens. I guess this happens because the implementation I try with is faulty.  Could you please look into the problem?
Best,
Shirley Holmes