New protobuf message for bulk table updates for enhancing table update rate

[Stavan Christian]

Saw a video of P4Language Consortium - Evolving P4 Runtime from Switch to DPU - Alan Lo & Milind Chabbi (https://www.youtube.com/watch?v=8EzR9FH2Wqk) -
on how new protocol is under plan to increase the rate of table updates made by a remote controller to a server.

Two questions: -
1. The discussion stands true for switches as well as DPU right? Asking because the video is entitled and conveys as this is an evolution to be done for DPU (And not only for DPU)
2. Is there a github issue created for this already? If yes, please share.

Regards,

Stavan

[Dhanasekar Rathinavel]

hi Alan Lo & Milind Chabbi,

I would like to add a few more questions on top of what Stavan has asked,

I would like to understand what do you mean by update in the term "update-rate".

      1. By update do you mean entry written successfully to the underlying switch hardware(ASIC/FPGA)?

      2. If yes , how did you ensure the entry was written into hardware successfully? 

with regards,...
Dhanasekar

--
You received this message because you are subscribed to the Google Groups "p4-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to p4-discuss+...@lists.p4.org.
To view this discussion on the web visit https://groups.google.com/a/lists.p4.org/d/msgid/p4-discuss/c96daec4-8d42-4fec-806f-10b8e3300e7fn%40lists.p4.org.

[Andy Fingerhut]

Hopefully Alan or someone else can confirm, but I believe I may know some of the answers here:

Question: The discussion stands true for switches as well as DPU right? Asking because the video is entitled and conveys as this is an evolution to be done for DPU (And not only for DPU)

Answer: Yes, that is true.  The changes suggested would improve performance of control plane API operations regardless of the target device, if the target device is not the bottleneck, i.e. if the bottleneck was the software in the P4Runtime API stack.  The main reason that DPU is mentioned as a motivation for the work is that previously P4-programmable switch ASICs tended to have tables with ~1M or fewer table entries, whereas most DPUs can have 10s of millions of entries, so the desire for higher update rates is stronger.  If a P4-programmable switch ASIC has 10s of millions of entries, the motivation would be the same.

Question: Is there a github issue created for this already? If yes, please share.

Answer: I do not think there is, although I believe that there is still a desire among multiple P4.org participants to see some kind of performance improvement like this make its way through the P4Runtime API specification, so it is likely to see some activity this calendar year.

Question: By update do you mean entry written successfully to the underlying switch hardware(ASIC/FPGA)?

Answer: Yes.  The goal is to increase the throughput of table operations through "the whole stack", from software down to hardware.  Thus my comment above that making improvements in the P4Runtime API layers of software is only an issue if it is the bottleneck -- if one's hardware design is the bottleneck, then there is not nearly as much motivation to improve efficiency in the software layers above it.

Question:  If yes , how did you ensure the entry was written into hardware successfully? 

Answer: The answer to this is specific to each particular SmartNIC/IPU/DPU hardware design.  e.g. if the table is updated in hardware by doing PCIe write operations, then when those are ACKed, the entries should be updated.  If the hardware is designed for injected control packets to perform table operations, then getting the response packets out of the hardware pipeline back into driver software would be an acknowledgement that the hardware operations are completed (perhaps also checking for error status/flags in those responses, too).

Andy Fingerhut

To view this discussion on the web visit https://groups.google.com/a/lists.p4.org/d/msgid/p4-discuss/CANW9tfqxXdT7CXu6YtF93TZep5nnut2Uz-jH5se%3DtvuL%2BQ7DKA%40mail.gmail.com.

[Stavan Christian]

Thank you @Andy for the prompt response!
Can you please let me knwo any existing issue on github of P4 or discussion which does address increment in table update rate?

[Dhanasekar Rathinavel]

Thanks for the clarification, Andy.

with regards,...
Dhanasekar

On Wed, Jul 12, 2023 at 1:37 AM Alan Lo <ala...@nvidia.com> wrote:

Hi Stavan,

 

Andy answered most of your questions. There are several areas where P4RT needs some rethinking, as DPUs have different requirements and use cases versus switches. The need for fast insertion (for example, due to VM migration events) required a different approach than protobuf messages over grpc. In many DPU deployments, the controller can be local. So the demo showed how to improve the insertion rate using a shared memory approach. We have not finished cleaning up and “productizing” this effort, but may have something ready for the community in the fall.

 

Thanks,

Alan

 

From: p4-di...@lists.p4.org <p4-di...@lists.p4.org> On Behalf Of Stavan Christian
Sent: Tuesday, July 11, 2023 12:01 PM
To: p4-discuss <p4-di...@lists.p4.org>
Cc: Andy Fingerhut <andy.fi...@gmail.com>; Stavan Christian <stavan.christ...@gmail.com>; p4-discuss <p4-di...@lists.p4.org>; Dhanasekar Rathinavel <sekar....@gmail.com>
Subject: Re: [P4-discuss] New protobuf message for bulk table updates for enhancing table update rate

 

External email: Use caution opening links or attachments

 

To view this discussion on the web visit https://groups.google.com/a/lists.p4.org/d/msgid/p4-discuss/b7a8cea9-ca46-4213-9391-b5512ba91050n%40lists.p4.org.

[Stavan Christian]

Hey @Andy,

Please let me know if any leads!

Regards,

Stavan

[Andy Fingerhut]

I am not aware of any Github issues that address this at this time.

I expect some will be created, probably later this calendar year, but I do not know by whom, or precisely when.

Andy