Re: [k8s-sig-net] "AI Gateway" Working Group Proposal

[Antonio Ojea]

Hi all,

Thanks Shane for putting this proposal together and to everyone for the enthusiasm. It's clear that the intersection of AI and networking on Kubernetes is an important space that is rapidly evolving, and it's great to see this Gateway API interest. 

I want to put some thoughts on this proposal, since I think there is a mismatch with the formal definition and purpose of a Kubernetes Working Group (WG). According to the official WG governance documentation, a Working Group is intended to facilitate communication and coordination across multiple SIGs to address a specific, time-limited problem that spans those SIGs.

The mission outlined in the proposal, describing the group as "effectively a 'branch' of the Gateway API and Gateway API Inference Extension (GIE) projects," suggests the work is tied to the existing Gateway API subproject within SIG Network. It sounds much more like a focused effort or a new workstream within the Gateway API community rather than a cross-SIG initiative.

The kind of collaboration being described here is something that the current community structure fully supports and encourages. You can absolutely form a focused group to drive the "AI Gateway" effort within the Gateway API subproject today.

In addition, for cross-SIG work related to serving AI/ML workloads, we already have wg-serving. If the scope of this AI Gateway workstream requires formal collaboration with other SIGs (like SIG Apps or SIG Node), wg-serving seems the best place to coordinate.

To be clear, I think this is a good idea, but I want to ensure we use the right community structures to help to keep efforts organized and avoid overlapping or redundancies with a formal Working Group.

Perhaps the best path forward would be to establish this as a dedicated focus area within the Gateway API project?

Thanks

(cc @steering  because WG need steering approval )

On Sat, 14 Jun 2025 at 03:36, Dan Sun <yuzi...@gmail.com> wrote:
>
> +1 on supporting creating this group. AI gateway has much larger scope as documented in the proposal, we need an API that's flexible to support the variety of options between self hosted model routing and cloud model providers. LLM traffic has unique characteristics (e.g. token based billing, diverse provider API, intelligent routing and fallback) that go beyond standard HTTP routing, and we need centralized management for the fleet of LLM model endpoints to monitor, secure and control the cost of LLM traffic.
>
> On Friday, June 13, 2025 at 10:36:06 AM UTC-4 Flynn - wrote:
>>
>> I would support creating this group. To Rob's point, GIE is very much focused on only inference, rather than the broader AI world, and I'd like to see some coordinated work looking at the whole space.
>> -- Flynn
>> On Jun 12 2025, at 5:13 PM, 'Rob Scott' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>>>
>>> Hey Shane,
>>>
>>> Thanks for sharing this proposal! The "AI Gateway" space is really exciting and I've definitely seen some interest in standardization here. My question would be if we need a new working group for this purpose. As your proposal mentions, we already have a group working on extending Gateway API for Inference. I'd be worried that creating another very similar group could spread us too thin as a SIG.
>>>
>>> Thanks,
>>>
>>> Rob
>>>
>>>
>>> On Thu, Jun 12, 2025 at 1:34 PM 'Shane Utt' via kubernetes-sig-network <kubernetes-...@googlegroups.com> wrote:
>>>>
>>>> Hello SIG Network,
>>>>
>>>> Hope everyone is having a good summer!
>>>>
>>>> With the explosive growth of AI/ML the last few years, and the subsequent success of the Gateway API Inference Extension (GIE) the intersection of networking an AI/ML continues to be a frontier for us in SIG Network. We all want to make sure Kubernetes becomes and remains the platform for running your AI workloads, and there's tons of work all over the SIGs pushing for that. To that end, It's always good to keep stepping back and taking a look at what's next there, and how we continue to achieve that as a SIG.
>>>>
>>>> There is a new proposal which aims at trying to harness this growth, and look into an area here that might be the next good focus point which (for better or worse) we've named "AI Gateway". If you're interested in the future standards of AI/ML networking on Kubernetes, inference, and "AI Gateways" check out the proposal here, and please provide your thoughts and feedback.
>>>>
>>>> Cheers,
>>>>
>>>> Shane
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
>>>> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/6737afa8-e84e-4180-bf19-246796297111n%40googlegroups.com.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
>>>
>>> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAGY4dknipvPePsiMOsxyuPbsGq6W-HtaSA-udJhs_oqnOyhcpw%40mail.gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/0c1c982c-0e35-4aad-aa35-cf8a18e61635n%40googlegroups.com.