Request for new repo: sigs.k8s.io/ingate

[James Strong]

Hi everyone,

The ingress sub-project maintainers have been discussing a gateway API implementation for a while now. We have continued our discussion with sig-network leads, at the gateway-api community meeting and our latest Kubcon Maintainer talk.

We aim to begin a new implementation in a new repo under Kubernetes-sigs. This implementation would be developed as a completely new entity with a new codebase and goals. It is not meant to be a reference implementation or a default one and will have a very tight scope: 

1. Core-ONLY Gateway API support 

1. (no extended at first, nothing implementation-specific, no policies)

2. Ingress support

1. Feature-compatible with the secure subset of ingress-nginx

Ingress-nginx Maintainers High-level Plan

1. New Project Name <- InGate

2. Announcement at KubeCon SLC

3. New Repo <- We are here

4. Setup New Community Meeting

5. Architecture discussion

6. Project plan for implementation.

1. Carryover what works, discard what doesn't from ingress-nginx

We would like to request a new repository for InGate in the Kubernetes-sig organization so that we can begin building InGate as a Gateway API sub-project and help users migrate from ingress-nginx into InGate.

Thank you,

James

--

James Strong https://jamesstrong.dev

[Carlos Tadeu Panato Jr]

+1

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAMMHB8U3eRTG0tLe4a%3Db%2BFWfdwMYho7DbjHZYSViynk-PzN%3DaA%40mail.gmail.com.

[antonio.o...@gmail.com]

>  Feature-compatible with the secure subset of ingress-nginx

can we enumarate this subset?

[James Strong]

Thanks, Antonio, for the question.

That statement implies that only features that are required to implement the API functionality.

If users want TCP/UDP route functionality, which is supported in ingress-nginx, they must migrate to the Gateway API since it is not part of the Ingress API.

Also, not allowing access to the nginx config via things like config and server snippets to inject arbitrary code via Lua has caused security issues in the past.

Ingress in InGate will be only that which is required to support the ingress API, nothing more, nothing less. Gateway API should have advanced functionality to help users migrate and advance the Gateway API spec.

Does that help clarify the statement? We have not done a full inventory but this is the general idea.

Thanks,

James

[Arko Dasgupta]

Hey James, 

Is Ingate's plan to implement only Core fields and APIs (e.g. HTTPRoute, GRPCRoute) or all experimental APIs and fields as well (TCPRoute, SessionPersistence etc) ?

And what about PolicyAttachments for NGINX features not available in Gateway API ?

I'm trying to understand the scope/goals of the new project that's going to live under kubernetes-sigs

[James Strong]

Core API's for now, experimental as time and contributors allow. Ingate's focus is cleaning up the ingress-nginx sins of the past.

If it is not in Gateway API, Ingate won't implement it; we do not want to get into the trouble we have with annotations and exposing functionality via annotations or lua, which, to me, is what PolicyAttachments describes.

if users need something experimental, they will have to find another implementation; InGate is not trying to be the end-all-be-all.

Thank,

James

[Antonio Ojea]

Reduced scope limited the Ingress API and the opportunity to be a trampoline for ingress-nginx users to jump into Gateway API is a +1 from me on the repo creation.

This was a very controversial topic but also an example of how this community has the best interest of the project and users at heart, and is focused on finding solutions.

Can we have a few more +1s? Otherwise we can apply lazy consensus, let's say , on Wed 27th ... and hey, silence means acknowledge, right? ;)

[Arnaud M.]

+1

[Adrian Moisey]

+1

As a heavy user of ingress-nginx, I'm looking forward to seeing what InGate brings, and being able to take use of Gateway API 

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/134306fc-e1e9-4d2d-93eb-6b4e2a3be9f2n%40googlegroups.com.

[Nick Young]

I'm a +1 on this too, I think that the ingress-nginx team have done a great job with talking to the rest of the Gateway API community about this, really excited to see how this goes.

Nick

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAAiwNsjbZBsWCJFb4LZCQVCQ6u-OM5N2JMHrS0m9w%2BSYDAdkhQ%40mail.gmail.com.

[Jintao Zhang]

+1

Jintao

[Shane Utt]

+1

An enormous thank you to the ingress-nginx maintainers and the community for all these years building and maintaining this service for the benefit of the entire Kubernetes community and users. I'm glad we have a way to help you continue that legacy, and I'm glad you're excited about it!

I'm in favor of this proposal, especially given the tight scoping. I think that tight scoping here is very appropriate and must always continue with this project (based on what we learned from ingress-nginx). Specifically, we should avoid all custom extension mechanisms that are not native to Gateway API.

We (the SIG Network leads) have received feedback regarding the placement of this project within the Kubernetes SIGs. It's important to recognize that Kubernetes SIGs encompass a diverse range of projects, many of which serve as foundational technologies that can be built upon by various users and organizations. Gateway API is an example of such a project. In contrast, the proposed InGate project occupies a distinct position as a more comprehensive solution. We have heard concerns from maintainers of other Gateway API implementations who worry that this particular implementation may receive preferential treatment due to its placement. We want to ensure that the community that we are aware and mindful of these concerns and are taking them into consideration. In particular, we expect InGate to be documented differently than how ingress-nginx is documented--for example: InGate will not get any special mention in the main Kubernetes documentation as ingress-nginx does today. I also personally support keeping the option open to reassess the project's placement as it grows--for example: our current expectation is that InGate will be a fully community driven project. If at some point the maintainers were to pursue commercial opportunities related to InGate, it may be reasonable to reconsider whether the project might be better suited for placement within the CNCF than Kubernetes SIGs.

All that said, I don't expect trouble as James and the rest of the crew have already been highly communicative, and kindly responsive to concerns that have been raised. Thank you for being great net citizens while trying to make this happen throughout the year, and thank you for all your work. Looking forward to InGate!

Sincerely,

Shane

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/22203297-4a0d-4726-881e-e9f30e9bb1b6n%40googlegroups.com.

[Benjamin Elder]

Having a neutral, boring, straightforward implementation of these APIs will be extremely useful for Kubernetes's own testing purposes.

We currently use ingress-nginx with kind pretty often, but I would be very happy to leverage a reduced-scope baseline implementation instead.

Thank you for driving this effort.

+1

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAAiwNsjbZBsWCJFb4LZCQVCQ6u-OM5N2JMHrS0m9w%2BSYDAdkhQ%40mail.gmail.com.

[Benjamin Elder]

I think the rest of this mostly makes sense but ...

> In particular, we expect InGate to be documented differently than how ingress-nginx is documented--for example: InGate will not get any special mention in the main Kubernetes documentation as ingress-nginx does today.

This is really a discussion for kubernetes/website and not for the creation of the subproject.
By this argument we could drop docs from kubernetes/website for kubeadm and other cluster creation tools because we have commercial and open source alternatives.
IF we make the decision to not document the ability to use a community hosted basic implementation, that decision should happen in kubernetes/website and not be tied to the repo request.

> If at some point the maintainers were to pursue commercial opportunities related to InGate, it may be reasonable to reconsider whether the project might be better suited for placement within the CNCF than Kubernetes SIGs.

This is a restriction we do not place on any other project within Kubernetes and I would strongly caution against anything perceived as a ban on commercial usage or support or a ban on anyone choosing to use or support it commercially contributing to the project.

With my steering hat: I would be concerned about projects being shut down or kicked out because of commercial competition concerns, that is not in the spirit of the project or in line with our community values.

I would instead focus on ingate implementing only the official Kubernetes APIs which leaves room for vendors to compete on extensions, performance, and other means in a way that the SIG will not host.

We should focus on the scope, not whether anyone offers any support.

We should also trust folks like James to be working in the best interest of the community in-line with "community over product or company".
I think having a neutral baseline will drive adoption of the APIs and create more room for competition anyhow.

If limiting the scope to the standard community APIs isn't sufficiently non-threatening to commercial vendors, I don't think they have a viable product and that shouldn't be our concern as a community. 

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAHEVDqOVekqApWUiXmqVfiKczudsEA5opvEppH_jkY5uCTGjaA%40mail.gmail.com.

[Dan Winship]

On 11/22/24 13:10, 'Benjamin Elder' via kubernetes-sig-network wrote:
> I think the rest of this mostly makes sense but ...
>
>> In particular, we expect InGate to be documented differently than how
> ingress-nginx is documented--for example: InGate will not get any
> special mention in the main Kubernetes documentation as
> ingress-nginx does today
>

> This is really a discussion for kubernetes/website and not for the
> creation of the subproject.

What we're trying to say is that the SIG Network Leads do not intend for
InGate to be considered a "default" or "blessed" implementation of
Gateway, and in particular, the fact that it happens to be in
kubernetes-sigs is not intended to indicate that they have any special
status relative to other implementations. (Likewise, we would not expect
our official documentation to suggest that people should run
https://github.com/kubernetes-sigs/blixt as their Gateway implementation
just because it is in kubernetes-sigs.)

While I suppose SIG Docs can decide to document whatever they want, SIG
Network would find it very surprising if they mentioned InGate specially
in the documentation, unless they had carefully tested all
twenty-something existing Gateway implementations and determined that
InGate was clearly the best and most user-friendly one, and thus worthy
of being called out specially in the documentation.

> I would instead focus on ingate implementing only the official
> Kubernetes APIs which leaves room for vendors to compete on extensions,
> performance, and other means in a way that the SIG will not host.

Indeed, SIG Network would be very happy if InGate did that, but we don't
want to tell James, Ricardo, and the rest of the ingress-nginx team that
they *have to* do that. So what we're saying (both to them *and* to
everyone watching at home) is, SIG Network is happy to sponsor a
subproject that provides a reference implementation of the official
APIs, but we would not necessarily be as happy to sponsor a subproject
that was actively commercially competing with other Gateway
implementations. So if, at some point, the InGate maintainers suddenly
decide they want to put Envoy out of business, we'd be happier if they
did that as a CNCF project rather than as a SIG Network subproject.

-- Dan

[Benjamin Elder]

> While I suppose SIG Docs can decide to document whatever they want, SIG
Network would find it very surprising if they mentioned InGate specially
in the documentation, unless they had carefully tested all
twenty-something existing Gateway implementations and determined that
InGate was clearly the best and most user-friendly one, and thus worthy
of being called out specially in the documentation.

The reason it might be documented is *not* because "it's the best", but specifically to avoid documenting "the best" as opposed to "we have something you can use to get started, we have no statement about the best production offering".

This is why we currently have other pages like https://kind.sigs.k8s.io/docs/user/ingress/, to stay out of that debate while having something users can actually run to develop with the standard APIs.

Refusing to document a working neutral approach makes things harder for users.

Again, this is really a distinct thread we should have somewhere else though.

> Indeed, SIG Network would be very happy if InGate did that, but we don't

want to tell James, Ricardo, and the rest of the ingress-nginx team that
they *have to* do that. So what we're saying (both to them *and* to
everyone watching at home) is, SIG Network is happy to sponsor a
subproject that provides a reference implementation of the official
APIs, but we would not necessarily be as happy to sponsor a subproject
that was actively commercially competing with other Gateway
implementations. So if, at some point, the InGate maintainers suddenly
decide they want to put Envoy out of business, we'd be happier if they
did that as a CNCF project rather than as a SIG Network subproject.

The sentiment is understood, but the implied policy is still problematic.

Kubernetes's projects are open for anyone to contribute, within the scope of the project.

If some contributors (not James, Ricardo who we know are not doing this, but some future contributors) choose to offer commercial support, that is their prerogative.

Limiting the scope to deconflict is reasonable and actually something you absolutely could require of the project as a SIG-TL.
Potentially removing projects because of the employment of contributors and external offerings from contributors (as opposed to what code we merge into the project and how the project itself is run and scoped) is not a reasonable precedent.

That distinction is important.

[Shane Utt]

+1 to what Dan said, plus:

> This is a restriction we do not place on any other project within Kubernetes and I would strongly caution against anything perceived as a ban on commercial usage or support or a ban on anyone choosing to use or support it commercially contributing to the project.
>

> The sentiment is understood, but the implied policy is still problematic.

There seems to be a misunderstanding that I suggested some kind of new formal policy. I very deliberately tried to use soft language there, and was not actually intending to push for any formal restriction or policy. Let me be clear that the language "I also personally support keeping the option open to reassess the project's placement as it grows" and "it may be reasonable to reconsider" was intentionally soft and meant to suggest opportunities for discussion, not policy.

> We should also trust folks like James to be working in the best interest of the community

I do, and I would have liked to think the last line of my response made it clear how much I do.

---

Based on your response Ben, I think a mistake of mine here was that I should have explained the controversy further. Without that, I can see how the suggestions might come across as arbitrary.

As InGate has gradually evolved into a proposal over the past year, we (including the ingress-nginx team, who have been very responsive) have been navigating a controversy that, as previously mentioned, has arisen from the numerous other implementations of the Gateway API. Community members from Gateway API have expressed (mostly in private) significant frustration regarding the existence of any Gateway API implementation that even slightly resembles an "official" one. This concern originates from the early days of Gateway API: back in 2019/2020, we established a sort of "social contract" with those joining the subproject to assist, agreeing not to create any "reference implementation" to foster a truly level playing field. Many of us believe that this social contract has been crucial to Gateway API's collaborative success, as contributors felt secure knowing that all implementations would be treated equally. For five years, we upheld that contract, but when the ingress-nginx team began expressing interest in implementing the Gateway API, we started hearing feedback about the perceived unfairness to others. At the same time, it didn't seem just to govern a long-standing group with an unwritten rule that they never had the opportunity to contest.

I hope this clarifies the situation a bit. If you were in a position where you had contributed to Gateway API for many years, relying on an implementation you created in good faith on a level playing field, and then encountered the proposal for InGate, would you not feel threatened or cheated? Would it not be disheartening to see such a project gain the privilege of being part of kubernetes-sigs, which is sure to attract many users simply by virtue of being an "official project"? A privilege that your project, which you've dedicated years to, can never attain? I suspect that those with this perspective might hesitate to voice their concerns in a public forum, as it could be intimidating. With that in mind, I hope you understand that the intention behind the comments above is to seek a compromise between these groups and ensure that both sides are heard. If they feel overlooked, it could foster distrust within the community, and our contributors are essential to our success.

Note that I appreciate your input, and if you have any suggestions given all the above they are very welcome as the situation is somewhat unique.

[Benjamin Elder]

Thanks Shane, and I think I understand the intent from the beginning.
I just want to be clear for "those watching at home" as well about what hard policy looks like, for the NEXT tricky situation, so these things are not arbitrary.

Thank you all for working through sorting this out.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAHEVDqOgOLFVrQLJ6DAG9vhbSFg8N1dmdsv-_ooLWiibXgxOJw%40mail.gmail.com.

[Tim Hockin]

> I hope this clarifies the situation a bit. If you were in a position where you had contributed to Gateway API for many years, relying on an implementation you created in good faith on a level playing field, and then encountered the proposal for InGate, would you not feel threatened or cheated?

With full acknowledgement of the situation, I think this is a little
dramatic. The lack of a reference implementation has been pretty
painful in many ways (I know you know that, but for the record).
We're trying to thread a needle here - having something we can use
subject ourselves to the consequences of our own decisions, but not
eat at the ecosystem. I would hope that commercially available
implementations have a lot more to offer than this would, but that's
not really my business.

I support having a minimalist implementation in-house, and also not
advertising it in any sort of preferential way. I would also ask - is
nginx really the best vehicle for this?

Tim

On Fri, Nov 22, 2024 at 11:59 AM 'Benjamin Elder' via
kubernetes-sig-network <kubernetes-...@googlegroups.com>
wrote:

> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/CAOZRXm8n%2BEqGhHkCvkD-LFs80XybzOJjb2HB8jx%2B1UULTLTnuw%40mail.gmail.com.

[Keith Mattix]

 > I support having a minimalist implementation in-house, and also not
advertising it in any sort of preferential way. I would also ask - is
nginx really the best vehicle for this?

+1 to both of these points; I see a lot of benefit in there being an in-house reference implementation, absent any kind of preferred status. However, if the stated surface area of ingate is the core Gateway API surface area only, is there any specific benefit to choosing NGINX as the data plane? If the goal were an NGINX-based Gateway API implementation, there is at least 1 open source option available that's maintained by F5. As I understand it, one of the main goals of ingate is to help ingress-nginx users migrate; couldn't https://github.com/kubernetes-sigs/ingress2gateway translate ingress-nginx specific annotations to ~any implementation? I think generally I'm seeing a lot of ideas for things that we hope ingate will solve in aggregate but my question is whether or not the proposed details of ingate are the best solution to any specific problem stated in this thread.

-Keith

[Antonio Ojea]

I think that the discussion of the implementation details are
independent of the repository request.
Since there are not outstanding comments my take is that the
repository request has been ACCEPTED.

Please follow the rules to request a new one
https://github.com/kubernetes/community/blob/master/github-management/kubernetes-repositories.md

> To view this discussion visit https://groups.google.com/d/msgid/kubernetes-sig-network/be3ee51e-4478-4579-bbca-3fc3d1600d1cn%40googlegroups.com.

[James Strong]

Thank you all for the discussion points. 

The request for the new repo is here https://github.com/kubernetes/org/issues/5280

Thanks,

James