AI Agent Abstraction in Kubernetes

[Joe Huang]

Hi everyone,

As we know, The Pod abstraction, an elegant layer atop containers, has been incredibly successful, forming the foundation for K8S/CNCF’s vast ecosystem(as illustrated in this AI-generated diagram — minor errors aside: https://github.com/ChaoyiHuang/aiagent/blob/main/images/pod-abstraction-in-K8S-CNCF.png).

Yet, the Kubernetes/CNCF ecosystem lacks a core abstraction for AI Agents—one flexible enough and Pod-like resource to support ANY existing framework (LangChain, ADK, OpenClaw, Hermes, etc.) and ANY future innovations. Without this, critical platform engineering capabilities for AI Agents remain fragmented.

To kickstart the conversation, I’ve drafted a proposal: AI Agent Abstraction in K8s (https://github.com/ChaoyiHuang/aiagent/blob/main/docs/design/ai-agent-abstraction-EN.md). It’s early-stage and very imperfect — I genuinely welcome all alternative ideas and proposals — this proposal isn’t about pushing a solution, but sparking discussion.

As Janet suggested, let’s discuss this in sig-apps Slack/mailing list. The goal is to discuss, collaborate, refine, and converge on a shared direction.

Let’s shape this together!

Best Regards

Chaoyi Huang (Joe Huang)

[Joe Huang]

Hi, all,

For better collaboration on the proposal, the "AI Agent Abstraction in Kubernetes"  markdown file was imported to google doc:

https://docs.google.com/document/d/1T37f_nTvJF5z8NYYqAF-F6eHbiO1FdEhYfd8jJDmqus/edit?usp=sharing

And now edit/comment are open for all. Your thoughts are welcom.

Best,

Chaoyi Huang(Joe Huang)