This Week in CN AI (19th April 2024)

Dive into this week’s key updates brought to you by the CNAI working group, including a new Kubernetes WG Serving proposal, advancements in AI monitoring with OpenTelemetry, and innovative projects like the Lingo Project and OPEA framework LLM engine. Explore community-driven initiatives such as the CNCF YouTube Summarizer and the evolving Cloud Native AI landscape and more!

Key Developments

1. WG Serving Proposal in Kubernetes Community

The Kubernetes community has introduced a new proposal for a Working Group (WG) Serving to enhance serving capabilities within the ecosystem. The proposal (#7823) involves collaboration across multiple SIGs including apps, architecture, node, scheduling, and autoscaling. This broad involvement indicates its potential wide-reaching impact on Kubernetes.

2. Advancement & Innovations in CNCF AI Technologies

  • The OpenTelemetry community has merged the first PR for defining semantic conventions for Large Language Models (LLMs). This initiative is part of a broader effort by the otel-llm-semconv-wg to standardize and monitor AI systems, paving the way for future AI-related telemetry enhancements.
  • Lingo Project: Introduced in the TAG-Runtime meeting, this project focuses on a lightweight ML model proxy and autoscaler specifically designed for Kubernetes, underscoring the ongoing integration of machine learning with cloud-native infrastructure.
  • The Open Policy Agent community demoed a proposed experimental AI/LLM engine :sparkles: (in addition to CEL and Rego engines) in the :opa: Gatekeeper community call that would simplify policy writing. Recording for those interested and just the demo here.
  • OPEA: Ecosystem Orchestration for GenAI Adoption: OPEA is an ecosystem orchestration framework that aims to integrate performant GenAI technologies and workflows, enabling faster GenAI adoption and delivering business value.

3. CNCF Community Activities

  • CNAI Mission/Vision: To clarify the shared understanding of the group’s scope, goals, and mission, and initiatives, a draft strategy document was created, as well as a new cloud native AI github project to track the work and highlight progress.
  • CNCF Youtube Summarizer Project: Discussed in recent community calls, this project aims to use AI to enhance content accessibility and summarization on CNCF’s YouTube channel.
  • Kubernetes Book Club: The community has begun exploring the book “Machine Learning on Kubernetes”, adapting its content to better fit current trends and the specific needs of the community.
  • Cloud Native AI Landscape: progress made on getting a PoC for a cloud native AI landscape up and running, contributions are welcome.

Conclusion

This week has been bustling with activity, highlighting the CNCF’s AI community’s ongoing efforts to integrate AI into the cloud native landscape. These developments are geared towards building a robust ecosystem equipped to handle the future demands of AI. Stay tuned for more updates as these projects progress and continue to shape the landscape.

If you’d like to contribute, we are happy to welcome new ideas and memebers to the community, you can find us fastest at #wg-artificial-intelligence.