EnMaaS: Red Hat’s pilot for open source inference

As frontier model inference costs soar, Red Hat has addressed the challenge head-on by proving that enterprise-grade AI does not require exclusive reliance on expensive commercial endpoints. Through Engineering Model-as-a-Service (EnMaaS), our internal MaaS pilot deployed on Red Hat AI, we united Praxis—a framework for building proxy servers which also provides an AI gateway built on that framework—with llm-d for distributed open weight model serving. Deploying this open source architecture across our global engineering organization proved that intelligent proxying and…

Quelle: Originalartikel öffnen