Heterogeneous Inference Endpoints

The managed inference tool registry stores endpoints of one or more types, including general-purpose language models, task-specific fine-tuned language models, image classifiers, speech recognition models, embedding models, retrieval models, and personal corpus models. Multiple endpoints of distinct types and distinct sizes may be co-resident in the tool registry concurrently, subject to local memory and storage constraints. Each endpoint is a managed inference endpoint subordinate to the agent and subject to governed lifecycle operations.

Disclosure Scope

This article describes subject matter disclosed in U.S. Provisional Application No. 64/070,239.