The outcome

Provide a useful model endpoint without granting consumers model-management, registration, log, cache, or administrative rights.

Step by step

A workflow you can repeat.

  1. 01

    Define task, exact model and license, engine, hardware, context, quality, safety, latency, concurrency, storage, network, user permissions, retention, and rollback targets.

  2. 02

    Install and pin Xinference, initialize the administrator before exposure, persist auth keys and database, then create a user and API key limited to model listing and inference.

  3. 03

    Verify and launch a pinned built-in or custom model with bounded devices and replicas, private access, explicit model UID, and no unreviewed remote or custom code.

  4. 04

    Run representative correctness, safety and load tests through the native and intended OpenAI-compatible clients, recording output parity, latency, memory, recovery, errors, and cost.

  5. 05

    Canary consumers, monitor model and permission activity, rotate keys, archive the manifest, and test stop, relaunch, cache cleanup, rollback, and deletion without granting write access.

Working standard

What good use looks like.

  • Separate inference from model-management rights.
  • Persist and back up auth state.
  • Vet all custom model code.

Official references

Check the current product documentation.