The outcome

Select a current production model using repeatable evidence rather than throughput alone.

Step by step

A workflow you can repeat.

  1. 01

    Define representative prompts, expected outputs, safety cases, context lengths, latency targets, and a scoring rubric.

  2. 02

    Review the current supported-model list and separate production offerings from previews or models with announced deprecation dates.

  3. 03

    Run identical tests with fixed parameters and record accuracy, time to first token, total latency, token use, refusals, and errors.

  4. 04

    Repeat at realistic concurrency and input sizes while monitoring organization rate-limit headers and bounded retry behavior.

  5. 05

    Pin the selected model ID, document compatible fallbacks and thresholds, and rerun the benchmark before any model migration.

Working standard

What good use looks like.

  • Evaluate production models separately from previews.
  • Measure end-to-end latency.
  • Track announced deprecations.

Official references

Check the current product documentation.