Open-model AI cloud
How to use
Together AI.
Comparing and serving open models through serverless inference, dedicated endpoints, evaluations, fine-tuning, and managed GPU infrastructure.
Together AI is a cloud platform for running, evaluating, fine-tuning, and deploying open generative AI models. Comparing and serving open models through serverless inference, dedicated endpoints, evaluations, fine-tuning, and managed GPU infrastructure. This guide covers the whole path in one place: official access, a first session that produces something reviewable, the checks that make output trustworthy, and the permissions worth limiting before you connect real work.
AI should make the work easier to inspect. If the workflow removes the source, the owner, or the review step, redesign the workflow.
Access & setup
Find, install, and sign in to Together AI
Get into the official Together AI experience with the right account and a setup you understand.
- 01
Start at https://www.together.ai/ and confirm the domain before entering account or payment information.
- 02
Availability: Available through a web platform, OpenAI-compatible APIs, SDKs, and a Python-based command-line interface.
- 03
A Together AI account and API key, network access, and a compatible programming or CLI environment for programmatic use.
- 04
Sign in with the account you intend to keep using, then review plan, data, notification, and permission settings.
- 05
Run one low-risk test task before connecting sensitive files, repositories, or workspace data.
- Use official download pages.
- Review permissions during setup.
- Keep installers and applications updated.
First session
Your first useful Together AI session
Learn the interaction loop using a small task with a clear outcome.
- 01
Create a development API key and keep it in an environment secret rather than code or command arguments.
- 02
Choose a current serverless model after defining task, modality, context, quality, safety, and budget requirements.
- 03
Run a minimal request, then evaluate a fixed set of representative and adversarial examples.
- 04
Record model ID, parameters, output quality, latency, token use, cost, and failures before production integration.
- State the outcome before the background.
- Provide the real source material.
- Review the result before expanding the task.
Quality control
Check the quality of Together AI output
Establish that a change is safe to run in production and reversible if it is not.
- 01
Restate the intended end state and the blast radius before applying anything.
- 02
Review the generated configuration line by line against the provider's current documentation.
- 03
Apply to a non-production environment first and confirm the observed result matches the intended one.
- 04
Confirm the rollback path works by actually exercising it, not by assuming it exists.
- 05
Check cost, scaling limits, and network exposure before the change reaches production traffic.
- Test the rollback, do not assume it.
- Check what a configuration exposes to the public internet.
- Watch cost and rate limits as closely as correctness.
Privacy & permissions
Use Together AI safely
Keep credentials, network exposure, and cost under deliberate control.
- 01
Use scoped, short-lived credentials, and never paste production secrets into a prompt or config file.
- 02
Confirm what each change exposes publicly before applying it, especially storage, databases, and admin endpoints.
- 03
Separate environments so a mistake in development cannot reach production data.
- 04
Set billing alerts and hard quotas before enabling autoscaling or usage-based services.
- 05
Review audit logs and revoke access for integrations that are no longer in use.
- Model availability, behavior, licensing, rate limits, and price vary. Protect credentials and datasets, evaluate outputs, isolate environments, monitor spend, and stop dedicated capacity when it is not needed.
- Follow your organisation's approved-use policy.
- Never treat fluent output as authorization to act.
Core workflows
Step-by-step ways to use Together AI for the work it does best.
Each workflow below has its own outcome, steps and review checkpoints.
Test open models against a task-shaped benchmark before integrating a serverless endpoint.
↓ 02 WorkflowMove from Together serverless inference to a dedicated endpointDecide whether reserved model capacity is justified and deploy it with measured scaling and cost controls.
↓Workflow 01
Select a Together AI model with a repeatable evaluation
Test open models against a task-shaped benchmark before integrating a serverless endpoint.
Outcome: Select a model based on measured quality, cost, latency, safety, and operational fit.
- 01
Define the workload, modalities, context, output schema, license constraints, safety cases, budget, and scoring method.
- 02
Shortlist current serverless models and verify capability, pricing, deprecation status, and provider documentation.
- 03
Run identical examples with fixed settings and record outputs, latency, token use, cost, refusals, and formatting failures.
- 04
Use a held-out evaluation set or the evaluation service to compare finalists without tuning prompts to individual examples.
- 05
Choose a primary and compatible fallback, pin model identifiers, and establish regression thresholds for future changes.
What good use looks like
- Evaluate your real task.
- Keep a held-out test set.
- Record the exact model and parameters.
Workflow 02
Move from Together serverless inference to a dedicated endpoint
Decide whether reserved model capacity is justified and deploy it with measured scaling and cost controls.
Outcome: Provide predictable inference for steady or customized workloads without leaving idle resources unmanaged.
- 01
Measure serverless traffic, latency, errors, utilization pattern, and cost, then define the dedicated endpoint's service objective.
- 02
Confirm model eligibility, license, hardware choices, regional needs, expected replicas, and hourly cost under each scaling scenario.
- 03
Create a staging endpoint with credentials stored in a secret manager and exercise it through the same inference API.
- 04
Load-test realistic concurrency, cold starts, scaling, timeouts, failures, and output quality before routing production traffic.
- 05
Roll out gradually, monitor utilization and spend, and stop or resize unused endpoints using a documented rollback path.
What good use looks like
- Justify dedicated capacity with measurements.
- Load-test before cutover.
- Stop idle endpoints deliberately.
Official references
Check the current product documentation.
Features, plan limits, availability, and data controls change. These official pages are the starting points used for this guide.