Enterprise AI inference
How to use
SambaNova Cloud.
Testing and integrating high-throughput hosted language models through a familiar API surface, with enterprise deployment options through SambaStack.
SambaNova Cloud is a hosted inference API for production and preview generative AI models. Testing and integrating high-throughput hosted language models through a familiar API surface, with enterprise deployment options through SambaStack. This guide covers the whole path in one place: official access, a first session that produces something reviewable, the checks that make output trustworthy, and the permissions worth limiting before you connect real work.
AI should make the work easier to inspect. If the workflow removes the source, the owner, or the review step, redesign the workflow.
Access & setup
Find, install, and sign in to SambaNova Cloud
Get into the official SambaNova Cloud experience with the right account and a setup you understand.
- 01
Start at https://cloud.sambanova.ai/ and confirm the domain before entering account or payment information.
- 02
Availability: SambaCloud is accessed through its web platform and API using Python, JavaScript, curl, the SambaNova SDK, or a compatible client.
- 03
A SambaCloud account or managed SambaStack access, an API key and base URL, network access, and a compatible development environment.
- 04
Sign in with the account you intend to keep using, then review plan, data, notification, and permission settings.
- 05
Run one low-risk test task before connecting sensitive files, repositories, or workspace data.
- Use official download pages.
- Review permissions during setup.
- Keep installers and applications updated.
First session
Your first useful SambaNova Cloud session
Learn the interaction loop using a small task with a clear outcome.
- 01
Generate a development API key, save it securely, and configure the documented base URL outside source code.
- 02
Choose a current production model rather than a preview model for any stability-sensitive workload.
- 03
Run a fixed benchmark and record output quality, latency, token use, formatting, and failures.
- 04
Inspect rate-limit headers and test quota, retry, validation, and fallback behavior before production use.
- State the outcome before the background.
- Provide the real source material.
- Review the result before expanding the task.
Quality control
Check the quality of SambaNova Cloud output
Establish that a model's output is reliable enough for the job you are handing it.
- 01
Restate the task, the acceptance criteria, and the failure cost before evaluating any response.
- 02
Test with a fixed evaluation set rather than ad-hoc prompts, so changes are measurable across model versions.
- 03
Check determinism and drift: record model version, parameters, and date, because providers update models underneath you.
- 04
Probe the failure modes deliberately — ambiguous inputs, adversarial phrasing, and out-of-scope requests.
- 05
Validate structure as well as content: confirm schemas, types, and required fields survive real inputs.
- Pin and record the model version with every result.
- Evaluate against a fixed set, not one-off prompts.
- Validate output structure before it reaches downstream code.
Privacy & permissions
Use SambaNova Cloud safely
Understand what leaves your system on every call, and what the provider keeps.
- 01
Classify the data in each request before it is sent to a hosted model.
- 02
Check the provider's retention and training terms for your specific plan, not the marketing page.
- 03
Store API keys in a secret manager, scope them per project, and rotate them on a schedule.
- 04
Strip or tokenise personal and regulated data before it reaches the request body.
- 05
Set rate limits, spend caps, and monitoring so a runaway loop is bounded and visible.
- Preview models have limited capacity and may disappear at short notice. Protect keys, pin exact model IDs, validate outputs, bound retries and tokens, inspect current per-user limits, and do not log sensitive prompts indiscriminately.
- Follow your organisation's approved-use policy.
- Never treat fluent output as authorization to act.
Core workflows
Step-by-step ways to use SambaNova Cloud for the work it does best.
Each workflow below has its own outcome, steps and review checkpoints.
Compare currently available production models while keeping preview offerings out of critical workloads.
↓ 02 WorkflowBuild a resilient SambaNova API clientUse the compatible API surface with secure keys, validated responses, retries, and model-change controls.
↓Workflow 01
Choose a SambaCloud model with a controlled benchmark
Compare currently available production models while keeping preview offerings out of critical workloads.
Outcome: Choose a model based on measured quality, speed, limits, and stability for the intended application.
- 01
Define task examples, expected outputs, context requirements, safety cases, latency targets, and a consistent scoring method.
- 02
Inspect the current SambaCloud model catalog and label each candidate as production or preview before testing.
- 03
Run identical prompts with fixed parameters and record output quality, latency, token use, formatting, refusals, and errors.
- 04
Test realistic request volume against current per-user limits and inspect response headers rather than relying on static quotas.
- 05
Select a production primary and compatible fallback, record exact model IDs, and schedule regression tests for catalog changes.
What good use looks like
- Keep preview models out of production paths.
- Test the actual workload.
- Read current response-limit headers.
Workflow 02
Build a resilient SambaNova API client
Use the compatible API surface with secure keys, validated responses, retries, and model-change controls.
Outcome: Integrate SambaCloud predictably while containing credential, quota, and output risks.
- 01
Create separate keys for development and production, store them as secrets, and configure the documented base URL server-side.
- 02
Set explicit model, timeout, output-token, concurrency, and retry settings instead of relying on client defaults.
- 03
Validate structured responses and any tool arguments before downstream use, with clear handling for refusals and partial streams.
- 04
Exercise rate limits, daily quota, invalid keys, unavailable models, timeouts, malformed output, and 5xx responses in staging.
- 05
Log request identifiers, model, latency, tokens, status, and remaining quota safely, then alert before capacity is exhausted.
What good use looks like
- Separate keys by environment.
- Use bounded backoff for transient failures.
- Never treat generated arguments as authorization.
Official references
Check the current product documentation.
Features, plan limits, availability, and data controls change. These official pages are the starting points used for this guide.