Production AI inference platform
How to use
FriendliAI.
Serving open models through instant APIs, deploying pinned Hugging Face models on dedicated GPUs, running private inference containers, and operating production inference with explicit scaling and observability.
FriendliAI is a AI inference platform with serverless model APIs, dedicated GPU endpoints, private containers, observability, and OpenAI-compatible interfaces. Serving open models through instant APIs, deploying pinned Hugging Face models on dedicated GPUs, running private inference containers, and operating production inference with explicit scaling and observability. This guide covers the whole path in one place: official access, a first session that produces something reviewable, the checks that make output trustworthy, and the permissions worth limiting before you connect real work.
AI should make the work easier to inspect. If the workflow removes the source, the owner, or the review step, redesign the workflow.
Access & setup
Find, install, and sign in to FriendliAI
Get into the official FriendliAI experience with the right account and a setup you understand.
- 01
Start at https://friendli.ai/ and confirm the domain before entering account or payment information.
- 02
Availability: FriendliAI is available through Friendli Suite in a browser, HTTPS and Python APIs, serverless endpoints, dedicated managed deployments, and on-premises containers.
- 03
A Friendli account and scoped personal API key, a project and deployment mode, licensed model weights, GPU and autoscaling choices where dedicated, privacy and logging settings, evaluation data, and cost controls.
- 04
Sign in with the account you intend to keep using, then review plan, data, notification, and permission settings.
- 05
Run one low-risk test task before connecting sensitive files, repositories, or workspace data.
- Use official download pages.
- Review permissions during setup.
- Keep installers and applications updated.
First session
Your first useful FriendliAI session
Learn the interaction loop using a small task with a clear outcome.
- 01
Define the workload, data class, model and revision, license, throughput, latency, context, GPU, scaling, logging, privacy, availability, budget, and rollback targets.
- 02
Use serverless for evaluation, dedicated endpoints for predictable managed capacity, or containers for stronger infrastructure control, documenting the shared-responsibility boundary.
- 03
Deploy with a scoped server-side key, pinned model revision, conservative token and batch limits, content logging disabled unless approved, and minimum safe replicas.
- 04
Load-test quality, latency, concurrency, cold start, autoscaling, failure, privacy, and cost, then canary the version, monitor it, and retain termination and rollback procedures.
- State the outcome before the background.
- Provide the real source material.
- Review the result before expanding the task.
Quality control
Check the quality of FriendliAI output
Establish that a change is safe to run in production and reversible if it is not.
- 01
Restate the intended end state and the blast radius before applying anything.
- 02
Review the generated configuration line by line against the provider's current documentation.
- 03
Apply to a non-production environment first and confirm the observed result matches the intended one.
- 04
Confirm the rollback path works by actually exercising it, not by assuming it exists.
- 05
Check cost, scaling limits, and network exposure before the change reaches production traffic.
- Test the rollback, do not assume it.
- Check what a configuration exposes to the public internet.
- Watch cost and rate limits as closely as correctness.
Privacy & permissions
Use FriendliAI safely
Keep credentials, network exposure, and cost under deliberate control.
- 01
Use scoped, short-lived credentials, and never paste production secrets into a prompt or config file.
- 02
Confirm what each change exposes publicly before applying it, especially storage, databases, and admin endpoints.
- 03
Separate environments so a mistake in development cannot reach production data.
- 04
Set billing alerts and hard quotas before enabling autoscaling or usage-based services.
- 05
Review audit logs and revoke access for integrations that are no longer in use.
- OpenAI compatibility does not guarantee identical tokenization, tools, errors, moderation, or output. Dedicated endpoint management APIs are marked beta, content logging is configurable, and a model repository can change unless its revision is pinned. Verify model rights, keep keys server-side, minimize logged content, separate environments and teams, cap autoscaling spend, test sleeping and failure states, and never update a production endpoint without an evaluated version and rollback.
- Follow your organisation's approved-use policy.
- Never treat fluent output as authorization to act.
Core workflows
Step-by-step ways to use FriendliAI for the work it does best.
Each workflow is a separate guide with its own steps and review checkpoints.
Use serverless evaluation to justify a pinned dedicated deployment with predictable capacity, observability, and rollback.
↗ 02 WorkflowOperate a governed Friendli dedicated endpointManage versions, autoscaling, teams, keys, logging, failures, and costs without making beta management APIs a single point of failure.
↗Official references
Check the current product documentation.
Features, plan limits, availability, and data controls change. These official pages are the starting points used for this guide.