Inference as a service, on dedicated GPUs.
Three planned services on single-tenant NVIDIA HGX systems: an API to open-weight models, managed hosting for a model you provide as weights, and fine-tuning on your data.
- Your applicationSends requests
- API endpointPrivate to your organization
- ModelOpen-weight, or your own weights
- Serving and operationsPackaging, scaling, monitoring
- ComputeSingle-tenant NVIDIA HGX B200 and B300
Three services. Start from what you have.
None of these services are available yet. They are planned for our first deployment and will be offered to design partners first.
What do you bring?
Inference as a service
For teams that want an open-weight model behind an API and do not want to operate GPUs.
How it works: Inference as a service- You bringRequests from your applications, and nothing else.
- We runAn open-weight model on GPUs reserved for you.
- You getAn API endpoint your applications call.
Managed model hosting
For teams that fine-tuned a model themselves and want it served without building a serving stack.
How it works: Managed model hosting- You bringThe weights of a model you trained or fine-tuned.
- We runPackaging, deployment, scaling, and monitoring.
- You getA private endpoint serving your model.
Fine-tuning service
For teams that have the data for a custom model and want someone else to run the training.
How it works: Fine-tuning service- You bringYour data, and a choice of base model.
- We runThe fine-tuning, and an evaluation agreed with you.
- You getThe fine-tuned weights, which we can also host.
Inference as a service: bring only requests.
We will run an open-weight model on GPUs reserved for you. Your applications call an API.
1. You choose an open-weight model
Tell us which model you need and how much traffic to expect. We will size the capacity with you.
2. We deploy it on GPUs reserved for you
The model will run on single-tenant systems. No other customer will share them.
3. Your applications call the API
We plan to offer an OpenAI-compatible API at launch. Requests will reach the model over a private endpoint.
Managed model hosting: bring the weights.
You provide the weights of a model you trained or fine-tuned. We will do the rest.
1. You send the weights
You will send the weights over a private transfer. We plan to accept standard formats such as safetensors.
2. We package and load-test the model
We will package the model for serving and load-test it on the hardware it will run on. We share the results with you.
3. The endpoint goes live
Your model will be served from a private endpoint, on hardware reserved for you.
4. We run it
We will monitor the endpoint and scale it inside your reserved capacity. Updates happen only with your approval.
Fine-tuning: bring the data.
You provide the data and pick a base model. We will fine-tune it, evaluate it with you, and return the weights.
1. You send data and choose a base model
You will send the training data over a private transfer. We choose an open-weight base model with you.
2. We agree on the method and the evaluation set
Before any training starts, we will agree on the fine-tuning method and the evaluation set with you.
3. We train and evaluate
We will fine-tune the model on single-tenant hardware and run the agreed evaluation. You get the evaluation results.
4. You receive the weights
The fine-tuned weights are delivered to you. We can host them as a managed model, or you can run them elsewhere.
What stays yours.
Commitments that apply to all three services.
Your weights and data remain yours
Weights and data you send us stay your property. Sending them to us does not change who owns them.
Nothing of yours trains another model
We will not use your weights, your data, or your prompts to train or improve any other model.
U.S. facilities, single-tenant hardware
Your weights and data will stay in U.S. facilities, on hardware that no other customer shares.
Fine-tuned weights are yours to take elsewhere
Weights we fine-tune for you are delivered to you. You can run them with us or with anyone else.
Deleted when the agreement ends, or when you ask
We will delete your data and weights when the agreement ends, or earlier at your request.
The same hardware as our GPU cloud.
All three services will run on single-tenant NVIDIA HGX B200 and B300 systems in U.S. facilities. These are the systems we offer as dedicated clusters.
Inference and model hosting questions
Which models will you serve?
Open-weight models, meaning models whose weights are published for anyone to run, and your own models. We have not published a model list, because the service is not running yet. Tell us which model you need and we will tell you whether we plan to serve it.
What format should our weights be in?
We plan to accept weights in standard formats such as safetensors. The list of accepted formats is not final. If your model is in another format, tell us and we will say whether we can take it.
Will our data or prompts be used to train anything?
No. We will not use your data, your weights, or your prompts to train or improve any other model. Data you send for fine-tuning is used only to fine-tune your model.
Can we take fine-tuned weights elsewhere?
Yes. Weights we fine-tune for you are yours. We deliver them to you, and you can run them with us or with anyone else.
Is the hardware shared with other customers?
No. Each customer’s models will run on single-tenant NVIDIA HGX systems reserved for that customer, in U.S. facilities.
When will these services be available?
None of them is available today. They are planned for our first deployment and will be offered to design partners first. We do not publish dates yet.
Talk to us about a model.
Tell us the model, the traffic you expect, or the data you have. An engineer will reply with how we would run it.
Thank you. Your inquiry is with our team.
Someone from our team will reply within two business days. If your request involves a solicitation deadline, mention the date in your reply and we will prioritize it.