How does billing work?

You incur costs for using Red Hat AI Inference. If you use model alignment, you also incur costs for IBM Cloud® Object Storage, which is used as a storage location.

If you deploy your model on another service, you incur additional charges from that service.

How is cost calculated in Red Hat AI Inference?

The cost from Red Hat AI Inference usage is based on metrics that are measured in tokens. Each token corresponds to a specific amount of computational power that is required for the processing tasks. The total number of tokens consumed directly influences the scale of data generation, model fine-tuning, or inference. This metric serves as a basis for our billing system, enabling users to monitor and control their costs according to the computational resources used.

Synthetic data generation (SDG)

Output tokens (SYN-DATA-TOKEN) are calculated by the volume of generated data produced by the service from the entire input taxonomy. The text is tokenized by using Hugging Face's tokenizer library with the tokenization information for the Mistral teacher model.

Model alignment training

Input tokens (MODEL-TRAIN-TOKEN) are calculated based on the amount of data fed that into the system for model alignment training, as well as the Granite base knowledge that is used to increase accuracy without knowledge loss. Because of the foundational knowledge that is used, there is a minimum cost.

Inference with a model

Inference costs are calculated separately for input and output tokens on a per-model basis. Input tokens represent your prompt or query sent to the model, while output tokens represent the model's generated response. Each model has its own pricing structure based on its size and computational requirements.

How do I find and track cost information as I train a model?

1. Before you begin running anything in Red Hat AI Inference, you can use the cost estimator to get an estimate of what the cost might be.

2. Upload your taxonomy

3. When you set the details for data generation, review the estimated cost before starting the job.

4. After the data is generated, you can also view the estimated Output tokens in the details for that data generation.

5. When you set the details for the training job, review the estimated cost before starting the job.

6. When the jobs are complete, you can review the actual costs in Billing and usage. You can view the costs for both Red Hat AI Inference and the IBM Cloud® Object Storage service, which is used as a storage location.

Are failed operations billed?

Failed operations are not billed. Successful operations and user canceled operations are billed, though user canceled operations are prorated based on the processing that completed.