Inference Instances

Get Inference models

This endpoint allows you to view available AI models for deployment, aiding in selecting the right model for your needs.

Check model features to match your specific project requirements.

get

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --url '/ai-inference/models' \
  --header 'x-api-key: text'

200

200

[
  {
    "id": "12ed7523-432c-48f5-b3cd-32e6726d07c8",
    "name": "stable-diffusion",
    "port": 8000,
    "defaultHardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
    "features": [
      "image"
    ]
  }
]

Get inference hardware

You have several hardware options available through the AI inference feature of Sesterce Cloud (you can consult this section for more information). This endpoint allows you to explore options for deploying AI instances, which are crucial for planning resources and manage latency rate.

Evaluate hardware capabilities to ensure optimal performance for your AI tasks.

get

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --url '/ai-inference/hardwares' \
  --header 'x-api-key: text'

200

200

[
  {
    "id": "59651ba4-657a-41d4-8c42-00f34f732375",
    "name": "1xL40S / 16 vCPU / 232GiB RAM",
    "cpu": 16000,
    "ram": 237568,
    "gpu": {
      "model": "NVIDIA-L40S",
      "count": 1,
      "memory": 48
    }
  }
]

Get Regions available for inference instances

Identify available regions for deploying AI instances, important for compliance and latency considerations.

The region choice is a crucial parameter for your inference endpoint hosting. It will determine the latency rate for your final end-users. Choose regions that align with your data residency and latency needs.

get

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --url '/ai-inference/regions' \
  --header 'x-api-key: text'

200

200

[
  {
    "id": 18,
    "name": "Singapore",
    "countryCode": "SG",
    "state": "ACTIVE",
    "capacities": [
      {
        "hardwareId": "30328755-51f0-4251-b535-029358700099",
        "capacity": 38
      },
      {
        "hardwareId": "dc172cc8-0035-4527-958b-775f951e8836",
        "capacity": 19
      }
    ]
  }
]

Create a Registry

A registry is necessary if you need to infere your own custom model, which is not publicly available. Click here to learn more about Registries on Sesterce Cloud AI Inference service!

post

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Body

urlstringrequired

Example: docker.io/library/user/image:tag

usernamestringrequired

Example: someusername

passwordstringrequired

Example: securepassword

namestringrequired

Example: example-registry

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request POST \
  --url '/ai-inference/registries' \
  --header 'x-api-key: text' \
  --header 'Content-Type: application/json' \
  --data '{
    "url": "docker.io/library/user/image:tag",
    "username": "someusername",
    "password": "securepassword",
    "name": "example-registry"
  }'

application/json

201

201

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-registry",
  "url": "docker.io/library/user/image:tag",
  "username": "someusername",
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Get the list of registries created

To manage your registries for storing and accessing AI models, use the following endpoint:

get

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --url '/ai-inference/registries' \
  --header 'x-api-key: text'

200

200

[
  {
    "_id": "6721058be81810b9dd045f40",
    "name": "example-registry",
    "url": "docker.io/library/user/image:tag",
    "createdAt": "2019-06-26T13:00:00.000Z",
    "updatedAt": "2019-06-26T13:00:00.000Z"
  }
]

Update a registry

To modify registry details to ensure they meet current security and access needs, use the following endpoint:

patch

Path parameters

idstringrequired

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Body

urlstringrequired

Example: docker.io/library/user/image:tag

usernamestringrequired

Example: someusername

passwordstringrequired

Example: securepassword

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request PATCH \
  --url '/ai-inference/registries/{id}' \
  --header 'x-api-key: text' \
  --header 'Content-Type: application/json' \
  --data '{
    "url": "docker.io/library/user/image:tag",
    "username": "someusername",
    "password": "securepassword"
  }'

application/json

204

404

204

No body

Registry successfully updated.

Delete a Registry

The following endpoint allows to remove outdated or unused registries to maintain a clean environment.

delete

Path parameters

idstringrequired

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request DELETE \
  --url '/ai-inference/registries/{id}' \
  --header 'x-api-key: text'

204

404

204

No body

Registry successfully deleted.

Create an inference instance

Time has come! You can now deploy a new AI inference instance to scale your applications and services, or deploy in production an existing model! Use the following endpoint to perform this action.

To create an inference instance, check that your credit balance is filled. Please check here our documentation to top up your balance.

post

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Body

modelIdstring | nullablerequired

Example: 201a99c3-7cd4-4831-865e-b261082fda4b

containerPortnumberrequired

Example: 80

namestringrequired

Example: example-inference-instance

descriptionstring | nullablerequired

Example: example description.

autoScalingConfigurationsobject[]required

podLifetimenumber | nullablerequired

Example: 120

envsobject | nullablerequired

Example: {"PORT":"3333"}

hardwareIdstringrequired

Example: 59651ba4-657a-41d4-8c42-00f34f732375

startupCommandstring | nullablerequired

Example: npx create-llama

registryIdstring | nullablerequired

Example: 6721058be81810b9dd045f40

apiKeyIdsstring[] | nullablerequired

Example: ["6721058be81810b9dd045f40"]

Responses

application/json

application/json

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request POST \
  --url '/ai-inference/instances' \
  --header 'x-api-key: text' \
  --header 'Content-Type: application/json' \
  --data '{
    "modelId": "201a99c3-7cd4-4831-865e-b261082fda4b",
    "containerPort": 80,
    "name": "example-inference-instance",
    "description": "example description.",
    "autoScalingConfigurations": [
      {
        "regionId": 18,
        "scale": {
          "min": 1,
          "max": 4,
          "cooldownPeriod": 300,
          "triggers": {
            "cpu": {
              "threshold": 92
            },
            "gpuMemory": {
              "threshold": 92
            },
            "gpuUtilization": {
              "threshold": 92
            },
            "memory": {
              "threshold": 92
            },
            "http": {
              "rate": 12,
              "window": 20
            }
          }
        }
      }
    ],
    "podLifetime": 120,
    "envs": {
      "PORT": "3333"
    },
    "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
    "startupCommand": "npx create-llama",
    "registryId": "6721058be81810b9dd045f40",
    "apiKeyIds": [
      "6721058be81810b9dd045f40"
    ]
  }'

application/json

201

400

404

201

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-inference-instance",
  "status": "ACTIVE",
  "hourlyPrice": 2.55,
  "features": [
    "chat"
  ],
  "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Start an inference instance

This endpoint allows you to activate an AI inference instance to begin processing tasks and data.

You can monitor startup times to assess performance efficiency.

post

Path parameters

idstringrequired

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request POST \
  --url '/ai-inference/instances/{id}/start' \
  --header 'x-api-key: text'

204

404

204

No body

Inference instance successfully started

Get the list of your Inference instances

Here is the endpoint to monitor your active AI instances to manage resources and performance.

get

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --url '/ai-inference/instances' \
  --header 'x-api-key: text'

200

200

[
  {
    "_id": "6721058be81810b9dd045f40",
    "name": "example-inference-instance",
    "status": "ACTIVE",
    "hourlyPrice": 2.55,
    "features": [
      "chat"
    ],
    "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
    "createdAt": "2019-06-26T13:00:00.000Z",
    "updatedAt": "2019-06-26T13:00:00.000Z"
  }
]

Get details about a specific Inference Instance

Retrieve detailed information about a specific AI instance for management and troubleshooting.

get

Path parameters

idstringrequired

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --url '/ai-inference/instances/{id}' \
  --header 'x-api-key: text'

200

404

200

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-inference-instance",
  "status": "ACTIVE",
  "hourlyPrice": 2.55,
  "features": [
    "chat"
  ],
  "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
  "containerPort": 80,
  "description": "some description",
  "podLifetime": 120,
  "containers": [
    {
      "regionId": 78,
      "scale": {
        "min": 1,
        "max": 4,
        "cooldownPeriod": 300,
        "triggers": {
          "cpu": {
            "threshold": 92
          },
          "gpuMemory": {
            "threshold": 92
          },
          "gpuUtilization": {
            "threshold": 92
          },
          "memory": {
            "threshold": 92
          },
          "http": {
            "rate": 12,
            "window": 20
          }
        }
      },
      "deployStatus": {
        "total": 2,
        "ready": 1
      },
      "errorMessage": "Unexpected error occurred while launching the instance. Please contact the support team."
    }
  ],
  "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
  "envs": {
    "PORT": "3333"
  },
  "startupCommand": "npx create-llama",
  "apiKeysIds": [
    "6721058be81810b9dd045f40"
  ],
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Preview AI instance pricing

This endpoints allows you to estimate costs for your running AI instances, helping in budget planning.

Sesterce Cloud AI inference service is based on an unlimited-token pricing. This means you are charged for a global hour price, whatever the use of your dedicated endpoint.

post

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Body

hardwareIdstringrequired

Example: 59651ba4-657a-41d4-8c42-00f34f732375

autoScalingConfigurationsobject[]required

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request POST \
  --url '/ai-inference/instances/pricing' \
  --header 'x-api-key: text' \
  --header 'Content-Type: application/json' \
  --data '{
    "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
    "autoScalingConfigurations": [
      {
        "regionId": 18,
        "scale": {
          "max": 4
        }
      }
    ]
  }'

application/json

201

201

{
  "currencyCode": "USD",
  "pricePerHour": 1.44,
  "pricePerMonth": 43.2
}

Update an inference instance

This endpoint allows you to modify existing AI instances to adapt to changing project needs. This is particularly useful is you need to update your hardware flavor and/or autoscaling limits according to the use of your dedicated endpoint :

patch

Path parameters

idstringrequired

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Body

modelIdstring | nullablerequired

Example: 201a99c3-7cd4-4831-865e-b261082fda4b

containerPortnumberrequired

Example: 80

descriptionstring | nullablerequired

Example: example description.

autoScalingConfigurationsobject[]required

podLifetimenumber | nullablerequired

Example: 120

envsobject | nullablerequired

Example: {"PORT":"3333"}

hardwareIdstringrequired

Example: 59651ba4-657a-41d4-8c42-00f34f732375

startupCommandstring | nullablerequired

Example: npx create-llama

registryIdstring | nullablerequired

Example: 6721058be81810b9dd045f40

apiKeyIdsstring[] | nullablerequired

Example: ["6721058be81810b9dd045f40"]

Responses

application/json

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request PATCH \
  --url '/ai-inference/instances/{id}' \
  --header 'x-api-key: text' \
  --header 'Content-Type: application/json' \
  --data '{
    "modelId": "201a99c3-7cd4-4831-865e-b261082fda4b",
    "containerPort": 80,
    "description": "example description.",
    "autoScalingConfigurations": [
      {
        "regionId": 18,
        "scale": {
          "min": 1,
          "max": 4,
          "cooldownPeriod": 300,
          "triggers": {
            "cpu": {
              "threshold": 92
            },
            "gpuMemory": {
              "threshold": 92
            },
            "gpuUtilization": {
              "threshold": 92
            },
            "memory": {
              "threshold": 92
            },
            "http": {
              "rate": 12,
              "window": 20
            }
          }
        }
      }
    ],
    "podLifetime": 120,
    "envs": {
      "PORT": "3333"
    },
    "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
    "startupCommand": "npx create-llama",
    "registryId": "6721058be81810b9dd045f40",
    "apiKeyIds": [
      "6721058be81810b9dd045f40"
    ]
  }'

application/json

200

404

200

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-inference-instance",
  "status": "ACTIVE",
  "hourlyPrice": 2.55,
  "features": [
    "chat"
  ],
  "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Stop an inference instance

If you need to pause an AI instance to conserve resources and manage costs, use the following endpoint:

post

Path parameters

idstringrequired

Header parameters

x-api-keystringrequired

The API Key secret should be sent through this header to authenticate the request.

Responses

application/json

cURL

JavaScript

Python

HTTP

cURL

curl -L \
  --request POST \
  --url '/ai-inference/instances/{id}/stop' \
  --header 'x-api-key: text'

204

404

204

No body

Inference instance successfully stopped