Inference Instances

Get Inference models

This endpoint allows you to view available AI models for deployment, aiding in selecting the right model for your needs.

Check model features to match your specific project requirements.

GET/ai-inference/models

Header parameters

Response

200

Body

application/json

id*string

Example: "12ed7523-432c-48f5-b3cd-32e6726d07c8"

name*string

Example: "stable-diffusion"

port*number

Example: 8000

defaultHardwareId*string

Example: "59651ba4-657a-41d4-8c42-00f34f732375"

features*array of enum

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/models', {
    method: 'GET',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

200

[
  {
    "id": "12ed7523-432c-48f5-b3cd-32e6726d07c8",
    "name": "stable-diffusion",
    "port": 8000,
    "defaultHardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
    "features": [
      "image"
    ]
  }
]

Get inference hardware

You have several hardware options available through the AI inference feature of Sesterce Cloud (you can consult this section for more information). This endpoint allows you to explore options for deploying AI instances, which are crucial for planning resources and manage latency rate.

Evaluate hardware capabilities to ensure optimal performance for your AI tasks.

GET/ai-inference/hardwares

Header parameters

Response

200

Body

application/json

id*string

Example: "59651ba4-657a-41d4-8c42-00f34f732375"

name*string

Example: "1xL40S / 16 vCPU / 232GiB RAM"

cpu*number

Example: 16000

ram*number

Example: 237568

gpu*nullable all of

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/hardwares', {
    method: 'GET',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

200

[
  {
    "id": "59651ba4-657a-41d4-8c42-00f34f732375",
    "name": "1xL40S / 16 vCPU / 232GiB RAM",
    "cpu": 16000,
    "ram": 237568,
    "gpu": {
      "model": "NVIDIA-L40S",
      "count": 1,
      "memory": 48
    }
  }
]

Get Regions available for inference instances

Identify available regions for deploying AI instances, important for compliance and latency considerations.

The region choice is a crucial parameter for your inference endpoint hosting. It will determine the latency rate for your final end-users. Choose regions that align with your data residency and latency needs.

GET/ai-inference/regions

Header parameters

Response

200

Body

application/json

id*number

Example: 18

name*string

Example: "Singapore"

countryCode*string

Example: "SG"

state*enum

Example: "ACTIVE"

NEWINACTIVEACTIVEMAINTENANCEDELETINGDELETION_FAILEDDELETED

capacities*array of InferenceRegionCapacityResponse (object)

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/regions', {
    method: 'GET',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

200

[
  {
    "id": 18,
    "name": "Singapore",
    "countryCode": "SG",
    "state": "ACTIVE",
    "capacities": [
      {
        "hardwareId": "30328755-51f0-4251-b535-029358700099",
        "capacity": 38
      },
      {
        "hardwareId": "dc172cc8-0035-4527-958b-775f951e8836",
        "capacity": 19
      }
    ]
  }
]

Create a Registry

A registry is necessary if you need to infere your own custom model, which is not publicly available. Click here to learn more about Registries on Sesterce Cloud AI Inference service!

POST/ai-inference/registries

Header parameters

Body

application/json

url*string

Example: "docker.io/library/user/image:tag"

username*string

Example: "someusername"

password*string

Example: "securepassword"

name*string

Example: "example-registry"

Response

201

Body

application/json

_id*string

Example: "6721058be81810b9dd045f40"

name*string

Example: "example-registry"

url*string

Example: "docker.io/library/user/image:tag"

username*string

Example: "someusername"

createdAt*string

Example: "2019-06-26T13:00:00.000Z"

updatedAt*string

Example: "2019-06-26T13:00:00.000Z"

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/registries', {
    method: 'POST',
    headers: {
      "x-api-key": "text",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "url": "docker.io/library/user/image:tag",
      "username": "someusername",
      "password": "securepassword",
      "name": "example-registry"
    }),
});
const data = await response.json();

Response

201

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-registry",
  "url": "docker.io/library/user/image:tag",
  "username": "someusername",
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Get the list of registries created

To manage your registries for storing and accessing AI models, use the following endpoint:

GET/ai-inference/registries

Header parameters

Response

200

Body

application/json

_id*string

Example: "6721058be81810b9dd045f40"

name*string

Example: "example-registry"

url*string

Example: "docker.io/library/user/image:tag"

createdAt*string

Example: "2019-06-26T13:00:00.000Z"

updatedAt*string

Example: "2019-06-26T13:00:00.000Z"

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/registries', {
    method: 'GET',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

200

[
  {
    "_id": "6721058be81810b9dd045f40",
    "name": "example-registry",
    "url": "docker.io/library/user/image:tag",
    "createdAt": "2019-06-26T13:00:00.000Z",
    "updatedAt": "2019-06-26T13:00:00.000Z"
  }
]

Update a registry

To modify registry details to ensure they meet current security and access needs, use the following endpoint:

PATCH/ai-inference/registries/{id}

Path parameters

id*string

Header parameters

Body

application/json

url*string

Example: "docker.io/library/user/image:tag"

username*string

Example: "someusername"

password*string

Example: "securepassword"

Response

204404

Registry successfully updated.

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/registries/{id}', {
    method: 'PATCH',
    headers: {
      "x-api-key": "text",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "url": "docker.io/library/user/image:tag",
      "username": "someusername",
      "password": "securepassword"
    }),
});
const data = await response.json();

Response

404

{
  "statusCode": 0,
  "timestamp": "text",
  "path": "text",
  "message": "text"
}

Delete a Registry

The following endpoint allows to remove outdated or unused registries to maintain a clean environment.

DELETE/ai-inference/registries/{id}

Path parameters

id*string

Header parameters

Response

204404

Registry successfully deleted.

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/registries/{id}', {
    method: 'DELETE',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

404

{
  "statusCode": 0,
  "timestamp": "text",
  "path": "text",
  "message": "text"
}

Create an inference instance

Time has come! You can now deploy a new AI inference instance to scale your applications and services, or deploy in production an existing model! Use the following endpoint to perform this action.

To create an inference instance, check that your credit balance is filled. Please check here our documentation to top up your balance.

POST/ai-inference/instances

Header parameters

Body

application/json

modelId*nullable string

Example: "201a99c3-7cd4-4831-865e-b261082fda4b"

containerPort*number

Example: 80

name*string

Example: "example-inference-instance"

description*nullable string

Example: "example description."

autoScalingConfigurations*array of AutoScaling (object)

podLifetime*nullable number

Example: 120

envs*nullable object

hardwareId*string

Example: "59651ba4-657a-41d4-8c42-00f34f732375"

startupCommand*nullable string

Example: "npx create-llama"

registryId*nullable string

Example: "6721058be81810b9dd045f40"

apiKeyIds*nullable array of string

Response

201400404

Body

application/json

_id*string

Example: "6721058be81810b9dd045f40"

name*string

Example: "example-inference-instance"

status*enum

Example: "ACTIVE"

NEWACTIVEPENDINGERRORDISABLEDDELETED

hourlyPrice*number

Example: 2.55

features*array of enum

address*string

Example: "https://iate-example-6672-d7c85154.ai.sesterce.dev/"

createdAt*string

Example: "2019-06-26T13:00:00.000Z"

updatedAt*string

Example: "2019-06-26T13:00:00.000Z"

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/instances', {
    method: 'POST',
    headers: {
      "x-api-key": "text",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "modelId": "201a99c3-7cd4-4831-865e-b261082fda4b",
      "containerPort": 80,
      "name": "example-inference-instance",
      "description": "example description.",
      "autoScalingConfigurations": [
        {
          "regionId": 18,
          "scale": {
            "min": 1,
            "max": 4,
            "cooldownPeriod": 300,
            "triggers": {
              "cpu": {
                "threshold": 92
              },
              "gpuMemory": {
                "threshold": 92
              },
              "gpuUtilization": {
                "threshold": 92
              },
              "memory": {
                "threshold": 92
              },
              "http": {
                "rate": 12,
                "window": 20
              }
            }
          }
        }
      ],
      "podLifetime": 120,
      "envs": {
        "PORT": "3333"
      },
      "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
      "startupCommand": "npx create-llama",
      "registryId": "6721058be81810b9dd045f40",
      "apiKeyIds": [
        "6721058be81810b9dd045f40"
      ]
    }),
});
const data = await response.json();

Response

201400404

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-inference-instance",
  "status": "ACTIVE",
  "hourlyPrice": 2.55,
  "features": [
    "chat"
  ],
  "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Start an inference instance

This endpoint allows you to activate an AI inference instance to begin processing tasks and data.

You can monitor startup times to assess performance efficiency.

POST/ai-inference/instances/{id}/start

Path parameters

id*string

Header parameters

Response

204404

Inference instance successfully started

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/instances/{id}/start', {
    method: 'POST',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

404

{
  "statusCode": 0,
  "timestamp": "text",
  "path": "text",
  "message": "text"
}

Get the list of your Inference instances

Here is the endpoint to monitor your active AI instances to manage resources and performance.

GET/ai-inference/instances

Header parameters

Response

200

Body

application/json

_id*string

Example: "6721058be81810b9dd045f40"

name*string

Example: "example-inference-instance"

status*enum

Example: "ACTIVE"

NEWACTIVEPENDINGERRORDISABLEDDELETED

hourlyPrice*number

Example: 2.55

features*array of enum

address*string

Example: "https://iate-example-6672-d7c85154.ai.sesterce.dev/"

createdAt*string

Example: "2019-06-26T13:00:00.000Z"

updatedAt*string

Example: "2019-06-26T13:00:00.000Z"

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/instances', {
    method: 'GET',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

200

[
  {
    "_id": "6721058be81810b9dd045f40",
    "name": "example-inference-instance",
    "status": "ACTIVE",
    "hourlyPrice": 2.55,
    "features": [
      "chat"
    ],
    "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
    "createdAt": "2019-06-26T13:00:00.000Z",
    "updatedAt": "2019-06-26T13:00:00.000Z"
  }
]

Get details about a specific Inference Instance

Retrieve detailed information about a specific AI instance for management and troubleshooting.

GET/ai-inference/instances/{id}

Path parameters

id*string

Header parameters

Response

200404

Body

application/json

_id*string

Example: "6721058be81810b9dd045f40"

name*string

Example: "example-inference-instance"

status*enum

Example: "ACTIVE"

NEWACTIVEPENDINGERRORDISABLEDDELETED

hourlyPrice*number

Example: 2.55

features*array of enum

address*string

Example: "https://iate-example-6672-d7c85154.ai.sesterce.dev/"

containerPort*number

Example: 80

description*nullable string

Example: "some description"

podLifetime*number

Example: 120

containers*array of Container (object)

hardwareId*string

Example: "59651ba4-657a-41d4-8c42-00f34f732375"

envs*object

startupCommand*nullable string

Example: "npx create-llama"

apiKeysIds*array of string

createdAt*string

Example: "2019-06-26T13:00:00.000Z"

updatedAt*string

Example: "2019-06-26T13:00:00.000Z"

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/instances/{id}', {
    method: 'GET',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

200404

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-inference-instance",
  "status": "ACTIVE",
  "hourlyPrice": 2.55,
  "features": [
    "chat"
  ],
  "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
  "containerPort": 80,
  "description": "some description",
  "podLifetime": 120,
  "containers": [
    {
      "regionId": 78,
      "scale": {
        "min": 1,
        "max": 4,
        "cooldownPeriod": 300,
        "triggers": {
          "cpu": {
            "threshold": 92
          },
          "gpuMemory": {
            "threshold": 92
          },
          "gpuUtilization": {
            "threshold": 92
          },
          "memory": {
            "threshold": 92
          },
          "http": {
            "rate": 12,
            "window": 20
          }
        }
      },
      "deployStatus": {
        "total": 2,
        "ready": 1
      },
      "errorMessage": "Unexpected error occurred while launching the instance. Please contact the support team."
    }
  ],
  "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
  "envs": {
    "PORT": "3333"
  },
  "startupCommand": "npx create-llama",
  "apiKeysIds": [
    "6721058be81810b9dd045f40"
  ],
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Preview AI instance pricing

This endpoints allows you to estimate costs for your running AI instances, helping in budget planning.

Sesterce Cloud AI inference service is based on an unlimited-token pricing. This means you are charged for a global hour price, whatever the use of your dedicated endpoint.

POST/ai-inference/instances/pricing

Header parameters

Body

application/json

hardwareId*string

Example: "59651ba4-657a-41d4-8c42-00f34f732375"

autoScalingConfigurations*array of AutoScalingPricing (object)

Response

201

Body

application/json

currencyCode*string

Example: "USD"

pricePerHour*number

Example: 1.44

pricePerMonth*number

Example: 43.2

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/instances/pricing', {
    method: 'POST',
    headers: {
      "x-api-key": "text",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
      "autoScalingConfigurations": [
        {
          "regionId": 18,
          "scale": {
            "max": 4
          }
        }
      ]
    }),
});
const data = await response.json();

Response

201

{
  "currencyCode": "USD",
  "pricePerHour": 1.44,
  "pricePerMonth": 43.2
}

Update an inference instance

This endpoint allows you to modify existing AI instances to adapt to changing project needs. This is particularly useful is you need to update your hardware flavor and/or autoscaling limits according to the use of your dedicated endpoint :

PATCH/ai-inference/instances/{id}

Path parameters

id*string

Header parameters

Body

application/json

modelId*nullable string

Example: "201a99c3-7cd4-4831-865e-b261082fda4b"

containerPort*number

Example: 80

description*nullable string

Example: "example description."

autoScalingConfigurations*array of AutoScaling (object)

podLifetime*nullable number

Example: 120

envs*nullable object

hardwareId*string

Example: "59651ba4-657a-41d4-8c42-00f34f732375"

startupCommand*nullable string

Example: "npx create-llama"

registryId*nullable string

Example: "6721058be81810b9dd045f40"

apiKeyIds*nullable array of string

Response

200404

Body

application/json

_id*string

Example: "6721058be81810b9dd045f40"

name*string

Example: "example-inference-instance"

status*enum

Example: "ACTIVE"

NEWACTIVEPENDINGERRORDISABLEDDELETED

hourlyPrice*number

Example: 2.55

features*array of enum

address*string

Example: "https://iate-example-6672-d7c85154.ai.sesterce.dev/"

createdAt*string

Example: "2019-06-26T13:00:00.000Z"

updatedAt*string

Example: "2019-06-26T13:00:00.000Z"

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/instances/{id}', {
    method: 'PATCH',
    headers: {
      "x-api-key": "text",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "modelId": "201a99c3-7cd4-4831-865e-b261082fda4b",
      "containerPort": 80,
      "description": "example description.",
      "autoScalingConfigurations": [
        {
          "regionId": 18,
          "scale": {
            "min": 1,
            "max": 4,
            "cooldownPeriod": 300,
            "triggers": {
              "cpu": {
                "threshold": 92
              },
              "gpuMemory": {
                "threshold": 92
              },
              "gpuUtilization": {
                "threshold": 92
              },
              "memory": {
                "threshold": 92
              },
              "http": {
                "rate": 12,
                "window": 20
              }
            }
          }
        }
      ],
      "podLifetime": 120,
      "envs": {
        "PORT": "3333"
      },
      "hardwareId": "59651ba4-657a-41d4-8c42-00f34f732375",
      "startupCommand": "npx create-llama",
      "registryId": "6721058be81810b9dd045f40",
      "apiKeyIds": [
        "6721058be81810b9dd045f40"
      ]
    }),
});
const data = await response.json();

Response

200404

{
  "_id": "6721058be81810b9dd045f40",
  "name": "example-inference-instance",
  "status": "ACTIVE",
  "hourlyPrice": 2.55,
  "features": [
    "chat"
  ],
  "address": "https://iate-example-6672-d7c85154.ai.sesterce.dev/",
  "createdAt": "2019-06-26T13:00:00.000Z",
  "updatedAt": "2019-06-26T13:00:00.000Z"
}

Stop an inference instance

If you need to pause an AI instance to conserve resources and manage costs, use the following endpoint:

POST/ai-inference/instances/{id}/stop

Path parameters

id*string

Header parameters

Response

204404

Inference instance successfully stopped

Request

JavaScriptCurlPython

const response = await fetch('/ai-inference/instances/{id}/stop', {
    method: 'POST',
    headers: {
      "x-api-key": "text"
    },
});
const data = await response.json();

Response

404

{
  "statusCode": 0,
  "timestamp": "text",
  "path": "text",
  "message": "text"
}