Installation

No software installation is required to use the Octomizer web interface, other than a web browser with an Internet connection.

To use the OctoML Python SDK, you will need Python 3.7 or later. To install the Python SDK, run:

$ python3 -m pip install octomizer-sdk --extra-index-url https://octo.jfrog.io/artifactory/api/pypi/pypi-local/simple

You can test that the installation was successful by running the command-line client, which will be on your PATH:

$ octomizer

The above command should show a help message and exit. See OctoML Command Line Interface for more details.

To use the OctoML SDK, you must first create an API token via the OctoML web UI. Navigate to the settings tab of your Account page and create a token there. Then, set the environment variable OCTOMIZER_API_TOKEN to the value of this token.

You can then test it with:

$ OCTOMIZER_API_TOKEN=<your-api-token> python3
>>> import octomizer.client as octoclient
>>> client = octoclient.OctomizerClient()
>>> client.get_current_user()
uuid: "7405756c-dcfe-4e1d-9fef-72d54f263612"
given_name: "Chadwick"
family_name: "Boseman"
email: "chad@octoml.ai"
account_uuid: "0e5c654e-3d8c-4b8c-84cd-18ac824f7b37"

This shows that your client is authenticated to the service.

Package Deployment

The packages produced by the octomizer have their own installation requirements.

CPU

Currently, OctoML uses ONNX version 1.10.2 and ONNX-RT version 1.9.0 with default CPU Execution Providers.

CUDA

Additionally CUDA targets (except for Jetson devices), require CUDA 11.5 and onnx_tensorrt 8.0 which calls into TensorRT 8.0. This ensures that any GPU-based benchmarking is inclusive of TensorRT capabilities.

NVIDIA Jetson AGX

Jetson AGX platform is currently in private beta

For NVIDIA Jetson AGX, CUDA 10.2 and cuDNN 8.0 are required.