Installation

No software installation is required to use the Octomizer web interface, other than a web browser with an Internet connection.

To use the Octomizer Python SDK, you will need Python 3.7 or later. To install the Python SDK, run:

$ python3 -m pip install octomizer-sdk --extra-index-url https://octo.jfrog.io/artifactory/api/pypi/pypi-local/simple

You can test that the installation was successful by running the command-line client, which will be on your PATH:

$ octomizer

The above command should show a help message and exit. See Octomizer Command Line Interface for more details.

To use the Octomizer SDK, you must first create an API token via the Octomizer web UI. Navigate to your account settings page and create a token there. Then, set the environment variable OCTOMIZER_API_TOKEN to the value of this token.

You can then test it with:

$ OCTOMIZER_API_TOKEN=<your-api-token> python3
>>> import octomizer.client as octoclient
>>> client = octoclient.OctomizerClient()
>>> client.get_current_user()
uuid: "7405756c-dcfe-4e1d-9fef-72d54f263612"
given_name: "Chadwick"
family_name: "Boseman"
email: "chad@octoml.ai"
account_uuid: "0e5c654e-3d8c-4b8c-84cd-18ac824f7b37"

This shows that your client is authenticated to the service.

Package Deployment

The packages produced by the octomizer have their own installation requirements.

CPU

Currently, the Octomizer uses ONNX version 1.8 and ONNX-RT version 1.8.0 with default CPU Execution Providers.

CUDA

Additionally CUDA targets (except for jetson devices), require onnx_tensorrt 7.0, which calls into TensorRT 7.0. This ensures that any GPU-based benchmarking is inclusive of TensorRT capabilities.

NVIDIA Jetson AGX

Jetson AGX platform is currently in private beta

For NVIDIA Jetson AGX, CUDA 10.2 and cuDNN 8.0 are required.