Installation¶
No software installation is required to use the Octomizer web interface, other than a web browser with an Internet connection.
To use the OctoML Python SDK, you will need Python 3.7 or later. To install the Python SDK, run:
$ pip install octomizer-sdk
You can test that the installation was successful by running the command-line client, which will be on your PATH:
$ octomizer
The above command should show a help message and exit. See OctoML CLI for more details.
To use the OctoML SDK, you must first create an API token via the
OctoML web UI. Navigate to the settings tab of your Account page and create a
token there. Then, set the environment variable OCTOMIZER_API_TOKEN
to the value of this token.
You can then test it with:
$ OCTOMIZER_API_TOKEN=<your-api-token> python3
>>> import octomizer.client as octoclient
>>> client = octoclient.OctomizerClient()
>>> client.get_current_user()
uuid: "7405756c-dcfe-4e1d-9fef-72d54f263612"
given_name: "Chadwick"
family_name: "Boseman"
email: "chad@octoml.ai"
account_uuid: "0e5c654e-3d8c-4b8c-84cd-18ac824f7b37"
This shows that your client is authenticated to the service.
Package Deployment¶
The packages produced by the octomizer have their own installation requirements.
CPU¶
Currently, OctoML uses ONNX version 1.10.2 and ONNX-RT version 1.9.0 with default CPU Execution Providers.
CUDA¶
Additionally CUDA targets (except for Jetson devices), require CUDA 11.5 and onnx_tensorrt 8.0 which calls into TensorRT 8.0. This ensures that any GPU-based benchmarking is inclusive of TensorRT capabilities.
NVIDIA Jetson AGX¶
Jetson AGX platform is currently in private beta
For NVIDIA Jetson AGX, CUDA 10.2 and cuDNN 8.0 are required.