Welcome to OctoML’s documentation!

The OctoML Platform is a machine learning model optimization and deployment service powered by octoml.ai. This documentation provides an overview of how to use the OctoML Platform via the web interface as well as the Python SDK.

Example usage

Here is a simple Python program to upload an ONNX model to the OctoML Platform, optimize it, and download the optimized Python package:

from octomizer import client, workflow
from octomizer.models import onnx_model

# Pass your API token below:
client = client.OctomizerClient(access_token=MY_ACCESS_TOKEN)

# Specify model file and input layer parameters.
model_file = "mnist.onnx"

# Upload the model to OctoML.
model = onnx_model.ONNXModel(client, name=model_file, model=model_file)

# Optimize the model. By default, the resulting package will be a Python wheel.
wrkflow = model.get_uploaded_model_variant().octomize(platform="broadwell")

# Save the workflow uuid somewhere so you can use it later
print(wrkflow.uuid)
After you receive an email notification about the completion of the acceleration workflow, you

can view performance benchmark metrics on the hardware you chose and download a packaged version of the accelerated model, either by invoking the following code or visiting the UI:

# Look up the workflow you previously launched using the workflow id
wkrflow = client.get_workflow("<INSERT WORKFLOW ID>")

# Save the resulting Python wheel to the current directory.
wrkflow.save_package(".")

# To view benchmark metrics, either visit the UI or invoke something similar to:
if wkrflow.completed() and wkrflow.has_benchmark_stage():
   engine = wkrflow.proto.benchmark_stage_spec.engine
   metrics = wkrflow.metrics()
   print(engine, metrics)
   # Save the resulting Python wheel
   wkrflow.save_package(".")

To specify saving another package type, one may instead execute something similar to:

from octomizer.package_type import PackageType
wrkflow.save_package(".", package_type=PackageType.LINUX_SHARED_OBJECT)

To see a list of possible hardware platforms, you may execute:

print(client.get_hardware_targets())

Please contact us if you have any questions or problems.