Welcome to OctoML’s documentation!¶
The OctoML Platform is a machine learning model optimization and deployment service powered by octoml.ai. This documentation provides an overview of how to use the OctoML Platform via the web interface as well as the Python SDK.
Here is a simple Python program to upload an ONNX model to the OctoML Platform, optimize it, and download the optimized Python package:
from octomizer import client, workflow from octomizer.models import onnx_model # Pass your API token below: client = client.OctomizerClient(access_token=MY_ACCESS_TOKEN) # Specify model file and input layer parameters. model_file = "mnist.onnx" # Upload the model to OctoML. model = onnx_model.ONNXModel(client, name=model_file, model=model_file) # Optimize the model. By default, the resulting package will be a Python wheel. wrkflow = model.get_uploaded_model_variant().octomize(platform="broadwell") # Save the workflow uuid somewhere so you can use it later print(wrkflow.uuid)
- After you receive an email notification about the completion of the acceleration workflow, you
can view performance benchmark metrics on the hardware you chose and download a packaged version of the accelerated model, either by invoking the following code or visiting the UI:
# Look up the workflow you previously launched using the workflow id wkrflow = client.get_workflow("<INSERT WORKFLOW ID>") # Save the resulting Python wheel to the current directory. wrkflow.save_package(".") # To view benchmark metrics, either visit the UI or invoke something similar to: if wkrflow.completed() and wkrflow.has_benchmark_stage(): engine = wkrflow.proto.benchmark_stage_spec.engine metrics = wkrflow.metrics() print(engine, metrics) # Save the resulting Python wheel wkrflow.save_package(".")
To specify saving another package type, one may instead execute something similar to:
from octomizer.package_type import PackageType wrkflow.save_package(".", package_type=PackageType.LINUX_SHARED_OBJECT)
To see a list of possible hardware platforms, you may execute:
Please contact us if you have any questions or problems.