Python SDKPython SDK inferenceOctoAI Python SDK at a glance The OctoAI Python SDK is intended to help you use OctoAI endpoints. At its simplest form, it allows you to run inferences against an endpoint by providing a dictionary with the necessary inputs. Python1import time2from octoai.client import OctoAI34client = OctoAI()56# It allows you to run inferences7output = client.infer(endpoint_url="your-endpoint-url", inputs={"keyword": "dictionary"})89# It also allows for inference streams for LLMs10for token in client.infer_stream("your-endpoint-url", inputs={"keyword": "dictionary"}):11 if token.get("object") == "chat.completion.chunk":12 # Do stuff with the token13 pass1415# And for server-side asynchronous inferences16future = client.infer_async("your-endpoint-url", {"keyword": "dictionary"})17# Typically, you'd collect additional futures then poll for status, but for the sake of example...18while not client.is_future_ready(future):19 time.sleep(1)20# Once the results are ready, you can use them in the same way as you21# typically do for demo endpoints22result = client.get_future_result(future)2324# And includes healthChecks25if client.health_check("your-healthcheck-url") == 200:26 # Run some inferences27 pass