使用 Cerebras 的结构化输出，一份完整的指导（含 instructor）¶

Cerebras 提供经过硬件加速、为高性能计算环境优化的 AI 模型。本指南将向您展示如何将 Instructor 与 Cerebras 模型结合使用，以实现类型安全、经过验证的响应。

快速入门¶

安装支持 Cerebras 的 Instructor

pip install "instructor[cerebras_cloud_sdk]"

简单用户示例（同步）¶

import instructor
from cerebras.cloud.sdk import Cerebras
from pydantic import BaseModel

client = instructor.from_cerebras(Cerebras())

# Enable instructor patches
client = instructor.from_cerebras(client)

class User(BaseModel):
    name: str
    age: int

# Create structured output
resp = client.chat.completions.create(
    model="llama3.1-70b",
    messages=[
        {
            "role": "user",
            "content": "Extract the name and age of the person in this sentence: John Smith is 29 years old.",
        }
    ],
    response_model=User,
)

print(resp)
#> User(name='John Smith', age=29)

简单用户示例（异步）¶

from cerebras.cloud.sdk import AsyncCerebras
import instructor
from pydantic import BaseModel
import asyncio

# Initialize async client
client = AsyncCerebras()

# Enable instructor patches
client = instructor.from_cerebras(client)

class User(BaseModel):
    name: str
    age: int

async def extract_user():
    resp = await client.chat.completions.create(
        model="llama3.1-70b",
        messages=[
            {
                "role": "user",
                "content": "Extract the name and age of the person in this sentence: John Smith is 29 years old.",
            }
        ],
        response_model=User,
    )
    return resp

# Run async function
resp = asyncio.run(extract_user())
print(resp)
#> User(name='John Smith', age=29)

嵌套示例¶

from pydantic import BaseModel
import instructor
from cerebras.cloud.sdk import Cerebras

client = instructor.from_cerebras(Cerebras())


class Address(BaseModel):
    street: str
    city: str
    country: str


class User(BaseModel):
    name: str
    age: int
    addresses: list[Address]


# Create structured output with nested objects
user = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": """
        Extract: Jason is 25 years old.
        He lives at 123 Main St, New York, USA
        and has a summer house at 456 Beach Rd, Miami, USA
    """,
        }
    ],
    model="llama3.1-70b",
    response_model=User,
)

print(user)
#> {
#>     'name': 'Jason',
#>     'age': 25,
#>     'addresses': [
#>         {
#>             'street': '123 Main St',
#>             'city': 'New York',
#>             'country': 'USA'
#>         },
#>         {
#>             'street': '456 Beach Rd',
#>             'city': 'Miami',
#>             'country': 'USA'
#>         }
#>     ]
#> }

流式支持¶

Instructor 提供了两种主要的流式输出响应方式：

可迭代对象（Iterables）：当您想流式输出同一类型的对象列表时，这非常有用（例如，使用结构化输出提取多个用户）。
部分流式传输（Partial Streaming）：当您想流式传输单个对象并希望在响应到达时立即开始处理它时，这非常有用。

目前，我们通过解析原始文本补全来支持 Cerebras 的部分流式传输。目前尚未实现函数调用流式传输。请确保在使用部分流式传输时设置了 mode=instructor.Mode.CEREBRAS_JSON。

import instructor
from cerebras.cloud.sdk import Cerebras, AsyncCerebras
from pydantic import BaseModel
from typing import Iterable

client = instructor.from_cerebras(Cerebras(), mode=instructor.Mode.CEREBRAS_JSON)


class Person(BaseModel):
    name: str
    age: int


resp = client.chat.completions.create_partial(
    model="llama3.1-70b",
    messages=[
        {
            "role": "user",
            "content": "Ivan is 27 and lives in Singapore",
        }
    ],
    response_model=Person,
    stream=True,
)

for person in resp:
    print(person)
    # > name=None age=None
    # > name='Ivan' age=None
    # > name='Ivan' age=27

可迭代对象示例¶

import instructor
from cerebras.cloud.sdk import Cerebras, AsyncCerebras
from pydantic import BaseModel
from typing import Iterable

client = instructor.from_cerebras(Cerebras(), mode=instructor.Mode.CEREBRAS_JSON)


class Person(BaseModel):
    name: str
    age: int


resp = client.chat.completions.create_iterable(
    model="llama3.1-70b",
    messages=[
        {
            "role": "user",
            "content": "Extract all users from this sentence : Chris is 27 and lives in San Francisco, John is 30 and lives in New York while their college roomate Jessica is 26 and lives in London",
        }
    ],
    response_model=Person,
    stream=True,
)

for person in resp:
    print(person)
    # > Person(name='Chris', age=27)
    # > Person(name='John', age=30)
    # > Person(name='Jessica', age=26)

Instructor 钩子¶

Instructor 提供了几个钩子来定制行为

验证钩子¶

from instructor import Instructor

def validation_hook(value, retry_count, exception):
    print(f"Validation failed {retry_count} times: {exception}")
    return retry_count < 3  # Retry up to 3 times

instructor.patch(client, validation_hook=validation_hook)

Instructor 模式¶

我们提供了几种模式，方便您使用 Cerebras 支持的不同响应模型。

instructor.Mode.CEREBRAS_JSON ：此模式将原始补全解析为有效的 JSON 对象。
instructor.Mode.CEREBRAS_TOOLS ：此模式使用 Cerebras 的工具调用模式向客户端返回结构化输出。

总的来说，我们推荐使用 Mode.CEREBRAS_TOOLS 模式，因为它最灵活且面向未来。它支持您在其中指定模式的最大特性集，并且使得操作起来更加容易。