流式传输部分响应

Literal

如果您使用的数据结构包含 literal 值，您需要确保导入 PartialLiteralMixin mixin。

from typing import Literal
from pydantic import BaseModel
from instructor.dsl.partial import PartialLiteralMixin


class User(BaseModel, PartialLiteralMixin):
    name: str
    age: int
    category: Literal["admin", "user", "guest"]


# The rest of your code below

这是因为如果 jiter 在流式传输 incomplete Literal 值时遇到该值，则会抛出错误。

字段级流式传输提供响应模型当前状态的增量快照，这些快照可立即使用。这种方法在渲染 UI 组件等上下文中尤为重要。

Instructor 通过使用 create_partial 支持这种模式。这使我们能够动态创建一个新类，该类将原始模型的所有字段视为 Optional。

理解部分响应

考虑当我们定义响应模型时会发生什么

from pydantic import BaseModel


class User(BaseModel):
    name: str
    age: int

如果我们从 OpenAI 流式输出 json，只有当对象完全返回时，我们才能进行解析！

{"name": "Jo
{"name": "John", "ag
{"name": "John", "age:
{"name": "John", "age": 25} # Completed

当指定 create_partial 并设置 stream=True 时，instructor 的响应将成为一个 Generator[T]。随着生成器产生结果，您可以迭代这些增量更新。生成器产生的最后一个值代表完整的提取！

{"name": "Jo                 => User(name="Jo", age=None)
{"name": "John", "ag         => User(name="John", age=None)
{"name": "John", "age:       => User(name="John", age=None)
{"name": "John", "age": 25}  => User(name="John", age=25)

有限的验证器支持

由于响应模型的流式传输特性，我们不支持验证器，因为它们无法应用于流式响应。

让我们来看一个流式传输会议信息提取的示例，该示例将用于在 react 组件中进行流式传输。

import instructor
from openai import OpenAI
from pydantic import BaseModel
from typing import List
from rich.console import Console

client = instructor.from_openai(OpenAI())

text_block = """
In our recent online meeting, participants from various backgrounds joined to discuss the upcoming tech conference. The names and contact details of the participants were as follows:

- Name: John Doe, Email: johndoe@email.com, Twitter: @TechGuru44
- Name: Jane Smith, Email: janesmith@email.com, Twitter: @DigitalDiva88
- Name: Alex Johnson, Email: alexj@email.com, Twitter: @CodeMaster2023

During the meeting, we agreed on several key points. The conference will be held on March 15th, 2024, at the Grand Tech Arena located at 4521 Innovation Drive. Dr. Emily Johnson, a renowned AI researcher, will be our keynote speaker.

The budget for the event is set at $50,000, covering venue costs, speaker fees, and promotional activities. Each participant is expected to contribute an article to the conference blog by February 20th.

A follow-up meetingis scheduled for January 25th at 3 PM GMT to finalize the agenda and confirm the list of speakers.
"""


class User(BaseModel):
    name: str
    email: str
    twitter: str


class MeetingInfo(BaseModel):
    users: List[User]
    date: str
    location: str
    budget: int
    deadline: str


extraction_stream = client.chat.completions.create_partial(
    model="gpt-4",
    response_model=MeetingInfo,
    messages=[
        {
            "role": "user",
            "content": f"Get the information about the meeting and the users {text_block}",
        },
    ],
    stream=True,
)


console = Console()

for extraction in extraction_stream:
    obj = extraction.model_dump()
    console.clear()
    console.print(obj)

print(extraction.model_dump_json(indent=2))
"""
{
  "users": [
    {
      "name": "John Doe",
      "email": "johndoe@email.com",
      "twitter": "@TechGuru44"
    },
    {
      "name": "Jane Smith",
      "email": "janesmith@email.com",
      "twitter": "@DigitalDiva88"
    },
    {
      "name": "Alex Johnson",
      "email": "alexj@email.com",
      "twitter": "@CodeMaster2023"
    }
  ],
  "date": "2024-03-15",
  "location": "Grand Tech Arena located at 4521 Innovation Drive",
  "budget": 50000,
  "deadline": "2024-02-20"
}
"""

这将输出以下内容

异步流式传输

我还想在这个示例中指出，instructor 也支持异步流式传输。当您想要流式传输响应模型并处理随着结果到来而处理它们时，这很有用，但您需要使用 async for 语法来迭代结果。

import instructor
from openai import AsyncOpenAI
from pydantic import BaseModel

client = instructor.from_openai(AsyncOpenAI())


class User(BaseModel):
    name: str
    age: int


async def print_partial_results():
    user = client.chat.completions.create_partial(
        model="gpt-4-turbo-preview",
        response_model=User,
        max_retries=2,
        stream=True,
        messages=[
            {"role": "user", "content": "Jason is 12 years old"},
        ],
    )
    async for m in user:
        print(m)
        #> name=None age=None
        #> name=None age=None
        #> name='' age=None
        #> name='Jason' age=None
        #> name='Jason' age=None
        #> name='Jason' age=None
        #> name='Jason' age=None
        #> name='Jason' age=12
        #> name='Jason' age=12


import asyncio

asyncio.run(print_partial_results())