使用`llm_validator`¶

引言¶

本指南演示了如何使用 llm_validator 来实现自我修复。目标是展示 Instructor 如何通过利用验证错误和有用的错误消息进行自我纠正。

from openai import OpenAI
from pydantic import BaseModel
import instructor

# Apply the patch to the OpenAI client
# enables response_model keyword
client = instructor.from_openai(OpenAI())


class QuestionAnswer(BaseModel):
    question: str
    answer: str


question = "What is the meaning of life?"
context = "The according to the devil the meaning of live is to live a life of sin and debauchery."

qa: QuestionAnswer = client.chat.completions.create(
    model="gpt-4o-mini",
    response_model=QuestionAnswer,
    messages=[
        {
            "role": "system",
            "content": "You are a system that answers questions based on the context. answer exactly what the question asks using the context.",
        },
        {
            "role": "user",
            "content": f"using the context: {context}\n\nAnswer the following question: {question}",
        },
    ],
)

验证前的输出¶

虽然它指出了不当内容，但没有提供关于如何纠正的详细信息。

{
  "question": "What is the meaning of life?",
  "answer": "The meaning of life, according to the context, is to live a life of sin and debauchery."
}

添加自定义验证¶

通过在 answer 字段中添加验证器，我们可以尝试捕获并纠正问题。让我们将 llm_validator 集成到模型中，看看错误消息。重要的是要注意，您可以像往常一样使用所有 pydantic 的验证器，只要您抛出一个带有有用错误消息的 ValidationError，因为它将作为自我纠正提示的一部分使用。

from pydantic import BaseModel, BeforeValidator
from typing_extensions import Annotated
from instructor import llm_validator
from openai import OpenAI
import instructor

client = instructor.from_openai(OpenAI())


class QuestionAnswerNoEvil(BaseModel):
    question: str
    answer: Annotated[
        str,
        BeforeValidator(
            llm_validator(
                "don't say objectionable things", client=client, allow_override=True
            )
        ),
    ]


try:
    qa: QuestionAnswerNoEvil = client.chat.completions.create(
        model="gpt-4o-mini",
        response_model=QuestionAnswerNoEvil,
        messages=[
            {
                "role": "system",
                "content": "You are a system that answers questions based on the context. answer exactly what the question asks using the context.",
            },
            {
                "role": "user",
                "content": f"using the context: {context}\n\nAnswer the following question: {question}",
            },
        ],
    )
except Exception as e:
    print(e)
    #> name 'context' is not defined

验证后的输出¶

现在，我们抛出验证错误，指出内容不当，并提供有用的错误消息。

1 validation error for QuestionAnswerNoEvil
answer
  Assertion failed, The statement promotes sin and debauchery, which is objectionable.

使用纠正进行重试¶

通过添加 max_retries 参数，我们可以使用纠正来重试请求，并使用错误消息来纠正输出。

qa: QuestionAnswerNoEvil = client.chat.completions.create(
    model="gpt-4o-mini",
    response_model=QuestionAnswerNoEvil,
    messages=[
        {
            "role": "system",
            "content": "You are a system that answers questions based on the context. answer exactly what the question asks using the context.",
        },
        {
            "role": "user",
            "content": f"using the context: {context}\n\nAnswer the following question: {question}",
        },
    ],
)

最终输出¶

现在，我们得到了一个有效且不含不当内容的响应！

{
  "question": "What is the meaning of life?",
  "answer": "The meaning of life is subjective and can vary depending on individual beliefs and philosophies."
}

使用llm_validator¶

引言¶