结合不同的专门化LLM

语言模型在需要不同推理能力的问题类型上难以泛化。通过结合各种不同的专门化语言模型，我们可以提高响应的质量。这是通过一种称为推理专家混合 (Mixture Of Reasoning Experts, MoRE) 的技术实现的。

在原始论文中，他们使用了四种不同的专家模型

1. 事实专家：这是一种通过 RAG 提示流程增强的模型。当它接收到查询时，它会从维基百科检索出最相关的 10 段内容，并在问题之前将它们附加到提示中。

2. 多跳专家：这是一种在每个演示后手动编写原理的专家模型，以引出问题的多步骤推理过程

3. 数学专家：这是一种为 GSM8k 数据集手动编写解释的专家模型，旨在使模型偏向不同的推理步骤

4. 常识专家：这是一种专家模型，提供了由 Codex 模型生成的 10 个不同的事实，这些事实被附加到问题之前的提示中

一旦每个专家生成了响应，他们就会使用随机森林分类器对其进行 0 到 1 的评分。然后用这个评分来选择最终答案并判断我们是否生成了一个足够好的答案（因为我们在每个点都有选择弃权的选项）

我们可以使用 instructor 实现一个简化版的 MoRE，并进行一些修改。

from openai import OpenAI
from pydantic import BaseModel, Field
import instructor
from textwrap import dedent

client = instructor.from_openai(OpenAI())


class MultihopExpert(BaseModel):
    chain_of_thought: str
    answer: str


class FactualExpert(BaseModel):
    answer: str


class ModelScore(BaseModel):
    score: float = Field(ge=0, lt=1)


def query_factual_expert(query: str, evidence: list[str]):
    formatted_evidence = "\n-".join(evidence)
    return client.chat.completions.create(
        model="gpt-4o",
        response_model=FactualExpert,
        messages=[
            {
                "role": "system",
                "content": dedent(
                    f"""
                <query>
                {query}
                </query>

                <evidences>
                {formatted_evidence}
                </evidences>
                """
                ),
            }
        ],
    )


def query_multihop_expert(query: str):
    return client.chat.completions.create(
        model="gpt-4o",
        response_model=MultihopExpert,
        messages=[
            {
                "role": "system",
                "content": dedent(
                    f"""
                <query>
                {query}
                </query>
                """
                ),
            }
        ],
    )


def score_answer(query: str, answer: str):
    return client.chat.completions.create(
        model="gpt-4o",
        response_model=ModelScore,
        messages=[
            {
                "role": "system",
                "content": """You are a helpful assistant that scores
                answers based on well they are able to answer a
                specific user query""",
            },
            {
                "role": "user",
                "content": f"""
                <user query>
                {query}
                </user query>

                <response>
                {answer}
                </response>
                """,
            },
        ],
    )


if __name__ == "__main__":
    query = """Who's the original singer of Help Me Make It
    Through The Night?"""
    evidences = [
        """Help Me Make It Through The Night is a country
        music ballad written and composed by Kris Kristofferson
        and released on his 1970 album 'Kristofferson'"""
    ]

    threshold = 0.8

    factual_expert_output = query_factual_expert(query, evidences)
    print(factual_expert_output.model_dump_json(indent=2))
    """
    {
      "answer": "The original singer of 'Help Me Make It Through the
      Night' is Kris Kristofferson, who released it on his 1970 album
      'Kristofferson'."
    }
    """

    multihop_expert_output = query_multihop_expert(query)
    print(multihop_expert_output.model_dump_json(indent=2))
    """
    {
      "chain_of_thought": "To identify the original singer of 'Help Me
      Make It Through The Night,' I need to look for the person who
      first recorded and released the song.",
      "answer": "The original singer of 'Help Me Make It Through
      The Night' is Kris Kristofferson."
    }
    """

    factual_expert_score = score_answer(query, factual_expert_output.answer)
    multihop_expert_score = score_answer(query, multihop_expert_output.answer)

    if max(factual_expert_score.score, multihop_expert_score.score) < threshold:
        answer = "Abstaining from responding"
    elif factual_expert_score.score > multihop_expert_score.score:
        answer = factual_expert_output.answer
    else:
        answer = multihop_expert_output.answer

    print(answer)
    """
    The original singer of 'Help Me Make It Through the Night' is Kris
    Kristofferson, who released it on his 1970 album 'Kristofferson'.
    """