[Python] How to exclude thinking steps when streaming from...

Discussão em 'Python' iniciado por Stack, Maio 5, 2025.

  1. Stack

    Stack Membro Participativo

    I am using the openai Python library to stream chat completions from the google/gemini-2.5-flash-preview:thinking model via the OpenRouter API.

    My goal is to get only the final assistant response, but this specific model variant :)thinking) prepends its internal thought process (e.g., "Thinking...", "Examining Request...", "Locating References...") directly into the main content stream before the actual answer:

    [​IMG]

    Example Code:

    from openai import OpenAI
    import os

    client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ.get("OPENROUTER_API_KEY"),
    )

    model_name = "google/gemini-2.5-flash-preview:thinking"
    messages_for_api = [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Explain gravity briefly."}
    ]

    try:
    response_stream = client.chat.completions.create(
    model=model_name,
    messages=messages_for_api,
    temperature=0.7,
    stream=True,
    # Note: Tried extra_body={'reasoning': {'exclude': True}} here, but it had no effect
    )

    print("\nResponse Stream Chunks (Illustrative):")
    full_response = ""
    # Example of observed output stream structure:
    # Chunk: "Thinking..."
    # Chunk: "Okay, the user wants to know about gravity."
    # ...

    for chunk in response_stream:
    if chunk.choices and chunk.choices[0].delta and chunk.choices[0].delta.content:
    content_part = chunk.choices[0].delta.content
    print(f"Received Chunk Content: {content_part}") # Shows thinking steps first
    full_response += content_part

    print(f"\n--- Final accumulated content (includes thinking steps) --- \n{full_response}")

    except Exception as e:
    print(f"\nAn error occurred: {e}")


    What I've Tried:


    1. Explicitly instructing the model in the system prompt not to output its thinking steps. This was ignored.


    2. Passing extra_body={'reasoning': {'exclude': True}} in the create call, based on OpenRouter documentation for controlling "Reasoning Tokens". This also had no effect on the output from this specific model.

    Question:

    Given that the thinking steps seem to be part of the main content stream for google/gemini-2.5-flash-preview:thinking, is there any way using the openai Python library and OpenRouter API parameters to prevent this specific model variant from outputting its thinking steps?

    Or, is the only practical solution to achieve a clean output to switch to a different model endpoint without the :thinking suffix (e.g., google/gemini-1.5-flash-latest)?

    Continue reading...

Compartilhe esta Página