fix(litellm): preserve reasoning_content for known reasoning model families

[daewoongoh]

Related GitHub Issue

Closes: #891

Description

LiteLLM's /v1/model/info response never exposes reasoning-related capability flags, so preserveReasoning was always resolving to false for reasoning/thinking models routed through a LiteLLM proxy (DeepSeek-Reasoner, GLM-4.7/5, Kimi-K2-Thinking, MiniMax-M2/M3, Qwen3-plus/max, etc.). As a result, LiteLLMHandler.createMessage always used convertToOpenAiMessages, so reasoning_content from a prior turn was never echoed back to the API after a tool-call turn. This differs from accessing the same underlying models directly through their native provider config (e.g. deepseek.ts already sets preserveReasoning: true), and some providers reject the request with a 400 error when required reasoning history is missing.

How:

• packages/types/src/providers/lite-llm.ts: Collected the exact alias/routed-model-name ids for every model that requires preserveReasoning: true in its native provider config (deepseek.ts, mimo.ts, moonshot.ts, bedrock.ts, fireworks.ts, zai.ts, minimax.ts, opencode-go.ts), and exported them as LITELLM_PRESERVE_REASONING_MODEL_IDS. isLiteLLMPreserveReasoningModel() checks a model name against this set with an exact (case-insensitive) match on the final /-delimited segment, so provider-prefixed routed names (e.g. deepseek/deepseek-reasoner, fireworks_ai/accounts/fireworks/models/kimi-k2p7-code) resolve correctly without over-matching unrelated aliases that merely contain a family fragment as a substring (e.g. glm-5-flash vs glm-5).
• src/api/providers/fetchers/litellm.ts: getLiteLLMModels now checks both model_name and litellm_params.model against isLiteLLMPreserveReasoningModel() independently while parsing each model, setting ModelInfo.preserveReasoning: true if either matches.
• src/api/providers/lite-llm.ts: createMessage now branches on info.preserveReasoning — true routes through convertToR1Format (with mergeToolResultText: true), otherwise it keeps using convertToOpenAiMessages. convertToR1Format folds any text following a tool_result into the preceding tool message instead of inserting a separate user message, which prevents reasoning-mode providers from dropping reasoning_content when a user message appears mid-turn.

Trade-off: the matching is an explicit model-id allowlist, not a query against LiteLLM's actual capabilities (which it doesn't expose). It's exact rather than pattern-based to avoid false positives on similarly-named non-reasoning variants, but that also means unrecognized aliases or renamed deployments simply won't match (documented in the lite-llm.ts comment).

Test Procedure

• pnpm vitest run (unit tests):
  • packages/types/src/__tests__/lite-llm.test.ts: verifies isLiteLLMPreserveReasoningModel matches every listed model id (including provider-prefixed routed names and case-insensitivity), and does not match unrelated ids or same-family substrings (e.g. glm-5-flash, deepseek-v4-mini).
  • src/api/providers/fetchers/__tests__/litellm.spec.ts: verifies getLiteLLMModels sets preserveReasoning: true when either the alias (model_name) or the routed model name (litellm_params.model) matches — with each field tested independently so a match on one alone can't be mistaken for the other being checked — and omits the field otherwise.
  • src/api/providers/__tests__/lite-llm.spec.ts: verifies convertToR1Format is used and reasoning_content survives on the outgoing request when preserveReasoning: true; and that the non-preserveReasoning branch sends no reasoning_content (reflecting that Task.buildCleanConversationHistory already strips the reasoning block upstream in that case).
• Manual repro: configure the LiteLLM provider with litellm_params.model routed to a reasoning model (e.g. deepseek/deepseek-reasoner), run a multi-turn task involving a tool call — previously the follow-up request dropped reasoning_content; it's now preserved.

Pre-Submission Checklist

• Issue Linked: (fill in once the issue is created)
• Scope: Limited to LiteLLM's reasoning_content preservation.
• Self-Review: Done.
• Testing: Added unit tests across 3 files.
• Documentation Impact: None (internal behavior change, no user-facing settings changed).
• Contribution Guidelines: Reviewed.

Documentation Updates

• No documentation updates are required.

Additional Notes

If a reasoning model family or renamed alias isn't caught by this heuristic, additional patterns can be appended to LITELLM_PRESERVE_REASONING_FRAGMENTS.

Get in Touch

hehegwk_23849

Summary by CodeRabbit

• New Features
  • Added automatic “preserve reasoning” detection for supported LiteLLM-routed models using an allowlist of underlying model IDs and case-insensitive matching.
  • Updated conversation formatting for reasoning-enabled models to use the correct message shaping, including proper tool-call handling and merging of trailing tool-result text.
• Bug Fixes
  • Prevented incorrect reasoning detection for unrelated or only-partially-matching model names.
• Tests
  • Added coverage for reasoning capability inference and for reasoning-enabled vs standard message conversion behavior (including exact message output expectations).

[coderabbitai]

📝 Walkthrough

Walkthrough

LiteLLM model aliases are matched against an explicit reasoning-model allowlist. Matching models receive preserveReasoning, selecting R1-format message conversion that retains reasoning content and merges tool results.

Changes

LiteLLM reasoning preservation

Layer / File(s)	Summary
Reasoning alias pattern packages/types/src/providers/lite-llm.ts, packages/types/src/__tests__/lite-llm.test.ts	Defines and tests case-insensitive allowlist matching for routed model aliases and excluded variants.
Model capability inference src/api/providers/fetchers/litellm.ts, src/api/providers/fetchers/__tests__/litellm.spec.ts	Infers preserveReasoning: true from response model names and routed aliases when returning LiteLLM models.
Reasoning-aware message conversion src/api/providers/lite-llm.ts, src/api/providers/__tests__/lite-llm.spec.ts	Uses R1-format conversion with tool-result merging for reasoning models and OpenAI conversion otherwise. Tests verify reasoning content and message shapes.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant getLiteLLMModels
  participant LiteLLMHandler
  participant convertToR1Format
  participant LiteLLM_API
  getLiteLLMModels->>getLiteLLMModels: Match model name and routed alias
  getLiteLLMModels-->>LiteLLMHandler: Return preserveReasoning metadata
  LiteLLMHandler->>convertToR1Format: Convert messages and merge tool results
  convertToR1Format-->>LiteLLMHandler: Return reasoning-preserving messages
  LiteLLMHandler->>LiteLLM_API: Send converted conversation

Loading

Possibly related PRs

• Zoo-Code-Org/Zoo-Code#892: Both changes update LiteLLM message conversion and test reasoning-content preservation.
• Zoo-Code-Org/Zoo-Code#449: Both modify LiteLLM reasoning handling, including LiteLLMHandler.createMessage.

Suggested labels: awaiting-review

Suggested reviewers: navedmerchant, hannesrudolph, jamesrobert20

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes address #891 by inferring preserveReasoning for known LiteLLM reasoning models and preserving reasoning_content across tool-call turns.
Out of Scope Changes check	✅ Passed	The diff stays focused on LiteLLM reasoning preservation and related tests, with no obvious unrelated changes.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The title clearly and concisely summarizes the main LiteLLM reasoning_content preservation change.
Description check	✅ Passed	The description matches the template with issue link, summary, test steps, checklist, and notes filled in.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[edelauna]

Thanks for this contribution, had a small comment on test and the regex in use.

[edelauna]

Could this still match glm-5 as a prefix for names like glm-5.1-flash or glm-5.3? If those should stay excluded, it may be worth making this fragment consume the accepted suffix fully and adding negative cases for those variants.

[daewoongoh]

fixed in d57a97a by dropping the regex/prefix approach entirely. isLiteLLMPreserveReasoningModel now does an exact match against LITELLM_PRESERVE_REASONING_MODEL_IDS via Set.has() on the full (lowercased) model id, so glm-5 no longer matches glm-5.1-flash, glm-5.3, etc. Added negative test cases for exactly these variants (glm-5-flash, glm-4.7-flash, glm-4.7-flashx, glm-4.8, etc.) in lite-llm.test.ts.

[edelauna]

Could this alias be something neutral that does not match the preserve-reasoning pattern? As written, this still passes if the implementation only checks model_name, so it does not quite prove that litellm_params.model is used.

[daewoongoh]

fixed in d57a97a — the alias (model_name) is now a neutral string that doesn't match the allowlist on its own (my-deepseek-alias, my-kimi-alias), so this test only passes if litellm_params.model is actually checked. Added a companion test for the reverse case (alias matches, routed model doesn't) to confirm both fields are checked independently.

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/api/providers/lite-llm.ts (1)

325-325: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Thread CompletePromptOptions into LiteLLM requests
completePrompt drops abortSignal and timeoutMs, so callers can’t cancel or time-limit LiteLLM completions. Pass them through to this.client.chat.completions.create(..., { signal, timeout }) and add coverage for abort/timeout behavior.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/api/providers/lite-llm.ts` at line 325, Update
LiteLLMProvider.completePrompt to propagate CompletePromptOptions.abortSignal
and timeoutMs into this.client.chat.completions.create as the request signal and
timeout, preserving existing completion behavior while enabling cancellation and
time limits. Add coverage verifying both abort and timeout options reach the
LiteLLM request.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@src/api/providers/lite-llm.ts`:
- Line 325: Update LiteLLMProvider.completePrompt to propagate
CompletePromptOptions.abortSignal and timeoutMs into
this.client.chat.completions.create as the request signal and timeout,
preserving existing completion behavior while enabling cancellation and time
limits. Add coverage verifying both abort and timeout options reach the LiteLLM
request.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 0ef8d091-a315-4b0e-9a30-01ada5a94cc1

📥 Commits

Reviewing files that changed from the base of the PR and between ce38822 and 9f9c53b.

📒 Files selected for processing (1)

• src/api/providers/lite-llm.ts