iTerm2 AI Chat sends deprecated functions/function_call instead of tools/tool_choice — breaks tool use with Mistral (and any non‑OpenAI provider)

[Markus Schicker]

iTerm2 AI Chat sends deprecated functions/function_call instead of tools/tool_choice — breaks tool use with Mistral (and any non‑OpenAI provider)

Environment

• iTerm2:  3.6.10
• AI plugin: installed and enabled
• Provider: Mistral AI (api.mistral.ai)
• API style configured in iTerm2: Chat Completions
• Features enabled: Function Calling, Streaming Responses

Summary

iTerm2’s Composer AI works fine against Mistral’s API. AI Chat, however, fails. After extensive debugging through a LiteLLM proxy, the root cause turns out to be that iTerm2’s AI Chat sends tool definitions using OpenAI’s legacy functions / function_call fields, which OpenAI itself deprecated in mid‑2023 in favour of tools / tool_choice. Mistral (and most non‑OpenAI providers) never implemented the legacy schema and reject the request, so tool calling silently degrades — the model just prints execute_command: {...} as plain text instead of issuing a real tool call iTerm2 can intercept and execute.

Direct symptoms in iTerm2

1. Initial error talking directly to Mistral:

An error occurred: Error from Unknown Platform: HTTP request failed with status 422.Composer worked, AI Chat did not.

1. After inserting a LiteLLM proxy with drop_params: false (to surface what Mistral was actually rejecting):

litellm.UnsupportedParamsError: mistral does not support parameters: ['functions', 'function_call'], for model=mistral-large-latest.This confirms iTerm2 is sending the legacy fields.

1. After dropping functions/function_call via drop_params: ["functions", "function_call"]:
   The request succeeds (HTTP 200), but the model produces execute_command: {"command": "..."} as plain assistant text content. iTerm2 does not execute it, because there is no structured tool_calls block in the response — there can’t be, since no tools array was ever sent.

In other words: iTerm2’s AI Chat appears to send only functions[], not the modern tools[] array. So when functions[] is stripped (which is the only way to get past Mistral’s validator), the model has no tool schema at all and falls back to imitating tool‑call syntax in prose.

What I tried (timeline)
#
Step
Result
1
Pointed iTerm2 directly at https://api.mistral.ai/v1/chat/completionswith mistral-large-latest, API style = Chat Completions, Function Calling + Streaming on
Composer ✅, AI Chat ❌ HTTP 422
2
Disabled Function Calling and Streaming in iTerm2’s model dialog
AI Chat no longer 422s, but is no longer agentic (no command execution at all)
3
Installed LiteLLM (uv tool install 'litellm[proxy]') as a translating proxy in front of Mistral
Required Python 3.13 — uvloop is not yet 3.14‑compatible, separate issue
4
Configured LiteLLM with drop_params: true and pointed iTerm2 at http://localhost:4000/chat/completions
422 went away, but AI Chat still printed execute_command: {...} as text — commands never executed
5
Switched to drop_params: false to force Mistral to reveal which fields it was rejecting
Got the explicit UnsupportedParamsError: ['functions', 'function_call'] from LiteLLM
6
Set per‑model drop_params: ["functions", "function_call"]
Requests succeed, but model still emits tool calls as text (Scenario B). Confirmed via --detailed_debug that the outgoing request to Mistral contains no tools array — meaning iTerm2 never sent one, only the legacy functions field
7
Tried models mistral-large-latest, mistral-small-latest, codestral-latest
Same behaviour across all of them — the issue is the request shape, not model capabilityRoot cause (as I understand it)

iTerm2’s AI Chat builds its outgoing request using the pre‑June‑2023 OpenAI function‑calling format:

{ "model": "...", "messages": [...], "functions": [ { "name": "execute_command", "description": "...", "parameters": {...} }, ... ], "function_call": "auto" }

The modern format (which Mistral, Anthropic, Gemini, Groq, Together, Fireworks, and basically every non‑OpenAI provider implements) is:

{ "model": "...", "messages": [...], "tools": [ { "type": "function", "function": { "name": "execute_command", "description": "...", "parameters": {...} } } ], "tool_choice": "auto" }

Because iTerm2 only emits the legacy shape, AI Chat tool calls are effectively OpenAI‑only. Anyone configuring AI Chat against a non‑OpenAI Chat Completions endpoint either gets a 422 (provider validates and rejects unknown fields) or a silent regression to text‑only chat (provider ignores unknown fields).

Composer works against Mistral because it doesn’t use tool calling — it’s a single‑shot text completion.

Requested fix

Update iTerm2’s AI Chat to emit tool definitions in the modern tools / tool_choice format. OpenAI’s API still accepts this form (it’s now the canonical one), so the change is backward‑compatible for OpenAI users while immediately unlocking AI Chat for every other OpenAI‑compatible provider — Mistral, Anthropic via OpenAI‑shim, Gemini, Groq, OpenRouter, LM Studio, Ollama, vLLM, etc.

Optionally: expose a setting like “Use legacy functions API” (default off) for anyone pinned to a very old OpenAI‑compatible endpoint.

[George Nachman]

Thanks for the incredibly thorough write-up. You diagnosed it correctly: AI Chat in 3.6.10 builds the Chat Completions request using the pre-2023 functions / function_call fields, which is fine against OpenAI itself but rejected (or silently degraded) by every other OpenAI-compatible provider that validates unknown fields.

Good news: this is already fixed on the main branch and ships in the current 3.7 beta. The relevant change is commit 8a2d22f6f (“Refactor AI parser/builder layer and add live test harness”), which switched ModernBodyRequestBuilder in sources/AITerm/CompletionsOpenAI.swift from:

functions: [...]

function_call: "auto"

to:

tools: [{ type: "function", function: {...} }, ...]

tool_choice: "auto"

If you want to verify before 3.7.0 GAm grab the latest nightly from https://iterm2.com/nightly/latest and point AI Chat directly at https://api.mistral.ai/v1/chat/completions with API style = Chat Completions, model=mistral-large-latest, Function Calling on, Streaming on. The LiteLLM proxy is no longer needed.

Two related notes worth mentioning:

- Composer was always fine because, as you suspected, it does not use tool calling at all. It is a single-shot completion that does not exercise the request-shape bug. 

- The “Use legacy functions API” toggle you suggested is a nice idea, but in practice no current vendor needs it (including OpenAI), so I'm not planning to add one. If anyone is pinned to a self-hosted server old enough that it only understands functions / function_call, the release notes will call that out so they know to upgrade the backend.

--
You received this message because you are subscribed to the Google Groups "iterm2-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iterm2-discus...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/iterm2-discuss/e5438ddb-0053-4dcd-8622-41ee03310e51n%40googlegroups.com.

[Markus Schicker]

Many thanks George - I've tested it - works perfectly.

Kind regards

Markus

PS: AI assisted me in writing the summary after my debug session ;-)