Configure Per-Request Timeouts
Support configurable longer timeouts for large-payload RPCs and model requests, including slow local model loading, generation, retries, and large outputs, while retaining shorter defaults for ordinary calls.
Explore challenge
The problem
Support configurable longer timeouts for large-payload RPCs and model requests, including slow local model loading, generation, retries, and large outputs, while retaining shorter defaults for ordinary calls.
Start with these discussions
- llm-pi-ai Timeout increase for local LLMsgithub · ▲17 · 21 comments
- 使用本地部署的模型时,如果重启过电脑,Vram内容被清空,则发起对话时模型要思考很久时间,如果超过五分钟就报错重试了,导致无法继续对话github · ▲1 · 4 comments
- dsh-client-connection:history 等大载荷 RPC 受全局 30s 超时限制,Web 端报「历史加载失败:signal timed out (internal)」—— timeoutPolicy 钩子已存在但无人接线github · ▲1 · 1 comments