mirror of
https://github.com/tiennm99/goclaw.git
synced 2026-10-04 16:13:47 +00:00
thotam
e2ec3370dd
Server-side webhook agent runs (sync, async, and admin test) used Stream:false, so OpenAI-compatible routers that only cache streaming requests never populated or served their prompt cache. Webhook runs paid full input-token price on every turn even with a stable session and an identical multi-turn prefix, while WS chat (Stream:true) got cache hits from the 3rd turn. Add gateway.webhook_stream (default true, env GOCLAW_WEBHOOK_STREAM) and apply it to all three run sites. ChatStream returns the fully assembled response, so the payload returned to the caller is unchanged. Set to false to restore non-streaming.
19 lines
705 B
Go
19 lines
705 B
Go
package webhooks
|
|
|
|
// DefaultStream is the fallback for the webhook internal-streaming flag when
|
|
// config is unset. Streaming is on by default so the upstream provider can
|
|
// populate/serve its prompt cache for server-side webhook agent runs (sync,
|
|
// async, and test). Non-streaming requests are not cached by some
|
|
// OpenAI-compatible routers, so webhook runs would otherwise always pay full
|
|
// input-token price even with a stable session.
|
|
const DefaultStream = true
|
|
|
|
// ResolveStream converts the optional gateway.webhook_stream config into a bool:
|
|
// nil → DefaultStream (true); otherwise the explicit value.
|
|
func ResolveStream(v *bool) bool {
|
|
if v == nil {
|
|
return DefaultStream
|
|
}
|
|
return *v
|
|
}
|