mirror of
https://github.com/tiennm99/ccs.git
synced 2026-07-16 02:11:28 +00:00
docs(image-analysis): document managed runtime flow
This commit is contained in:
@@ -151,6 +151,7 @@ The dashboard provides visual management for all account types:
|
||||
> Setting `websearch.enabled: false` disables the managed local runtime, but CCS still suppresses Anthropic's native `WebSearch` on third-party backends because those providers cannot execute it correctly.
|
||||
|
||||
> **Image backend visibility:** `ccs config image-analysis --set-fallback <backend>` defines the backend CCS should use when a profile alias cannot be inferred directly. Use `--set-profile-backend <profile> <backend>` and `--clear-profile-backend <profile>` for explicit per-profile mappings. In the dashboard, the global `Settings -> Image` section now shows the shared backend routing state, while each profile editor keeps a compact `Image` status card that links back to those global controls.
|
||||
> Third-party launches now expose a first-class local `ImageAnalysis` MCP tool when the runtime is ready, route requests directly to the resolved CCS provider path, and fall back to native `Read` when the managed runtime is unavailable.
|
||||
|
||||
> **Copilot config behavior:** Opening the dashboard or other read-only Copilot endpoints does not rewrite `~/.ccs/copilot.settings.json`. If CCS detects deprecated Copilot model IDs such as `raptor-mini`, it shows warnings immediately and only persists replacements when you explicitly save the Copilot configuration.
|
||||
|
||||
@@ -687,6 +688,38 @@ See [docs/websearch.md](./docs/websearch.md) for detailed configuration and trou
|
||||
|
||||
<br>
|
||||
|
||||
## Image Analysis
|
||||
|
||||
Third-party profiles (Gemini, Codex, GLM bridge profiles, Copilot, and similar routes) now use a first-class local `ImageAnalysis` MCP tool instead of relying on a denied `Read` hook as the normal experience.
|
||||
|
||||
### How It Works
|
||||
|
||||
| Profile Type | Image Method |
|
||||
|--------------|--------------|
|
||||
| Claude (native) | Native Claude vision / native `Read` |
|
||||
| Third-party profiles | CCS local MCP `ImageAnalysis` tool when available |
|
||||
| Third-party when runtime unavailable | Native `Read` fallback |
|
||||
|
||||
### Direct Provider Routing
|
||||
|
||||
When the managed tool is used, CCS resolves the backend before launch and posts image-analysis requests directly to the provider-scoped CCS route:
|
||||
|
||||
```text
|
||||
/api/provider/<backend>/v1/messages
|
||||
```
|
||||
|
||||
That path goes from Claude -> `ccs-image-analysis.ImageAnalysis` -> CCS/CLIProxy provider routing. It does not bounce through Claude Code, a helper CLI, or a second model wrapper.
|
||||
|
||||
### Prompting and Fallback
|
||||
|
||||
CCS appends a short steering hint telling Claude to prefer `ImageAnalysis` over `Read` for local image and PDF files. The tool uses editable prompt templates from `~/.ccs/prompts/image-analysis/` and automatically picks `default`, `screenshot`, or `document`.
|
||||
|
||||
If the local runtime, auth, or proxy path is unavailable, CCS keeps the launch non-fatal and falls back to native `Read`. The legacy `Read` hook remains only as a compatibility fallback when CCS can install it safely.
|
||||
|
||||
See [docs/image-analysis.md](./docs/image-analysis.md) for configuration, routing details, and troubleshooting.
|
||||
|
||||
<br>
|
||||
|
||||
## Remote CLIProxy
|
||||
|
||||
CCS v7.x supports connecting to remote CLIProxyAPI instances, enabling:
|
||||
|
||||
@@ -54,7 +54,7 @@ src/
|
||||
│ │ └── [subcommand files...]
|
||||
│ ├── cliproxy-command.ts # CLIProxy subcommand handling
|
||||
│ ├── config-command.ts # Config management commands
|
||||
│ ├── config-image-analysis-command.ts # Image analysis hook config (NEW v7.34)
|
||||
│ ├── config-image-analysis-command.ts # First-class ImageAnalysis config (NEW v7.34)
|
||||
│ ├── named-command-router.ts # Reusable named-command dispatcher
|
||||
│ ├── doctor-command.ts # Health diagnostics
|
||||
│ ├── env-command.ts # Export shell env vars for third-party tools (v7.39)
|
||||
@@ -149,7 +149,7 @@ src/
|
||||
│ ├── index.ts # Barrel export
|
||||
│ ├── checks/ # Diagnostic checks
|
||||
│ │ ├── index.ts
|
||||
│ │ └── image-analysis-check.ts # Image hook validation (NEW v7.34)
|
||||
│ │ └── image-analysis-check.ts # ImageAnalysis runtime validation (NEW v7.34)
|
||||
│ └── repair/ # Auto-repair logic
|
||||
│ └── index.ts
|
||||
│
|
||||
@@ -169,15 +169,17 @@ src/
|
||||
│ │ └── spinners.ts # Progress spinners
|
||||
│ ├── websearch/ # Search tool integrations
|
||||
│ │ └── index.ts
|
||||
│ ├── hooks/ # Claude Code hooks (NEW v7.34)
|
||||
│ ├── hooks/ # Claude Code compatibility hooks (NEW v7.34)
|
||||
│ │ ├── index.ts
|
||||
│ │ ├── image-analyzer-hook-installer.ts
|
||||
│ │ ├── image-analyzer-hook-configuration.ts
|
||||
│ │ ├── image-analyzer-profile-hook-injector.ts
|
||||
│ │ └── get-image-analysis-hook-env.ts
|
||||
│ ├── image-analysis/ # Image analysis hook utilities (NEW v7.34)
|
||||
│ ├── image-analysis/ # ImageAnalysis MCP/runtime utilities (NEW v7.34)
|
||||
│ │ ├── index.ts
|
||||
│ │ └── hook-installer.ts
|
||||
│ │ ├── hook-installer.ts
|
||||
│ │ ├── mcp-installer.ts
|
||||
│ │ └── claude-tool-args.ts
|
||||
│ └── [utility files...]
|
||||
│
|
||||
└── web-server/ # Express web server (heavily modularized)
|
||||
@@ -636,4 +638,5 @@ tests/
|
||||
- [System Architecture](./system-architecture/index.md) - High-level architecture diagrams
|
||||
- [Project Roadmap](./project-roadmap.md) - Modularization phases and future work
|
||||
- [WebSearch](./websearch.md) - WebSearch feature documentation
|
||||
- [Image Analysis](./image-analysis.md) - First-class ImageAnalysis runtime documentation
|
||||
- [CLAUDE.md](../CLAUDE.md) - AI-facing development guidance
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
# Image Analysis Configuration Guide
|
||||
|
||||
CCS provides first-class image and PDF analysis for third-party Claude launches that do not have reliable native vision support.
|
||||
|
||||
## How Image Analysis Works
|
||||
|
||||
Native Claude accounts keep Anthropic's own vision flow.
|
||||
|
||||
Third-party profiles now use a CCS-managed local MCP tool named `ImageAnalysis` when the runtime is available. CCS also appends a short steering hint so Claude prefers that tool over `Read` for local image and PDF files.
|
||||
|
||||
If the managed runtime, auth, or proxy path is unavailable, CCS falls back to native `Read` instead of failing the whole launch. The old `Read` hook remains only as a compatibility fallback when it can be installed safely.
|
||||
|
||||
## Routing Model
|
||||
|
||||
ImageAnalysis requests go straight to the CCS-managed provider route:
|
||||
|
||||
```text
|
||||
Claude -> ccs-image-analysis MCP -> CCS provider route -> /api/provider/<backend>/v1/messages
|
||||
```
|
||||
|
||||
Important:
|
||||
- CCS does not relay image analysis through Claude Code, another CLI, or a second model wrapper.
|
||||
- For bridge-backed settings profiles, CCS resolves the backend and provider path before launch.
|
||||
- CCS avoids leaking a profile's ordinary third-party `ANTHROPIC_BASE_URL` or token into image analysis unless that profile is explicitly using a CLIProxy bridge.
|
||||
|
||||
## Profile Behavior
|
||||
|
||||
| Profile Type | Image Method |
|
||||
|--------------|--------------|
|
||||
| Claude `default` / `account` | Native Claude vision / native `Read` |
|
||||
| Third-party settings / CLIProxy / Copilot | CCS local `ImageAnalysis` MCP tool when ready |
|
||||
| Third-party when runtime unavailable | Native `Read` fallback |
|
||||
|
||||
## Configuration
|
||||
|
||||
Configure via dashboard (`Settings -> Image`) or `~/.ccs/config.yaml`:
|
||||
|
||||
```yaml
|
||||
image_analysis:
|
||||
enabled: true
|
||||
timeout: 60
|
||||
fallback_backend: agy
|
||||
provider_models:
|
||||
agy: gemini-3-1-flash-preview
|
||||
codex: gpt-5.1-codex-mini
|
||||
ghcp: claude-haiku-4.5
|
||||
```
|
||||
|
||||
Useful commands:
|
||||
|
||||
```bash
|
||||
ccs config image-analysis
|
||||
ccs config image-analysis --enable
|
||||
ccs config image-analysis --disable
|
||||
ccs config image-analysis --set-fallback agy
|
||||
ccs config image-analysis --set-profile-backend glm agy
|
||||
ccs config image-analysis --clear-profile-backend glm
|
||||
```
|
||||
|
||||
## Prompt Templates
|
||||
|
||||
CCS installs editable prompt templates at:
|
||||
|
||||
```text
|
||||
~/.ccs/prompts/image-analysis/
|
||||
```
|
||||
|
||||
Templates:
|
||||
- `default.txt`
|
||||
- `screenshot.txt`
|
||||
- `document.txt`
|
||||
|
||||
CCS automatically selects `screenshot` for screenshot-like filenames, `document` for PDFs, and `default` otherwise.
|
||||
|
||||
## Runtime Environment
|
||||
|
||||
Key runtime env vars:
|
||||
|
||||
| Variable | Purpose |
|
||||
|----------|---------|
|
||||
| `CCS_IMAGE_ANALYSIS_SKIP` | Disable image analysis for the current launch |
|
||||
| `CCS_IMAGE_ANALYSIS_RUNTIME_BASE_URL` | Explicit CCS runtime base URL |
|
||||
| `CCS_IMAGE_ANALYSIS_RUNTIME_PATH` | Provider route such as `/api/provider/agy` |
|
||||
| `CCS_IMAGE_ANALYSIS_RUNTIME_API_KEY` | Explicit CCS runtime auth key |
|
||||
| `CCS_IMAGE_ANALYSIS_MODEL` | Force a single image-analysis model |
|
||||
| `CCS_DEBUG` | Verbose runtime logging |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Claude still uses `Read`
|
||||
|
||||
- Confirm `ccs config image-analysis` shows `enabled: true`
|
||||
- Check the active profile resolves to a configured backend
|
||||
- Run with `CCS_DEBUG=1` to see runtime preparation details
|
||||
|
||||
### ImageAnalysis is not exposed
|
||||
|
||||
- Verify CLIProxy auth for the resolved backend
|
||||
- Verify the local or remote CLIProxy target is reachable
|
||||
- Check `~/.claude.json` and inherited account configs for `ccs-image-analysis`
|
||||
|
||||
### I need to prove requests are going directly to the provider route
|
||||
|
||||
Run with `CCS_DEBUG=1` and inspect the resolved runtime path. The request target should be provider-scoped, for example:
|
||||
|
||||
```text
|
||||
/api/provider/agy/v1/messages
|
||||
```
|
||||
@@ -1,6 +1,6 @@
|
||||
# CCS Product Development Requirements (PDR)
|
||||
|
||||
Last Updated: 2026-03-24
|
||||
Last Updated: 2026-04-02
|
||||
|
||||
## Product Overview
|
||||
|
||||
@@ -10,7 +10,7 @@ Last Updated: 2026-03-24
|
||||
|
||||
**Description**: CLI wrapper enabling seamless switching between multiple Claude accounts and alternative AI providers (GLM, Gemini, Codex, OpenRouter, Qwen, Kimi, DeepSeek) with a React-based dashboard for configuration management. Supports both local and remote CLIProxyAPI instances, hybrid quota management, and official Claude channel runtime setup for Telegram, Discord, and iMessage.
|
||||
|
||||
**Current Version**: v7.34.x (Image Analysis Hook + Performance Improvements)
|
||||
**Current Version**: v7.34.x+ (First-class ImageAnalysis MCP tooling, WebSearch MCP, performance improvements)
|
||||
|
||||
---
|
||||
|
||||
@@ -36,8 +36,9 @@ CCS provides:
|
||||
4. **API Profiles**: GLM, Kimi, OpenRouter, any Anthropic-compatible API
|
||||
5. **Visual Dashboard**: React SPA for configuration management
|
||||
6. **Automatic WebSearch**: First-class local WebSearch tool with deterministic provider chain for third-party providers
|
||||
7. **Usage Analytics**: Token tracking, cost analysis, model breakdown
|
||||
8. **Official Claude Channels**: Runtime auto-enable plus dashboard token/config flow for Telegram, Discord, and macOS-only iMessage
|
||||
7. **Automatic Image Analysis**: First-class local ImageAnalysis tool with direct provider routing for third-party profiles
|
||||
8. **Usage Analytics**: Token tracking, cost analysis, model breakdown
|
||||
9. **Official Claude Channels**: Runtime auto-enable plus dashboard token/config flow for Telegram, Discord, and macOS-only iMessage
|
||||
|
||||
---
|
||||
|
||||
@@ -99,6 +100,13 @@ CCS provides:
|
||||
- Keep Gemini CLI, OpenCode, and Grok as optional legacy fallback
|
||||
- Graceful fallback chain
|
||||
|
||||
### FR-007A: First-Class Image Analysis
|
||||
- Expose a CCS-managed local `ImageAnalysis` MCP tool for third-party profiles that need provider-backed vision
|
||||
- Resolve the provider route before launch and send requests directly to `/api/provider/<backend>/v1/messages`
|
||||
- Use editable prompt templates for `default`, `screenshot`, and `document` analysis modes
|
||||
- Keep the old `Read` hook as compatibility fallback only, not the primary user experience
|
||||
- Fall back to native `Read` without failing the whole launch when the managed runtime is unavailable
|
||||
|
||||
### FR-008: Remote CLIProxy Support
|
||||
- Connect to remote CLIProxyAPI instances
|
||||
- CLI flags for proxy configuration (--proxy-host, --proxy-port, etc.)
|
||||
@@ -277,10 +285,11 @@ CCS provides:
|
||||
- [x] Entrypoint with privilege dropping
|
||||
|
||||
### v7.34 Release (Complete)
|
||||
- [x] Image Analysis Hook for vision model proxying
|
||||
- [x] Auto-injection for agy, gemini, codex, cliproxy profiles
|
||||
- [x] Skip hook for Claude Sub accounts (native vision)
|
||||
- [x] CLIProxy fallback with deprecated block-image-read
|
||||
- [x] First-class `ImageAnalysis` MCP tool for third-party launches
|
||||
- [x] Direct provider-scoped routing for image analysis requests
|
||||
- [x] Prompt template selection for default / screenshot / document flows
|
||||
- [x] Hook fallback retained only for compatibility
|
||||
- [x] Non-fatal native `Read` fallback when managed runtime is unavailable
|
||||
- [x] `ccs config image-analysis` CLI command
|
||||
- [x] Doctor integration for hook validation
|
||||
- [x] 791-line E2E test suite for image analysis
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# CCS Project Roadmap
|
||||
|
||||
Last Updated: 2026-04-01
|
||||
Last Updated: 2026-04-02
|
||||
|
||||
Forward-looking roadmap documenting current priorities, GitHub issues, and future feature plans.
|
||||
|
||||
@@ -41,6 +41,7 @@ All major modularization work is complete. The codebase evolved from monolithic
|
||||
|
||||
### Recent Fixes
|
||||
|
||||
- **2026-04-02**: Third-party image and PDF analysis now follows the same first-class local-tool model as WebSearch. CCS provisions `ccs-image-analysis` as a managed MCP tool, routes requests directly to provider-scoped CCS endpoints such as `/api/provider/agy/v1/messages`, keeps editable prompt templates under `~/.ccs/prompts/image-analysis/`, and demotes the old `Read` hook to a best-effort compatibility fallback. Launches now stay non-fatal and fall back to native `Read` when the managed runtime cannot be prepared.
|
||||
- **2026-04-01**: The `Compatible -> Codex CLI` dashboard now exposes manual long-context controls for `model_context_window` and `model_auto_compact_token_limit`. CCS reads and patches those upstream Codex config keys directly, adds official guidance that GPT-5.4 long context is experimental and opt-in, and keeps the behavior manual-only so the dashboard never auto-fills or auto-saves long-context values for the user.
|
||||
- **2026-03-30**: **#862** Third-party WebSearch now uses a first-class CCS-managed MCP tool path instead of relying on a denied native Anthropic `WebSearch` call as the normal UX. CCS provisions `ccs-websearch` into `~/.claude.json`, syncs it into isolated account configs when needed, suppresses native `WebSearch` on third-party launches, preserves the provider order `Exa -> Tavily -> Brave -> DuckDuckGo -> legacy CLI fallback`, and keeps the old hook runtime only as shared provider plumbing plus compatibility fallback. Uninstall cleanup now also removes the managed WebSearch MCP runtime.
|
||||
- **2026-03-28**: **#773** CCS now ships a dedicated `Compatible -> Codex CLI` dashboard route with a real split-view control center. The page detects the local Codex binary, keeps overview/docs guidance, and adds guided editors for the user-owned `~/.codex/config.toml` layer: top-level runtime defaults, project trust, profiles, model providers, MCP servers, and supported feature flags. Structured saves intentionally normalize TOML formatting and drop comments, so the raw editor remains the fidelity escape hatch. Follow-up fixes added immediate raw snapshot refresh, refresh/discard recovery for stale raw drafts, dirty raw-editor guarding for structured controls, project-trust path validation, read-only handling for unreadable config files, preservation of unsupported upstream values such as granular `approval_policy`, and feature reset-to-default support. CCS still warns that transient runtime overrides such as `codex -c key=value` and `CCS_CODEX_API_KEY` may change effective behavior without persisting into the file.
|
||||
|
||||
@@ -537,29 +537,29 @@ Image Analysis Hook enables vision model proxying through CLIProxy with automati
|
||||
Claude CLI processes image request
|
||||
|
|
||||
v
|
||||
Hook intercepts image request
|
||||
Claude prefers ImageAnalysis MCP tool
|
||||
|
|
||||
v
|
||||
Vision Model Proxying (via CLIProxyAPI)
|
||||
CCS provider-backed image analysis
|
||||
|
|
||||
+---> Gemini, Codex, AGY support vision
|
||||
+---> Provider route resolved before launch
|
||||
|
|
||||
+---> Kiro (Claude native vision)
|
||||
+---> Direct request to /api/provider/<backend>/v1/messages
|
||||
|
|
||||
+---> Skip for Claude Sub accounts (native vision)
|
||||
+---> Native Read fallback if runtime/auth/proxy is unavailable
|
||||
|
|
||||
v
|
||||
Vision response returned to Claude CLI
|
||||
Text description returned to Claude CLI
|
||||
```
|
||||
|
||||
### Hook Environment
|
||||
### Runtime Environment
|
||||
|
||||
```typescript
|
||||
// getImageAnalysisHookEnv()
|
||||
{
|
||||
ANTHROPIC_IMAGE_HOOK_URL: 'http://localhost:8317/api/image-analysis',
|
||||
// or for remote proxy:
|
||||
ANTHROPIC_IMAGE_HOOK_URL: 'https://proxy.example.com:8317/api/image-analysis',
|
||||
CCS_IMAGE_ANALYSIS_RUNTIME_BASE_URL: 'http://127.0.0.1:8317',
|
||||
CCS_IMAGE_ANALYSIS_RUNTIME_PATH: '/api/provider/agy',
|
||||
CCS_IMAGE_ANALYSIS_RUNTIME_API_KEY: 'ccs-internal-managed',
|
||||
}
|
||||
```
|
||||
|
||||
@@ -567,12 +567,12 @@ Image Analysis Hook enables vision model proxying through CLIProxy with automati
|
||||
|
||||
| Provider | Vision Support | Notes |
|
||||
|----------|---|---|
|
||||
| Gemini | ✓ | Via CLIProxy image analysis |
|
||||
| Codex | ✓ | Via CLIProxy image analysis |
|
||||
| Antigravity | ✓ | Via CLIProxy image analysis |
|
||||
| Kiro | ✓ | Native Claude vision (no proxy needed) |
|
||||
| Copilot | ✗ | Not supported |
|
||||
| GLM/Kimi | ✗ | Requires direct API implementation |
|
||||
| Gemini | ✓ | Via CCS ImageAnalysis provider route |
|
||||
| Codex | ✓ | Via CCS ImageAnalysis provider route |
|
||||
| Antigravity | ✓ | Via CCS ImageAnalysis provider route |
|
||||
| Kiro | ✓ | Via mapped CCS provider route when configured |
|
||||
| Copilot | ✓ | Via mapped ghcp provider route |
|
||||
| GLM/Kimi | ✓ | Via explicit or fallback backend mapping |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -578,7 +578,7 @@ Run ${color('ccs config', 'command')} for web dashboard`.trim();
|
||||
|
||||
// Image Analysis
|
||||
printSubSection(
|
||||
'Image Analysis (CLIProxy vision)',
|
||||
'Image Analysis (first-class local tool)',
|
||||
[
|
||||
['ccs config image-analysis', 'Show current settings'],
|
||||
['ccs config image-analysis --enable', 'Enable for CLIProxy providers'],
|
||||
@@ -586,9 +586,9 @@ Run ${color('ccs config', 'command')} for web dashboard`.trim();
|
||||
['ccs config image-analysis --timeout 120', 'Set analysis timeout'],
|
||||
['ccs config image-analysis --set-model <p> <m>', 'Set provider model'],
|
||||
['', ''],
|
||||
['Note:', 'When enabled, images/PDFs are analyzed via vision models'],
|
||||
['', 'instead of passing raw data to Claude. Works with CLIProxy'],
|
||||
['', 'providers (agy, gemini, codex, kiro, ghcp).'],
|
||||
['Note:', 'When ready, third-party launches expose the local ImageAnalysis MCP tool'],
|
||||
['', 'and route requests directly to the resolved CCS provider path.'],
|
||||
['', 'If runtime/auth/proxy is unavailable, CCS falls back to native Read.'],
|
||||
],
|
||||
writeLine
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user