goclaw/docs/packages-pip-npm.md

pip + npm Package Updates (Phase 2a)

Extends the Phase 1 GitHub binary update flow to system-wide pip and npm packages. Closes #900 (Phase 2a).

See also: GitHub binary updates · apk system package updates

---

Overview

When the gateway is running in Standard edition with pip3 and/or npm on PATH, GET /v1/packages/updates includes pip and npm update results alongside GitHub binaries. The UI shows a per-source pill filter; sources without a binary on PATH are hidden automatically.

pip scope: system-wide (--break-system-packages). pip venv / user-site is not supported in Phase 2a.

npm scope: global (--global). Per-project node_modules are not touched.

---

Command Matrix

Source	Check command	Update command	Check timeout	Update timeout
pip	pip3 list --outdated --format json --break-system-packages	pip3 install --upgrade --no-cache-dir --break-system-packages --upgrade-strategy only-if-needed <name>	30 s	5 min
npm	npm outdated --global --json	npm install --global <name>@<version>	30 s	5 min

Pre-release pip check appends --pre in a secondary call (see Pre-Release Handling).

---

Behavior

pip

• pip3 list --outdated --format json emits a JSON array; each element has name, version, latest_version, latest_filetype.
• Exit code is always 0 whether or not updates exist.
• stderr is classified via ClassifyPipStderr into sentinel errors (see Error Classes).

npm

npm's exit-code semantics are non-standard:

Condition	Exit code	Interpretation
No outdated packages	0	No updates
Outdated packages found	1	Updates — parse JSON stdout
Real npm error (ERESOLVE, network, etc.)	1	stderr contains npm ERR!
Ambiguous (exit 1, no stdout, no stderr)	1	Treated as no-updates

The checker inspects exit code and stderr for npm ERR! before deciding whether exit 1 means "updates available" or "real error".

---

Pre-Release Handling

pip

Two-call merge strategy:

1. Primary call (stable only, no --pre): baseline list of outdated packages.
2. If any currently-installed package has a pre-release version (IsPipPreRelease()): Secondary call with --pre to surface the best available upgrade target.
3. Results are merged by package name; when a name appears in both, the entry with the lexicographically higher latest_version wins.

Gate: IsPipPreRelease matches PEP 440 patterns — a, b, rc, dev, .pre, .preview (case-insensitive, digits optional).

npm

Single-call strategy with a skip gate:

• If latest contains an npm pre-release label (-alpha, -beta, -rc, -pre, -preview, -dev, -nightly, -snapshot) and current does not → entry is skipped.
• If current is already a pre-release and latest is too → entry kept (user on pre-release channel receives the newest pre-release update).

This prevents unexpected upgrades from stable channels to unstable channels.

---

Availability Detection

A source is considered available when two gates both pass:

1. Binary present: exec.LookPath("pip3") / exec.LookPath("npm") succeeds.
2. Edition allows it: edition.Current().SupportsPipNpm == true (always true for Standard; always false for Lite desktop).

When a source is unavailable:

• Its checker returns UpdateCheckResult{Available: false} (no error, no updates).
• UpdateRegistry.Availability() maps that source to false.
• GET /v1/packages/updates response includes "availability": {"pip": false, "npm": false}.
• The frontend hides that source from the filter bar.

Lite edition: gateway_packages_wiring.go checks edition.Current().SupportsPipNpm before calling RegisterChecker / RegisterExecutor. Pip and npm checkers are never instantiated — registry.Sources() returns ["github"] only.

---

Error Classes

Sentinel errors are defined in internal/skills/pkg_update_helpers.go.

pip sentinels

Sentinel	Trigger pattern in stderr	i18n key
ErrUpdatePipExternallyManaged	externally-managed-environment / EXTERNALLY-MANAGED	packages.update.pip.externally_managed
ErrUpdatePipPermission	Permission denied / EACCES	packages.update.pip.permission
ErrUpdatePipNotFound	No matching distribution / Could not find a version	packages.update.pip.not_found
ErrUpdatePipNetwork	Read timed out / ConnectionError / network	packages.update.pip.network
ErrUpdatePipConflict	incompatible / dependency resolver / Shallow backtracking	packages.update.pip.conflict

npm sentinels

Sentinel	Trigger pattern in stderr	i18n key
ErrUpdateNpmPermission	EACCES	packages.update.npm.permission
ErrUpdateNpmConflict	ERESOLVE	packages.update.npm.conflict
ErrUpdateNpmNetwork	ETIMEDOUT / ENOTFOUND / getaddrinfo	packages.update.npm.network
ErrUpdateNpmTargetMissing	ETARGET	packages.update.npm.target_missing
ErrUpdateNpmNotFound	E404 / 404 / not in this registry	packages.update.npm.not_found

Unclassified stderr returns a generic wrapped error with a truncated reason (≤ 500 chars, ANSI-stripped).

---

Runbook

Symptom	Fix
pip EACCES — gateway lacks write to site-packages	Run gateway as an owner of /usr/lib/python3/dist-packages, or set PIP_TARGET=/app/data/.pip + add it to PYTHONPATH
npm EACCES — global prefix owned by root	npm config set prefix ~/.npm-global; add ~/.npm-global/bin to PATH in entrypoint
npm ERESOLVE — peer conflict blocks install	SSH into container: npm install -g <name>@<version> --legacy-peer-deps; re-check will clear the entry
pip externally-managed (PEP 668)	Set env var PIP_BREAK_SYSTEM_PACKAGES=1, or upgrade pip to ≥ 23.3 (respects the CLI flag without the env var)

---

Minimum Versions

Runtime	Minimum	Recommended	Notes
pip	20.0	≥ 23.3	--format json requires 20+; --break-system-packages without env var requires 23.3+
npm	6.0	≥ 10	Older versions may not emit JSON exit 1 correctly
Node.js	12	≥ 18 LTS	npm 10 requires Node 18+

---

Shared Locker

InstallSingleDep (skill dep install) and PipUpdateExecutor.Update / NpmUpdateExecutor.Update (update apply) share a single PackageLocker instance injected via SetSharedPackageLocker.

This means concurrent pip install requests (from a skill) and pip upgrade requests (from the update flow) are serialized by the same per-key mutex. The lock key is the bare package name (e.g. "requests") scoped to the source ("pip" or "npm").

Operators must not bypass the gateway and call pip install directly in parallel with gateway operations — doing so defeats the shared lock and risks a partial-install race.

---

Fixture Regeneration

Test fixtures capture pip3 list --outdated --format json and npm outdated -g --json output. When the environment's package versions change, regenerate them:

# pip fixture — include pip version in filename for drift tracking
pip3 --version  # e.g., pip 24.0
pip3 list --outdated --format json --break-system-packages \
  > internal/skills/testdata/pip_outdated_pip24.json

# npm fixture — include npm version in filename
npm --version   # e.g., 10.5.0
npm outdated --global --json \
  > internal/skills/testdata/npm_outdated_npm10.json
# Note: npm exits 1 when packages are outdated — that's expected.

# Update test cases to reference the new filename and expected values.

Fixture files are version-stamped in their names so drift between CI environments is detectable by git diff.