One GitHub connection feeds both a Knowledge source and an agent tool.
Users connect with a personal access token, checked against GitHub and
named after the account, or, when an admin registers a GitHub App
(GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET, GITHUB_APP_SLUG), with Sign in
with GitHub. App tokens expire after eight hours and are refreshed before
each sync or tool call; tokens with no expiry are never treated as expired.
The catalog gains an optional second sign-in method (oauth_settings,
exposed as sign_in_methods) without changing any other connector.
Sync lists the repositories the connection can read (the token's own, or
the App installations') and ingests one with the connection's token. The
tool is GitHub's read-only MCP server (api.githubcopilot.com/mcp/readonly):
setup discovers its actions and creates it bound to the connection, and
the executor sends the connection's token only to that server.
Adds a v2 credential envelope next to the v1 tool-secret helpers:
AES-256-GCM, a master key derived once per process from
ENCRYPTION_SECRET_KEY, and a per-record key from HKDF over the owner's
id, which is also the associated data, so a blob moved onto another
user's row does not decrypt. The envelope names its key, so
ENCRYPTION_SECRET_KEY_PREVIOUS keeps old rows readable during a
rotation.
Log redaction now also covers token_info, tokens and client_info, and
the API warns at startup when the public default key is in use.
TRACES_* settings for the per-request trace timeline and its GenAI OTel
export. opentelemetry-api was only transitive; the tracing package
imports it directly.
Rename the unused *_cost_per_token capability fields to USD per 1M tokens,
add prompt-cache read/write rates, and ship list prices for the hosted
catalogs. The old per-token keys still load, scaled, with a warning.
docsgpt/pricing.py turns a call's token bins into a USD cost. Models with
no declared rate cost $0 unless QUOTA_UNPRICED_RATE_PER_MILLION is set.
The ASGI message events route now accepts the same scopes as its Flask
sibling (conversations:read or chat:run) through a shared constant. A token
request that fails routing gets Flask's 404/405 instead of a 403. Creating a
token retires an expired token that still held the name. allowed_ids uses
is_pat instead of a bare literal. PAT_ENABLED is documented as the master
switch it is: turning it off stops every existing token from authenticating.
The docs explain that sources are matched by name (oldest wins) and point CI
flows at sources upload --replace.
A personal_access_tokens table (migration 0032) holds scoped user-level API
credentials. Only the SHA-256 of the secret is stored, like device session
tokens. Lookups exclude revoked and expired tokens and the tokens of
deactivated users. PAT_* settings cover the feature switch, default and
maximum lifetime, the operator opt-in for non-expiring tokens and the
per-user cap.
Graph retrieval tied plain vector search at best and never beat it. Measured
across five corpora, the bottleneck was seeding, not the graph: the walk
started from nodes whose embeddings were computed from bare entity names, and
a whole question shares almost nothing with a name like "Quill".
Extraction now embeds each node from "name (type): description" and each
relationship as the fact it asserts ("Alder streams_to Quill: ..."), stored on
a new nullable graph_edges.fact_embedding column that ensure_vector_schema adds
in place. Entity names are canonicalised (case, punctuation, word breaks and a
cautious plural) so "VECTOR_STORE" and "vector stores" land on one node. Extraction calls run
concurrently (GRAPHRAG_EXTRACTION_WORKERS, default 8) while embedding and graph
writes stay serial on the task thread, so ordering and idempotency are
unchanged; that measured 8.4x faster with identical output.
Retrieval gains per-source options, stored under retrieval.graph and read live
at query time:
- seed_strategy: start from matching entities (default) or matching
relationships, which can reach an entity the question never names;
- passage_nodes (on): walk the source's passages alongside entities, with
PageRank damping 0.5 instead of 0.85;
- blend_vector (on): fuse the graph ranking with the source's vector ranking
by reciprocal rank.
The defaults are the measured-best configuration. Through GraphRAGRetriever,
the new seeding moved recall@4 from 0.41 to 0.68 on a multi-hop corpus and
from 0.50 to 1.00 on the docs corpus, and regressed none of the corpora
measured. Existing graphs keep name-only embeddings until rebuilt.
The docs site failed to build: a bare "<= 1" in the prose of the
generated page is parsed by MDX as the start of a JSX tag ("Unexpected
character '=' before name"). Constraints are rendered as code spans now,
where MDX leaves them alone, and a test rejects any bare <, { or } outside
a code span so a future description cannot reintroduce the failure.
Verified with a local next build of the docs site.
Review follow-up. The per-group secret validators normalised a hand-picked
list of API keys, which left other optional credentials and overrides
(OPEN_ROUTER_API_KEY, S3 and Daytona keys, ELASTIC_PASSWORD, the OIDC
trio, connector client ids, MICROSOFT_AUTHORITY, MCP_OAUTH_REDIRECT_URI)
holding the literal "None" or "" a .env file spells "unset" with, so
truthiness checks and fallbacks downstream saw a value. One rule on the
group base replaces those lists: every Optional[str] field maps "", "None"
and whitespace to None and strips real values. Plain str fields are left
alone. The OIDC required-settings check therefore also rejects those
spellings.
EMBEDDINGS_POOLING is Literal["cls", "mean"] with case-insensitive
parsing; its consumer silently ignored anything else.
Bounds added where the consumer rejects or misbehaves on the value:
SCHEDULE_RUN_OUTPUT_RETENTION_DAYS and MESSAGE_EVENTS_RETENTION_DAYS (the
cleanup repositories raise on <= 0), EMBEDDINGS_DELEGATE_TIMEOUT, the
remote-device idle/pairing/invocation TTLs and CELERY_VISIBILITY_TIMEOUT
(> 0), REMOTE_DEVICE_CMD_QUEUE_TTL_SECONDS (> 605, the documented drain
deadline), GRAPHRAG_MAX_CHUNKS_FOR_EXTRACTION (>= 0; negative would slice
the pending list from the end).
The generated reference now renders generic type arguments
(dict[str, int] rather than dict).
About 85 call sites read a setting as getattr(settings, "NAME", fallback),
each carrying its own copy of the default. Every one of those names is a
field with a default on the model, so the fallback could never apply to
the real settings object; it only masked drift. Two had drifted:
- OPENAI_PROMPT_CACHE_KEY defaults to True on the model but the reader
fell back to False, and two test stubs relied on that.
- SharePoint's MICROSOFT_AUTHORITY fallback to
https://login.microsoftonline.com/<tenant> never fired, because the
attribute always exists (as None), so MSAL got authority=None. The
connector now derives the tenant authority when the setting is unset,
as its test always assumed.
Four places read EMBEDDINGS_KEY straight from os.environ, skipping the
"None"/"" normalisation the model applies; they read the setting now.
Test stubs that replaced a module's settings with a SimpleNamespace list
every setting the code under test reads.
SAGEMAKER_REGION, SAGEMAKER_ACCESS_KEY and SAGEMAKER_SECRET_KEY survive
only as a fallback for the S3_* credentials. They carry
Field(deprecated=...) now, so any read emits a DeprecationWarning naming
the replacement and the generated reference shows the notice. The S3
store is the one sanctioned reader; it silences that warning locally
because it already logs its own operator-facing one when the fallback
is actually used.
DEFAULT_MAX_HISTORY was referenced nowhere. RETRIEVERS_ENABLED was read by
no code at all, while two docs pages described it as an enforced
allow-list; both the setting and those claims are removed.
The "AUTH_TYPE=oidc requires OIDC_ISSUER, OIDC_CLIENT_ID and
OIDC_FRONTEND_URL" check lived in app.py, so it only ran when the Flask
app was imported; a worker or script with the same misconfiguration
started fine. It is now a model validator on the auth group and runs
wherever Settings is loaded, with the same message.
DEPLOYMENT_TYPE, which app.py read straight from the environment to
decide whether a missing JWT_SECRET_KEY is fatal, is a documented
setting on the server group now, so it shows up in the reference like
every other variable the app reads.
Enum-like settings whose allowed values were only listed in a comment are
now Literal types, so a typo fails at startup with a message naming the
allowed values instead of falling through to a default with a warning
(or, for VECTOR_STORE, failing on first use):
AUTH_TYPE, VECTOR_STORE, STORAGE_TYPE, URL_STRATEGY, OCR_BACKEND,
OCR_ENGINE, SANDBOX_BACKEND, DOC_PARSER_ENGINE, TTS_PROVIDER, STT_PROVIDER
Each keeps a before-validator that strips and lower-cases the value, since
the registries that consume them already lower-cased at the use site, and
AUTH_TYPE maps the "None"/"none"/"" spellings a .env file carries to None
(it was the string "None" before, which only worked because nothing
compared against it). An empty TTS/STT provider still means "off".
LLM_PROVIDER stays a plain str because providers are plugin-extensible.
Containers are typed (dict[str, int], list[str], dict[str, Any]) instead
of bare dict/list, six fields that were Optional with a non-None default
are plain, and integer settings whose description already states a range
carry it as a constraint (ge=0 for "0 disables", ge=1 for counts that
cannot be zero, 0 < threshold <= 1).
The hand-maintained settings page documented 95 of 258 settings and
.env-template 42, and both drifted as fields were added. The field
descriptions now live on the model, so the reference is rendered from it:
python -m docsgpt.core.settings.reference --write
writes docs/content/Deploying/Settings-Reference.mdx, one section per
settings group with each field's type, default, constraints, aliases and
description. --check reports a stale page, and tests/core/test_settings.py
fails when the checked-in page no longer matches the definitions, so a
new setting cannot land undocumented.
test_settings.py also pins the composition contract: every group field is
a flat Settings attribute, no field is defined twice, every field has a
description, and the secret-normalising validator of every group is
applied (the case that a shared method name would silently drop).
The App Configuration page points at the reference instead of at
settings.py, and the reference is listed in the Deploying navigation.
arc53-machine