mirror of
https://github.com/tiennm99/miti99bot.git
synced 2026-08-06 08:23:17 +00:00
docs(portfolio): document cost basis and P&L
This commit is contained in:
@@ -51,6 +51,28 @@ The bot validates syntax, tickers, and arithmetic safety, but does not look up
|
||||
notices or prevent duplicate calls. The caller is responsible for verifying the
|
||||
notice and avoiding accidental repeated adjustments.
|
||||
|
||||
### Stock and coin P&L accounting
|
||||
|
||||
Stock and coin portfolios persist the total remaining cost basis for each open
|
||||
position. Buys add their actual spend. Partial sells remove basis using the
|
||||
weighted-average method and report realized P&L; full sells remove the position
|
||||
and its basis. Stock share dividends add shares without adding cost, which
|
||||
lowers the derived average price, while cash dividends do not change position
|
||||
basis.
|
||||
|
||||
`/stock_portfolio` and `/coin_portfolio` show average entry price and
|
||||
unrealized P&L for each priced position. `Account P&L` remains the broader
|
||||
account value minus all top-ups, so it also reflects realized proceeds,
|
||||
dividend cash, and idle cash. If any current quote is unavailable, totals are
|
||||
marked partial and numeric Account P&L is withheld.
|
||||
|
||||
On startup, enabled stock and coin modules scan every stored portfolio. Legacy
|
||||
holdings without basis are initialized at that startup's current market quote,
|
||||
giving them zero initial unrealized P&L. The migration uses optimistic writes,
|
||||
records completion in the shared `system` collection, and still verifies the
|
||||
invariant on every boot. Startup fails rather than accepting trades when a
|
||||
required legacy quote or migration write is unavailable.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
@@ -60,7 +82,7 @@ internal/telegram/ Telegram long-polling bot wrapper
|
||||
internal/cron/ in-process cron scheduler
|
||||
internal/modules/ Module framework, registry, dispatchers, modules
|
||||
internal/storage/ typed DocStore[T] (Provider + Typed); mongodb runtime + memory (tests). Values persist as flattened native BSON root documents
|
||||
internal/systemstate/ shared `system` collection helper for future startup migrations
|
||||
internal/systemstate/ shared `system` collection helper for startup migration records
|
||||
compose.yml Coolify self-host stack (single bot service)
|
||||
docs/deploy-coolify-selfhosted.md Self-host deploy and operations guide
|
||||
```
|
||||
|
||||
@@ -101,8 +101,12 @@ Successful GIF replies include the result behind Telegram spoiler formatting.
|
||||
> counts and creates indexes on startup. Deleted legacy command rows are
|
||||
> retained with `deleted: true`; `/stats` queries filter those rows from visible
|
||||
> results. A historical `system` collection may remain in MongoDB with completed
|
||||
> migration records and can be reused if a future one-time startup migration is
|
||||
> needed.
|
||||
> migration records. Stock and coin documents store `costBasis` by symbol. On
|
||||
> every startup, enabled paper-trading modules verify that each positive holding
|
||||
> has a valid basis; legacy holdings are initialized from current quotes before
|
||||
> Telegram handlers are installed. The bot intentionally fails startup if a
|
||||
> required quote or migration write fails. Completed records remain in `system`
|
||||
> as audit history, but do not suppress later invariant scans.
|
||||
|
||||
## 2. Coolify
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
# Stock and Coin Cost Basis P&L Journal
|
||||
|
||||
## Context
|
||||
|
||||
Stock and coin portfolios tracked account funding and holdings but not the
|
||||
remaining acquisition cost of each position, so they could not distinguish
|
||||
realized sale results from unrealized open-position performance.
|
||||
|
||||
## What Changed
|
||||
|
||||
- Added persisted per-symbol total remaining `costBasis` to stock and coin
|
||||
portfolios, with invariant checks on load, update, and save paths.
|
||||
- Buys add actual spend; partial sells remove proportional weighted-average
|
||||
basis and report realized P&L; full sells remove the position and its basis.
|
||||
- Portfolio views now show average entry price and per-position unrealized P&L.
|
||||
Account P&L remains total account value minus top-ups and is withheld when
|
||||
missing quotes make valuation partial.
|
||||
- Stock share dividends preserve total basis while increasing quantity; cash
|
||||
dividends and top-ups remain outside position basis.
|
||||
- Sorted, bounded portfolio replies retain summaries while omitting excess
|
||||
position lines before Telegram's message limit.
|
||||
|
||||
## Migration Safety
|
||||
|
||||
- Enabled modules scan every portfolio before handlers are installed on every
|
||||
boot, even when a completion marker already exists.
|
||||
- Missing legacy basis is initialized from a complete current quote set, giving
|
||||
each migrated position zero initial unrealized P&L without repricing rows that
|
||||
already have basis.
|
||||
- Versioned writes retry conflicts; the shared `system` marker is written only
|
||||
after all rows succeed and remains an audit record rather than a scan bypass.
|
||||
- A two-minute overall deadline bounds startup. Invalid data, noncanonical
|
||||
symbols, missing quotes, exhausted conflicts, or storage failures abort
|
||||
startup instead of allowing trades with unknown basis.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Persist total remaining cost, deriving average entry price as basis divided by
|
||||
held quantity; do not persist trade lots or cumulative realized P&L.
|
||||
- Migrate only loaded modules, so disabled-module data waits until re-enabled.
|
||||
- Preserve all command names, parameter contracts, and existing portfolio data.
|
||||
|
||||
## Verification
|
||||
|
||||
- Passed: focused stock and coin accounting, migration, output, and reply-budget
|
||||
tests.
|
||||
- Passed: MongoDB 8 Testcontainers migration and idempotency coverage.
|
||||
- Passed: `go test ./...`
|
||||
- Passed: `go vet ./...`
|
||||
- Passed: `go build ./...`
|
||||
- Passed: `golangci-lint run`
|
||||
|
||||
## Operational Impact
|
||||
|
||||
- Deployments with legacy holdings may perform market-price lookups and writes
|
||||
during startup; a required provider or database failure intentionally keeps
|
||||
the bot offline until initialization can complete safely.
|
||||
- Subsequent healthy boots still verify invariants but do not reprice completed
|
||||
positions.
|
||||
+84
@@ -0,0 +1,84 @@
|
||||
---
|
||||
phase: 1
|
||||
title: Design persistence and migration
|
||||
status: completed
|
||||
priority: P1
|
||||
effort: medium
|
||||
dependencies: []
|
||||
---
|
||||
|
||||
# Phase 1: Design persistence and migration
|
||||
|
||||
## Overview
|
||||
|
||||
Extend both portfolio documents and add retry-safe startup migrations that
|
||||
initialize legacy holdings from current quotes before command handlers run.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Functional: persist `costBasis` as ticker/coin → total remaining cost.
|
||||
- Functional: initialize missing legacy basis as `quantity × current price`.
|
||||
- Functional: preserve any positive basis already written by a previous retry.
|
||||
- Functional: scan every boot even after completion; the marker is an audit
|
||||
record and never suppresses invariant checks.
|
||||
- Non-functional: startup fails on a required quote/storage error; migration is
|
||||
idempotent and bounded by a migration-wide deadline.
|
||||
|
||||
## Architecture
|
||||
|
||||
`cmd/server` checks which modules are loaded, then invokes each module's
|
||||
`InitStore` before `modules.Install`. Every boot lists `user:` documents and
|
||||
checks `positive holding => finite positive basis`. Migration fetches each
|
||||
required symbol once, verifies the complete requested set, and writes only
|
||||
missing basis using bounded CAS reload/retry. The shared `system` marker records
|
||||
completion but does not skip future scans. Existing populated basis acts as
|
||||
per-position retry progress.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: `internal/modules/stock/portfolio.go`
|
||||
- Modify: `internal/modules/coin/portfolio.go`
|
||||
- Create: `internal/modules/stock/startup.go`
|
||||
- Create: `internal/modules/coin/startup.go`
|
||||
- Modify: `cmd/server/main.go`
|
||||
- Create: stock/coin startup memory and MongoDB tests
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Add and defensively initialize `CostBasis map[string]float64` with
|
||||
`json:"costBasis" bson:"costBasis"` in both portfolio types.
|
||||
2. Validate before normalization. Reject corrupt basis, noncanonical symbols,
|
||||
canonical-key collisions, and non-finite holdings; do not delete or merge
|
||||
ambiguous legacy assets.
|
||||
3. Implement injectable stock and coin migrations with stable `system` keys,
|
||||
`user:` listing, unique-symbol quote caching, complete quote-set validation,
|
||||
bounded CAS reload/retry, progress logs, and a completion marker written last.
|
||||
4. Wire migration only for loaded modules after registry construction and
|
||||
before handler installation. Return startup-fatal errors with module/symbol
|
||||
context but no credentials.
|
||||
5. Apply an overall startup-migration timeout; timeout is startup-fatal and
|
||||
leaves the marker incomplete.
|
||||
6. Test empty, legacy, mixed, partial-retry, post-marker missing rows, partial
|
||||
quote maps, noncanonical/corrupt rows, CAS conflicts, timeout, storage
|
||||
failure, and idempotent second-boot cases in memory and MongoDB 8.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] New portfolios always contain initialized basis maps.
|
||||
- [ ] Existing documents without `costBasis` still decode.
|
||||
- [ ] Each legacy symbol receives migration-time current-price basis.
|
||||
- [ ] Retry never overwrites basis initialized by an earlier attempt.
|
||||
- [ ] Any required failure prevents the completion marker and aborts startup.
|
||||
- [ ] Disabled modules do not call external quote providers.
|
||||
- [ ] A completion marker never hides a later holding with missing basis.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- External quote outages can block startup by explicit owner choice; provider
|
||||
timeouts bound the delay and logs identify the module/symbol.
|
||||
- Partial writes are recoverable because populated per-symbol basis is never
|
||||
repriced and the global marker is written only after all rows succeed.
|
||||
- Runtime validation fails closed if an old writer or restored row reintroduces
|
||||
a positive holding without valid basis after startup.
|
||||
- Storage listing remains unpaginated; the migration-wide deadline bounds the
|
||||
current small, one-replica deployment without broad storage-API scope.
|
||||
+81
@@ -0,0 +1,81 @@
|
||||
---
|
||||
phase: 2
|
||||
title: Implement weighted-average accounting
|
||||
status: completed
|
||||
priority: P1
|
||||
effort: medium
|
||||
dependencies:
|
||||
- 1
|
||||
---
|
||||
|
||||
# Phase 2: Implement weighted-average accounting
|
||||
|
||||
## Overview
|
||||
|
||||
Update stock and coin trade mutations to maintain remaining cost basis and show
|
||||
realized P&L on every successful sale.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Functional: buys add actual transaction spend to the symbol's total basis.
|
||||
- Functional: sells remove proportional basis and report realized P&L.
|
||||
- Functional: full exits remove both holding and basis keys.
|
||||
- Functional: stock share dividends change quantity but not total basis.
|
||||
- Non-functional: rejected trades and failed saves leave portfolio state intact.
|
||||
- Non-functional: positive holdings with absent, nonpositive, or non-finite
|
||||
basis fail closed instead of producing fabricated accounting.
|
||||
|
||||
## Architecture
|
||||
|
||||
For a sale, capture pre-sale holding and basis, then calculate:
|
||||
|
||||
```text
|
||||
sold_basis = total_basis × sold_quantity / held_quantity
|
||||
realized_pnl = proceeds - sold_basis
|
||||
realized_pct = realized_pnl / sold_basis × 100
|
||||
```
|
||||
|
||||
The remaining average price is unchanged after a partial sale. A share dividend
|
||||
adds shares without cost, so `basis / new_quantity` automatically lowers the
|
||||
average price. No lot ledger or cumulative realized-P&L field is introduced.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: stock/coin `portfolio.go`, trade handlers, and format helpers
|
||||
- Modify: stock dividend handler tests to assert unchanged total basis
|
||||
- Modify: stock/coin portfolio and handler tests
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Add small portfolio methods to add purchase basis and remove proportional
|
||||
basis with finite/overflow guards appropriate to each module's precision.
|
||||
2. Update buys so quantity, cash deduction, and basis addition persist in the
|
||||
same mutation.
|
||||
3. Update sells so quantity, cash credit, and basis reduction persist together;
|
||||
append `Realized P&L` amount and percentage to the reply.
|
||||
4. Determine coin full exit from normalized post-sale holdings. If quantity
|
||||
disappears at the dust threshold, assign all remaining basis to that sale
|
||||
and delete the basis key. Keep monetary validation separate from quantity
|
||||
dust normalization.
|
||||
5. Assert cash/share/combined dividend behavior: cash never changes basis and
|
||||
share additions preserve total basis.
|
||||
6. Cover weighted repeated buys, gain/loss/breakeven partial sells, full sells,
|
||||
insufficient holdings, overflow/invalid data, conflicts, and save failure.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] `average price == remaining basis / remaining quantity` after every buy,
|
||||
sell, and share dividend.
|
||||
- [ ] Stock and coin sell replies report correct realized P&L.
|
||||
- [ ] Full exits leave no holding or basis entry.
|
||||
- [ ] Failed/rejected operations do not mutate persisted state.
|
||||
- [ ] Missing or corrupt runtime basis blocks affected accounting operations
|
||||
with a clear retry/operator message.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Stock uses float64 currency already; calculations must reject non-finite or
|
||||
unsafe results. Coin retains its existing dust normalization.
|
||||
- Mutation order must not delete quantity before sold basis is calculated.
|
||||
- Existing account `Meta.Invested` continues to mean deposits and must not be
|
||||
repurposed as position basis.
|
||||
+86
@@ -0,0 +1,86 @@
|
||||
---
|
||||
phase: 3
|
||||
title: Verify P&L behavior and compatibility
|
||||
status: completed
|
||||
priority: P1
|
||||
effort: medium
|
||||
dependencies:
|
||||
- 1
|
||||
- 2
|
||||
---
|
||||
|
||||
# Phase 3: Verify P&L behavior and compatibility
|
||||
|
||||
## Overview
|
||||
|
||||
Expose average entry price and unrealized position P&L in both portfolio views,
|
||||
document the accounting model, and run full compatibility verification.
|
||||
|
||||
## Requirements
|
||||
|
||||
- Functional: each priced holding shows average price and unrealized P&L.
|
||||
- Functional: retain cash, total value, invested deposits, and account P&L.
|
||||
- Functional: unavailable quotes degrade gracefully without inventing P&L;
|
||||
aggregate account P&L is suppressed when any holding is unpriced.
|
||||
- Non-functional: output remains deterministic and within Telegram limits.
|
||||
|
||||
## Architecture
|
||||
|
||||
Position metrics derive from persisted remaining basis:
|
||||
|
||||
```text
|
||||
average_price = basis / quantity
|
||||
unrealized_pnl = current_value - basis
|
||||
unrealized_pct = unrealized_pnl / basis × 100
|
||||
```
|
||||
|
||||
Label position performance `Unrealized P&L` and the existing net-worth minus
|
||||
top-ups metric `Account P&L` so dividends, cash, and realized proceeds are not
|
||||
confused with open-position performance. If any holding lacks a quote, label
|
||||
priced values as partial and do not render aggregate account P&L.
|
||||
|
||||
## Related Code Files
|
||||
|
||||
- Modify: `internal/modules/stock/handlers.go`, `stats_test.go`
|
||||
- Modify: `internal/modules/coin/views.go` and view/handler tests
|
||||
- Modify: `README.md`, `docs/deploy-coolify-selfhosted.md`
|
||||
- Create: completion journal during finalization
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. Render average basis and signed unrealized P&L for each priced stock/coin
|
||||
position. Add explicit stock sorting; retain coin sorting.
|
||||
2. Keep existing total-value and invested calculations; relabel the final line
|
||||
as `Account P&L` and add aggregate `Unrealized P&L` for priced positions.
|
||||
3. For unavailable current quotes, show stored average price when valid, mark
|
||||
current value/P&L unavailable, label priced totals partial, and suppress
|
||||
aggregate account P&L.
|
||||
4. Add a conservative Telegram reply budget with deterministic truncation and
|
||||
test worst-case multi-position output.
|
||||
5. Update exact output, missing-price, reply-budget, migration compatibility,
|
||||
and command behavior tests.
|
||||
6. Document startup migration, weighted-average basis, realized vs unrealized
|
||||
P&L, and dividend effects.
|
||||
7. Run `gofmt`, focused stock/coin/server tests, MongoDB Testcontainers tests,
|
||||
`go test -count=1 ./...`, `go vet ./...`, `go build ./...`, and
|
||||
`golangci-lint run`; complete independent test/debug/review gates.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Each open position shows average price and unrealized amount/percentage.
|
||||
- [ ] Portfolio retains account-level P&L with a clearer label.
|
||||
- [ ] Missing prices do not fabricate P&L or break the reply.
|
||||
- [ ] Stock output is sorted and both portfolio replies remain within Telegram
|
||||
limits under worst-case supported holdings.
|
||||
- [ ] Legacy memory and MongoDB documents remain readable and migrate once.
|
||||
- [ ] All focused and repository-wide verification passes.
|
||||
- [ ] No command name, parameter, stats-history, or price-provider contract
|
||||
changes.
|
||||
|
||||
## Risk Assessment
|
||||
|
||||
- Users may confuse account and position metrics; explicit labels and README
|
||||
examples define both.
|
||||
- Missing quotes make totals partial today; the new output must label them
|
||||
partial and suppress account P&L rather than report a false loss.
|
||||
- Longer replies require budget tests for multi-asset coin portfolios.
|
||||
@@ -0,0 +1,108 @@
|
||||
---
|
||||
title: Stock and Coin Cost Basis P&L
|
||||
description: >-
|
||||
Persist weighted-average cost basis, migrate legacy holdings at startup
|
||||
prices, and expose realized/unrealized P&L.
|
||||
status: completed
|
||||
priority: P1
|
||||
branch: main
|
||||
tags:
|
||||
- stock
|
||||
- coin
|
||||
- mongodb
|
||||
- migration
|
||||
- pnl
|
||||
blockedBy: []
|
||||
blocks: []
|
||||
created: '2026-07-21T08:08:21.155Z'
|
||||
createdBy: 'ck:plan'
|
||||
source: skill
|
||||
---
|
||||
|
||||
# Stock and Coin Cost Basis P&L
|
||||
|
||||
## Overview
|
||||
|
||||
Add a per-symbol remaining cost basis to stock and coin portfolios. New buys
|
||||
increase basis, partial sells remove proportional weighted-average basis, and
|
||||
full sells remove it. Portfolio views show average entry price and unrealized
|
||||
P&L per position; sell replies show realized P&L for that sale while preserving
|
||||
the existing account-level P&L.
|
||||
|
||||
Legacy holdings migrate before handlers are installed. Each missing basis is
|
||||
initialized from the current quote, so its unrealized P&L starts at zero.
|
||||
Migration is idempotent, marker-backed, and fail-fast: the bot does not accept
|
||||
trades if required quotes or storage writes fail.
|
||||
|
||||
## Phases
|
||||
|
||||
| Phase | Name | Status |
|
||||
|-------|------|--------|
|
||||
| 1 | [Design persistence and migration](./phase-01-design-persistence-and-migration.md) | Completed |
|
||||
| 2 | [Implement weighted-average accounting](./phase-02-implement-weighted-average-accounting.md) | Completed |
|
||||
| 3 | [Verify P&L behavior and compatibility](./phase-03-verify-p-l-behavior-and-compatibility.md) | Completed |
|
||||
|
||||
## Dependencies
|
||||
|
||||
- No active plan dependencies. The completed stock-dividend implementation
|
||||
defines the share-dividend behavior this plan must preserve.
|
||||
- MongoDB 8 Testcontainers are required for persisted migration verification;
|
||||
existing Docker-unavailable warning/skip behavior remains unchanged.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Persist total remaining cost per symbol, not a mutable average-price field.
|
||||
- Derive average price as `basis / held quantity`.
|
||||
- Use proportional weighted-average basis for partial sells.
|
||||
- Keep total stock basis unchanged when share dividends add quantity.
|
||||
- Keep cash dividends and top-ups outside position basis.
|
||||
- Do not persist cumulative realized P&L or trade lots.
|
||||
- Run a module migration only when that module is loaded; disabled-module data
|
||||
migrates the next time the module is enabled.
|
||||
- Abort startup on migration failure to avoid operating with unknown basis.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Stock and coin portfolios persist remaining cost basis without breaking
|
||||
legacy BSON/JSON decoding.
|
||||
- Repeated buys produce the correct weighted average; partial/full sells update
|
||||
basis exactly and report realized gain or loss.
|
||||
- Startup maintenance scans every boot, initializes only missing legacy basis
|
||||
from one complete current quote per symbol, never reprices completed work,
|
||||
and writes its system marker only after success.
|
||||
- Share dividends add quantity without adding cost; cash dividends do not
|
||||
change position basis.
|
||||
- Portfolio output shows average price and per-position unrealized P&L plus the
|
||||
existing account-level P&L.
|
||||
- No command names or parameter contracts change.
|
||||
- Focused tests, MongoDB migration tests, full tests, vet, build, and lint pass.
|
||||
|
||||
## Red Team Review
|
||||
|
||||
### Session — 2026-07-21
|
||||
|
||||
**Findings:** 12 deduplicated (9 accepted, 3 rejected)
|
||||
|
||||
| Finding | Severity | Disposition | Applied To |
|
||||
|---|---|---|---|
|
||||
| Completion marker cannot enforce future row invariants | Critical | Accept | Phases 1–2 |
|
||||
| Missing quotes fabricate aggregate account P&L | Critical | Accept | Phase 3 |
|
||||
| Migration has no overall deadline | High | Accept | Phase 1 |
|
||||
| Partial stock quote maps can look successful | High | Accept | Phase 1 |
|
||||
| Corrupt/noncanonical legacy symbols are ambiguous | High | Accept | Phase 1 |
|
||||
| Migration conflicts lack retry semantics | Medium | Accept | Phase 1 |
|
||||
| Coin quantity dust can orphan monetary basis | Medium | Accept | Phase 2 |
|
||||
| Invalid basis can be silently normalized | Medium | Accept | Phases 1–2 |
|
||||
| Stock output order and reply size are unsafe | Medium | Accept | Phase 3 |
|
||||
| Storage-wide pagination and migration lease | High | Reject | Deadline and fail-closed runtime validation fit the one-replica deployment |
|
||||
| Persist quote manifests and correction tooling | High | Reject | Outside paper-trading scope; validate complete finite quotes and log context |
|
||||
| Restrict portfolio commands to private chats | Medium | Reject | Pre-existing visibility contract outside this accounting change |
|
||||
|
||||
### Whole-Plan Consistency Sweep
|
||||
|
||||
- Files reread: `plan.md` and all three phase files.
|
||||
- Decision deltas checked: every-boot invariant scan, fail-closed runtime checks,
|
||||
bounded startup, quote completeness, corrupt-symbol rejection, CAS retry,
|
||||
dust cleanup, missing-price suppression, sorting, and reply bounds.
|
||||
- Reconciled stale references: marker short-circuiting and partial aggregate P&L.
|
||||
- Unresolved contradictions: 0.
|
||||
Reference in New Issue
Block a user