tiennm99/miti99bot
1
0
Fork 0
mirror of https://github.com/tiennm99/miti99bot.git synced 2026-09-08 10:21:07 +00:00
Code Issues Packages Projects Releases Wiki Activity

docs(stock): record ssi provider research

Browse Source
This commit is contained in:
tiennm99
2026-06-25 15:52:43 +07:00
parent 78e51157d7
commit fca5b8961f
2 changed files with 502 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plans/reports/260625-1510-vn-stock-price-provider-research.md
+320
View File
@@ -0,0 +1,320 @@
---
type: research-report
topic: vn-stock-price-provider
created: 2026-06-25 15:10 Asia/Saigon
status: done
---
# Research Report: VN Stock Price Provider
## Executive Summary
Need better provider for `/stock_stats`. Current KBS endpoint works locally but failed in Lambda symptoms: every holding `(no price)`, total value only cash. Increasing timeout alone is not enough. Single provider + no last-good cache is the real fragility.
Best practical path: add a provider chain with **SSI iBoard chart history as primary**, existing **KBS as fallback**, and **DynamoDB last-good price cache** as final fallback. Keep Yahoo Finance chart endpoint as **opt-in emergency fallback only**, because Yahoo terms are hostile to automated collection.
No fully official, documented, free VN stock quote API found. All usable no-key endpoints are broker/app internals. Therefore design must assume endpoint breakage: provider interface, short deadlines, source labels, health logs, cache.
## Research Methodology
- Conducted: 2026-06-25.
- Repo context: Go bot on AWS Lambda 256 MB, 10s webhook handler, DynamoDB KV.
- Gemini: enabled in config, but CLI auth/trust failed. Used WebSearch + direct endpoint probes.
- Sources/probes consulted:
- SSI iBoard app: https://iboard.ssi.com.vn/
- SSI chart endpoint: `https://iboard-api.ssi.com.vn/statistics/charts/history?symbol=TCB&resolution=D&from=1781136000&to=1782345600`
- KBSV site / KB Buddy WTS: https://www.kbsec.com.vn/
- KBS current endpoint: `https://kbbuddywts.kbsec.com.vn/iis-server/investment/stocks/TCB/data_day?sdate=11-06-2026&edate=25-06-2026`
- Yahoo chart endpoint: `https://query1.finance.yahoo.com/v8/finance/chart/TCB.VN?range=5d&interval=1d`
- Yahoo Terms: https://legal.yahoo.com/us/en/yahoo/terms/otos/index.html
- VNDIRECT probe: `https://finfo-api.vndirect.com.vn/v4/stock_prices?q=code:TCB&sort=date:desc&size=1`
- FireAnt probe: `https://restv2.fireant.vn/symbols/TCB/quote`
- TCBS guessed probes under `https://apipubaws.tcbs.com.vn/stock-insight/...`
## Scope
Need daily/current-ish close price for VN listed tickers used by paper portfolio stats. Must be cheap/free, no private auth if possible, JSON, simple Go client, safe under Lambda webhook budget. Out of scope: licensed professional market-data feed, realtime trading-grade data, websocket streaming, paid vendor contract.
## Key Findings
### 1. Current KBS Provider
Current code uses KBS:
```text
https://kbbuddywts.kbsec.com.vn/iis-server/investment/stocks/{TICKER}/data_day?sdate=DD-MM-YYYY&edate=DD-MM-YYYY
```
Probe result for held tickers on 2026-06-25:
| Symbol | Status | Time ms | Price VND |
|---|---:|---:|---:|
| MWG | 200 | 245 | 77,200 |
| SSI | 200 | 201 | 26,500 |
| TCB | 200 | 220 | 33,400 |
| VND | 200 | 179 | 17,350 |
| FPT | 200 | 254 | 71,000 |
| HPG | 200 | 203 | 23,400 |
| MSN | 200 | 164 | 71,500 |
Good: returns VND unscaled, already integrated, matches expected holdings.
Bad: no formal public API contract found, current Lambda symptom suggests endpoint/domain path can be unreliable from AWS. Keep as fallback, not sole source.
### 2. SSI iBoard Chart History
Endpoint tested:
```text
https://iboard-api.ssi.com.vn/statistics/charts/history?symbol={TICKER}&resolution=D&from={unix_seconds}&to={unix_seconds}
```
Response shape:
```json
{
"code": "SUCCESS",
"message": "Get chart history data success",
"data": {
"t": [1781136000],
"c": [33.4],
"o": [],
"h": [],
"l": [],
"v": []
},
"status": "ok"
}
```
Important: close values are in thousands of VND. Multiply by `1000`.
Probe result for held tickers:
| Symbol | Status | Time ms | Rows | Price VND |
|---|---:|---:|---:|---:|
| MWG | 200 | 329 | 11 | 77,200 |
| SSI | 200 | 336 | 11 | 26,500 |
| TCB | 200 | 355 | 11 | 33,400 |
| VND | 200 | 308 | 11 | 17,350 |
| FPT | 200 | 353 | 11 | 71,000 |
| HPG | 200 | 306 | 11 | 23,400 |
| MSN | 200 | 277 | 11 | 71,500 |
Good: same values as KBS, simple JSON, no key, stable chart-history shape, official SSI app domain.
Bad: no formal API docs found. Batch query with comma-separated symbols returned empty arrays, so one request per symbol.
Verdict: **best default primary provider** for this bot.
### 3. Yahoo Finance Chart
Endpoint tested:
```text
https://query1.finance.yahoo.com/v8/finance/chart/{TICKER}.VN?range=5d&interval=1d
```
Probe result:
| Symbol | Status | Time ms | Rows | Price VND |
|---|---:|---:|---:|---:|
| MWG.VN | 200 | 145 | 5 | 77,200 |
| SSI.VN | 200 | 111 | 5 | 26,500 |
| TCB.VN | 200 | 97 | 5 | 33,400 |
| VND.VN | 200 | 120 | 5 | 17,350 |
| FPT.VN | 200 | 114 | 5 | 71,000 |
| HPG.VN | 200 | 262 | 5 | 23,400 |
| MSN.VN | 200 | 252 | 5 | 71,500 |
Good: fastest in local probe, returns direct VND, broad exchange metadata.
Bad: batch quote endpoint returned 401. Yahoo terms restrict automated collection and reuse without permission, and services are provided as-is with no support/reliability warranty. Do not make this default unless user accepts terms risk.
Verdict: **opt-in fallback only**, disabled by default.
### 4. VNDIRECT / TCBS / FireAnt
VNDIRECT `finfo-api.vndirect.com.vn` refused connection from local probe. TCBS guessed `stock-insight` endpoints returned 404. FireAnt guessed quote endpoint returned 404. Existing FireAnt usage in repo is for income events, auth-aware, not a known quote API.
Verdict: not useful until exact maintained endpoints/docs are found.
## Comparative Analysis
| Provider | Works for 7 holdings | Auth | Unit | Batch | Contract risk | Recommendation |
|---|---:|---|---|---|---|---|
| SSI iBoard history | yes | none | thousand VND | no | medium | primary |
| KBS data_day | yes | none | VND | no | medium | fallback |
| Yahoo chart | yes | none for chart | VND | no | high | opt-in fallback |
| VNDIRECT finfo | no in probe | unknown | unknown | unknown | high | skip |
| TCBS apipubaws | no in probe | unknown | unknown | unknown | high | skip |
| FireAnt quote | no in probe | maybe auth | unknown | unknown | high | skip |
## Recommended Architecture
```mermaid
flowchart LR
A[stock_stats ticker] --> B[SSI provider]
B -->|ok| Z[store last-good quote]
B -->|fail| C[KBS provider]
C -->|ok| Z
C -->|fail| D{Yahoo enabled?}
D -->|yes| E[Yahoo chart provider]
D -->|no| F[last-good quote cache]
E -->|ok| Z
E -->|fail| F
F -->|hit| G[render cached/stale label]
F -->|miss| H[render no price]
```
### Provider Order
1. `SSIProvider` primary.
2. `KBSProvider` fallback.
3. `YahooProvider` fallback only if env flag enables it, e.g. `STOCK_YAHOO_PRICE_FALLBACK=1`.
4. KV last-good quote cache.
### Why This Order
- SSI and KBS are both broker app endpoints. SSI is a separate domain/path from the current failing KBS path.
- Yahoo is technically good but policy-risky. Keep it explicit, not hidden.
- Cache is mandatory. It solves the actual user-facing disaster: all prices zero during transient provider failure.
## Implementation Recommendations
### Quick Start
1. Replace concrete `PriceClient.FetchPrice` internals with provider chain.
2. Add `StockPrice` struct:
```go
type StockPrice struct {
Symbol string
VND float64
Source string
UpdatedAt int64
Stale bool
}
```
3. Add provider interface:
```go
type PriceProvider interface {
FetchPrice(ctx context.Context, ticker string) (StockPrice, error)
}
```
4. Implement `SSIProvider`.
5. Convert current KBS logic into `KBSProvider`.
6. Add optional `YahooProvider`, gated by env.
7. Add KV cache:
```text
price:{TICKER} -> {symbol, vnd, source, updatedAt}
```
8. `/stock_buy` and `/stock_sell`: require fresh provider quote; do not use stale cache for trades.
9. `/stock_stats`: live provider first, stale cache allowed, label stale.
### SSI Parsing Rules
```go
// data.c is chronological and quoted in thousand VND.
latest := resp.Data.C[len(resp.Data.C)-1] * 1000
```
Validation:
- `code == "SUCCESS"` or `status == "ok"`.
- `len(data.c) > 0`.
- latest price finite and positive.
- ticker regex already exists; keep it.
- endpoint HTTPS only unless localhost test server.
### Timeout
Do not simply return to 10s per fetch.
Recommended:
- per provider call: `2s` to `3s`
- whole stats fetch context: current `chathelper.FetchContext`
- sequential fetch remains fine for small portfolios
- if later optimizing: bounded worker pool with provider cache, not unlimited fanout
### Display
Fresh:
```text
TCB x4200 @ 33.400 VND = 140.280.000 VND (SSI)
```
Cached:
```text
TCB x4200 @ 33.400 VND = 140.280.000 VND (cached SSI, 25/06 15:10)
```
No cache:
```text
TCB x4200 (no price)
```
## Security Considerations
- Keep ticker regex `^[A-Z0-9]{1,16}$`.
- No arbitrary provider URL from user input.
- If env overrides are added, require HTTPS except localhost tests.
- Do not log response bodies; log provider, ticker, status, duration, error class.
- Yahoo: terms restrict automated collection. Keep disabled by default.
- Cache poisoning risk low if only providers write cache, but validate positive finite prices and reasonable upper bound.
## Performance Insights
For current 7 holdings, local 2026-06-25 probes:
- SSI: 277-355ms per symbol.
- KBS: 164-254ms per symbol.
- Yahoo chart: 97-262ms per symbol.
All are acceptable for a personal bot if sequential and if reply reserve remains. Cache makes worst-case provider outage cheap.
## Common Pitfalls
- Treating SSI close as direct VND. It is thousand VND.
- Using stale cache for buy/sell. That can make trades unfair.
- Hiding cached values without label. Users will trust stale P&L as fresh.
- Adding Yahoo by default. Legal/availability risk.
- Removing KBS. Keep it; working fallback is valuable.
## Recommended Next Steps
1. Implement `SSIProvider` + `KBSProvider` chain.
2. Add KV last-good quote cache.
3. Add tests:
- SSI happy path parses `33.4` as `33400`.
- SSI failure falls through to KBS.
- all live providers fail + cache hit renders cached valuation.
- all fail + cache miss keeps `(no price)`.
- buy/sell does not use stale cache.
4. Optional: add Yahoo provider behind env flag only.
5. After deploy, inspect CloudWatch counts by provider and error class.
## Resources & References
- SSI iBoard app: https://iboard.ssi.com.vn/
- SSI chart endpoint example: https://iboard-api.ssi.com.vn/statistics/charts/history?symbol=TCB&resolution=D&from=1781136000&to=1782345600
- KBSV site, links to KB Buddy WTS smart price board: https://www.kbsec.com.vn/
- KBS endpoint example: https://kbbuddywts.kbsec.com.vn/iis-server/investment/stocks/TCB/data_day?sdate=11-06-2026&edate=25-06-2026
- Yahoo chart endpoint example: https://query1.finance.yahoo.com/v8/finance/chart/TCB.VN?range=5d&interval=1d
- Yahoo Terms: https://legal.yahoo.com/us/en/yahoo/terms/otos/index.html
## Unresolved Questions
- Enable Yahoo fallback at all, or keep only SSI + KBS + cache?
- Need HNX/UPCOM coverage beyond current holdings before implementation?
- Accept 7-day stale quote max age for stats display?
plans/reports/260625-1523-ssi-direct-current-price-research.md
+182
View File
@@ -0,0 +1,182 @@
---
type: research-report
topic: ssi-direct-current-price-api
created: 2026-06-25 15:23 Asia/Saigon
status: done
---
# Research Report: SSI Direct Current Price API
## Executive Summary
SSI has a direct current quote API. Do not use the chart-history endpoint for current portfolio valuation if this endpoint stays available.
Use:
- Single symbol: `GET https://iboard-query.ssi.com.vn/stock/{TICKER}`
- Batch: `POST https://iboard-query.ssi.com.vn/stock/multiple`
For `/stock_stats`, batch endpoint is the better fit: one HTTP call for all holdings, direct VND prices, no date math, no "close array last element" parsing.
## Research Methodology
- Conducted: 2026-06-25.
- Gemini: enabled in local config but auth/trust failed; used WebSearch + direct endpoint inspection.
- Inspected SSI app shell: https://iboard.ssi.com.vn/
- Inspected main app bundle: `/build/static/js/h7pe3rzS.js`
- Found `API_SERVICES_NAME.STOCK_QUERY`, base `https://iboard-query.ssi.com.vn`, and stock query calls:
- `/stock/{symbol}`
- `/stock/multiple`
- `/stock/stock-info`
- `/system/time`
- Live-probed single and batch endpoints.
## Key Findings
### 1. Single-Symbol Current Quote
Example:
```sh
curl -sS 'https://iboard-query.ssi.com.vn/stock/TCB' \
-H 'Accept: application/json' \
-H 'Origin: https://iboard.ssi.com.vn' \
-H 'Referer: https://iboard.ssi.com.vn/' \
-H 'User-Agent: Mozilla/5.0 (miti99bot)'
```
Observed result:
- Status: `200`
- Content-Type: `application/json; charset=utf-8`
- Time: ~309ms local
- `data.matchedPrice`: `33400`
- `data.tradingDate`: `20260625`
- `data.tradingCurrencyISOCode`: `VND`
Important fields:
| Field | Meaning | Use |
|---|---|---|
| `matchedPrice` | current/last matched price, VND | primary current price |
| `expectedMatchedPrice` | expected/derived matched price, VND | fallback only if matched empty |
| `refPrice` | reference price | do not use for valuation |
| `best1Bid` / `best1Offer` | order book top | do not use for valuation |
| `tradingDate` | YYYYMMDD market date | display/cache freshness |
| `tradingStatus` | market/security status | log/display optional |
### 2. Batch Current Quote
Example:
```sh
curl -sS 'https://iboard-query.ssi.com.vn/stock/multiple' \
-X POST \
-H 'Accept: application/json' \
-H 'Content-Type: application/x-www-form-urlencoded' \
-H 'Origin: https://iboard.ssi.com.vn' \
-H 'Referer: https://iboard.ssi.com.vn/' \
-H 'User-Agent: Mozilla/5.0 (miti99bot)' \
--data 'stocks=MWG&stocks=SSI&stocks=TCB&stocks=VND&stocks=FPT&stocks=HPG&stocks=MSN'
```
Observed result:
- Status: `200`
- Time: ~312ms local
- `data` count: `7`
- Prices direct VND, no scaling.
Observed prices:
| Symbol | matchedPrice | expectedMatchedPrice | refPrice | tradingDate | exchange |
|---|---:|---:|---:|---|---|
| FPT | 71000 | 71000 | 70800 | 20260625 | hose |
| HPG | 23400 | 23400 | 23500 | 20260625 | hose |
| MSN | 71500 | 71500 | 71500 | 20260625 | hose |
| MWG | 77200 | 77200 | 77800 | 20260625 | hose |
| SSI | 26500 | 26500 | 26700 | 20260625 | hose |
| TCB | 33400 | 33400 | 32500 | 20260625 | hose |
| VND | 17350 | 17350 | 17600 | 20260625 | hose |
### 3. Chart History Still Useful, But Not Best For Current Price
Previous chart endpoint:
```text
https://iboard-api.ssi.com.vn/statistics/charts/history?symbol=TCB&resolution=D&from=1781136000&to=1782345600
```
It works, but:
- requires `from` / `to` Unix seconds
- returns arrays
- close values are thousand VND, so multiply by `1000`
- one request per symbol
Direct batch quote avoids all of that.
## Recommendation
Use SSI direct batch as primary for `/stock_stats`:
```text
POST https://iboard-query.ssi.com.vn/stock/multiple
Content-Type: application/x-www-form-urlencoded
body: stocks=MWG&stocks=SSI&...
parse: data[].matchedPrice
```
Provider order:
1. SSI direct batch for stats.
2. SSI single direct for buy/sell or one-off fetches.
3. KBS as fallback.
4. Last-good DynamoDB cache as final fallback for stats only.
## Implementation Notes
- `matchedPrice > 0`: use it.
- If `matchedPrice <= 0` and `expectedMatchedPrice > 0`: optional fallback, label source maybe `SSI expected`.
- Never use `refPrice` as current valuation.
- Keep headers: `Origin`, `Referer`, `User-Agent`, `Accept`.
- Batch limit in SSI bundle appears to be `CONFIG_LIMIT_STOCK_MULTIPLE_QUERY=400`, far above this bot's typical portfolio size.
- Still treat this as unofficial app-internal API. Add KBS/cache fallback.
## Suggested Go Shape
```go
type SSIStockQuote struct {
StockSymbol string `json:"stockSymbol"`
MatchedPrice float64 `json:"matchedPrice"`
ExpectedMatchedPrice float64 `json:"expectedMatchedPrice"`
RefPrice float64 `json:"refPrice"`
TradingDate string `json:"tradingDate"`
TradingStatus string `json:"tradingStatus"`
Exchange string `json:"exchange"`
}
```
## Next Steps
1. Implement `SSIDirectProvider`.
2. Add `FetchPrices(ctx, []string)` for batch stats path.
3. Keep single `FetchPrice(ctx, ticker)` wrapper for buy/sell.
4. Add tests for:
- batch parses `matchedPrice` directly as VND
- `refPrice` is ignored
- batch partial result handles missing ticker
- fallback to KBS/cache on SSI failure
## References
- SSI iBoard app: https://iboard.ssi.com.vn/
- Single quote endpoint example: https://iboard-query.ssi.com.vn/stock/TCB
- Batch quote endpoint: https://iboard-query.ssi.com.vn/stock/multiple
- Chart history endpoint example: https://iboard-api.ssi.com.vn/statistics/charts/history?symbol=TCB&resolution=D&from=1781136000&to=1782345600
## Unresolved Questions
- Should buy/sell use SSI direct quote immediately, or keep KBS until stats is stabilized?
- Should `expectedMatchedPrice` be allowed when `matchedPrice` is zero during pre-open/ATO/ATC?