Architecture: - Restructure into 5 subsystems: data/, features/, market/, signals/, execution/, apps/ - Unified ts_code conversion in core/codes.py (idempotent, kills 4 duplicate copies) - analytics_conn() + kline_glob() — zero hardcoded DB/Parquet paths - Fixed double-suffix bug (.SZ.SZ) in backfill pipeline root cause Signal Intelligence (the moat): - 14 signal types: EMA52, Vegas, Chan, ORB, Gap, NR7, Inside Bar - 640K+ historical signal instances across 8 backfilled types - Multi-signal Expectancy Engine with breadth-similarity matching - Signal backfill CLI: ashare-dp backfill signals Market Intelligence: - 8 engines: State, Leadership, Opportunity, Flow, Sentiment, Memory, Knowledge Graph, Recommendations - Real limit-up/down sentiment via akshare (108 ZT, 19 DT, 52 broken board) - Knowledge Graph: 8 themes × 30+ concepts with keyword matching - Money-flow stock recommendations with entry/stop/target trade plans Dashboard Command Center: - Decision-first layout: COMMAND → WHERE → WHY → RISK → EXPECTANCY - Multi-signal Expectancy comparison table (8 types ranked by WR) - Theme Map visualization with rotation detection - Intraday Replay infrastructure (30min state snapshots) - RECOMMENDATIONS card with actionable trade plans Trading Memory: - trade_log table + POST/GET/PUT API for trade recording - Performance stats aggregation Code Quality: - 0 hardcoded DB paths, 0 REPLACE hacks, 0 dead ts_code copies - EMA52 screening deduplicated (CLI + scheduler share one function) - read_parquet_sql() helper for 28 duplicate patterns - 6 bugs fixed from code review (NR7 window, theme matching, column indices, etc.) Co-Authored-By: Claude <noreply@anthropic.com>
5.2 KiB
CLAUDE.md — A-Share Data Platform (Trading OS)
Project Overview
A-share trading operating system built on Parquet + DuckDB with REST/WebSocket APIs. Architecture follows a 5-subsystem design: Data Platform → Market Intelligence → Signal Intelligence → Execution Intelligence → Presentation. Dashboard at /dashboard renders a Trading Command Center.
Tech Stack
- Data:
akshare(East Money / Sina), Parquet (Zstd), DuckDB (analytics) - API: FastAPI + uvicorn, single
/api/v1/dashboard/stateendpoint + K-line/stock/calendar REST - CLI: Typer (
ashare-dp backfill daily|minute|industry|signals ...) - Scheduler: APScheduler (EOD 15:05, EMA52 15:10)
- Config: pydantic-settings (
.env)
Project Structure (v10 — 5 Subsystems)
src/ashare_dp/
├── config.py # Settings
├── core/ # Shared kernel
│ ├── models.py # Freq enum, INDEX_CODES, INDEX_SINA_SYMBOLS
│ ├── codes.py # ★ Single ts_code conversion module (IDEMPOTENT)
│ ├── calendar.py # Trading calendar, market state, Beijing TZ
│ └── exceptions.py
├── domain/ # Ontology — shared contracts
│ ├── state.py # MarketState (7-dim continuous vector)
│ ├── context.py # TradingContext, Playbook, Expectancy, Opportunity
│ ├── events.py # RiskEvent
│ ├── features.py # FeatureDefinition
│ ├── leadership.py # LeaderState enum
│ └── signal.py # SignalType, SignalInstance
├── data/ # ═══ DATA PLATFORM ═══
│ ├── sources/ # akshare_client, index, industry
│ ├── pipelines/ # backfill, eod, realtime
│ └── store/ # database (get_db, analytics_conn, kline_glob), repository, partitioning, schema
├── features/ # Feature Store (6 registered features, all use analytics_conn + kline_glob)
├── market/ # ═══ MARKET INTELLIGENCE ═══
│ ├── state.py # infer_market_state
│ ├── leadership.py # assess_leaders (lifecycle per industry)
│ ├── opportunity.py # rank_opportunities
│ ├── flow.py # compute_flow (money flow graph)
│ ├── sentiment.py # Phase 2 placeholder
│ └── memory.py # StateStore (state_snapshot table)
├── signals/ # ═══ SIGNAL INTELLIGENCE ═══ (the moat)
│ ├── detectors.py # EMA52 cross detection + shared screening logic
│ ├── store.py # signal_instance CRUD (to be extracted from detectors)
│ └── expectancy.py # get_expectancy(state) — single entry, fallback chain internal
├── execution/ # ═══ EXECUTION INTELLIGENCE ═══
│ ├── playbook.py # build_playbook (State → strategies/bias/holding)
│ ├── risk.py # RiskRule engine + evaluate_risks
│ └── brief.py # build_brief + brief_to_api_dict (single serialization point)
└── apps/ # ═══ PRESENTATION ═══
├── api/ # app.py, routers/, websocket/, dashboard/
├── cli/ # main.py + backfill/serve/query/eod/screening commands
└── scheduler/ # scheduler.py, jobs.py
Key Conventions
ts_code conversion (CRITICAL)
Always use from ashare_dp.core.codes import to_ts_code — the single idempotent implementation. Never write local _code_to_ts_code() copies. to_ts_code() is safe to call on any format: bare codes, already-formatted ts_codes, Sina symbols, even legacy corrupted .SZ.SZ values.
Database connections
- DuckDB tables (stock_info, trading_calendar, signal_instance): use
get_db()context manager fromdata.store.database - Analytics queries (features, engines): use
analytics_conn()for raw DuckDB connection — the single sanctioned way. Never hardcode"data/duckdb/ashare.db" - Parquet globs: use
kline_glob()orpartition_glob(freq)— never hardcode paths
Architecture boundaries
data/knows nothing about tradingfeatures/computes features, never classifies regimesmarket/infers state, knows nothing about signalssignals/queries historical expectancy, knows nothing about executionexecution/maps state to strategies, assembles TradingBriefapps/only renders, never reasons
MarketState is a continuous vector (not enum)
7 dimensions: trend, fear, liquidity, rotation, participation, volatility, breadth. Each 0.0–1.0. Display labels derived downstream only.
API Response
Single endpoint produces all dashboard data: GET /api/v1/dashboard/state. Response versioned ("version": "1.0"). Serialization in execution/brief.py::brief_to_api_dict() — the single serialization point. Router only orchestrates engine calls.
Running
pip install -e ".[dev]"
ashare-dp backfill init # Schema + stock list + trading calendar
ashare-dp backfill daily # Daily/weekly/monthly + indices
ashare-dp backfill industry # Industry classifications
ashare-dp backfill signals # EMA52 signal detection + store
ashare-dp serve start # API + scheduler + realtime
open http://localhost:8000/dashboard