v10: Trading OS — 5-subsystem architecture + Signal Intelligence + Dashboard Command Center

Architecture:
- Restructure into 5 subsystems: data/, features/, market/, signals/, execution/, apps/
- Unified ts_code conversion in core/codes.py (idempotent, kills 4 duplicate copies)
- analytics_conn() + kline_glob() — zero hardcoded DB/Parquet paths
- Fixed double-suffix bug (.SZ.SZ) in backfill pipeline root cause

Signal Intelligence (the moat):
- 14 signal types: EMA52, Vegas, Chan, ORB, Gap, NR7, Inside Bar
- 640K+ historical signal instances across 8 backfilled types
- Multi-signal Expectancy Engine with breadth-similarity matching
- Signal backfill CLI: ashare-dp backfill signals

Market Intelligence:
- 8 engines: State, Leadership, Opportunity, Flow, Sentiment, Memory, Knowledge Graph, Recommendations
- Real limit-up/down sentiment via akshare (108 ZT, 19 DT, 52 broken board)
- Knowledge Graph: 8 themes × 30+ concepts with keyword matching
- Money-flow stock recommendations with entry/stop/target trade plans

Dashboard Command Center:
- Decision-first layout: COMMAND → WHERE → WHY → RISK → EXPECTANCY
- Multi-signal Expectancy comparison table (8 types ranked by WR)
- Theme Map visualization with rotation detection
- Intraday Replay infrastructure (30min state snapshots)
- RECOMMENDATIONS card with actionable trade plans

Trading Memory:
- trade_log table + POST/GET/PUT API for trade recording
- Performance stats aggregation

Code Quality:
- 0 hardcoded DB paths, 0 REPLACE hacks, 0 dead ts_code copies
- EMA52 screening deduplicated (CLI + scheduler share one function)
- read_parquet_sql() helper for 28 duplicate patterns
- 6 bugs fixed from code review (NR7 window, theme matching, column indices, etc.)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
jackyu66git
2026-07-06 12:11:42 +08:00
co-authored by Claude
parent ac5d538a04
commit cc95bbb638
83 changed files with 6328 additions and 728 deletions
+117
View File
@@ -0,0 +1,117 @@
"""Index daily K-line backfill via Sina API.
Fetches 7 major A-share indices and stores them in the same
Hive-partitioned Parquet format as stock K-lines.
Uses akshare.stock_zh_index_daily() which works via Sina
(verified accessible outside China).
"""
from __future__ import annotations
from datetime import date, datetime
import pandas as pd
from loguru import logger
from ashare_dp.core.models import INDEX_SINA_SYMBOLS, Freq
from ashare_dp.data.store.repository import KLineRepository
def backfill_indices(
repo: KLineRepository | None = None,
start_date: str = "19900101",
end_date: str | None = None,
) -> dict[str, int]:
"""Fetch and store index daily K-line history.
Args:
repo: KLineRepository instance. Created if None.
start_date: Start date in YYYYMMDD format.
end_date: End date in YYYYMMDD format (default: today).
Returns:
Dict mapping ts_code → number of bars stored.
"""
if repo is None:
repo = KLineRepository()
if end_date is None:
end_date = date.today().strftime("%Y%m%d")
import akshare as ak
results: dict[str, int] = {}
for sina_sym, ts_code in INDEX_SINA_SYMBOLS.items():
try:
logger.info(f"Fetching index {sina_sym} ({ts_code})...")
df = ak.stock_zh_index_daily(symbol=sina_sym)
if df is None or df.empty:
logger.warning(f"No data for {sina_sym}")
results[ts_code] = 0
continue
# Normalize columns to match K-line standard schema
df = _normalize_index_df(df, ts_code)
# Filter by date range
df["trade_date"] = pd.to_datetime(df["trade_date"])
df = df[(df["trade_date"] >= start_date) & (df["trade_date"] <= end_date)]
if df.empty:
logger.warning(f"No data in range for {sina_sym}")
results[ts_code] = 0
continue
# Store
repo.write_klines(df, Freq.d1)
results[ts_code] = len(df)
logger.info(f" {sina_sym}: stored {len(df)} bars, "
f"{df['trade_date'].min().date()} ~ {df['trade_date'].max().date()}")
except Exception as e:
logger.error(f"Failed to backfill index {sina_sym}: {e}")
results[ts_code] = 0
total = sum(results.values())
logger.info(f"Index backfill complete: {total} total bars across {len(results)} indices")
return results
def _normalize_index_df(df: pd.DataFrame, ts_code: str) -> pd.DataFrame:
"""Normalize index DataFrame to match stock K-line standard schema.
Sina index columns: date, open, high, low, close, volume
Missing: amount (not available for indices from Sina)
"""
df = df.copy()
# Standardize column names
col_map = {
"date": "trade_time",
"open": "open",
"high": "high",
"low": "low",
"close": "close",
"volume": "volume",
}
df = df.rename(columns=col_map)
# Add required columns
df["ts_code"] = ts_code
df["trade_date"] = pd.to_datetime(df["trade_time"])
df["freq"] = Freq.d1.value
# Index data from Sina doesn't have amount
if "amount" not in df.columns:
df["amount"] = 0.0
# Ensure volume is int
df["volume"] = df["volume"].fillna(0).astype("int64")
# Select and order standard columns
std_cols = ["ts_code", "trade_time", "trade_date", "open", "high", "low", "close", "volume", "amount", "freq"]
df = df[[c for c in std_cols if c in df.columns]]
return df