v10: Trading OS — 5-subsystem architecture + Signal Intelligence + Dashboard Command Center
Architecture: - Restructure into 5 subsystems: data/, features/, market/, signals/, execution/, apps/ - Unified ts_code conversion in core/codes.py (idempotent, kills 4 duplicate copies) - analytics_conn() + kline_glob() — zero hardcoded DB/Parquet paths - Fixed double-suffix bug (.SZ.SZ) in backfill pipeline root cause Signal Intelligence (the moat): - 14 signal types: EMA52, Vegas, Chan, ORB, Gap, NR7, Inside Bar - 640K+ historical signal instances across 8 backfilled types - Multi-signal Expectancy Engine with breadth-similarity matching - Signal backfill CLI: ashare-dp backfill signals Market Intelligence: - 8 engines: State, Leadership, Opportunity, Flow, Sentiment, Memory, Knowledge Graph, Recommendations - Real limit-up/down sentiment via akshare (108 ZT, 19 DT, 52 broken board) - Knowledge Graph: 8 themes × 30+ concepts with keyword matching - Money-flow stock recommendations with entry/stop/target trade plans Dashboard Command Center: - Decision-first layout: COMMAND → WHERE → WHY → RISK → EXPECTANCY - Multi-signal Expectancy comparison table (8 types ranked by WR) - Theme Map visualization with rotation detection - Intraday Replay infrastructure (30min state snapshots) - RECOMMENDATIONS card with actionable trade plans Trading Memory: - trade_log table + POST/GET/PUT API for trade recording - Performance stats aggregation Code Quality: - 0 hardcoded DB paths, 0 REPLACE hacks, 0 dead ts_code copies - EMA52 screening deduplicated (CLI + scheduler share one function) - read_parquet_sql() helper for 28 duplicate patterns - 6 bugs fixed from code review (NR7 window, theme matching, column indices, etc.) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -33,3 +33,4 @@ logs/
|
|||||||
# Virtual env
|
# Virtual env
|
||||||
.venv/
|
.venv/
|
||||||
venv/
|
venv/
|
||||||
|
.gstack/
|
||||||
|
|||||||
@@ -1,94 +1,92 @@
|
|||||||
# CLAUDE.md - A-Share Data Platform
|
# CLAUDE.md — A-Share Data Platform (Trading OS)
|
||||||
|
|
||||||
## Project Overview
|
## Project Overview
|
||||||
|
|
||||||
A-share (Chinese stock market) data platform providing 9 K-line frequencies via Parquet + DuckDB storage with REST and WebSocket APIs. Greenfield project, currently at v0.1.0.
|
A-share trading operating system built on Parquet + DuckDB with REST/WebSocket APIs. Architecture follows a 5-subsystem design: Data Platform → Market Intelligence → Signal Intelligence → Execution Intelligence → Presentation. Dashboard at `/dashboard` renders a Trading Command Center.
|
||||||
|
|
||||||
## Tech Stack
|
## Tech Stack
|
||||||
|
|
||||||
- **Data source**: `akshare` (wraps East Money and Sina APIs)
|
- **Data**: `akshare` (East Money / Sina), Parquet (Zstd), DuckDB (analytics)
|
||||||
- **Storage**: Parquet (Zstd compression, Hive-partitioned `year=YYYY/month=MM/day=DD/`)
|
- **API**: FastAPI + uvicorn, single `/api/v1/dashboard/state` endpoint + K-line/stock/calendar REST
|
||||||
- **Query engine**: DuckDB (embedded OLAP, `read_parquet` with `hive_partitioning=true, union_by_name=true`)
|
- **CLI**: Typer (`ashare-dp backfill daily|minute|industry|signals ...`)
|
||||||
- **Web**: FastAPI + uvicorn
|
- **Scheduler**: APScheduler (EOD 15:05, EMA52 15:10)
|
||||||
- **CLI**: Typer
|
- **Config**: pydantic-settings (`.env`)
|
||||||
- **Scheduler**: APScheduler 3.x (AsyncIOScheduler)
|
|
||||||
- **Config**: pydantic-settings (reads `.env`)
|
|
||||||
- **Logging**: loguru
|
|
||||||
|
|
||||||
## Project Structure
|
## Project Structure (v10 — 5 Subsystems)
|
||||||
|
|
||||||
```
|
```
|
||||||
src/ashare_dp/
|
src/ashare_dp/
|
||||||
├── config.py # Pydantic Settings (all config via env vars)
|
├── config.py # Settings
|
||||||
├── core/
|
├── core/ # Shared kernel
|
||||||
│ ├── models.py # Freq enum with storage_dir property, freq groupings
|
│ ├── models.py # Freq enum, INDEX_CODES, INDEX_SINA_SYMBOLS
|
||||||
│ ├── calendar.py # Trading calendar, market state, Beijing TZ
|
│ ├── codes.py # ★ Single ts_code conversion module (IDEMPOTENT)
|
||||||
│ └── exceptions.py # AShareDPError hierarchy
|
│ ├── calendar.py # Trading calendar, market state, Beijing TZ
|
||||||
├── data/
|
│ └── exceptions.py
|
||||||
│ ├── akshare_client.py # AKShare wrapper: dual backend, retry, rate limit
|
├── domain/ # Ontology — shared contracts
|
||||||
│ ├── backfill.py # Historical backfill (daily/weekly/monthly + minute)
|
│ ├── state.py # MarketState (7-dim continuous vector)
|
||||||
│ ├── eod.py # End-of-day batch pull (all stocks, all freqs)
|
│ ├── context.py # TradingContext, Playbook, Expectancy, Opportunity
|
||||||
│ └── realtime.py # Background async spot poller → WebSocket broadcast
|
│ ├── events.py # RiskEvent
|
||||||
├── storage/
|
│ ├── features.py # FeatureDefinition
|
||||||
│ ├── database.py # DuckDB singleton (get_db)
|
│ ├── leadership.py # LeaderState enum
|
||||||
│ ├── schema.py # DDL: stock_info, trading_calendar
|
│ └── signal.py # SignalType, SignalInstance
|
||||||
│ ├── repository.py # KLineRepository: write_klines, read_klines, get_latest
|
├── data/ # ═══ DATA PLATFORM ═══
|
||||||
│ └── partitioning.py # Hive partition path builder
|
│ ├── sources/ # akshare_client, index, industry
|
||||||
├── api/
|
│ ├── pipelines/ # backfill, eod, realtime
|
||||||
│ ├── app.py # FastAPI factory + lifespan + embedded docs HTML
|
│ └── store/ # database (get_db, analytics_conn, kline_glob), repository, partitioning, schema
|
||||||
│ ├── deps.py # FastAPI DI (get_repo)
|
├── features/ # Feature Store (6 registered features, all use analytics_conn + kline_glob)
|
||||||
│ ├── routers/ # stocks, kline, realtime, calendar routers
|
├── market/ # ═══ MARKET INTELLIGENCE ═══
|
||||||
│ └── websocket/
|
│ ├── state.py # infer_market_state
|
||||||
│ ├── manager.py # ConnectionManager: subscribe/broadcast with async lock
|
│ ├── leadership.py # assess_leaders (lifecycle per industry)
|
||||||
│ └── handlers.py # WS message dispatch (/ws/realtime)
|
│ ├── opportunity.py # rank_opportunities
|
||||||
├── scheduler/
|
│ ├── flow.py # compute_flow (money flow graph)
|
||||||
│ ├── scheduler.py # APScheduler setup (EOD 15:05, health check 08:00)
|
│ ├── sentiment.py # Phase 2 placeholder
|
||||||
│ └── jobs.py # Job implementations
|
│ └── memory.py # StateStore (state_snapshot table)
|
||||||
└── cli/
|
├── signals/ # ═══ SIGNAL INTELLIGENCE ═══ (the moat)
|
||||||
├── main.py # Typer root: ashare-dp {backfill, serve, query, version}
|
│ ├── detectors.py # EMA52 cross detection + shared screening logic
|
||||||
├── backfill_cmd.py
|
│ ├── store.py # signal_instance CRUD (to be extracted from detectors)
|
||||||
├── serve_cmd.py
|
│ └── expectancy.py # get_expectancy(state) — single entry, fallback chain internal
|
||||||
└── query_cmd.py
|
├── execution/ # ═══ EXECUTION INTELLIGENCE ═══
|
||||||
|
│ ├── playbook.py # build_playbook (State → strategies/bias/holding)
|
||||||
|
│ ├── risk.py # RiskRule engine + evaluate_risks
|
||||||
|
│ └── brief.py # build_brief + brief_to_api_dict (single serialization point)
|
||||||
|
└── apps/ # ═══ PRESENTATION ═══
|
||||||
|
├── api/ # app.py, routers/, websocket/, dashboard/
|
||||||
|
├── cli/ # main.py + backfill/serve/query/eod/screening commands
|
||||||
|
└── scheduler/ # scheduler.py, jobs.py
|
||||||
```
|
```
|
||||||
|
|
||||||
## Critical Design Decisions
|
|
||||||
|
|
||||||
### Freq.storage_dir property
|
|
||||||
macOS APFS is case-insensitive, so `kline_1m` and `kline_1M` collide. The `Freq.M1` (monthly) uses `storage_dir = "1mon"` to disambiguate. Always use `freq.storage_dir` for filesystem paths, never `freq.value`.
|
|
||||||
|
|
||||||
### Batched writes to avoid file overwrites
|
|
||||||
`write_klines` groups all stocks for a day into a single `data.parquet` file. The backfill and EOD pipelines collect all DataFrames first, then write once per day/freq. Never write per-stock-per-day files.
|
|
||||||
|
|
||||||
### Dual backend with auto-fallback
|
|
||||||
`AKShareClient._resolve_backend()` probes East Money reachability once and caches the result. If unreachable (geo-blocked outside China), falls back to Sina. `get_stock_list()` and `get_trading_calendar()` are Sina-only.
|
|
||||||
|
|
||||||
### SQL parameterization
|
|
||||||
`read_klines()` and `get_latest()` use DuckDB parameterized queries (`$1`, `$2`) for user-supplied values (ts_code, dates). Do NOT use f-string interpolation for user input.
|
|
||||||
|
|
||||||
### 2h derivation
|
|
||||||
2-hour K-lines are derived on-the-fly from 1h data via `_read_2h()` using pandas resampling. No Parquet storage for 2h.
|
|
||||||
|
|
||||||
## Key Conventions
|
## Key Conventions
|
||||||
|
|
||||||
- Stock codes: `ts_code` format is `"000001.SZ"` (6-digit code + exchange suffix). Internal API calls use 6-digit numeric strings.
|
### ts_code conversion (CRITICAL)
|
||||||
- Exchange mapping: codes starting with `6`/`9` → SH, `4`/`8`/`92` → BJ, rest → SZ
|
**Always use `from ashare_dp.core.codes import to_ts_code`** — the single idempotent implementation. Never write local `_code_to_ts_code()` copies. `to_ts_code()` is safe to call on any format: bare codes, already-formatted ts_codes, Sina symbols, even legacy corrupted `.SZ.SZ` values.
|
||||||
- Backend-specific column normalization: Sina returns English columns (`date`, `open`, etc.), EM returns Chinese. `_normalize_hist_df` and `_normalize_min_df` handle both.
|
|
||||||
- Proxy: env vars cleared + `urllib.request.getproxies` monkey-patched at module import time in `akshare_client.py`
|
### Database connections
|
||||||
- All async state in `ConnectionManager` is protected by `asyncio.Lock`
|
- **DuckDB tables** (stock_info, trading_calendar, signal_instance): use `get_db()` context manager from `data.store.database`
|
||||||
|
- **Analytics queries** (features, engines): use `analytics_conn()` for raw DuckDB connection — the single sanctioned way. Never hardcode `"data/duckdb/ashare.db"`
|
||||||
|
- **Parquet globs**: use `kline_glob()` or `partition_glob(freq)` — never hardcode paths
|
||||||
|
|
||||||
|
### Architecture boundaries
|
||||||
|
- `data/` knows nothing about trading
|
||||||
|
- `features/` computes features, never classifies regimes
|
||||||
|
- `market/` infers state, knows nothing about signals
|
||||||
|
- `signals/` queries historical expectancy, knows nothing about execution
|
||||||
|
- `execution/` maps state to strategies, assembles TradingBrief
|
||||||
|
- `apps/` only renders, never reasons
|
||||||
|
|
||||||
|
### MarketState is a continuous vector (not enum)
|
||||||
|
7 dimensions: trend, fear, liquidity, rotation, participation, volatility, breadth. Each 0.0–1.0. Display labels derived downstream only.
|
||||||
|
|
||||||
|
### API Response
|
||||||
|
Single endpoint produces all dashboard data: `GET /api/v1/dashboard/state`. Response versioned (`"version": "1.0"`). Serialization in `execution/brief.py::brief_to_api_dict()` — the single serialization point. Router only orchestrates engine calls.
|
||||||
|
|
||||||
## Running
|
## Running
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install -e ".[dev]"
|
pip install -e ".[dev]"
|
||||||
ashare-dp backfill init # First time: create tables, load stock list
|
ashare-dp backfill init # Schema + stock list + trading calendar
|
||||||
ashare-dp backfill daily # Backfill all daily/weekly/monthly history
|
ashare-dp backfill daily # Daily/weekly/monthly + indices
|
||||||
ashare-dp serve start # Start API + scheduler + realtime poller
|
ashare-dp backfill industry # Industry classifications
|
||||||
ashare-dp query stats # Check data status
|
ashare-dp backfill signals # EMA52 signal detection + store
|
||||||
```
|
ashare-dp serve start # API + scheduler + realtime
|
||||||
|
open http://localhost:8000/dashboard
|
||||||
## Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pytest
|
|
||||||
ruff check src/
|
|
||||||
```
|
```
|
||||||
|
|||||||
+1
-1
@@ -28,7 +28,7 @@ dev = [
|
|||||||
]
|
]
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
ashare-dp = "ashare_dp.cli.main:app"
|
ashare-dp = "ashare_dp.apps.cli.main:app"
|
||||||
|
|
||||||
[build-system]
|
[build-system]
|
||||||
requires = ["setuptools>=75.0"]
|
requires = ["setuptools>=75.0"]
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
"""PRESENTATION — API, CLI, scheduler. Renders, never reasons."""
|
||||||
@@ -10,12 +10,14 @@ from fastapi import FastAPI, HTTPException, Query
|
|||||||
from fastapi.responses import HTMLResponse
|
from fastapi.responses import HTMLResponse
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.api.routers import stocks, kline, realtime, calendar
|
from ashare_dp.apps.api.routers import stocks, kline, realtime, calendar
|
||||||
|
from ashare_dp.apps.api.dashboard.router import router as dashboard_router
|
||||||
|
from ashare_dp.apps.api.dashboard.html import DASHBOARD_HTML
|
||||||
from ashare_dp.core.models import Freq
|
from ashare_dp.core.models import Freq
|
||||||
from ashare_dp.storage.repository import KLineRepository
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
from ashare_dp.api.websocket.handlers import router as ws_router
|
from ashare_dp.apps.api.websocket.handlers import router as ws_router
|
||||||
from ashare_dp.storage.database import get_db
|
from ashare_dp.data.store.database import get_db
|
||||||
from ashare_dp.storage.schema import DDL_STATEMENTS
|
from ashare_dp.data.store.schema import DDL_STATEMENTS
|
||||||
|
|
||||||
DOCS_HTML = r"""
|
DOCS_HTML = r"""
|
||||||
<!DOCTYPE html>
|
<!DOCTYPE html>
|
||||||
@@ -425,7 +427,7 @@ async def lifespan(app: FastAPI):
|
|||||||
|
|
||||||
# Start realtime poller
|
# Start realtime poller
|
||||||
try:
|
try:
|
||||||
from ashare_dp.data.realtime import poller
|
from ashare_dp.data.pipelines.realtime import poller
|
||||||
await poller.start()
|
await poller.start()
|
||||||
app.state.poller = poller
|
app.state.poller = poller
|
||||||
logger.info("Realtime poller started")
|
logger.info("Realtime poller started")
|
||||||
@@ -565,6 +567,11 @@ def create_app() -> FastAPI:
|
|||||||
"""EMA52 screening results page."""
|
"""EMA52 screening results page."""
|
||||||
return EMA52_HTML
|
return EMA52_HTML
|
||||||
|
|
||||||
|
@app.get("/dashboard", response_class=HTMLResponse)
|
||||||
|
async def dashboard_page():
|
||||||
|
"""Trading OS Dashboard — professional trader decision support."""
|
||||||
|
return DASHBOARD_HTML
|
||||||
|
|
||||||
@app.get("/api/v1/screening/ema52")
|
@app.get("/api/v1/screening/ema52")
|
||||||
async def api_ema52_screening(
|
async def api_ema52_screening(
|
||||||
freq: str = Query("1d", description="Frequency: 1d or 1w"),
|
freq: str = Query("1d", description="Frequency: 1d or 1w"),
|
||||||
@@ -609,6 +616,7 @@ def create_app() -> FastAPI:
|
|||||||
app.include_router(kline.router, prefix="/api/v1")
|
app.include_router(kline.router, prefix="/api/v1")
|
||||||
app.include_router(realtime.router, prefix="/api/v1")
|
app.include_router(realtime.router, prefix="/api/v1")
|
||||||
app.include_router(calendar.router, prefix="/api/v1")
|
app.include_router(calendar.router, prefix="/api/v1")
|
||||||
|
app.include_router(dashboard_router, prefix="/api/v1")
|
||||||
app.include_router(ws_router)
|
app.include_router(ws_router)
|
||||||
|
|
||||||
return app
|
return app
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""Dashboard package."""
|
||||||
@@ -0,0 +1,359 @@
|
|||||||
|
"""Dashboard HTML — Trading Command Center.
|
||||||
|
|
||||||
|
5 sections: COMMAND → ACTION → WHERE → WHY → RISK → EXPECTANCY
|
||||||
|
Everything else collapsed as Diagnostics.
|
||||||
|
"""
|
||||||
|
|
||||||
|
DASHBOARD_HTML = """<!DOCTYPE html>
|
||||||
|
<html lang="zh-CN">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8"><meta name="viewport" content="width=device-width,initial-scale=1.0">
|
||||||
|
<title>Trading OS</title>
|
||||||
|
<style>
|
||||||
|
:root{--bg:#0d1117;--card:#161b22;--border:#30363d;--text:#e6edf3;--muted:#8b949e;--accent:#58a6ff;--green:#3fb950;--red:#f85149;--orange:#d2991d;--purple:#a371f7}
|
||||||
|
*{box-sizing:border-box;margin:0;padding:0}
|
||||||
|
body{font-family:-apple-system,BlinkMacSystemFont,'Segoe UI','PingFang SC','Microsoft YaHei',sans-serif;background:var(--bg);color:var(--text);padding:18px;line-height:1.5;max-width:1040px;margin:0 auto}
|
||||||
|
.header{display:flex;justify-content:space-between;align-items:center;margin-bottom:8px}
|
||||||
|
.header h1{font-size:13px;font-weight:600;letter-spacing:-.3px}
|
||||||
|
.date{font-size:10px;color:var(--muted)}
|
||||||
|
.loading{text-align:center;padding:50px;color:var(--muted)}
|
||||||
|
.spinner{display:inline-block;width:22px;height:22px;border:2px solid rgba(255,255,255,.1);border-top-color:var(--accent);border-radius:50%;animation:spin .8s linear infinite}
|
||||||
|
@keyframes spin{to{transform:rotate(360deg)}}
|
||||||
|
.error{text-align:center;padding:30px;color:var(--red);font-size:13px}
|
||||||
|
|
||||||
|
.card{background:var(--card);border:1px solid var(--border);border-radius:8px;padding:14px;margin-bottom:8px}
|
||||||
|
.sec-label{font-size:9px;font-weight:700;color:var(--muted);text-transform:uppercase;letter-spacing:1px;margin-bottom:6px}
|
||||||
|
.good{color:var(--green)}.bad{color:var(--red)}.warn{color:var(--orange)}.muted{color:var(--muted)}
|
||||||
|
.tag{display:inline-block;padding:2px 8px;border-radius:4px;font-size:10px;font-weight:500;margin:1px 3px 1px 0}
|
||||||
|
.tag.buy{background:rgba(63,185,80,.15);color:var(--green)}.tag.avoid{background:rgba(248,81,73,.15);color:var(--red)}
|
||||||
|
.tag.bias{font-size:12px;font-weight:700;padding:3px 10px;border-radius:5px}
|
||||||
|
.tag.LONG{background:rgba(63,185,80,.18);color:var(--green)}
|
||||||
|
.tag.CASH{background:rgba(139,148,158,.18);color:var(--muted)}
|
||||||
|
|
||||||
|
/* Rows */
|
||||||
|
.row{display:grid;grid-template-columns:1fr 1fr;gap:8px;margin-bottom:8px}
|
||||||
|
@media(max-width:800px){.row{grid-template-columns:1fr}}
|
||||||
|
|
||||||
|
/* COMMAND */
|
||||||
|
.cmd-mode{font-size:26px;font-weight:800;margin-bottom:2px}
|
||||||
|
.cmd-play{font-size:12px;color:var(--muted);margin-bottom:6px}
|
||||||
|
.cmd-meta{display:flex;gap:12px;flex-wrap:wrap;font-size:10px;color:var(--muted);margin-top:6px;padding-top:6px;border-top:1px solid rgba(255,255,255,.06)}
|
||||||
|
|
||||||
|
/* WHERE */
|
||||||
|
.where-item{display:flex;align-items:center;gap:6px;padding:6px 8px;background:rgba(255,255,255,.02);border-radius:5px;margin-bottom:4px}
|
||||||
|
.where-stars{font-size:11px;color:var(--orange);width:55px;flex-shrink:0;letter-spacing:-1px}
|
||||||
|
.where-name{font-weight:600;font-size:12px;min-width:70px}
|
||||||
|
.where-detail{font-size:9px;color:var(--muted);flex:1}
|
||||||
|
.where-leader{font-size:9px;color:var(--accent)}
|
||||||
|
|
||||||
|
/* WHY — flow */
|
||||||
|
.flow-line{font-family:monospace;font-size:10px;line-height:1.6;white-space:pre;color:var(--muted)}
|
||||||
|
.flow-line .fr{color:var(--red)}.flow-line .to{color:var(--green)}.flow-line .ar{color:var(--muted)}
|
||||||
|
|
||||||
|
/* RISK */
|
||||||
|
.risk-item{padding:6px 8px;border-radius:4px;margin-bottom:3px;font-size:10px;display:flex;align-items:flex-start;gap:5px}
|
||||||
|
.risk-crit{background:rgba(248,81,73,.1);border:1px solid rgba(248,81,73,.2)}
|
||||||
|
.risk-warn{background:rgba(210,153,29,.1);border:1px solid rgba(210,153,29,.2)}
|
||||||
|
.risk-badge{font-size:7px;font-weight:700;padding:1px 4px;border-radius:2px;flex-shrink:0}
|
||||||
|
.risk-crit .risk-badge{background:rgba(248,81,73,.2);color:var(--red)}
|
||||||
|
.risk-warn .risk-badge{background:rgba(210,153,29,.2);color:var(--orange)}
|
||||||
|
|
||||||
|
/* EXPECTANCY */
|
||||||
|
.exp-grid{display:grid;grid-template-columns:repeat(4,1fr);gap:4px;text-align:center}
|
||||||
|
.exp-stat{padding:4px 2px;background:rgba(255,255,255,.02);border-radius:3px}
|
||||||
|
.exp-stat .val{font-size:15px;font-weight:700}.exp-stat .lbl{font-size:8px;color:var(--muted)}
|
||||||
|
.exp-table{width:100%;border-collapse:collapse;font-size:10px;margin-top:6px}
|
||||||
|
.exp-table th{color:var(--muted);text-align:left;padding:3px 4px;border-bottom:1px solid rgba(255,255,255,.06);font-weight:500}
|
||||||
|
.exp-table td{padding:2px 4px;border-bottom:1px solid rgba(255,255,255,.03)}
|
||||||
|
.exp-mini-bar{display:inline-block;height:8px;border-radius:2px;min-width:2px;vertical-align:middle;margin-right:4px}
|
||||||
|
|
||||||
|
/* Diagnostics */
|
||||||
|
.diag-toggle{cursor:pointer;user-select:none;font-size:9px;color:var(--muted);padding:3px 0;margin-bottom:8px}
|
||||||
|
.diag-toggle:hover{color:var(--text)}
|
||||||
|
.diag-content{display:none}
|
||||||
|
.diag-content.open{display:block}
|
||||||
|
.diag-grid{display:grid;grid-template-columns:repeat(auto-fit,minmax(170px,1fr));gap:6px}
|
||||||
|
.diag-card{background:rgba(255,255,255,.015);border:1px solid rgba(255,255,255,.04);border-radius:5px;padding:6px}
|
||||||
|
.diag-card h4{font-size:9px;color:var(--muted);margin-bottom:4px}
|
||||||
|
.metric{display:flex;justify-content:space-between;padding:1px 0;font-size:9px;border-bottom:1px solid rgba(255,255,255,.02)}.metric:last-child{border:none}
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div class="header"><div><h1>Trading OS</h1></div><div class="date" id="trade-date">--</div></div>
|
||||||
|
<div id="app"><div class="loading"><div class="spinner"></div><p>Loading...</p></div></div>
|
||||||
|
<script>
|
||||||
|
var A='/api/v1/dashboard/state';
|
||||||
|
function S(n){var s='';for(var i=0;i<5;i++)s+=i<n?'★':'☆';return s;}
|
||||||
|
function P(v,d){return v!=null&&!isNaN(v)?v:(d||0);}
|
||||||
|
function Q(v){return v==null?'--':v;}
|
||||||
|
|
||||||
|
function L(){
|
||||||
|
fetch(A).then(function(r){if(!r.ok)throw Error('HTTP '+r.status);return r.json();})
|
||||||
|
.then(function(d){R(d);}).catch(function(e){document.getElementById('app').innerHTML='<div class=\"error\">'+e.message+'</div>';setTimeout(L,10000);});
|
||||||
|
}
|
||||||
|
|
||||||
|
function R(d){
|
||||||
|
document.getElementById('trade-date').textContent=d.trade_date||'--';
|
||||||
|
var pb=d.playbook||{},exp=d.expectancy||{},alloc=d.allocation||{};
|
||||||
|
var opps=d.opportunities||[],risks=d.risks||[],leaders=d.leaders||{};
|
||||||
|
var mf=d.money_flow||{},flows=mf.flows||[],netIn=mf.net_inflow||{},netOut=mf.net_outflow||{};
|
||||||
|
var ev=d.evidence||{},state=ev.state||{},dims=state.dimensions||{},feat=ev.features||{};
|
||||||
|
var b=feat.breadth_vector||{},v=feat.volume_vector||{},tr=feat.trend_vector||{};
|
||||||
|
var wr=P(exp.win_rate*100,0),ar=P(exp.avg_return*100,0);
|
||||||
|
var dd=P(exp.max_drawdown*100,0),hold=P(exp.avg_holding_days,0);
|
||||||
|
var samples=P(exp.sample_count,0),sigType=Q(exp.signal_type);
|
||||||
|
var trend=P(dims.trend,0),fear=P(dims.fear,0),liq=P(dims.liquidity,0);
|
||||||
|
var conf=P(state.confidence||pb.confidence,0.5);
|
||||||
|
var cashP=P(alloc.cash_pct*100,30).toFixed(0),trendP=P(alloc.trend_pct*100,40).toFixed(0);
|
||||||
|
|
||||||
|
// Leaders
|
||||||
|
var alive=[],broken=[];
|
||||||
|
for(var k in leaders){var st=leaders[k].state||'';if(st==='leading'||st==='growing'||st==='birth')alive.push(k);if(st==='dead'||st==='breaking')broken.push(k);}
|
||||||
|
|
||||||
|
// Mode
|
||||||
|
var ms=conf>0.6&&wr>55?5:conf>0.45&&wr>40?4:conf>0.3?3:wr>35?2:1;
|
||||||
|
var ml=ms>=4?'积极做':ms>=3?'轻仓试错':ms>=2?'谨慎':'保命';
|
||||||
|
var mc=ms>=4?'good':ms>=3?'warn':'bad';
|
||||||
|
|
||||||
|
// ACTIONS — derive from Playbook
|
||||||
|
var acts=[],avs=[];
|
||||||
|
(pb.suitable_strategies||[]).forEach(function(s){acts.push('<span class=\"tag buy\">✓ '+s+'</span>');});
|
||||||
|
(pb.unsuitable_strategies||[]).forEach(function(s){avs.push('<span class=\"tag avoid\">✗ '+s+'</span>');});
|
||||||
|
|
||||||
|
// WHERE
|
||||||
|
var wh='';
|
||||||
|
opps.slice(0,6).forEach(function(o,i){
|
||||||
|
var ld=o.leader_stock||'';
|
||||||
|
wh+='<div class=\"where-item\">'+
|
||||||
|
'<span class=\"where-stars\">'+S(Math.round(o.score/20))+'</span>'+
|
||||||
|
'<span class=\"where-name\">'+o.name+'</span>'+
|
||||||
|
'<span class=\"where-detail\">'+(ld?'<span class=\"where-leader\">'+ld+'</span> · ':'')+o.lifecycle+' · '+o.persistence_days+'d</span>'+
|
||||||
|
'</div>';
|
||||||
|
});
|
||||||
|
|
||||||
|
// WHY — flow
|
||||||
|
var af=[];
|
||||||
|
for(var k in netIn)af.push({name:k,chg:netIn[k],dir:'in'});
|
||||||
|
for(var k in netOut)af.push({name:k,chg:netOut[k],dir:'out'});
|
||||||
|
af.sort(function(a,b){return Math.abs(b.chg)-Math.abs(a.chg);});
|
||||||
|
var flowL='',seen={};
|
||||||
|
af.slice(0,8).forEach(function(f){if(seen[f.name])return;seen[f.name]=1;
|
||||||
|
var arrow='',n=Math.round(Math.abs(f.chg)*12);
|
||||||
|
for(var i=0;i<Math.min(n,10);i++)arrow+=f.dir==='in'?'↑':'↓';
|
||||||
|
flowL+='<span class=\"'+(f.dir==='in'?'to':'fr')+'\">'+f.name+'</span> <span class=\"ar\">'+arrow+'</span> <span style=\"font-size:9px;color:var(--muted)\">'+(f.chg>0?'+':'')+(f.chg*100).toFixed(1)+'%</span>\\n';
|
||||||
|
});
|
||||||
|
|
||||||
|
// RISK
|
||||||
|
var rh='<span style=\"font-size:10px;color:var(--green)\">✓ None</span>';
|
||||||
|
if(risks.length){rh='';risks.forEach(function(r){var l=r.level==='danger'?'risk-crit':'risk-warn';rh+='<div class=\"risk-item '+l+'\"><span class=\"risk-badge\">'+r.level.toUpperCase().substring(0,4)+'</span>'+r.message+'</div>';})}
|
||||||
|
|
||||||
|
// Focus/Avoid
|
||||||
|
var ft='',at='';
|
||||||
|
(pb.focus_sectors||[]).forEach(function(s){ft+='<span class=\"tag buy\">'+s+'</span> ';});
|
||||||
|
(pb.avoid_sectors||[]).forEach(function(s){at+='<span class=\"tag avoid\">'+s+'</span> ';});
|
||||||
|
|
||||||
|
// WHY text
|
||||||
|
var wy=[];
|
||||||
|
if(trend>0.5)wy.push('趋势向上');else if(trend>0.3)wy.push('震荡');else wy.push('偏弱');
|
||||||
|
if(alive.length)wy.push('龙头健康');
|
||||||
|
if(broken.length)wy.push(broken.length+'龙头破位');
|
||||||
|
if(fear>0.6)wy.push('恐慌偏高');
|
||||||
|
if(v.vs_5d>1.1)wy.push('放量');
|
||||||
|
if(v.vs_5d<0.85)wy.push('缩量');
|
||||||
|
var sent=ev.sentiment||{};
|
||||||
|
if(sent.profit_effect&&sent.profit_effect!=='pending_data')wy.push('赚钱效应:'+sent.profit_effect);
|
||||||
|
|
||||||
|
// Build
|
||||||
|
var h='';
|
||||||
|
|
||||||
|
// ═══ COMMAND + WHERE ═══
|
||||||
|
h+='<div class=\"row\">';
|
||||||
|
h+='<div class=\"card\" style=\"background:linear-gradient(135deg,#1a1f2e,#161b22)\">';
|
||||||
|
h+='<div class=\"sec-label\">COMMAND</div>';
|
||||||
|
h+='<div class=\"cmd-mode\"><span class=\"'+mc+'\">'+S(ms)+'</span></div>';
|
||||||
|
h+='<div style=\"font-size:13px;font-weight:700\">'+ml+'</div>';
|
||||||
|
h+='<div class=\"cmd-play\">'+Q(pb.suitable_strategies[0]||pb.bias)+' · '+Q(pb.holding_time)+'</div>';
|
||||||
|
h+='<div style=\"margin-bottom:4px\">'+acts.join(' ')+'</div>';
|
||||||
|
if(avs.length)h+='<div style=\"margin-bottom:2px\">'+avs.join(' ')+'</div>';
|
||||||
|
h+='<div class=\"cmd-meta\"><span>'+wy.join(' · ')+'</span></div>';
|
||||||
|
h+='<div class=\"cmd-meta\" style=\"border-top:none;padding-top:2px\"><span>Agr '+(conf*100).toFixed(0)+' Hlth '+(trend*100).toFixed(0)+' Fear '+(fear*100).toFixed(0)+' Liq '+(liq*100).toFixed(0)+' '+Q(pb.bias)+' C'+cashP+'/T'+trendP+'/T20</span></div>';
|
||||||
|
h+='</div>';
|
||||||
|
|
||||||
|
h+='<div class=\"card\">';
|
||||||
|
h+='<div class=\"sec-label\">WHERE</div>';
|
||||||
|
h+=wh||'<span class=\"muted\" style=\"font-size:10px\">--</span>';
|
||||||
|
h+='</div></div>';
|
||||||
|
|
||||||
|
// ═══ WHY + RISK ═══
|
||||||
|
h+='<div class=\"row\">';
|
||||||
|
h+='<div class=\"card\">';
|
||||||
|
h+='<div class=\"sec-label\">WHY — Money Flow</div>';
|
||||||
|
h+='<div class=\"flow-line\">'+(flowL||'No flow data')+'</div>';
|
||||||
|
if(ft)h+='<div style=\"margin-top:6px;font-size:9px\"><span class=\"muted\">Focus:</span> '+ft+'</div>';
|
||||||
|
if(at)h+='<div style=\"font-size:9px\"><span class=\"muted\">Avoid:</span> '+at+'</div>';
|
||||||
|
h+='</div>';
|
||||||
|
|
||||||
|
// Theme Map
|
||||||
|
var tg=d.theme_graph||{},themes=tg.themes||[];
|
||||||
|
h+='<div class=\"card\">';
|
||||||
|
h+='<div class=\"sec-label\">THEMES</div>';
|
||||||
|
if(themes.length){
|
||||||
|
themes.slice(0,5).forEach(function(t,i){
|
||||||
|
var stars='';for(var j=0;j<5;j++)stars+=j<Math.round(t.score/20)?'★':'☆';
|
||||||
|
h+='<div style=\"display:flex;justify-content:space-between;align-items:center;padding:3px 0;font-size:11px\">'+
|
||||||
|
'<span style=\"font-weight:600\">'+t.name+'</span>'+
|
||||||
|
'<span style=\"color:var(--orange);font-size:10px\">'+stars+'</span>'+
|
||||||
|
'<span style=\"font-size:10px;color:var(--muted)\">'+t.momentum+'</span></div>';
|
||||||
|
if(t.concepts&&t.concepts.length){
|
||||||
|
h+='<div style=\"font-size:9px;color:var(--muted);margin-bottom:2px;padding-left:8px\">'+
|
||||||
|
t.concepts.slice(0,4).map(function(c){return c.name}).join(' → ')+'</div>';
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}else{h+='<span class=\"muted\" style=\"font-size:10px\">No active themes</span>';}
|
||||||
|
if(tg.edges&&tg.edges.length){
|
||||||
|
h+='<div style=\"margin-top:4px;font-size:9px;color:var(--muted)\">'+
|
||||||
|
tg.edges.slice(0,2).map(function(e){return e.source+' ⇢ '+e.target}).join(' · ')+'</div>';
|
||||||
|
}
|
||||||
|
h+='</div></div>';
|
||||||
|
|
||||||
|
// Recommendations
|
||||||
|
var recs=d.recommendations||[];
|
||||||
|
h+='<div class=\"card\" style=\"margin-bottom:8px\">';
|
||||||
|
h+='<div class=\"sec-label\">RECOMMENDATIONS — Money Flow Picks</div>';
|
||||||
|
if(recs.length){
|
||||||
|
h+='<div style=\"display:grid;grid-template-columns:repeat(auto-fill,minmax(300px,1fr));gap:6px\">';
|
||||||
|
recs.forEach(function(r,i){
|
||||||
|
var cls=r.score>70?'good':r.score>50?'warn':'muted';
|
||||||
|
h+='<div style=\"padding:8px 10px;background:rgba(255,255,255,.02);border-radius:6px;font-size:11px\">';
|
||||||
|
h+='<div style=\"display:flex;justify-content:space-between;align-items:center;margin-bottom:4px\">';
|
||||||
|
h+='<span><strong>'+(r.name||r.ts_code)+'</strong> <span style=\"font-size:10px;color:var(--muted)\">'+r.ts_code+' · '+r.sector+'</span></span>';
|
||||||
|
h+='<span class=\"'+cls+'\" style=\"font-weight:700\">'+r.score.toFixed(0)+'</span></div>';
|
||||||
|
h+='<div style=\"display:flex;gap:12px;font-size:10px;color:var(--muted);margin-bottom:4px\">';
|
||||||
|
h+='<span>Entry: <span class=\"good\">'+r.entry+'</span></span>';
|
||||||
|
h+='<span>Stop: <span class=\"bad\">'+r.stop+'</span></span>';
|
||||||
|
h+='<span>T1: <span class=\"good\">'+r.target1+'</span></span>';
|
||||||
|
h+='<span>T2: '+r.target2+'</span></div>';
|
||||||
|
h+='<div style=\"display:flex;justify-content:space-between;font-size:10px\">';
|
||||||
|
h+='<span class=\"muted\">5d:'+(r.metrics.ret_5d>0?'+':'')+r.metrics.ret_5d.toFixed(1)+'% Vol:'+r.metrics.vol_ratio.toFixed(1)+'x '+r.metrics.trend+'</span>';
|
||||||
|
h+='<span style=\"color:var(--accent)\">R:R '+r.rr_ratio.toFixed(1)+'</span></div>';
|
||||||
|
h+='<div style=\"font-size:10px;color:var(--green);margin-top:3px\">'+r.action+'</div>';
|
||||||
|
h+='</div>';
|
||||||
|
});
|
||||||
|
h+='</div>';
|
||||||
|
}else{h+='<span class=\"muted\" style=\"font-size:11px\">No recommendations — insufficient data</span>';}
|
||||||
|
h+='</div>';
|
||||||
|
|
||||||
|
h+='<div class=\"row\"><div class=\"card\">';
|
||||||
|
h+='<div class=\"sec-label\">RISK</div>';
|
||||||
|
h+=rh;
|
||||||
|
h+='</div></div>';
|
||||||
|
|
||||||
|
// ═══ EXPECTANCY ═══
|
||||||
|
h+='<div class=\"card\" style=\"opacity:0.8\">';
|
||||||
|
var expLabel=samples>0?'— '+sigType+' · '+samples+' samples':'— Phase 1 Heuristic · 0 samples';
|
||||||
|
var expColor=samples>0?'var(--green)':'var(--orange)';
|
||||||
|
h+='<div class=\"sec-label\">EXPECTANCY <span style=\"color:'+expColor+';font-weight:400\">'+expLabel+'</span></div>';
|
||||||
|
h+='<div class=\"exp-grid\">';
|
||||||
|
h+='<div class=\"exp-stat\"><div class=\"val '+(wr>50?'good':'warn')+'\">'+wr.toFixed(0)+'%</div><div class=\"lbl\">Est. WR</div></div>';
|
||||||
|
h+='<div class=\"exp-stat\"><div class=\"val '+(ar>0?'good':'bad')+'\">'+(ar>0?'+':'')+ar.toFixed(1)+'%</div><div class=\"lbl\">Est. Ret</div></div>';
|
||||||
|
h+='<div class=\"exp-stat\"><div class=\"val bad\">'+dd.toFixed(1)+'%</div><div class=\"lbl\">Est. DD</div></div>';
|
||||||
|
h+='<div class=\"exp-stat\"><div class=\"val\">'+hold.toFixed(1)+'d</div><div class=\"lbl\">Est. Hold</div></div>';
|
||||||
|
h+='</div>';
|
||||||
|
|
||||||
|
// All expectancies comparison table
|
||||||
|
var allExp=d.all_expectancies||[];
|
||||||
|
if(allExp.length>0){
|
||||||
|
h+='<table class=\"exp-table\"><tr><th>Signal</th><th>WR</th><th>Samples</th><th>Avg Ret</th><th>Max DD</th></tr>';
|
||||||
|
allExp.forEach(function(e){
|
||||||
|
var ew=e.win_rate*100,ea=e.avg_return*100,ed=e.max_drawdown*100;
|
||||||
|
var barW=Math.max(2,ew);
|
||||||
|
h+='<tr>'+
|
||||||
|
'<td>'+e.signal_type+'</td>'+
|
||||||
|
'<td><span class=\"exp-mini-bar\" style=\"width:'+barW+'px;background:'+(ew>50?'var(--green)':ew>40?'var(--orange)':'var(--red)')+'\"></span>'+(e.sample_count>0?ew.toFixed(0)+'%':'--')+'</td>'+
|
||||||
|
'<td style=\"color:var(--muted)\">'+(e.sample_count>0?e.sample_count.toLocaleString():'heuristic')+'</td>'+
|
||||||
|
'<td class=\"'+(ea>0?'good':'bad')+'\">'+(e.sample_count>0?(ea>0?'+':'')+ea.toFixed(1)+'%':'--')+'</td>'+
|
||||||
|
'<td class=\"bad\">'+(e.sample_count>0?ed.toFixed(1)+'%':'--')+'</td>'+
|
||||||
|
'</tr>';
|
||||||
|
});
|
||||||
|
h+='</table>';
|
||||||
|
}
|
||||||
|
h+='</div>';
|
||||||
|
|
||||||
|
// ═══ DIAGNOSTICS (always collapsed) ═══
|
||||||
|
var idx=(tr.index_details||[]).slice(0,7);
|
||||||
|
var idxH='';idx.forEach(function(i){idxH+='<div class=\"metric\"><span>'+i.name+'</span><span class=\"'+(i.pct_chg>0?'good':'bad')+'\">'+(i.pct_chg>0?'+':'')+i.pct_chg.toFixed(2)+'%</span></div>';});
|
||||||
|
h+='<div class=\"diag-toggle\" onclick=\"var e=document.getElementById(\\'diag\\');e.classList.toggle(\\'open\\');this.textContent=e.classList.contains(\\'open\\')?\\'▼ Diagnostics\\':\\'▶ Diagnostics\\'\">▶ Diagnostics</div>';
|
||||||
|
h+='<div class=\"diag-content\" id=\"diag\"><div class=\"diag-grid\">';
|
||||||
|
h+='<div class=\"diag-card\"><h4>Breadth</h4><div class=\"metric\"><span>Adv/Dec</span><span><span class=\"good\">'+Q(b.advances)+'</span>/<span class=\"bad\">'+Q(b.declines)+'</span></span></div><div class=\"metric\"><span>Ratio</span><span>'+((b.advance_ratio||0)*100).toFixed(1)+'%</span></div><div class=\"metric\"><span>New High</span><span>'+((b.new_high_ratio||0)*100).toFixed(1)+'%</span></div></div>';
|
||||||
|
h+='<div class=\"diag-card\"><h4>Volume</h4><div class=\"metric\"><span>Turnover</span><span>'+Q(v.total_turnover_yi)+' 亿</span></div><div class=\"metric\"><span>vs 5d</span><span>'+((v.vs_5d-1)*100).toFixed(1)+'%</span></div><div class=\"metric\"><span>Tier</span><span>'+Q(v.tier)+'</span></div></div>';
|
||||||
|
h+='<div class=\"diag-card\"><h4>State</h4>';['trend','fear','liquidity','rotation','participation','volatility','breadth'].forEach(function(k){h+='<div class=\"metric\"><span>'+k+'</span><span>'+((dims[k]||0)*100).toFixed(0)+'</span></div>';});h+='</div>';
|
||||||
|
h+='<div class=\"diag-card\"><h4>Indices</h4>'+idxH+'<div style=\"font-size:8px;color:var(--muted);margin-top:2px\">Resonance: '+((tr.resonance||0)*100).toFixed(0)+'%</div></div>';
|
||||||
|
var sent=ev.sentiment||{};
|
||||||
|
if(sent.status==='live'){
|
||||||
|
h+='<div class=\"diag-card\"><h4>Sentiment</h4>';
|
||||||
|
h+='<div class=\"metric\"><span>Profit Effect</span><span class=\"'+(sent.profit_effect==='Strong'||sent.profit_effect==='Average'?'good':'bad')+'\">'+Q(sent.profit_effect)+' ('+Q(sent.profit_effect_score)+')</span></div>';
|
||||||
|
h+='<div class=\"metric\"><span>Limit Up/Down</span><span><span class=\"good\">'+Q(sent.limit_up_count)+'</span>/<span class=\"bad\">'+Q(sent.limit_down_count)+'</span></span></div>';
|
||||||
|
h+='<div class=\"metric\"><span>Broken Board</span><span>'+Q(sent.broken_board_count)+' ('+((sent.broken_board_rate||0)*100).toFixed(0)+'%)</span></div>';
|
||||||
|
h+='<div class=\"metric\"><span>Consecutive</span><span>'+Q(sent.consecutive_count)+' ('+((sent.consecutive_board_rate||0)*100).toFixed(0)+'%)</span></div>';
|
||||||
|
if(sent.max_consecutive)h+='<div class=\"metric\"><span>Max Board</span><span>'+sent.max_consecutive+' 连板</span></div>';
|
||||||
|
h+='</div>';
|
||||||
|
}
|
||||||
|
h+='</div></div>';
|
||||||
|
|
||||||
|
// Replay section
|
||||||
|
h+='<div id=\"replay-section\" style=\"margin-top:8px\"><span class=\"muted\" style=\"font-size:9px\">No intraday data (starts after first trading session)</span></div>';
|
||||||
|
|
||||||
|
document.getElementById('app').innerHTML=h;
|
||||||
|
|
||||||
|
// Load replay data
|
||||||
|
loadReplay(d.trade_date);
|
||||||
|
}
|
||||||
|
L();setInterval(L,60000);
|
||||||
|
|
||||||
|
// ═══ REPLAY ═══
|
||||||
|
function loadReplay(dateStr){
|
||||||
|
fetch('/api/v1/dashboard/replay?date_str='+dateStr)
|
||||||
|
.then(function(r){return r.json();})
|
||||||
|
.then(function(d){
|
||||||
|
if(d.snapshots&&d.snapshots.length>0)renderReplay(d.snapshots);
|
||||||
|
})
|
||||||
|
.catch(function(){});
|
||||||
|
}
|
||||||
|
function renderReplay(snaps){
|
||||||
|
if(snaps.length<2)return;
|
||||||
|
var el=document.getElementById('replay-section');
|
||||||
|
if(!el)return;
|
||||||
|
var h='<div class=\"sec-title\">REPLAY — Intraday Timeline</div>';
|
||||||
|
h+='<div style=\"display:flex;align-items:center;gap:8px;margin-bottom:4px\">';
|
||||||
|
h+='<input type=\"range\" min=\"0\" max=\"'+(snaps.length-1)+'\" value=\"'+(snaps.length-1)+'\" ';
|
||||||
|
h+='oninput=\"updateReplay(this.value)\" style=\"flex:1;accent-color:var(--accent)\" id=\"replay-slider\">';
|
||||||
|
h+='<span id=\"replay-time\" style=\"font-size:10px;color:var(--muted);width:45px\">'+snaps[snaps.length-1].timestamp.slice(11,16)+'</span></div>';
|
||||||
|
h+='<div id=\"replay-bars\" style=\"display:flex;flex-direction:column;gap:2px\">';
|
||||||
|
var last=snaps[snaps.length-1].dimensions;
|
||||||
|
for(var k in last){
|
||||||
|
h+='<div style=\"display:flex;align-items:center;gap:6px\"><span style=\"font-size:9px;color:var(--muted);width:55px\">'+k+'</span>';
|
||||||
|
h+='<div style=\"flex:1;height:8px;background:rgba(255,255,255,.06);border-radius:2px;overflow:hidden\">';
|
||||||
|
h+='<div id=\"replay-'+k+'\" style=\"height:100%;background:var(--accent);border-radius:2px;width:'+(last[k]*100).toFixed(0)+'%\" class=\"rp-bar\"></div></div>';
|
||||||
|
h+='<span id=\"replay-val-'+k+'\" style=\"font-size:9px;width:24px;text-align:right\">'+(last[k]*100).toFixed(0)+'</span></div>';
|
||||||
|
}
|
||||||
|
h+='</div>';
|
||||||
|
el.innerHTML=h;
|
||||||
|
// Store snaps for slider updates
|
||||||
|
window._replaySnaps=snaps;
|
||||||
|
}
|
||||||
|
function updateReplay(idx){
|
||||||
|
var snaps=window._replaySnaps;
|
||||||
|
if(!snaps)return;
|
||||||
|
var s=snaps[parseInt(idx)];
|
||||||
|
document.getElementById('replay-time').textContent=s.timestamp.slice(11,16);
|
||||||
|
var dims=s.dimensions;
|
||||||
|
for(var k in dims){
|
||||||
|
var bar=document.getElementById('replay-'+k);
|
||||||
|
var val=document.getElementById('replay-val-'+k);
|
||||||
|
if(bar)bar.style.width=(dims[k]*100).toFixed(0)+'%';
|
||||||
|
if(val)val.textContent=(dims[k]*100).toFixed(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
</body>
|
||||||
|
</html>"""
|
||||||
@@ -0,0 +1,235 @@
|
|||||||
|
"""Dashboard REST API — thin orchestrator.
|
||||||
|
|
||||||
|
Calls engines → assembles TradingBrief → serializes via brief_to_api_dict.
|
||||||
|
No inference, no manual JSON building.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
from fastapi import APIRouter, HTTPException
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.core.calendar import BEIJING_TZ
|
||||||
|
from ashare_dp.data.store.database import get_db, analytics_conn, kline_glob
|
||||||
|
from ashare_dp.domain.context import Allocation
|
||||||
|
from ashare_dp.execution.brief import build_brief, brief_to_api_dict, generate_narrative
|
||||||
|
from ashare_dp.execution.playbook import build_playbook
|
||||||
|
from ashare_dp.execution.risk import evaluate_risks
|
||||||
|
from ashare_dp.features import registry
|
||||||
|
from ashare_dp.market.flow import compute_flow
|
||||||
|
from ashare_dp.market.leadership import assess_leaders
|
||||||
|
from ashare_dp.market.memory import state_store
|
||||||
|
from ashare_dp.market.opportunity import rank_opportunities
|
||||||
|
from ashare_dp.market.knowledge import activate_themes
|
||||||
|
from ashare_dp.market.recommendations import get_recommendations
|
||||||
|
from ashare_dp.market.sentiment import assess_sentiment
|
||||||
|
from ashare_dp.market.state import infer_market_state
|
||||||
|
from ashare_dp.signals.expectancy import get_expectancy, get_all_expectancies
|
||||||
|
|
||||||
|
router = APIRouter(tags=["dashboard"])
|
||||||
|
|
||||||
|
|
||||||
|
def _get_latest_trade_date() -> date:
|
||||||
|
conn = analytics_conn()
|
||||||
|
try:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT MAX(trade_date) FROM read_parquet('{kline_glob()}', "
|
||||||
|
f"hive_partitioning=true, union_by_name=true)"
|
||||||
|
).fetchone()
|
||||||
|
if row and row[0]:
|
||||||
|
return date.fromisoformat(str(row[0])[:10])
|
||||||
|
raise HTTPException(status_code=503, detail="No K-line data available")
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/dashboard/state")
|
||||||
|
async def dashboard_state():
|
||||||
|
"""Complete market state — single atomic endpoint, versioned schema."""
|
||||||
|
trade_date = _get_latest_trade_date()
|
||||||
|
now = datetime.now(BEIJING_TZ)
|
||||||
|
|
||||||
|
# ── L1: Market Understanding ──
|
||||||
|
try:
|
||||||
|
features = registry.compute_all(None, trade_date)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Feature computation failed: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=f"Feature computation: {e}")
|
||||||
|
|
||||||
|
state = infer_market_state(features, trade_date, timestamp=now)
|
||||||
|
|
||||||
|
# ── L2: Market Interpretation (each engine independent, failures degrade) ──
|
||||||
|
leaders, opportunities, money_flow = {}, [], None
|
||||||
|
try:
|
||||||
|
leaders = assess_leaders(trade_date)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Leadership engine failed: {e}")
|
||||||
|
try:
|
||||||
|
opportunities = rank_opportunities(trade_date)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Opportunity engine failed: {e}")
|
||||||
|
try:
|
||||||
|
money_flow = compute_flow(trade_date)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Flow engine failed: {e}")
|
||||||
|
sentiment = assess_sentiment(trade_date)
|
||||||
|
|
||||||
|
# Knowledge Graph — theme activation
|
||||||
|
theme_graph = activate_themes(trade_date, leaders=leaders, opportunities=opportunities)
|
||||||
|
|
||||||
|
# Stock Recommendations
|
||||||
|
try:
|
||||||
|
recommendations = get_recommendations(trade_date, top_n=10)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Recommendations failed: {e}")
|
||||||
|
recommendations = []
|
||||||
|
|
||||||
|
# ── L3: Trading Intelligence ──
|
||||||
|
playbook = build_playbook(state, leaders, opportunities)
|
||||||
|
expectancy = get_expectancy(state) # fallback chain lives inside
|
||||||
|
all_expectancies = get_all_expectancies(state)
|
||||||
|
|
||||||
|
allocation = Allocation(
|
||||||
|
cash_pct=round(max(state.fear * 0.6, 0.1), 2),
|
||||||
|
trend_pct=round(max(state.trend * 0.6, 0.1), 2),
|
||||||
|
trial_pct=0.2,
|
||||||
|
reasoning="恐惧偏高,保留现金缓冲" if state.fear > 0.5 else "趋势明确,积极参与",
|
||||||
|
)
|
||||||
|
narrative = generate_narrative(state, leaders, opportunities, money_flow)
|
||||||
|
|
||||||
|
brief = build_brief(
|
||||||
|
trade_date=trade_date, state=state, playbook=playbook,
|
||||||
|
expectancy=expectancy, leaders=leaders, opportunities=opportunities,
|
||||||
|
money_flow=money_flow, allocation=allocation, narrative=narrative,
|
||||||
|
timestamp=now,
|
||||||
|
)
|
||||||
|
|
||||||
|
try:
|
||||||
|
risk_events = evaluate_risks(features, timestamp=now)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Risk evaluation failed: {e}")
|
||||||
|
risk_events = []
|
||||||
|
|
||||||
|
# Persist state snapshot for Market Memory
|
||||||
|
try:
|
||||||
|
with get_db(read_only=False) as db:
|
||||||
|
state_store.save(db, "market", "1d", now, state)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"StateStore save failed: {e}")
|
||||||
|
|
||||||
|
return brief_to_api_dict(brief, risk_events, features, sentiment, all_expectancies, theme_graph, recommendations)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/dashboard/trades")
|
||||||
|
async def record_trade(request: dict):
|
||||||
|
"""Record a trade for Trading Memory."""
|
||||||
|
if not request.get("ts_code") or not request.get("entry_price"):
|
||||||
|
raise HTTPException(status_code=400, detail="ts_code and entry_price are required")
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
with get_db(read_only=False) as db:
|
||||||
|
db.execute(
|
||||||
|
"""INSERT INTO trade_log (ts_code, trade_date, entry_price, signal_type,
|
||||||
|
position_pct, state_trend, state_fear, state_liquidity, state_breadth,
|
||||||
|
tags, notes)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
|
||||||
|
(request.get("ts_code"), request.get("trade_date"),
|
||||||
|
request.get("entry_price"), request.get("signal_type", ""),
|
||||||
|
request.get("position_pct", 0), request.get("state_trend", 0),
|
||||||
|
request.get("state_fear", 0), request.get("state_liquidity", 0),
|
||||||
|
request.get("state_breadth", 0), request.get("tags", ""),
|
||||||
|
request.get("notes", "")),
|
||||||
|
)
|
||||||
|
return {"status": "recorded"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/dashboard/trades")
|
||||||
|
async def list_trades(limit: int = 20, closed: bool | None = None):
|
||||||
|
"""List recent trades with performance stats."""
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
with get_db(read_only=True) as db:
|
||||||
|
wheres = []
|
||||||
|
params = []
|
||||||
|
if closed is not None:
|
||||||
|
wheres.append("closed = ?")
|
||||||
|
params.append(closed)
|
||||||
|
where = ("WHERE " + " AND ".join(wheres)) if wheres else ""
|
||||||
|
rows = db.query(
|
||||||
|
f"SELECT * FROM trade_log {where} ORDER BY created_at DESC LIMIT ?",
|
||||||
|
tuple(params) + (limit,),
|
||||||
|
)
|
||||||
|
# Stats
|
||||||
|
stats_rows = db.query(
|
||||||
|
"SELECT COUNT(*) as total, AVG(return_pct) as avg_ret, "
|
||||||
|
"AVG(CASE WHEN return_pct>0 THEN 1.0 ELSE 0.0 END) as win_rate "
|
||||||
|
"FROM trade_log WHERE closed=TRUE"
|
||||||
|
)
|
||||||
|
stats = {"total": 0, "avg_return": 0, "win_rate": 0}
|
||||||
|
if stats_rows and stats_rows[0][0] > 0:
|
||||||
|
stats = {"total": int(stats_rows[0][0]),
|
||||||
|
"avg_return": round(float(stats_rows[0][1] or 0), 2),
|
||||||
|
"win_rate": round(float(stats_rows[0][2] or 0), 4)}
|
||||||
|
|
||||||
|
trades = []
|
||||||
|
for r in rows:
|
||||||
|
trades.append({
|
||||||
|
"id": r[0], "ts_code": r[1], "trade_date": str(r[2]) if r[2] else "",
|
||||||
|
"entry_price": r[3], "signal_type": r[5],
|
||||||
|
"return_pct": r[10], "holding_days": r[11],
|
||||||
|
"closed": bool(r[20]), "tags": r[18], "notes": r[19],
|
||||||
|
})
|
||||||
|
return {"trades": trades, "stats": stats}
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/dashboard/trades/{trade_id}/close")
|
||||||
|
async def close_trade(trade_id: int, request: dict):
|
||||||
|
"""Close a trade with exit price and reason."""
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
with get_db(read_only=False) as db:
|
||||||
|
db.execute(
|
||||||
|
"""UPDATE trade_log SET exit_price=?, exit_reason=?, return_pct=?,
|
||||||
|
holding_days=?, closed=TRUE
|
||||||
|
WHERE id=?""",
|
||||||
|
(request.get("exit_price"), request.get("exit_reason", "manual"),
|
||||||
|
request.get("return_pct"), request.get("holding_days", 0), trade_id),
|
||||||
|
)
|
||||||
|
return {"status": "closed"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/dashboard/replay")
|
||||||
|
async def dashboard_replay(date_str: str = ""):
|
||||||
|
"""Return all intraday state snapshots for a given date (YYYY-MM-DD)."""
|
||||||
|
from datetime import date as _date
|
||||||
|
if not date_str:
|
||||||
|
date_str = _get_latest_trade_date().isoformat()
|
||||||
|
|
||||||
|
try:
|
||||||
|
with get_db(read_only=True) as db:
|
||||||
|
rows = db.query(
|
||||||
|
"SELECT timestamp, state FROM state_snapshot "
|
||||||
|
"WHERE entity = 'market' AND timeframe = '30min' "
|
||||||
|
"AND CAST(timestamp AS DATE) = ? "
|
||||||
|
"ORDER BY timestamp ASC",
|
||||||
|
(date_str,),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
rows = []
|
||||||
|
|
||||||
|
snapshots = []
|
||||||
|
for ts, state_json in rows:
|
||||||
|
try:
|
||||||
|
import json
|
||||||
|
state = json.loads(state_json)
|
||||||
|
snapshots.append({
|
||||||
|
"timestamp": ts.isoformat() if hasattr(ts, 'isoformat') else str(ts),
|
||||||
|
"dimensions": state.get("dimensions", {}),
|
||||||
|
})
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return {
|
||||||
|
"date": date_str,
|
||||||
|
"count": len(snapshots),
|
||||||
|
"snapshots": snapshots,
|
||||||
|
}
|
||||||
@@ -1,6 +1,6 @@
|
|||||||
"""FastAPI dependency injection."""
|
"""FastAPI dependency injection."""
|
||||||
|
|
||||||
from ashare_dp.storage.repository import KLineRepository
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
|
|
||||||
|
|
||||||
def get_repo() -> KLineRepository:
|
def get_repo() -> KLineRepository:
|
||||||
@@ -6,9 +6,9 @@ from typing import Optional
|
|||||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.api.deps import get_repo
|
from ashare_dp.apps.api.deps import get_repo
|
||||||
from ashare_dp.core.models import BACKFILLABLE_FREQS, DERIVED_FREQS, INTRADAY_FREQS, Freq
|
from ashare_dp.core.models import BACKFILLABLE_FREQS, DERIVED_FREQS, INTRADAY_FREQS, Freq
|
||||||
from ashare_dp.storage.repository import KLineRepository
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
|
|
||||||
router = APIRouter(prefix="/klines", tags=["klines"])
|
router = APIRouter(prefix="/klines", tags=["klines"])
|
||||||
|
|
||||||
@@ -6,8 +6,9 @@ from typing import Optional
|
|||||||
from fastapi import APIRouter, HTTPException, Query
|
from fastapi import APIRouter, HTTPException, Query
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.core.codes import to_ts_code
|
||||||
from ashare_dp.core.calendar import BEIJING_TZ, determine_market_state
|
from ashare_dp.core.calendar import BEIJING_TZ, determine_market_state
|
||||||
from ashare_dp.data.akshare_client import AKShareClient
|
from ashare_dp.data.sources.akshare_client import AKShareClient
|
||||||
|
|
||||||
router = APIRouter(prefix="/realtime", tags=["realtime"])
|
router = APIRouter(prefix="/realtime", tags=["realtime"])
|
||||||
|
|
||||||
@@ -45,14 +46,7 @@ async def get_spot(
|
|||||||
|
|
||||||
# Build ts_code from code
|
# Build ts_code from code
|
||||||
if "code" in df.columns:
|
if "code" in df.columns:
|
||||||
def _to_ts_code(c):
|
df["ts_code"] = df["code"].apply(to_ts_code)
|
||||||
c = str(c).zfill(6)
|
|
||||||
if c.startswith(("6", "9")):
|
|
||||||
return f"{c}.SH"
|
|
||||||
elif c.startswith(("8", "4")):
|
|
||||||
return f"{c}.BJ"
|
|
||||||
return f"{c}.SZ"
|
|
||||||
df["ts_code"] = df["code"].apply(_to_ts_code)
|
|
||||||
|
|
||||||
# Filter by requested codes
|
# Filter by requested codes
|
||||||
if codes:
|
if codes:
|
||||||
@@ -6,7 +6,7 @@ from typing import Optional
|
|||||||
from fastapi import APIRouter, HTTPException, Query
|
from fastapi import APIRouter, HTTPException, Query
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.storage.database import get_db
|
from ashare_dp.data.store.database import get_db
|
||||||
|
|
||||||
router = APIRouter(prefix="/stocks", tags=["stocks"])
|
router = APIRouter(prefix="/stocks", tags=["stocks"])
|
||||||
|
|
||||||
+1
-1
@@ -7,7 +7,7 @@ from datetime import datetime
|
|||||||
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
|
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.api.websocket.manager import manager
|
from ashare_dp.apps.api.websocket.manager import manager
|
||||||
from ashare_dp.core.calendar import BEIJING_TZ, determine_market_state
|
from ashare_dp.core.calendar import BEIJING_TZ, determine_market_state
|
||||||
|
|
||||||
router = APIRouter()
|
router = APIRouter()
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
"""Backfill CLI subcommands."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
import typer
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.core.models import BACKFILLABLE_FREQS, INTRADAY_FREQS
|
||||||
|
from ashare_dp.data.sources.akshare_client import AKShareClient
|
||||||
|
from ashare_dp.data.pipelines.backfill import BackfillPipeline
|
||||||
|
from ashare_dp.data.sources.index import backfill_indices
|
||||||
|
from ashare_dp.data.sources.industry import fetch_industry_mapping, store_industry_mapping
|
||||||
|
|
||||||
|
backfill_app = typer.Typer()
|
||||||
|
|
||||||
|
|
||||||
|
@backfill_app.command("init")
|
||||||
|
def init_db():
|
||||||
|
"""Initialize database schema."""
|
||||||
|
pipeline = BackfillPipeline()
|
||||||
|
pipeline.init_db()
|
||||||
|
pipeline.load_stock_list()
|
||||||
|
pipeline.load_trading_calendar()
|
||||||
|
typer.echo("Database initialized with stock list and trading calendar")
|
||||||
|
|
||||||
|
|
||||||
|
@backfill_app.command("daily")
|
||||||
|
def backfill_daily(
|
||||||
|
start: str = typer.Option("19900101", help="Start date YYYYMMDD"),
|
||||||
|
end: str = typer.Option(None, help="End date YYYYMMDD (default: today)"),
|
||||||
|
workers: int = typer.Option(10, help="Number of worker threads"),
|
||||||
|
symbols: str = typer.Option(None, help="Comma-separated stock symbols (default: all)"),
|
||||||
|
skip_index: bool = typer.Option(False, help="Skip index K-line backfill"),
|
||||||
|
):
|
||||||
|
"""Backfill daily/weekly/monthly K-line data."""
|
||||||
|
|
||||||
|
start_date = datetime.strptime(start, "%Y%m%d").date()
|
||||||
|
end_date = datetime.strptime(end, "%Y%m%d").date() if end else date.today()
|
||||||
|
|
||||||
|
sym_list = [s.strip() for s in symbols.split(",")] if symbols else None
|
||||||
|
|
||||||
|
pipeline = BackfillPipeline(max_workers=workers)
|
||||||
|
pipeline.init_db()
|
||||||
|
pipeline.load_stock_list()
|
||||||
|
pipeline.load_trading_calendar()
|
||||||
|
|
||||||
|
results = pipeline.backfill_daily_weekly_monthly(
|
||||||
|
symbols=sym_list,
|
||||||
|
start_date=start_date,
|
||||||
|
end_date=end_date,
|
||||||
|
)
|
||||||
|
|
||||||
|
typer.echo(f"\nBackfill complete:")
|
||||||
|
for freq, stats in results.items():
|
||||||
|
typer.echo(
|
||||||
|
f" {freq}: {stats['records']} records, "
|
||||||
|
f"{stats['completed']} stocks ok, {stats['failed']} failed"
|
||||||
|
)
|
||||||
|
|
||||||
|
# Index backfill
|
||||||
|
if not skip_index and sym_list is None:
|
||||||
|
typer.echo("\nBackfilling index data...")
|
||||||
|
idx_results = backfill_indices(
|
||||||
|
start_date=start_date.strftime("%Y%m%d"),
|
||||||
|
end_date=end_date.strftime("%Y%m%d"),
|
||||||
|
)
|
||||||
|
for ts_code, count in idx_results.items():
|
||||||
|
typer.echo(f" {ts_code}: {count} bars")
|
||||||
|
elif skip_index:
|
||||||
|
typer.echo("\nSkipping index backfill (--skip-index)")
|
||||||
|
|
||||||
|
|
||||||
|
@backfill_app.command("minute")
|
||||||
|
def backfill_minute(
|
||||||
|
days: int = typer.Option(30, help="Number of calendar days to look back"),
|
||||||
|
workers: int = typer.Option(5, help="Number of worker threads"),
|
||||||
|
symbols: str = typer.Option(None, help="Comma-separated stock symbols (default: all)"),
|
||||||
|
):
|
||||||
|
"""Backfill recent minute K-line data (limited API history)."""
|
||||||
|
|
||||||
|
sym_list = [s.strip() for s in symbols.split(",")] if symbols else None
|
||||||
|
|
||||||
|
pipeline = BackfillPipeline(max_workers=workers)
|
||||||
|
pipeline.init_db()
|
||||||
|
pipeline.load_stock_list()
|
||||||
|
|
||||||
|
results = pipeline.backfill_minute(
|
||||||
|
symbols=sym_list,
|
||||||
|
days_back=days,
|
||||||
|
)
|
||||||
|
|
||||||
|
typer.echo(f"\nMinute backfill complete ({days} day lookback):")
|
||||||
|
for freq, stats in results.items():
|
||||||
|
typer.echo(
|
||||||
|
f" {freq}: {stats['records']} records, "
|
||||||
|
f"{stats['completed']} stocks ok, {stats['failed']} failed"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@backfill_app.command("industry")
|
||||||
|
def backfill_industry():
|
||||||
|
"""Fetch and store industry classifications for all stocks."""
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
from ashare_dp.data.store.schema import DDL_STATEMENTS
|
||||||
|
|
||||||
|
typer.echo("Ensuring schema...")
|
||||||
|
with get_db(read_only=False) as db:
|
||||||
|
for ddl in DDL_STATEMENTS:
|
||||||
|
try:
|
||||||
|
db.execute(ddl)
|
||||||
|
except Exception:
|
||||||
|
pass # Table may already exist
|
||||||
|
|
||||||
|
typer.echo("Fetching industry classifications...")
|
||||||
|
df = fetch_industry_mapping()
|
||||||
|
count = store_industry_mapping(df)
|
||||||
|
typer.echo(f"Industry data stored: {count} stocks classified")
|
||||||
|
|
||||||
|
|
||||||
|
@backfill_app.command("signals")
|
||||||
|
def backfill_signals(
|
||||||
|
days: int = typer.Option(500, help="Calendar days to scan for historical signals"),
|
||||||
|
):
|
||||||
|
"""Detect and store historical trading signals (all types)."""
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
from ashare_dp.data.store.schema import DDL_STATEMENTS
|
||||||
|
from ashare_dp.signals.detectors import (
|
||||||
|
detect_ema52_signals, detect_vegas_signals,
|
||||||
|
detect_chan_signals, detect_orb_signals,
|
||||||
|
detect_gap_signals, detect_nr7_signals, detect_ib_signals,
|
||||||
|
)
|
||||||
|
from ashare_dp.signals.store import store_signals
|
||||||
|
|
||||||
|
typer.echo("Ensuring schema...")
|
||||||
|
with get_db(read_only=False) as db:
|
||||||
|
for ddl in DDL_STATEMENTS:
|
||||||
|
try:
|
||||||
|
db.execute(ddl)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
detectors = [
|
||||||
|
("EMA52", detect_ema52_signals),
|
||||||
|
("Vegas", detect_vegas_signals),
|
||||||
|
("Chan", detect_chan_signals),
|
||||||
|
("ORB", detect_orb_signals),
|
||||||
|
("Gap", detect_gap_signals),
|
||||||
|
("NR7", detect_nr7_signals),
|
||||||
|
("IB", detect_ib_signals),
|
||||||
|
]
|
||||||
|
|
||||||
|
total = 0
|
||||||
|
for name, detector in detectors:
|
||||||
|
typer.echo(f"Scanning {days} days for {name} signals...")
|
||||||
|
df = detector(lookback_days=days)
|
||||||
|
if df.empty:
|
||||||
|
typer.echo(f" {name}: 0 signals")
|
||||||
|
continue
|
||||||
|
count = store_signals(df)
|
||||||
|
total += count
|
||||||
|
by_type = df["signal_type"].value_counts().to_dict()
|
||||||
|
typer.echo(f" {name}: {count} stored {dict(by_type)}")
|
||||||
|
|
||||||
|
typer.echo(f"Total signals stored: {total}")
|
||||||
@@ -8,7 +8,7 @@ import typer
|
|||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.core.calendar import BEIJING_TZ
|
from ashare_dp.core.calendar import BEIJING_TZ
|
||||||
from ashare_dp.data.eod import EODPipeline
|
from ashare_dp.data.pipelines.eod import EODPipeline
|
||||||
|
|
||||||
eod_app = typer.Typer()
|
eod_app = typer.Typer()
|
||||||
|
|
||||||
@@ -4,11 +4,11 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import typer
|
import typer
|
||||||
|
|
||||||
from ashare_dp.cli.backfill_cmd import backfill_app
|
from ashare_dp.apps.cli.backfill_cmd import backfill_app
|
||||||
from ashare_dp.cli.serve_cmd import serve_app
|
from ashare_dp.apps.cli.serve_cmd import serve_app
|
||||||
from ashare_dp.cli.query_cmd import query_app
|
from ashare_dp.apps.cli.query_cmd import query_app
|
||||||
from ashare_dp.cli.eod_cmd import eod_app
|
from ashare_dp.apps.cli.eod_cmd import eod_app
|
||||||
from ashare_dp.cli.screening_cmd import screening_app
|
from ashare_dp.apps.cli.screening_cmd import screening_app
|
||||||
|
|
||||||
app = typer.Typer(
|
app = typer.Typer(
|
||||||
name="ashare-dp",
|
name="ashare-dp",
|
||||||
@@ -7,8 +7,8 @@ from datetime import date, datetime
|
|||||||
import typer
|
import typer
|
||||||
|
|
||||||
from ashare_dp.core.models import Freq
|
from ashare_dp.core.models import Freq
|
||||||
from ashare_dp.storage.database import get_db
|
from ashare_dp.data.store.database import get_db
|
||||||
from ashare_dp.storage.repository import KLineRepository
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
|
|
||||||
query_app = typer.Typer()
|
query_app = typer.Typer()
|
||||||
|
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
"""Screening CLI subcommand — EMA52 scanning and other screens."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import typer
|
||||||
|
|
||||||
|
screening_app = typer.Typer()
|
||||||
|
|
||||||
|
|
||||||
|
@screening_app.command("ema52")
|
||||||
|
def ema52(
|
||||||
|
threshold: float = typer.Option(
|
||||||
|
None, "--threshold", "-t",
|
||||||
|
help="EMA52 proximity threshold (e.g. 0.03 = ±3%%). Default from .env",
|
||||||
|
),
|
||||||
|
):
|
||||||
|
"""Run EMA52 screening for 1d and 1w."""
|
||||||
|
from ashare_dp.signals.detectors import run_ema52_screening
|
||||||
|
|
||||||
|
results = run_ema52_screening(threshold=threshold)
|
||||||
|
for freq, count in results.items():
|
||||||
|
typer.echo(f" {freq}: {count} stocks near EMA52")
|
||||||
|
typer.echo("EMA52 screening complete")
|
||||||
@@ -30,7 +30,7 @@ def serve(
|
|||||||
# The FastAPI lifespan handles DB init, scheduler start, and poller start
|
# The FastAPI lifespan handles DB init, scheduler start, and poller start
|
||||||
|
|
||||||
uvicorn.run(
|
uvicorn.run(
|
||||||
"ashare_dp.api.app:create_app",
|
"ashare_dp.apps.api.app:create_app",
|
||||||
host=h,
|
host=h,
|
||||||
port=p,
|
port=p,
|
||||||
reload=reload,
|
reload=reload,
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
"""Scheduler job implementations."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.core.calendar import BEIJING_TZ
|
||||||
|
from ashare_dp.core.models import Freq
|
||||||
|
from ashare_dp.data.pipelines.eod import EODPipeline
|
||||||
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
|
|
||||||
|
|
||||||
|
async def eod_pull_job():
|
||||||
|
"""End-of-day data pull. Triggered at 15:05 Beijing time on trading days."""
|
||||||
|
now = datetime.now(BEIJING_TZ)
|
||||||
|
today = now.date()
|
||||||
|
|
||||||
|
if now.weekday() >= 5:
|
||||||
|
logger.info("EOD: Weekend, skipping")
|
||||||
|
return
|
||||||
|
|
||||||
|
logger.info(f"EOD job starting for {today.isoformat()}...")
|
||||||
|
pipeline = EODPipeline()
|
||||||
|
|
||||||
|
try:
|
||||||
|
symbols = pipeline.get_active_symbols()
|
||||||
|
logger.info(f"EOD: {len(symbols)} active stocks")
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"EOD: Failed to get stock list: {e}")
|
||||||
|
return
|
||||||
|
|
||||||
|
try:
|
||||||
|
pipeline.pull_daily(symbols, today)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"EOD daily pull failed: {e}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
pipeline.pull_minute(symbols, today)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"EOD minute pull failed: {e}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
pipeline.pull_weekly_monthly(symbols, today)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"EOD weekly/monthly pull failed: {e}")
|
||||||
|
|
||||||
|
logger.info(f"EOD job completed for {today.isoformat()}")
|
||||||
|
|
||||||
|
|
||||||
|
async def health_check_job():
|
||||||
|
"""Daily health check: report database statistics."""
|
||||||
|
logger.info("Health check running...")
|
||||||
|
repo = KLineRepository()
|
||||||
|
try:
|
||||||
|
logger.info(
|
||||||
|
f"Health: 1d={repo.count_records(Freq.d1)}, "
|
||||||
|
f"1h={repo.count_records(Freq.h1)}"
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Health check failed: {e}")
|
||||||
|
|
||||||
|
|
||||||
|
async def ema52_screening_job():
|
||||||
|
"""EOD EMA52 screening — delegates to shared implementation."""
|
||||||
|
from ashare_dp.signals.detectors import run_ema52_screening
|
||||||
|
run_ema52_screening()
|
||||||
@@ -7,7 +7,7 @@ from apscheduler.triggers.cron import CronTrigger
|
|||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.core.calendar import BEIJING_TZ
|
from ashare_dp.core.calendar import BEIJING_TZ
|
||||||
from ashare_dp.data.akshare_client import AKShareClient
|
from ashare_dp.data.sources.akshare_client import AKShareClient
|
||||||
|
|
||||||
|
|
||||||
class Scheduler:
|
class Scheduler:
|
||||||
@@ -19,7 +19,7 @@ class Scheduler:
|
|||||||
|
|
||||||
def start(self):
|
def start(self):
|
||||||
"""Start the scheduler and register jobs."""
|
"""Start the scheduler and register jobs."""
|
||||||
from ashare_dp.scheduler.jobs import eod_pull_job, ema52_screening_job, health_check_job
|
from ashare_dp.apps.scheduler.jobs import eod_pull_job, ema52_screening_job, health_check_job
|
||||||
|
|
||||||
# EOD job: 15:05 Beijing time, Mon-Fri
|
# EOD job: 15:05 Beijing time, Mon-Fri
|
||||||
self._scheduler.add_job(
|
self._scheduler.add_job(
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
"""Backfill CLI subcommands."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from datetime import date, datetime
|
|
||||||
|
|
||||||
import typer
|
|
||||||
from loguru import logger
|
|
||||||
|
|
||||||
from ashare_dp.core.models import BACKFILLABLE_FREQS, INTRADAY_FREQS
|
|
||||||
from ashare_dp.data.akshare_client import AKShareClient
|
|
||||||
from ashare_dp.data.backfill import BackfillPipeline
|
|
||||||
|
|
||||||
backfill_app = typer.Typer()
|
|
||||||
|
|
||||||
|
|
||||||
@backfill_app.command("init")
|
|
||||||
def init_db():
|
|
||||||
"""Initialize database schema."""
|
|
||||||
pipeline = BackfillPipeline()
|
|
||||||
pipeline.init_db()
|
|
||||||
pipeline.load_stock_list()
|
|
||||||
pipeline.load_trading_calendar()
|
|
||||||
typer.echo("Database initialized with stock list and trading calendar")
|
|
||||||
|
|
||||||
|
|
||||||
@backfill_app.command("daily")
|
|
||||||
def backfill_daily(
|
|
||||||
start: str = typer.Option("19900101", help="Start date YYYYMMDD"),
|
|
||||||
end: str = typer.Option(None, help="End date YYYYMMDD (default: today)"),
|
|
||||||
workers: int = typer.Option(10, help="Number of worker threads"),
|
|
||||||
symbols: str = typer.Option(None, help="Comma-separated stock symbols (default: all)"),
|
|
||||||
):
|
|
||||||
"""Backfill daily/weekly/monthly K-line data."""
|
|
||||||
|
|
||||||
start_date = datetime.strptime(start, "%Y%m%d").date()
|
|
||||||
end_date = datetime.strptime(end, "%Y%m%d").date() if end else date.today()
|
|
||||||
|
|
||||||
sym_list = [s.strip() for s in symbols.split(",")] if symbols else None
|
|
||||||
|
|
||||||
pipeline = BackfillPipeline(max_workers=workers)
|
|
||||||
pipeline.init_db()
|
|
||||||
pipeline.load_stock_list()
|
|
||||||
pipeline.load_trading_calendar()
|
|
||||||
|
|
||||||
results = pipeline.backfill_daily_weekly_monthly(
|
|
||||||
symbols=sym_list,
|
|
||||||
start_date=start_date,
|
|
||||||
end_date=end_date,
|
|
||||||
)
|
|
||||||
|
|
||||||
typer.echo(f"\nBackfill complete:")
|
|
||||||
for freq, stats in results.items():
|
|
||||||
typer.echo(
|
|
||||||
f" {freq}: {stats['records']} records, "
|
|
||||||
f"{stats['completed']} stocks ok, {stats['failed']} failed"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@backfill_app.command("minute")
|
|
||||||
def backfill_minute(
|
|
||||||
days: int = typer.Option(30, help="Number of calendar days to look back"),
|
|
||||||
workers: int = typer.Option(5, help="Number of worker threads"),
|
|
||||||
symbols: str = typer.Option(None, help="Comma-separated stock symbols (default: all)"),
|
|
||||||
):
|
|
||||||
"""Backfill recent minute K-line data (limited API history)."""
|
|
||||||
|
|
||||||
sym_list = [s.strip() for s in symbols.split(",")] if symbols else None
|
|
||||||
|
|
||||||
pipeline = BackfillPipeline(max_workers=workers)
|
|
||||||
pipeline.init_db()
|
|
||||||
pipeline.load_stock_list()
|
|
||||||
|
|
||||||
results = pipeline.backfill_minute(
|
|
||||||
symbols=sym_list,
|
|
||||||
days_back=days,
|
|
||||||
)
|
|
||||||
|
|
||||||
typer.echo(f"\nMinute backfill complete ({days} day lookback):")
|
|
||||||
for freq, stats in results.items():
|
|
||||||
typer.echo(
|
|
||||||
f" {freq}: {stats['records']} records, "
|
|
||||||
f"{stats['completed']} stocks ok, {stats['failed']} failed"
|
|
||||||
)
|
|
||||||
@@ -1,168 +0,0 @@
|
|||||||
"""Screening CLI subcommand: EMA52 scanning and other screens."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import typer
|
|
||||||
from loguru import logger
|
|
||||||
|
|
||||||
from ashare_dp.storage.database import get_db
|
|
||||||
|
|
||||||
screening_app = typer.Typer()
|
|
||||||
|
|
||||||
|
|
||||||
@screening_app.command("ema52")
|
|
||||||
def ema52(
|
|
||||||
threshold: float = typer.Option(
|
|
||||||
None, "--threshold", "-t",
|
|
||||||
help="EMA52 proximity threshold (e.g. 0.03 = ±3%%). Default from .env",
|
|
||||||
),
|
|
||||||
):
|
|
||||||
"""Run EMA52 screening for 1d and 1w.
|
|
||||||
|
|
||||||
Scans all stocks, computes EMA52 from close prices,
|
|
||||||
and saves results near EMA52 to the ema52_screening table.
|
|
||||||
"""
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
from ashare_dp.config import Settings
|
|
||||||
from ashare_dp.core.models import Freq
|
|
||||||
|
|
||||||
settings = Settings()
|
|
||||||
if threshold is None:
|
|
||||||
threshold = settings.ema52_threshold
|
|
||||||
|
|
||||||
with get_db(read_only=False) as db:
|
|
||||||
# Get all ts_codes and names
|
|
||||||
stocks = db.query(
|
|
||||||
"SELECT ts_code, name FROM stock_info ORDER BY ts_code"
|
|
||||||
)
|
|
||||||
symbols = [row[0] for row in stocks]
|
|
||||||
symbol_names = {row[0]: row[1] for row in stocks}
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
f"EMA52 screening: {len(symbols)} stocks, "
|
|
||||||
f"threshold=±{threshold * 100:.1f}%"
|
|
||||||
)
|
|
||||||
|
|
||||||
for freq, freq_label, bars_needed in [
|
|
||||||
(Freq.d1, "1d", 100),
|
|
||||||
(Freq.w1, "1w", 60),
|
|
||||||
]:
|
|
||||||
try:
|
|
||||||
from ashare_dp.storage.partitioning import partition_glob
|
|
||||||
glob = partition_glob(freq)
|
|
||||||
sql = f"""
|
|
||||||
WITH ranked AS (
|
|
||||||
SELECT *, ROW_NUMBER() OVER (
|
|
||||||
PARTITION BY ts_code ORDER BY trade_time DESC
|
|
||||||
) as rn
|
|
||||||
FROM read_parquet('{glob}', hive_partitioning=true, union_by_name=true)
|
|
||||||
)
|
|
||||||
SELECT * FROM ranked WHERE rn <= {bars_needed}
|
|
||||||
ORDER BY ts_code, trade_time ASC
|
|
||||||
"""
|
|
||||||
df = _query_parquet(sql)
|
|
||||||
if df.empty:
|
|
||||||
logger.warning(f"EMA52: no data for {freq_label}")
|
|
||||||
continue
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EMA52: failed to read {freq_label}: {e}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
df["trade_time"] = pd.to_datetime(df["trade_time"])
|
|
||||||
df = df.sort_values(["ts_code", "trade_time"])
|
|
||||||
|
|
||||||
results = []
|
|
||||||
for ts_code, group in df.groupby("ts_code"):
|
|
||||||
if ts_code not in symbol_names:
|
|
||||||
continue
|
|
||||||
group = group.tail(bars_needed)
|
|
||||||
if len(group) < 26:
|
|
||||||
continue
|
|
||||||
|
|
||||||
closes = group["close"].astype(float).values
|
|
||||||
ema_values = _compute_ema(closes, 52)
|
|
||||||
if len(ema_values) == 0:
|
|
||||||
continue
|
|
||||||
|
|
||||||
latest_close = closes[-1]
|
|
||||||
latest_ema = ema_values[-1]
|
|
||||||
if latest_ema <= 0:
|
|
||||||
continue
|
|
||||||
distance = (latest_close - latest_ema) / latest_ema
|
|
||||||
|
|
||||||
# Compute daily amount in 亿 CNY
|
|
||||||
latest_amount = float(group["amount"].iloc[-1])
|
|
||||||
amount_yi = latest_amount / 100_000_000 # 元 → 亿
|
|
||||||
|
|
||||||
if abs(distance) <= threshold:
|
|
||||||
results.append({
|
|
||||||
"trade_date": group["trade_date"].iloc[-1],
|
|
||||||
"ts_code": ts_code,
|
|
||||||
"name": symbol_names[ts_code],
|
|
||||||
"freq": freq_label,
|
|
||||||
"close_price": round(latest_close, 2),
|
|
||||||
"ema52": round(latest_ema, 2),
|
|
||||||
"distance_pct": round(distance * 100, 2),
|
|
||||||
"amount": round(amount_yi, 2),
|
|
||||||
})
|
|
||||||
|
|
||||||
if results:
|
|
||||||
# Sort by amount DESC (largest turnover first)
|
|
||||||
results.sort(key=lambda r: r.get("amount", 0) or 0, reverse=True)
|
|
||||||
try:
|
|
||||||
with get_db(read_only=False) as wdb:
|
|
||||||
wdb.conn.execute(
|
|
||||||
"DELETE FROM ema52_screening WHERE freq = ?",
|
|
||||||
[freq_label],
|
|
||||||
)
|
|
||||||
wdb.conn.executemany(
|
|
||||||
"""INSERT INTO ema52_screening
|
|
||||||
(trade_date, ts_code, name, freq, close_price, ema52, distance_pct, amount)
|
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
|
||||||
ON CONFLICT (trade_date, ts_code, freq) DO UPDATE SET
|
|
||||||
close_price=excluded.close_price,
|
|
||||||
ema52=excluded.ema52,
|
|
||||||
distance_pct=excluded.distance_pct,
|
|
||||||
amount=excluded.amount,
|
|
||||||
name=excluded.name,
|
|
||||||
updated_at=now()""",
|
|
||||||
[(r["trade_date"], r["ts_code"], r["name"],
|
|
||||||
r["freq"], r["close_price"], r["ema52"], r["distance_pct"], r["amount"])
|
|
||||||
for r in results],
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EMA52: failed to save {freq_label}: {e}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
f"EMA52 {freq_label}: {len(results)} near "
|
|
||||||
f"(±{threshold * 100:.1f}%)"
|
|
||||||
)
|
|
||||||
|
|
||||||
logger.info("EMA52 screening complete")
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
def _query_parquet(sql: str) -> "pd.DataFrame":
|
|
||||||
"""Execute a SQL query against Parquet files using a transient in-memory DuckDB."""
|
|
||||||
import duckdb
|
|
||||||
import pandas as pd
|
|
||||||
conn = duckdb.connect()
|
|
||||||
try:
|
|
||||||
return conn.execute(sql).fetchdf()
|
|
||||||
finally:
|
|
||||||
conn.close()
|
|
||||||
|
|
||||||
|
|
||||||
def _compute_ema(values: "np.ndarray", period: int) -> "np.ndarray":
|
|
||||||
"""Compute EMA for a 1D array of values."""
|
|
||||||
import numpy as np
|
|
||||||
if len(values) < period:
|
|
||||||
return np.array([])
|
|
||||||
alpha = 2.0 / (period + 1)
|
|
||||||
ema = np.zeros(len(values))
|
|
||||||
ema[period - 1] = np.mean(values[:period])
|
|
||||||
for i in range(period, len(values)):
|
|
||||||
ema[i] = alpha * values[i] + (1 - alpha) * ema[i - 1]
|
|
||||||
return ema
|
|
||||||
@@ -0,0 +1,134 @@
|
|||||||
|
"""ts_code conversion — the single implementation for the entire platform.
|
||||||
|
|
||||||
|
All code ↔ ts_code ↔ sina_symbol conversions live here.
|
||||||
|
Every function is IDEMPOTENT: feeding an already-converted value
|
||||||
|
returns it unchanged (this prevents the double-suffix corruption
|
||||||
|
that previously produced values like "000001.SZ.SZ").
|
||||||
|
|
||||||
|
Formats:
|
||||||
|
- bare code: "000001" (6-digit numeric string)
|
||||||
|
- ts_code: "000001.SZ" (code + exchange suffix)
|
||||||
|
- sina symbol: "sz000001" (lowercase exchange prefix + code)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
_TS_CODE_RE = re.compile(r"^(\d{6})\.(SH|SZ|BJ)$")
|
||||||
|
_SINA_RE = re.compile(r"^(sh|sz|bj)(\d{6})$")
|
||||||
|
|
||||||
|
VALID_SUFFIXES = (".SH", ".SZ", ".BJ")
|
||||||
|
|
||||||
|
|
||||||
|
def exchange_for_code(code: str) -> str:
|
||||||
|
"""Determine exchange suffix from a bare 6-digit code.
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- 4xxxxx, 8xxxxx, 92xxxx → BJ (Beijing)
|
||||||
|
- 6xxxxx, 9xxxxx → SH (Shanghai)
|
||||||
|
- everything else → SZ (Shenzhen)
|
||||||
|
"""
|
||||||
|
code = str(code).strip()
|
||||||
|
if code.startswith(("4", "8")) or code.startswith("92"):
|
||||||
|
return "BJ"
|
||||||
|
if code.startswith(("6", "9")):
|
||||||
|
return "SH"
|
||||||
|
return "SZ"
|
||||||
|
|
||||||
|
|
||||||
|
def to_ts_code(value: str) -> str:
|
||||||
|
"""Convert any supported format to ts_code. IDEMPOTENT.
|
||||||
|
|
||||||
|
"000001" → "000001.SZ"
|
||||||
|
"000001.SZ" → "000001.SZ" (unchanged — never double-suffixes)
|
||||||
|
"sz000001" → "000001.SZ"
|
||||||
|
"000001.SZ.SZ" → "000001.SZ" (repairs legacy corruption)
|
||||||
|
"""
|
||||||
|
value = str(value).strip()
|
||||||
|
|
||||||
|
# Repair legacy double suffix first
|
||||||
|
value = normalize_ts_code(value)
|
||||||
|
|
||||||
|
# Already a valid ts_code?
|
||||||
|
if _TS_CODE_RE.match(value.upper()):
|
||||||
|
return value.upper()
|
||||||
|
|
||||||
|
# Sina symbol format?
|
||||||
|
m = _SINA_RE.match(value.lower())
|
||||||
|
if m:
|
||||||
|
prefix, code = m.groups()
|
||||||
|
return f"{code}.{prefix.upper()}"
|
||||||
|
|
||||||
|
# Bare numeric code
|
||||||
|
if value.isdigit():
|
||||||
|
code = value.zfill(6)
|
||||||
|
return f"{code}.{exchange_for_code(code)}"
|
||||||
|
|
||||||
|
# Unknown format — return as-is (caller's problem)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_ts_code(ts_code: str) -> str:
|
||||||
|
"""Strip duplicated exchange suffixes. IDEMPOTENT.
|
||||||
|
|
||||||
|
"000001.SZ.SZ" → "000001.SZ"
|
||||||
|
"000001.SZ" → "000001.SZ"
|
||||||
|
"""
|
||||||
|
ts_code = str(ts_code).strip()
|
||||||
|
upper = ts_code.upper()
|
||||||
|
for suffix in VALID_SUFFIXES:
|
||||||
|
doubled = suffix + suffix
|
||||||
|
if upper.endswith(doubled):
|
||||||
|
return upper[: -len(suffix)]
|
||||||
|
return ts_code
|
||||||
|
|
||||||
|
|
||||||
|
def to_sina_symbol(value: str) -> str:
|
||||||
|
"""Convert to Sina symbol format. IDEMPOTENT.
|
||||||
|
|
||||||
|
"000001.SZ" → "sz000001"
|
||||||
|
"000001" → "sz000001"
|
||||||
|
"sz000001" → "sz000001" (unchanged)
|
||||||
|
"""
|
||||||
|
value = str(value).strip()
|
||||||
|
if _SINA_RE.match(value.lower()):
|
||||||
|
return value.lower()
|
||||||
|
ts = to_ts_code(value)
|
||||||
|
m = _TS_CODE_RE.match(ts)
|
||||||
|
if m:
|
||||||
|
code, exchange = m.groups()
|
||||||
|
return f"{exchange.lower()}{code}"
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def bare_code(value: str) -> str:
|
||||||
|
"""Extract the bare 6-digit code from any format.
|
||||||
|
|
||||||
|
"000001.SZ" → "000001"
|
||||||
|
"sz000001" → "000001"
|
||||||
|
"""
|
||||||
|
ts = to_ts_code(value)
|
||||||
|
m = _TS_CODE_RE.match(ts)
|
||||||
|
if m:
|
||||||
|
return m.group(1)
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def derive_market(ts_code: str) -> str:
|
||||||
|
"""Derive market segment (板块) from ts_code.
|
||||||
|
|
||||||
|
6xxxxx.SH (non-688) → 主板
|
||||||
|
688xxx.SH → 科创板
|
||||||
|
0xxxxx.SZ → 主板
|
||||||
|
3xxxxx.SZ → 创业板
|
||||||
|
*.BJ → 北交所
|
||||||
|
"""
|
||||||
|
ts = to_ts_code(ts_code).upper()
|
||||||
|
if ts.endswith(".SH"):
|
||||||
|
return "科创板" if ts.startswith("688") else "主板"
|
||||||
|
if ts.endswith(".SZ"):
|
||||||
|
return "创业板" if ts.startswith("3") else "主板"
|
||||||
|
if ts.endswith(".BJ"):
|
||||||
|
return "北交所"
|
||||||
|
return "其他"
|
||||||
@@ -85,3 +85,30 @@ INTRADAY_FREQS: tuple[Freq, ...] = (Freq.m1, Freq.m5, Freq.m15, Freq.m30, Freq.h
|
|||||||
|
|
||||||
# Frequencies that are derived at query time
|
# Frequencies that are derived at query time
|
||||||
DERIVED_FREQS: tuple[Freq, ...] = (Freq.h2,)
|
DERIVED_FREQS: tuple[Freq, ...] = (Freq.h2,)
|
||||||
|
|
||||||
|
# ── Index codes ──
|
||||||
|
# ts_code format → Chinese name
|
||||||
|
INDEX_CODES: dict[str, str] = {
|
||||||
|
"000001.SH": "上证指数",
|
||||||
|
"399001.SZ": "深证成指",
|
||||||
|
"399006.SZ": "创业板指",
|
||||||
|
"000688.SH": "科创50",
|
||||||
|
"000300.SH": "沪深300",
|
||||||
|
"000905.SH": "中证500",
|
||||||
|
"000852.SH": "中证1000",
|
||||||
|
}
|
||||||
|
|
||||||
|
# Sina API symbol format for index backfill
|
||||||
|
INDEX_SINA_SYMBOLS: dict[str, str] = {
|
||||||
|
"sh000001": "000001.SH",
|
||||||
|
"sz399001": "399001.SZ",
|
||||||
|
"sz399006": "399006.SZ",
|
||||||
|
"sh000688": "000688.SH",
|
||||||
|
"sh000300": "000300.SH",
|
||||||
|
"sh000905": "000905.SH",
|
||||||
|
"sh000852": "000852.SH",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# derive_market moved to ashare_dp.core.codes
|
||||||
|
from ashare_dp.core.codes import derive_market # noqa: F401, E402
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
"""DATA PLATFORM — data acquisition, ETL, and storage. Knows nothing about trading."""
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
"""ETL pipelines: backfill, EOD, realtime."""
|
||||||
@@ -18,16 +18,17 @@ import pyarrow.parquet as pq
|
|||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.config import Settings
|
from ashare_dp.config import Settings
|
||||||
|
from ashare_dp.core.codes import to_ts_code
|
||||||
from ashare_dp.core.exceptions import StorageError
|
from ashare_dp.core.exceptions import StorageError
|
||||||
from ashare_dp.core.models import (
|
from ashare_dp.core.models import (
|
||||||
BACKFILLABLE_FREQS,
|
BACKFILLABLE_FREQS,
|
||||||
INTRADAY_FREQS,
|
INTRADAY_FREQS,
|
||||||
Freq,
|
Freq,
|
||||||
)
|
)
|
||||||
from ashare_dp.data.akshare_client import AKShareClient
|
from ashare_dp.data.sources.akshare_client import AKShareClient
|
||||||
from ashare_dp.storage.database import get_db
|
from ashare_dp.data.store.database import get_db
|
||||||
from ashare_dp.storage.repository import KLineRepository, _standardize_df
|
from ashare_dp.data.store.repository import KLineRepository, _standardize_df
|
||||||
from ashare_dp.storage.partitioning import partition_glob
|
from ashare_dp.data.store.partitioning import partition_glob
|
||||||
|
|
||||||
settings = Settings()
|
settings = Settings()
|
||||||
|
|
||||||
@@ -78,16 +79,6 @@ LEGACY_MIN_COLUMN_MAPPING = {
|
|||||||
REQUIRED_COLS = ["ts_code", "trade_time", "open", "high", "low", "close", "volume", "amount"]
|
REQUIRED_COLS = ["ts_code", "trade_time", "open", "high", "low", "close", "volume", "amount"]
|
||||||
|
|
||||||
|
|
||||||
def _to_ts_code(symbol: str) -> str:
|
|
||||||
"""Convert 6-digit symbol to ts_code with exchange suffix."""
|
|
||||||
code = str(symbol).zfill(6)
|
|
||||||
if code.startswith(("4", "8")) or code.startswith("92"):
|
|
||||||
return f"{code}.BJ"
|
|
||||||
elif code.startswith("6") or code.startswith("9"):
|
|
||||||
return f"{code}.SH"
|
|
||||||
return f"{code}.SZ"
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_hist_df(df: pd.DataFrame, symbol: str, freq: Freq) -> pd.DataFrame:
|
def _normalize_hist_df(df: pd.DataFrame, symbol: str, freq: Freq) -> pd.DataFrame:
|
||||||
"""Normalize daily K-line output to standard K-line schema."""
|
"""Normalize daily K-line output to standard K-line schema."""
|
||||||
df = df.copy()
|
df = df.copy()
|
||||||
@@ -103,7 +94,7 @@ def _normalize_hist_df(df: pd.DataFrame, symbol: str, freq: Freq) -> pd.DataFram
|
|||||||
break
|
break
|
||||||
|
|
||||||
if "ts_code" not in df.columns:
|
if "ts_code" not in df.columns:
|
||||||
df["ts_code"] = _to_ts_code(symbol)
|
df["ts_code"] = to_ts_code(symbol)
|
||||||
|
|
||||||
if "trade_time" in df.columns:
|
if "trade_time" in df.columns:
|
||||||
df["trade_time"] = pd.to_datetime(df["trade_time"])
|
df["trade_time"] = pd.to_datetime(df["trade_time"])
|
||||||
@@ -135,7 +126,7 @@ def _normalize_min_df(df: pd.DataFrame, symbol: str) -> pd.DataFrame:
|
|||||||
break
|
break
|
||||||
|
|
||||||
if "ts_code" not in df.columns:
|
if "ts_code" not in df.columns:
|
||||||
df["ts_code"] = _to_ts_code(symbol)
|
df["ts_code"] = to_ts_code(symbol)
|
||||||
|
|
||||||
if "trade_time" in df.columns:
|
if "trade_time" in df.columns:
|
||||||
df["trade_time"] = pd.to_datetime(df["trade_time"])
|
df["trade_time"] = pd.to_datetime(df["trade_time"])
|
||||||
@@ -189,16 +180,6 @@ def _resample_daily_to_period(df: pd.DataFrame, freq: Freq) -> pd.DataFrame:
|
|||||||
return result
|
return result
|
||||||
|
|
||||||
|
|
||||||
def _code_to_ts_code(code: str) -> str:
|
|
||||||
"""Convert bare 6-digit code to ts_code format: '000001' -> '000001.SZ'."""
|
|
||||||
code = str(code).zfill(6)
|
|
||||||
if code.startswith(("4", "8")) or code.startswith("92"):
|
|
||||||
return f"{code}.BJ"
|
|
||||||
elif code.startswith("6") or code.startswith("9"):
|
|
||||||
return f"{code}.SH"
|
|
||||||
return f"{code}.SZ"
|
|
||||||
|
|
||||||
|
|
||||||
WRITE_BATCH = 500 # Write to disk every N stocks to limit memory
|
WRITE_BATCH = 500 # Write to disk every N stocks to limit memory
|
||||||
|
|
||||||
|
|
||||||
@@ -217,7 +198,7 @@ class BackfillPipeline:
|
|||||||
|
|
||||||
def init_db(self):
|
def init_db(self):
|
||||||
"""Initialize database schema (tables and views)."""
|
"""Initialize database schema (tables and views)."""
|
||||||
from ashare_dp.storage.schema import DDL_STATEMENTS
|
from ashare_dp.data.store.schema import DDL_STATEMENTS
|
||||||
|
|
||||||
logger.info("Initializing database schema...")
|
logger.info("Initializing database schema...")
|
||||||
with get_db(read_only=False) as db:
|
with get_db(read_only=False) as db:
|
||||||
@@ -394,8 +375,8 @@ class BackfillPipeline:
|
|||||||
chunk = pd.concat(batch_frames, ignore_index=True)
|
chunk = pd.concat(batch_frames, ignore_index=True)
|
||||||
# Standardize: add freq, keep only standard columns
|
# Standardize: add freq, keep only standard columns
|
||||||
chunk["freq"] = Freq.d1.value
|
chunk["freq"] = Freq.d1.value
|
||||||
# Convert bare codes (000001) to ts_code format (000001.SZ)
|
# Normalize ts_code (idempotent — never double-suffixes)
|
||||||
chunk["ts_code"] = chunk["ts_code"].astype(str).apply(_code_to_ts_code)
|
chunk["ts_code"] = chunk["ts_code"].astype(str).apply(to_ts_code)
|
||||||
chunk = _standardize_df(chunk)
|
chunk = _standardize_df(chunk)
|
||||||
total_records += len(chunk)
|
total_records += len(chunk)
|
||||||
table = pa.Table.from_pandas(chunk, preserve_index=False)
|
table = pa.Table.from_pandas(chunk, preserve_index=False)
|
||||||
@@ -451,6 +432,7 @@ class BackfillPipeline:
|
|||||||
logger.info(f"Merging {batch_idx} temp batches into final parquet...")
|
logger.info(f"Merging {batch_idx} temp batches into final parquet...")
|
||||||
t0 = time.monotonic()
|
t0 = time.monotonic()
|
||||||
temp_glob = str(temp_dir / "batch_*.parquet")
|
temp_glob = str(temp_dir / "batch_*.parquet")
|
||||||
|
import duckdb
|
||||||
copy_conn = duckdb.connect()
|
copy_conn = duckdb.connect()
|
||||||
copy_conn.execute(f"""
|
copy_conn.execute(f"""
|
||||||
COPY (
|
COPY (
|
||||||
@@ -535,6 +517,7 @@ class BackfillPipeline:
|
|||||||
logger.info(f"Deriving {freq.value} with single DuckDB scan...")
|
logger.info(f"Deriving {freq.value} with single DuckDB scan...")
|
||||||
t0 = time.monotonic()
|
t0 = time.monotonic()
|
||||||
try:
|
try:
|
||||||
|
import duckdb
|
||||||
derive_conn = duckdb.connect()
|
derive_conn = duckdb.connect()
|
||||||
df = derive_conn.execute(sql).fetchdf()
|
df = derive_conn.execute(sql).fetchdf()
|
||||||
derive_conn.close()
|
derive_conn.close()
|
||||||
@@ -14,9 +14,9 @@ from loguru import logger
|
|||||||
|
|
||||||
from ashare_dp.core.calendar import BEIJING_TZ
|
from ashare_dp.core.calendar import BEIJING_TZ
|
||||||
from ashare_dp.core.models import BACKFILLABLE_FREQS, INTRADAY_FREQS, Freq
|
from ashare_dp.core.models import BACKFILLABLE_FREQS, INTRADAY_FREQS, Freq
|
||||||
from ashare_dp.data.akshare_client import AKShareClient
|
from ashare_dp.data.sources.akshare_client import AKShareClient
|
||||||
from ashare_dp.data.backfill import _normalize_hist_df, _normalize_min_df, _resample_daily_to_period
|
from ashare_dp.data.pipelines.backfill import _normalize_hist_df, _normalize_min_df, _resample_daily_to_period
|
||||||
from ashare_dp.storage.repository import KLineRepository
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
|
|
||||||
|
|
||||||
class EODPipeline:
|
class EODPipeline:
|
||||||
@@ -11,32 +11,13 @@ from datetime import datetime
|
|||||||
|
|
||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.api.websocket.manager import manager
|
from ashare_dp.core.codes import to_ts_code
|
||||||
|
|
||||||
|
from ashare_dp.apps.api.websocket.manager import manager
|
||||||
from ashare_dp.core.calendar import BEIJING_TZ, MarketState, determine_market_state
|
from ashare_dp.core.calendar import BEIJING_TZ, MarketState, determine_market_state
|
||||||
from ashare_dp.data.akshare_client import AKShareClient
|
from ashare_dp.data.sources.akshare_client import AKShareClient
|
||||||
|
|
||||||
|
|
||||||
def _code_to_ts_code(code: str) -> str:
|
|
||||||
"""Convert a raw stock code to ts_code format.
|
|
||||||
|
|
||||||
Handles both Sina format (with exchange prefix, e.g. 'sh600000')
|
|
||||||
and numeric-only format (e.g. '000001').
|
|
||||||
"""
|
|
||||||
code = str(code)
|
|
||||||
# Sina format: 'sh600000', 'sz000001', 'bj920000'
|
|
||||||
if code[:2].isalpha() and len(code) == 8:
|
|
||||||
prefix = code[:2].lower()
|
|
||||||
num = code[2:]
|
|
||||||
exchange_map = {"sh": "SH", "sz": "SZ", "bj": "BJ"}
|
|
||||||
return f"{num}.{exchange_map.get(prefix, prefix.upper())}"
|
|
||||||
# Numeric format
|
|
||||||
code = code.zfill(6)
|
|
||||||
if code.startswith(("4", "8")) or code.startswith("92"):
|
|
||||||
return f"{code}.BJ"
|
|
||||||
elif code.startswith("6") or code.startswith("9"):
|
|
||||||
return f"{code}.SH"
|
|
||||||
return f"{code}.SZ"
|
|
||||||
|
|
||||||
|
|
||||||
class RealtimePoller:
|
class RealtimePoller:
|
||||||
"""Background task that polls market data and broadcasts to subscribers."""
|
"""Background task that polls market data and broadcasts to subscribers."""
|
||||||
@@ -50,6 +31,7 @@ class RealtimePoller:
|
|||||||
self._poll_interval = poll_interval
|
self._poll_interval = poll_interval
|
||||||
self._running = False
|
self._running = False
|
||||||
self._task: asyncio.Task | None = None
|
self._task: asyncio.Task | None = None
|
||||||
|
self._last_snapshot: datetime | None = None
|
||||||
|
|
||||||
async def start(self):
|
async def start(self):
|
||||||
"""Start the polling loop."""
|
"""Start the polling loop."""
|
||||||
@@ -105,7 +87,7 @@ class RealtimePoller:
|
|||||||
if df is not None and not df.empty:
|
if df is not None and not df.empty:
|
||||||
# Normalize and filter
|
# Normalize and filter
|
||||||
if "代码" in df.columns:
|
if "代码" in df.columns:
|
||||||
df["ts_code"] = df["代码"].apply(_code_to_ts_code)
|
df["ts_code"] = df["代码"].apply(to_ts_code)
|
||||||
|
|
||||||
spot_data = {}
|
spot_data = {}
|
||||||
for _, row in df.iterrows():
|
for _, row in df.iterrows():
|
||||||
@@ -129,6 +111,17 @@ class RealtimePoller:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.error(f"Spot poll error: {e}")
|
logger.error(f"Spot poll error: {e}")
|
||||||
|
|
||||||
|
# Intraday state snapshot every 30 minutes
|
||||||
|
if state == MarketState.TRADING and (
|
||||||
|
self._last_snapshot is None
|
||||||
|
or (now - self._last_snapshot).total_seconds() >= 1800
|
||||||
|
):
|
||||||
|
try:
|
||||||
|
await asyncio.to_thread(_save_intraday_snapshot, now)
|
||||||
|
self._last_snapshot = now
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Intraday snapshot failed: {e}")
|
||||||
|
|
||||||
# Heartbeat every 30 seconds
|
# Heartbeat every 30 seconds
|
||||||
if (now - last_heartbeat).total_seconds() >= 30:
|
if (now - last_heartbeat).total_seconds() >= 30:
|
||||||
await manager.broadcast({
|
await manager.broadcast({
|
||||||
@@ -144,5 +137,73 @@ class RealtimePoller:
|
|||||||
await asyncio.sleep(self._get_poll_interval(state))
|
await asyncio.sleep(self._get_poll_interval(state))
|
||||||
|
|
||||||
|
|
||||||
|
def _save_intraday_snapshot(now: datetime):
|
||||||
|
"""Compute and save a lightweight intraday MarketState from spot data."""
|
||||||
|
import json
|
||||||
|
from ashare_dp.data.store.database import analytics_conn
|
||||||
|
from ashare_dp.data.sources.akshare_client import AKShareClient
|
||||||
|
|
||||||
|
client = AKShareClient()
|
||||||
|
df = client.get_spot()
|
||||||
|
if df is None or df.empty:
|
||||||
|
return
|
||||||
|
|
||||||
|
# Lightweight breadth from spot: advance/decline from pct_chg
|
||||||
|
pct_col = None
|
||||||
|
for c in ["涨跌幅", "pct_chg"]:
|
||||||
|
if c in df.columns:
|
||||||
|
pct_col = c
|
||||||
|
break
|
||||||
|
if pct_col is None:
|
||||||
|
return
|
||||||
|
|
||||||
|
pct_vals = df[pct_col].dropna().astype(float)
|
||||||
|
total = len(pct_vals)
|
||||||
|
if total == 0:
|
||||||
|
return
|
||||||
|
advances = int((pct_vals > 0).sum())
|
||||||
|
declines = int((pct_vals < 0).sum())
|
||||||
|
advance_ratio = advances / total
|
||||||
|
|
||||||
|
# Compute volume if available
|
||||||
|
total_amount = 0.0
|
||||||
|
amt_col = None
|
||||||
|
for c in ["成交额", "amount"]:
|
||||||
|
if c in df.columns:
|
||||||
|
amt_col = c
|
||||||
|
break
|
||||||
|
if amt_col:
|
||||||
|
total_amount = float(df[amt_col].dropna().sum())
|
||||||
|
|
||||||
|
# Lightweight state
|
||||||
|
state_dict = {
|
||||||
|
"version": "1.0",
|
||||||
|
"source": "intraday_spot",
|
||||||
|
"timestamp": now.isoformat(),
|
||||||
|
"trade_date": now.date().isoformat(),
|
||||||
|
"dimensions": {
|
||||||
|
"trend": round(advance_ratio, 4),
|
||||||
|
"fear": round(1.0 - advance_ratio, 4),
|
||||||
|
"liquidity": 0.5,
|
||||||
|
"rotation": 0.0,
|
||||||
|
"participation": round(advance_ratio, 4),
|
||||||
|
"volatility": 0.0,
|
||||||
|
"breadth": round(advance_ratio, 4),
|
||||||
|
},
|
||||||
|
"confidence": 0.5,
|
||||||
|
"quality": 0.5,
|
||||||
|
}
|
||||||
|
|
||||||
|
conn = analytics_conn(read_only=False)
|
||||||
|
try:
|
||||||
|
conn.execute(
|
||||||
|
"INSERT OR REPLACE INTO state_snapshot (entity, timeframe, timestamp, state) "
|
||||||
|
"VALUES (?, ?, ?, ?)",
|
||||||
|
("market", "30min", now, json.dumps(state_dict, ensure_ascii=False)),
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
# Global poller instance
|
# Global poller instance
|
||||||
poller = RealtimePoller()
|
poller = RealtimePoller()
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
"""External data sources: akshare, index, industry."""
|
||||||
+3
-29
@@ -24,6 +24,7 @@ import urllib.request
|
|||||||
from loguru import logger
|
from loguru import logger
|
||||||
|
|
||||||
from ashare_dp.config import Settings
|
from ashare_dp.config import Settings
|
||||||
|
from ashare_dp.core.codes import to_sina_symbol, to_ts_code
|
||||||
from ashare_dp.core.exceptions import DataSourceError
|
from ashare_dp.core.exceptions import DataSourceError
|
||||||
|
|
||||||
settings = Settings()
|
settings = Settings()
|
||||||
@@ -66,33 +67,6 @@ def _to_akshare_date(d: date) -> str:
|
|||||||
return d.strftime("%Y%m%d")
|
return d.strftime("%Y%m%d")
|
||||||
|
|
||||||
|
|
||||||
def _code_to_sina_symbol(code: str) -> str:
|
|
||||||
"""Convert numeric stock code to Sina symbol format.
|
|
||||||
|
|
||||||
'000001' -> 'sz000001', '600000' -> 'sh600000', '920000' -> 'bj920000'
|
|
||||||
"""
|
|
||||||
code = str(code).zfill(6)
|
|
||||||
if code.startswith(("4", "8")) or code.startswith("92"):
|
|
||||||
return f"bj{code}"
|
|
||||||
elif code.startswith("6") or code.startswith("9"):
|
|
||||||
return f"sh{code}"
|
|
||||||
else:
|
|
||||||
return f"sz{code}"
|
|
||||||
|
|
||||||
|
|
||||||
def _sina_symbol_to_ts_code(symbol: str) -> str:
|
|
||||||
"""Convert Sina symbol to ts_code format.
|
|
||||||
|
|
||||||
'sh600000' -> '600000.SH', 'sz000001' -> '000001.SZ', 'bj920000' -> '920000.BJ'
|
|
||||||
"""
|
|
||||||
match = re.match(r'^([a-z]+)(\d{6})$', str(symbol))
|
|
||||||
if match:
|
|
||||||
prefix, code = match.groups()
|
|
||||||
exchange = prefix.upper()
|
|
||||||
return f"{code}.{exchange}"
|
|
||||||
return str(symbol)
|
|
||||||
|
|
||||||
|
|
||||||
class AKShareClient:
|
class AKShareClient:
|
||||||
"""Wrapper around akshare with retry, error handling, and dual backend.
|
"""Wrapper around akshare with retry, error handling, and dual backend.
|
||||||
|
|
||||||
@@ -228,7 +202,7 @@ class AKShareClient:
|
|||||||
self, symbol: str, start_date: str, end_date: str, adjust: str
|
self, symbol: str, start_date: str, end_date: str, adjust: str
|
||||||
) -> pd.DataFrame:
|
) -> pd.DataFrame:
|
||||||
import akshare as ak
|
import akshare as ak
|
||||||
sina_symbol = _code_to_sina_symbol(symbol)
|
sina_symbol = to_sina_symbol(symbol)
|
||||||
return self._retry(lambda: ak.stock_zh_a_daily(
|
return self._retry(lambda: ak.stock_zh_a_daily(
|
||||||
symbol=sina_symbol,
|
symbol=sina_symbol,
|
||||||
start_date=start_date,
|
start_date=start_date,
|
||||||
@@ -286,7 +260,7 @@ class AKShareClient:
|
|||||||
start_date: str | None, end_date: str | None,
|
start_date: str | None, end_date: str | None,
|
||||||
) -> pd.DataFrame:
|
) -> pd.DataFrame:
|
||||||
import akshare as ak
|
import akshare as ak
|
||||||
sina_symbol = _code_to_sina_symbol(symbol)
|
sina_symbol = to_sina_symbol(symbol)
|
||||||
df = self._retry(lambda: ak.stock_zh_a_minute(
|
df = self._retry(lambda: ak.stock_zh_a_minute(
|
||||||
symbol=sina_symbol, period=period,
|
symbol=sina_symbol, period=period,
|
||||||
))
|
))
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
"""Index daily K-line backfill via Sina API.
|
||||||
|
|
||||||
|
Fetches 7 major A-share indices and stores them in the same
|
||||||
|
Hive-partitioned Parquet format as stock K-lines.
|
||||||
|
|
||||||
|
Uses akshare.stock_zh_index_daily() which works via Sina
|
||||||
|
(verified accessible outside China).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.core.models import INDEX_SINA_SYMBOLS, Freq
|
||||||
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
|
|
||||||
|
|
||||||
|
def backfill_indices(
|
||||||
|
repo: KLineRepository | None = None,
|
||||||
|
start_date: str = "19900101",
|
||||||
|
end_date: str | None = None,
|
||||||
|
) -> dict[str, int]:
|
||||||
|
"""Fetch and store index daily K-line history.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
repo: KLineRepository instance. Created if None.
|
||||||
|
start_date: Start date in YYYYMMDD format.
|
||||||
|
end_date: End date in YYYYMMDD format (default: today).
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dict mapping ts_code → number of bars stored.
|
||||||
|
"""
|
||||||
|
if repo is None:
|
||||||
|
repo = KLineRepository()
|
||||||
|
|
||||||
|
if end_date is None:
|
||||||
|
end_date = date.today().strftime("%Y%m%d")
|
||||||
|
|
||||||
|
import akshare as ak
|
||||||
|
|
||||||
|
results: dict[str, int] = {}
|
||||||
|
|
||||||
|
for sina_sym, ts_code in INDEX_SINA_SYMBOLS.items():
|
||||||
|
try:
|
||||||
|
logger.info(f"Fetching index {sina_sym} ({ts_code})...")
|
||||||
|
df = ak.stock_zh_index_daily(symbol=sina_sym)
|
||||||
|
if df is None or df.empty:
|
||||||
|
logger.warning(f"No data for {sina_sym}")
|
||||||
|
results[ts_code] = 0
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Normalize columns to match K-line standard schema
|
||||||
|
df = _normalize_index_df(df, ts_code)
|
||||||
|
|
||||||
|
# Filter by date range
|
||||||
|
df["trade_date"] = pd.to_datetime(df["trade_date"])
|
||||||
|
df = df[(df["trade_date"] >= start_date) & (df["trade_date"] <= end_date)]
|
||||||
|
|
||||||
|
if df.empty:
|
||||||
|
logger.warning(f"No data in range for {sina_sym}")
|
||||||
|
results[ts_code] = 0
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Store
|
||||||
|
repo.write_klines(df, Freq.d1)
|
||||||
|
results[ts_code] = len(df)
|
||||||
|
logger.info(f" {sina_sym}: stored {len(df)} bars, "
|
||||||
|
f"{df['trade_date'].min().date()} ~ {df['trade_date'].max().date()}")
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Failed to backfill index {sina_sym}: {e}")
|
||||||
|
results[ts_code] = 0
|
||||||
|
|
||||||
|
total = sum(results.values())
|
||||||
|
logger.info(f"Index backfill complete: {total} total bars across {len(results)} indices")
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_index_df(df: pd.DataFrame, ts_code: str) -> pd.DataFrame:
|
||||||
|
"""Normalize index DataFrame to match stock K-line standard schema.
|
||||||
|
|
||||||
|
Sina index columns: date, open, high, low, close, volume
|
||||||
|
Missing: amount (not available for indices from Sina)
|
||||||
|
"""
|
||||||
|
df = df.copy()
|
||||||
|
|
||||||
|
# Standardize column names
|
||||||
|
col_map = {
|
||||||
|
"date": "trade_time",
|
||||||
|
"open": "open",
|
||||||
|
"high": "high",
|
||||||
|
"low": "low",
|
||||||
|
"close": "close",
|
||||||
|
"volume": "volume",
|
||||||
|
}
|
||||||
|
df = df.rename(columns=col_map)
|
||||||
|
|
||||||
|
# Add required columns
|
||||||
|
df["ts_code"] = ts_code
|
||||||
|
df["trade_date"] = pd.to_datetime(df["trade_time"])
|
||||||
|
df["freq"] = Freq.d1.value
|
||||||
|
|
||||||
|
# Index data from Sina doesn't have amount
|
||||||
|
if "amount" not in df.columns:
|
||||||
|
df["amount"] = 0.0
|
||||||
|
|
||||||
|
# Ensure volume is int
|
||||||
|
df["volume"] = df["volume"].fillna(0).astype("int64")
|
||||||
|
|
||||||
|
# Select and order standard columns
|
||||||
|
std_cols = ["ts_code", "trade_time", "trade_date", "open", "high", "low", "close", "volume", "amount", "freq"]
|
||||||
|
df = df[[c for c in std_cols if c in df.columns]]
|
||||||
|
|
||||||
|
return df
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
"""Industry classification data fetch and storage.
|
||||||
|
|
||||||
|
Fetches industry classifications from akshare and stores in DuckDB.
|
||||||
|
Used by rotation features and dashboard sector analysis.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.core.codes import to_ts_code
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_industry_mapping() -> pd.DataFrame:
|
||||||
|
"""Fetch stock-to-industry mapping from akshare.
|
||||||
|
|
||||||
|
Tries East Money first (richer data), falls back to Sina.
|
||||||
|
Returns DataFrame with columns: ts_code, industry_name
|
||||||
|
"""
|
||||||
|
logger.info("Fetching industry classifications...")
|
||||||
|
|
||||||
|
try:
|
||||||
|
import akshare as ak
|
||||||
|
# Try East Money industry constituent data
|
||||||
|
df = ak.stock_board_industry_cons_em()
|
||||||
|
if df is not None and not df.empty:
|
||||||
|
logger.info(f"East Money industry data: {len(df)} rows")
|
||||||
|
# Expected columns vary by akshare version
|
||||||
|
# Common: '代码', '名称', '所属行业' or similar
|
||||||
|
cols = df.columns.tolist()
|
||||||
|
logger.debug(f"Industry columns: {cols}")
|
||||||
|
|
||||||
|
# Normalize to ts_code + industry_name
|
||||||
|
code_col = None
|
||||||
|
name_col = None
|
||||||
|
|
||||||
|
for c in cols:
|
||||||
|
if "代码" in c or c.lower() in ("code", "symbol"):
|
||||||
|
code_col = c
|
||||||
|
if "行业" in c or c.lower() in ("industry", "board"):
|
||||||
|
name_col = c
|
||||||
|
|
||||||
|
if code_col and name_col:
|
||||||
|
result = pd.DataFrame({
|
||||||
|
"code": df[code_col].astype(str).str.zfill(6),
|
||||||
|
"industry_name": df[name_col].astype(str),
|
||||||
|
})
|
||||||
|
# Convert code to ts_code format
|
||||||
|
result["ts_code"] = result["code"].apply(to_ts_code)
|
||||||
|
result = result[["ts_code", "industry_name"]].drop_duplicates(subset=["ts_code"])
|
||||||
|
logger.info(f"Normalized {len(result)} stock-industry mappings")
|
||||||
|
return result
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"East Money industry fetch failed: {e}")
|
||||||
|
|
||||||
|
# Fallback: try Sina
|
||||||
|
try:
|
||||||
|
import akshare as ak
|
||||||
|
df = ak.stock_info_a_code_name()
|
||||||
|
if df is not None and not df.empty:
|
||||||
|
# Sina stock_info doesn't have industry, but we can try other endpoints
|
||||||
|
logger.info("Sina fallback: no industry data available from basic stock info")
|
||||||
|
except Exception as e2:
|
||||||
|
logger.warning(f"Sina fallback also failed: {e2}")
|
||||||
|
|
||||||
|
# Last resort: derive a simple industry grouping from stock names
|
||||||
|
logger.warning("No industry API available. Using minimal fallback.")
|
||||||
|
return _minimal_fallback()
|
||||||
|
|
||||||
|
|
||||||
|
def _minimal_fallback() -> pd.DataFrame:
|
||||||
|
"""Generate minimal industry mapping from existing stock_info.
|
||||||
|
|
||||||
|
Phase 1: if no API works, use market + simple heuristics.
|
||||||
|
The rotation module will fall back to market-based segments.
|
||||||
|
"""
|
||||||
|
with get_db(read_only=True) as db:
|
||||||
|
result = db.query("SELECT ts_code, name, market FROM stock_info")
|
||||||
|
rows = []
|
||||||
|
for ts_code, name, market in result:
|
||||||
|
industry = _guess_industry_from_name(name, ts_code)
|
||||||
|
rows.append({"ts_code": ts_code, "industry_name": industry})
|
||||||
|
|
||||||
|
df = pd.DataFrame(rows)
|
||||||
|
logger.info(f"Generated {len(df)} fallback industry mappings")
|
||||||
|
return df
|
||||||
|
|
||||||
|
|
||||||
|
def _guess_industry_from_name(name: str, ts_code: str) -> str:
|
||||||
|
"""Guess industry from stock name using keyword matching."""
|
||||||
|
name = str(name)
|
||||||
|
keywords = [
|
||||||
|
("银行", "银行"),
|
||||||
|
("证券", "证券"),
|
||||||
|
("保险", "保险"),
|
||||||
|
("房地产", "房地产"),
|
||||||
|
("医药", "医药"),
|
||||||
|
("汽车", "汽车"),
|
||||||
|
("钢铁", "钢铁"),
|
||||||
|
("煤炭", "煤炭"),
|
||||||
|
("电力", "电力"),
|
||||||
|
("石油", "石油"),
|
||||||
|
("化工", "化工"),
|
||||||
|
("有色", "有色"),
|
||||||
|
("食品", "食品饮料"),
|
||||||
|
("饮料", "食品饮料"),
|
||||||
|
("白酒", "白酒"),
|
||||||
|
("家电", "家电"),
|
||||||
|
("半导体", "半导体"),
|
||||||
|
("芯片", "半导体"),
|
||||||
|
("通信", "通信"),
|
||||||
|
("软件", "软件"),
|
||||||
|
("计算机", "计算机"),
|
||||||
|
("传媒", "传媒"),
|
||||||
|
("军工", "军工"),
|
||||||
|
("航空", "军工"),
|
||||||
|
("机器人", "机器人"),
|
||||||
|
("光伏", "光伏"),
|
||||||
|
("锂电", "新能源"),
|
||||||
|
("电池", "新能源"),
|
||||||
|
("风电", "新能源"),
|
||||||
|
("储能", "新能源"),
|
||||||
|
("新能源", "新能源"),
|
||||||
|
("AI", "AI"),
|
||||||
|
("算力", "AI"),
|
||||||
|
("光模块", "AI"),
|
||||||
|
("PCB", "PCB"),
|
||||||
|
("船舶", "船舶"),
|
||||||
|
("港口", "交通运输"),
|
||||||
|
("铁路", "交通运输"),
|
||||||
|
("公路", "交通运输"),
|
||||||
|
("航空运输", "交通运输"),
|
||||||
|
("建筑", "建筑"),
|
||||||
|
("建材", "建材"),
|
||||||
|
("环保", "环保"),
|
||||||
|
]
|
||||||
|
for keyword, industry in keywords:
|
||||||
|
if keyword in name:
|
||||||
|
return industry
|
||||||
|
return "其他"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
def store_industry_mapping(df: pd.DataFrame) -> int:
|
||||||
|
"""Store industry mappings in DuckDB stock_industry table.
|
||||||
|
|
||||||
|
Returns number of rows stored.
|
||||||
|
"""
|
||||||
|
with get_db(read_only=False) as db:
|
||||||
|
try:
|
||||||
|
for _, row in df.iterrows():
|
||||||
|
db.execute(
|
||||||
|
"INSERT OR REPLACE INTO stock_industry (ts_code, industry_name) VALUES (?, ?)",
|
||||||
|
(row["ts_code"], row["industry_name"]),
|
||||||
|
)
|
||||||
|
count_result = db.query("SELECT COUNT(*) FROM stock_industry")
|
||||||
|
count = count_result[0][0] if count_result else 0
|
||||||
|
logger.info(f"Stored {count} industry mappings in stock_industry")
|
||||||
|
return int(count)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Failed to store industry data: {e}")
|
||||||
|
return 0
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
"""Storage layer: DuckDB + Parquet."""
|
||||||
|
from ashare_dp.data.store.database import get_db, Database, analytics_conn
|
||||||
|
from ashare_dp.data.store.repository import KLineRepository
|
||||||
|
from ashare_dp.data.store.partitioning import partition_glob, partition_path
|
||||||
|
|
||||||
|
__all__ = ["get_db", "Database", "analytics_conn", "KLineRepository", "partition_glob", "partition_path"]
|
||||||
@@ -99,3 +99,31 @@ def get_db(read_only: bool = True) -> Iterator[Database]:
|
|||||||
yield db
|
yield db
|
||||||
finally:
|
finally:
|
||||||
db.close()
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
def analytics_conn(read_only: bool = True) -> "duckdb.DuckDBPyConnection":
|
||||||
|
"""Raw DuckDB connection for analytics queries (features, engines).
|
||||||
|
|
||||||
|
The single sanctioned way to open a DuckDB connection outside the
|
||||||
|
Database wrapper. Uses configured duckdb_path — never hardcode paths.
|
||||||
|
Caller must close() the connection.
|
||||||
|
"""
|
||||||
|
return duckdb.connect(settings.duckdb_path, read_only=read_only)
|
||||||
|
|
||||||
|
|
||||||
|
def kline_glob(freq=None) -> str:
|
||||||
|
"""Parquet glob for K-line data at the given frequency (default: daily).
|
||||||
|
|
||||||
|
Convenience re-export so analytics modules don't hardcode paths.
|
||||||
|
"""
|
||||||
|
from ashare_dp.core.models import Freq
|
||||||
|
from ashare_dp.data.store.partitioning import partition_glob
|
||||||
|
return partition_glob(freq or Freq.d1)
|
||||||
|
|
||||||
|
|
||||||
|
def read_parquet_sql(glob_var: str) -> str:
|
||||||
|
"""Standard read_parquet SQL snippet with hive partitioning.
|
||||||
|
|
||||||
|
Single place for the 28 duplicate 'read_parquet({glob}, hive_partitioning=true, union_by_name=true)' patterns.
|
||||||
|
"""
|
||||||
|
return f"read_parquet('{glob_var}', hive_partitioning=true, union_by_name=true)"
|
||||||
@@ -15,8 +15,8 @@ from loguru import logger
|
|||||||
from ashare_dp.config import Settings
|
from ashare_dp.config import Settings
|
||||||
from ashare_dp.core.exceptions import QueryError, StorageError
|
from ashare_dp.core.exceptions import QueryError, StorageError
|
||||||
from ashare_dp.core.models import DERIVED_FREQS, Freq
|
from ashare_dp.core.models import DERIVED_FREQS, Freq
|
||||||
from ashare_dp.storage.database import get_db
|
from ashare_dp.data.store.database import get_db
|
||||||
from ashare_dp.storage.partitioning import (
|
from ashare_dp.data.store.partitioning import (
|
||||||
ensure_partition_dir,
|
ensure_partition_dir,
|
||||||
partition_glob,
|
partition_glob,
|
||||||
)
|
)
|
||||||
@@ -0,0 +1,138 @@
|
|||||||
|
"""Database schema: DDL statements for DuckDB tables and views."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
# DDL for persistent tables
|
||||||
|
DDL_STATEMENTS = [
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS stock_info (
|
||||||
|
ts_code VARCHAR(9) PRIMARY KEY,
|
||||||
|
symbol VARCHAR(6) NOT NULL,
|
||||||
|
name VARCHAR(40) NOT NULL,
|
||||||
|
exchange VARCHAR(2) NOT NULL,
|
||||||
|
area VARCHAR(20),
|
||||||
|
industry VARCHAR(40),
|
||||||
|
list_date DATE,
|
||||||
|
delist_date DATE,
|
||||||
|
market VARCHAR(10),
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_stock_symbol ON stock_info(symbol)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_stock_exchange ON stock_info(exchange)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS trading_calendar (
|
||||||
|
trade_date DATE PRIMARY KEY,
|
||||||
|
is_trading_day BOOLEAN NOT NULL DEFAULT TRUE,
|
||||||
|
week_day TINYINT NOT NULL,
|
||||||
|
year SMALLINT NOT NULL,
|
||||||
|
month TINYINT NOT NULL
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE SEQUENCE IF NOT EXISTS seq_ema52_id
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS ema52_screening (
|
||||||
|
id BIGINT PRIMARY KEY DEFAULT nextval('seq_ema52_id'),
|
||||||
|
trade_date DATE NOT NULL,
|
||||||
|
ts_code VARCHAR(9) NOT NULL,
|
||||||
|
name VARCHAR(40),
|
||||||
|
freq VARCHAR(3) NOT NULL,
|
||||||
|
close_price DOUBLE NOT NULL,
|
||||||
|
ema52 DOUBLE NOT NULL,
|
||||||
|
distance_pct DOUBLE NOT NULL,
|
||||||
|
amount DOUBLE,
|
||||||
|
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
UNIQUE(trade_date, ts_code, freq)
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ema52_date ON ema52_screening(trade_date)
|
||||||
|
""",
|
||||||
|
# Migration: add amount column if upgrading from older schema
|
||||||
|
"""
|
||||||
|
ALTER TABLE ema52_screening ADD COLUMN IF NOT EXISTS amount DOUBLE
|
||||||
|
""",
|
||||||
|
# ── Dashboard / Trading OS tables ──
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS stock_industry (
|
||||||
|
ts_code VARCHAR(9) NOT NULL,
|
||||||
|
industry_name VARCHAR(40) NOT NULL,
|
||||||
|
industry_code VARCHAR(20),
|
||||||
|
PRIMARY KEY (ts_code)
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_stock_industry_name ON stock_industry(industry_name)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS state_snapshot (
|
||||||
|
entity VARCHAR(20) NOT NULL,
|
||||||
|
timeframe VARCHAR(10) NOT NULL,
|
||||||
|
timestamp TIMESTAMP NOT NULL,
|
||||||
|
state JSON NOT NULL,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
PRIMARY KEY (entity, timeframe, timestamp)
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
# ── Signal Intelligence ──
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS signal_instance (
|
||||||
|
id BIGINT PRIMARY KEY DEFAULT nextval('seq_signal_id'),
|
||||||
|
signal_type VARCHAR(30) NOT NULL,
|
||||||
|
ts_code VARCHAR(15) NOT NULL,
|
||||||
|
trade_date DATE NOT NULL,
|
||||||
|
signal_price DOUBLE NOT NULL,
|
||||||
|
-- Market state at signal time
|
||||||
|
state_trend DOUBLE,
|
||||||
|
state_fear DOUBLE,
|
||||||
|
state_liquidity DOUBLE,
|
||||||
|
state_rotation DOUBLE,
|
||||||
|
state_participation DOUBLE,
|
||||||
|
state_volatility DOUBLE,
|
||||||
|
state_breadth DOUBLE,
|
||||||
|
-- Forward outcomes
|
||||||
|
return_5d DOUBLE,
|
||||||
|
return_10d DOUBLE,
|
||||||
|
return_20d DOUBLE,
|
||||||
|
max_return_5d DOUBLE,
|
||||||
|
max_drawdown_5d DOUBLE,
|
||||||
|
holding_days INTEGER,
|
||||||
|
outcome_known BOOLEAN DEFAULT FALSE,
|
||||||
|
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
||||||
|
UNIQUE(signal_type, ts_code, trade_date)
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE SEQUENCE IF NOT EXISTS seq_signal_id
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_signal_type_date ON signal_instance(signal_type, trade_date)
|
||||||
|
""",
|
||||||
|
"""
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_signal_ts_code ON signal_instance(ts_code, trade_date)
|
||||||
|
""",
|
||||||
|
# ── Trading Memory ──
|
||||||
|
"""
|
||||||
|
CREATE TABLE IF NOT EXISTS trade_log (
|
||||||
|
id BIGINT PRIMARY KEY DEFAULT nextval('seq_trade_id'),
|
||||||
|
ts_code VARCHAR(15) NOT NULL, trade_date DATE NOT NULL,
|
||||||
|
entry_price DOUBLE NOT NULL, entry_time TIMESTAMP,
|
||||||
|
signal_type VARCHAR(30), position_pct DOUBLE DEFAULT 0,
|
||||||
|
exit_price DOUBLE, exit_time TIMESTAMP, exit_reason VARCHAR(20),
|
||||||
|
return_pct DOUBLE, holding_days INTEGER,
|
||||||
|
max_favorable DOUBLE, max_adverse DOUBLE,
|
||||||
|
state_trend DOUBLE, state_fear DOUBLE, state_liquidity DOUBLE, state_breadth DOUBLE,
|
||||||
|
tags VARCHAR(200), notes VARCHAR(500),
|
||||||
|
closed BOOLEAN DEFAULT FALSE, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
||||||
|
)
|
||||||
|
""",
|
||||||
|
"""CREATE SEQUENCE IF NOT EXISTS seq_trade_id""",
|
||||||
|
"""CREATE INDEX IF NOT EXISTS idx_trade_date ON trade_log(trade_date)""",
|
||||||
|
"""CREATE INDEX IF NOT EXISTS idx_trade_signal ON trade_log(signal_type)""",
|
||||||
|
]
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
"""Domain objects for the A-Share Data Platform.
|
||||||
|
|
||||||
|
These are the foundational types shared by all modules:
|
||||||
|
Feature Store, Market State Engine, Decision Engine, Dashboard, and future
|
||||||
|
Signal Expectancy Engine / AI Agent / Backtester.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from ashare_dp.domain.state import MarketState
|
||||||
|
from ashare_dp.domain.events import RiskEvent, RotationEdge, RotationGraph
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
from ashare_dp.domain.context import (
|
||||||
|
TradingContext, Playbook, Expectancy, Opportunity,
|
||||||
|
Allocation, FlowEdge, FlowGraph,
|
||||||
|
)
|
||||||
|
from ashare_dp.domain.leadership import LeaderState
|
||||||
|
|
||||||
|
from ashare_dp.domain.signal import SignalType, SignalInstance
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"MarketState",
|
||||||
|
"RiskEvent",
|
||||||
|
"RotationEdge",
|
||||||
|
"RotationGraph",
|
||||||
|
"FeatureDefinition",
|
||||||
|
"TradingContext",
|
||||||
|
"Playbook",
|
||||||
|
"Expectancy",
|
||||||
|
"Opportunity",
|
||||||
|
"Allocation",
|
||||||
|
"FlowEdge",
|
||||||
|
"FlowGraph",
|
||||||
|
"LeaderState",
|
||||||
|
"SignalType",
|
||||||
|
"SignalInstance",
|
||||||
|
]
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
"""TradingContext and related domain objects — the true center of the Trading OS.
|
||||||
|
|
||||||
|
MarketState describes the market. TradingContext describes a trading decision.
|
||||||
|
Signal Expectancy Engine queries history against this.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, date
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from ashare_dp.domain.state import MarketState
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Allocation:
|
||||||
|
"""Suggested capital allocation based on market state."""
|
||||||
|
cash_pct: float = 0.4
|
||||||
|
trend_pct: float = 0.4
|
||||||
|
trial_pct: float = 0.2
|
||||||
|
reasoning: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Playbook:
|
||||||
|
"""Complete trading playbook — not just a recommendation list.
|
||||||
|
|
||||||
|
Tells the trader: what strategies work, what to avoid,
|
||||||
|
how long to hold, how much to risk.
|
||||||
|
"""
|
||||||
|
suitable_strategies: list[str] = field(default_factory=list)
|
||||||
|
unsuitable_strategies: list[str] = field(default_factory=list)
|
||||||
|
holding_time: str = "2~4 Days"
|
||||||
|
risk_budget: str = "Medium"
|
||||||
|
position_sizing: str = "Standard"
|
||||||
|
focus_sectors: list[str] = field(default_factory=list)
|
||||||
|
avoid_sectors: list[str] = field(default_factory=list)
|
||||||
|
bias: str = "NEUTRAL"
|
||||||
|
confidence: float = 0.5
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Expectancy:
|
||||||
|
"""Historical expectancy for a signal in the current market context.
|
||||||
|
|
||||||
|
This is the moat. Query: Signal × MarketState → historical outcomes.
|
||||||
|
Phase 1: rule-based heuristic. Phase 2: real historical stats.
|
||||||
|
"""
|
||||||
|
signal_type: str = ""
|
||||||
|
win_rate: float = 0.0
|
||||||
|
avg_return: float = 0.0
|
||||||
|
max_drawdown: float = 0.0
|
||||||
|
avg_holding_days: float = 0.0
|
||||||
|
sample_count: int = 0
|
||||||
|
confidence: float = 0.0
|
||||||
|
similar_states: int = 0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Opportunity:
|
||||||
|
"""A trading opportunity — not an industry score, a tradable object.
|
||||||
|
|
||||||
|
AI Agent can consume this directly without re-reasoning.
|
||||||
|
"""
|
||||||
|
name: str = ""
|
||||||
|
score: float = 0.0
|
||||||
|
lifecycle: str = "" # "Early" | "Accelerating" | "Peak" | "Declining"
|
||||||
|
persistence_days: int = 0
|
||||||
|
catalyst: str = "" # "资金轮动" | "政策催化" | "业绩驱动"
|
||||||
|
leader_stock: str = ""
|
||||||
|
invalidation: str = "" # what would invalidate this opportunity
|
||||||
|
risk: str = ""
|
||||||
|
top_stocks: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class FlowEdge:
|
||||||
|
"""Money flow edge in the flow graph."""
|
||||||
|
source: str = ""
|
||||||
|
target: str = ""
|
||||||
|
magnitude: str = "" # "↓↓→↑↑"
|
||||||
|
strength: float = 0.0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class FlowGraph:
|
||||||
|
"""Money flow as a directed graph."""
|
||||||
|
flows: list[FlowEdge] = field(default_factory=list)
|
||||||
|
net_inflow: dict[str, float] = field(default_factory=dict)
|
||||||
|
net_outflow: dict[str, float] = field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TradingContext:
|
||||||
|
"""The universal context for any trading decision.
|
||||||
|
|
||||||
|
Signal Expectancy Engine queries history against this object.
|
||||||
|
Every downstream consumer (Dashboard, AI, Backtest) reads this,
|
||||||
|
not raw features or individual engine outputs.
|
||||||
|
"""
|
||||||
|
timestamp: datetime = field(default_factory=datetime.now)
|
||||||
|
trade_date: date = field(default_factory=date.today)
|
||||||
|
|
||||||
|
# L1: Market State
|
||||||
|
state: Optional[MarketState] = None
|
||||||
|
|
||||||
|
# L2: Interpretation
|
||||||
|
leaders: dict = field(default_factory=dict) # industry → LeaderState
|
||||||
|
opportunities: list[Opportunity] = field(default_factory=list)
|
||||||
|
money_flow: Optional[FlowGraph] = None
|
||||||
|
|
||||||
|
# L3: Intelligence
|
||||||
|
playbook: Optional[Playbook] = None
|
||||||
|
expectancy: Optional[Expectancy] = None
|
||||||
|
allocation: Optional[Allocation] = None
|
||||||
|
risk_budget: str = "Medium"
|
||||||
|
|
||||||
|
# Narrative
|
||||||
|
narrative: str = ""
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
"""RiskEvent, RotationEdge, RotationGraph — structured domain events.
|
||||||
|
|
||||||
|
Dashboard renders them, AI explains them, Backtest counts them.
|
||||||
|
NOT raw dicts or strings.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RiskEvent:
|
||||||
|
"""Structured risk warning emitted by the Risk Rule Engine.
|
||||||
|
|
||||||
|
Dashboard renders the message with color-coded severity.
|
||||||
|
AI Agent can explain the event context.
|
||||||
|
Backtester can count event frequency by type.
|
||||||
|
"""
|
||||||
|
|
||||||
|
type: str # "BreadthCollapse", "VolumeDrying", "LeaderFailure", etc.
|
||||||
|
severity: float # 0.0–1.0, how severe
|
||||||
|
level: str # "warning" | "danger"
|
||||||
|
affected: str # "All" | "主板" | "AI" | specific entity
|
||||||
|
confidence: float # 0.0–1.0, how confident the rule is in this detection
|
||||||
|
message: str # human-readable description
|
||||||
|
payload: dict = field(default_factory=dict) # type-specific data
|
||||||
|
timestamp: datetime = field(default_factory=datetime.now)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
return {
|
||||||
|
"type": self.type,
|
||||||
|
"severity": round(self.severity, 4),
|
||||||
|
"level": self.level,
|
||||||
|
"affected": self.affected,
|
||||||
|
"confidence": round(self.confidence, 4),
|
||||||
|
"message": self.message,
|
||||||
|
"timestamp": self.timestamp.isoformat(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RotationEdge:
|
||||||
|
"""Directed edge in the rotation graph.
|
||||||
|
|
||||||
|
Represents capital flowing FROM one sector TO another,
|
||||||
|
inferred from turnover changes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
source: str # sector name, e.g. "AI"
|
||||||
|
target: str # sector name, e.g. "机器人"
|
||||||
|
strength: float # 0.0–1.0, how strong the flow
|
||||||
|
confidence: float # 0.0–1.0
|
||||||
|
source_change: float # turnover change % for source (negative = outflow)
|
||||||
|
target_change: float # turnover change % for target (positive = inflow)
|
||||||
|
persistence_days: int # how many consecutive days this flow persisted
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RotationGraph:
|
||||||
|
"""Sector rotation as a directed graph.
|
||||||
|
|
||||||
|
Nodes = industries/concepts.
|
||||||
|
Edges = capital flow direction + strength.
|
||||||
|
|
||||||
|
Future: AI can do graph search over this structure.
|
||||||
|
"""
|
||||||
|
|
||||||
|
nodes: list[str] # all industry/concept names
|
||||||
|
edges: list[RotationEdge] # directed flows, sorted by strength desc
|
||||||
|
timestamp: datetime = field(default_factory=datetime.now)
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
"""FeatureDefinition — metadata for a registered feature.
|
||||||
|
|
||||||
|
Each feature in the Feature Registry is described by one of these.
|
||||||
|
The Engine code only knows about FeatureDefinitions, never imports
|
||||||
|
individual feature functions directly.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class FeatureDefinition:
|
||||||
|
"""Metadata for a registered market feature.
|
||||||
|
|
||||||
|
Adding a new feature means registering a FeatureDefinition.
|
||||||
|
No engine code changes needed.
|
||||||
|
"""
|
||||||
|
|
||||||
|
name: str # unique identifier, e.g. "breadth_vector"
|
||||||
|
category: str # "breadth", "volume", "trend", "fear", "rotation", "participation"
|
||||||
|
description: str # human-readable explanation
|
||||||
|
dependencies: list[str] = field(default_factory=list) # feature names this depends on
|
||||||
|
compute: Callable | None = None # the actual computation function (set after registration)
|
||||||
|
version: str = "1.0"
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
"""Knowledge Graph domain model — Theme → Concept → Industry → Leader.
|
||||||
|
|
||||||
|
The ontology that connects abstract trading themes (e.g. "AI", "新能源")
|
||||||
|
to concrete industries and stocks. Used by narrative generation, rotation
|
||||||
|
detection, and future AI agent reasoning.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Concept:
|
||||||
|
"""A mid-level concept within a theme — e.g. "算力" within "AI"."""
|
||||||
|
name: str # "算力"
|
||||||
|
industries: list[str] = field(default_factory=list) # industry names from stock_industry
|
||||||
|
keywords: list[str] = field(default_factory=list) # keywords to match stock names
|
||||||
|
leaders: list[str] = field(default_factory=list) # representative stocks
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Theme:
|
||||||
|
"""A high-level trading theme — e.g. "AI", "新能源", "机器人"."""
|
||||||
|
name: str
|
||||||
|
concepts: list[Concept] = field(default_factory=list)
|
||||||
|
relevance_score: float = 0.0 # 0-100, how active this theme is today
|
||||||
|
momentum: str = "" # "↑↑" | "↑" | "→" | "↓" | "↓↓"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ThemeEdge:
|
||||||
|
"""Directed edge: money flowing from one theme to another."""
|
||||||
|
source: str
|
||||||
|
target: str
|
||||||
|
strength: float = 0.0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ThemeGraph:
|
||||||
|
"""The complete theme landscape for today."""
|
||||||
|
themes: list[Theme] = field(default_factory=list)
|
||||||
|
edges: list[ThemeEdge] = field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════
|
||||||
|
# Curated A-Share Theme Knowledge Base
|
||||||
|
# ═══════════════════════════════════════════════
|
||||||
|
|
||||||
|
THEME_KNOWLEDGE: list[Theme] = [
|
||||||
|
Theme(name="AI", concepts=[
|
||||||
|
Concept(name="算力", industries=[], keywords=["算力", "服务器", "数据中心", "GPU", "CP", "曙光", "浪潮", "紫光", "中科曙光", "工业富联"], leaders=["工业富联", "中科曙光"]),
|
||||||
|
Concept(name="PCB", industries=[], keywords=["PCB", "电路板", "印制板", "沪电", "深南电路", "鹏鼎", "景旺", "胜宏"], leaders=["沪电股份", "深南电路"]),
|
||||||
|
Concept(name="光模块", industries=[], keywords=["光模块", "光通信", "中际", "新易盛", "天孚", "旭创", "剑桥", "光迅"], leaders=["中际旭创", "新易盛"]),
|
||||||
|
Concept(name="AI应用", industries=[], keywords=["AI应用", "大模型", "智能", "软件", "科大讯飞", "商汤", "云从", "格灵"], leaders=["科大讯飞"]),
|
||||||
|
Concept(name="半导体", industries=[], keywords=["半导体", "芯片", "晶圆", "封装", "设备", "韦尔", "兆易", "北方华创", "中微", "长电"], leaders=["北方华创"]),
|
||||||
|
]),
|
||||||
|
Theme(name="机器人", concepts=[
|
||||||
|
Concept(name="减速器", industries=[], keywords=["减速器", "谐波", "绿的", "双环传动", "中大力德"], leaders=["绿的谐波"]),
|
||||||
|
Concept(name="伺服电机", industries=[], keywords=["伺服", "电机", "汇川", "禾川", "埃斯顿", "雷赛"], leaders=["汇川技术", "埃斯顿"]),
|
||||||
|
Concept(name="传感器", industries=[], keywords=["传感器", "视觉", "力觉", "奥普特", "天准"], leaders=["奥普特"]),
|
||||||
|
Concept(name="本体集成", industries=[], keywords=["机器人", "新松", "拓斯达", "埃夫特", "博实"], leaders=["新松机器人"]),
|
||||||
|
]),
|
||||||
|
Theme(name="新能源", concepts=[
|
||||||
|
Concept(name="光伏", industries=[], keywords=["光伏", "隆基", "通威", "晶澳", "天合", "晶科", "阳光电源", "锦浪"], leaders=["隆基绿能", "阳光电源"]),
|
||||||
|
Concept(name="锂电", industries=[], keywords=["锂电", "宁德", "比亚迪", "亿纬", "国轩", "赣锋", "天齐", "华友", "中伟"], leaders=["宁德时代"]),
|
||||||
|
Concept(name="风电", industries=[], keywords=["风电", "金风", "明阳", "运达", "东方电缆", "大金重工"], leaders=["金风科技"]),
|
||||||
|
Concept(name="储能", industries=[], keywords=["储能", "派能", "科士达", "盛弘", "固德威", "德业"], leaders=["派能科技"]),
|
||||||
|
]),
|
||||||
|
Theme(name="消费", concepts=[
|
||||||
|
Concept(name="白酒", industries=[], keywords=["白酒", "茅台", "五粮液", "泸州老窖", "汾酒", "洋河", "古井"], leaders=["贵州茅台"]),
|
||||||
|
Concept(name="家电", industries=[], keywords=["家电", "美的", "格力", "海尔", "海信", "老板"], leaders=["美的集团"]),
|
||||||
|
Concept(name="食品饮料", industries=[], keywords=["食品", "饮料", "伊利", "海天", "双汇", "桃李", "安井"], leaders=["伊利股份"]),
|
||||||
|
]),
|
||||||
|
Theme(name="证券金融", concepts=[
|
||||||
|
Concept(name="证券", industries=[], keywords=["证券", "中信", "华泰", "国泰", "海通", "东方财富"], leaders=["东方财富"]),
|
||||||
|
Concept(name="保险", industries=[], keywords=["保险", "平安", "太保", "人寿", "新华"], leaders=["中国平安"]),
|
||||||
|
Concept(name="银行", industries=[], keywords=["银行", "招商", "兴业", "平安银行", "宁波银行"], leaders=["招商银行"]),
|
||||||
|
]),
|
||||||
|
Theme(name="军工", concepts=[
|
||||||
|
Concept(name="航空", industries=[], keywords=["航空", "发动机", "飞机", "沈飞", "西飞", "航发", "中航"], leaders=["中航沈飞"]),
|
||||||
|
Concept(name="船舶", industries=[], keywords=["船舶", "中国船舶", "中船", "重工"], leaders=["中国船舶"]),
|
||||||
|
Concept(name="导弹武器", industries=[], keywords=["导弹", "兵器", "光电", "航天"], leaders=["航天电器"]),
|
||||||
|
]),
|
||||||
|
Theme(name="医药", concepts=[
|
||||||
|
Concept(name="创新药", industries=[], keywords=["创新药", "恒瑞", "百济", "信达", "君实", "荣昌"], leaders=["恒瑞医药"]),
|
||||||
|
Concept(name="CXO", industries=[], keywords=["CXO", "药明", "康龙", "泰格", "凯莱英"], leaders=["药明康德"]),
|
||||||
|
Concept(name="医疗器械", industries=[], keywords=["医疗", "器械", "迈瑞", "联影", "鱼跃", "微创"], leaders=["迈瑞医疗"]),
|
||||||
|
]),
|
||||||
|
Theme(name="汽车", concepts=[
|
||||||
|
Concept(name="整车", industries=[], keywords=["汽车", "整车", "比亚迪", "长城", "长安", "吉利", "上汽"], leaders=["比亚迪"]),
|
||||||
|
Concept(name="零部件", industries=[], keywords=["零部件", "配件", "华域", "均胜", "拓普", "三花"], leaders=["拓普集团"]),
|
||||||
|
Concept(name="智能驾驶", industries=[], keywords=["智能驾驶", "自动驾驶", "ADAS", "德赛西威", "中科创达", "经纬恒润"], leaders=["德赛西威"]),
|
||||||
|
]),
|
||||||
|
]
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
"""LeaderLifecycle — full lifecycle model for sector leaders.
|
||||||
|
|
||||||
|
Not just Alive/Dead. Traders track the entire lifecycle:
|
||||||
|
Birth → Growing → Leading → Exhausted → Breaking → Dead → Recovering
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from enum import Enum
|
||||||
|
|
||||||
|
|
||||||
|
class LeaderState(str, Enum):
|
||||||
|
"""Full leader lifecycle — not binary Alive/Dead."""
|
||||||
|
BIRTH = "birth" # first breakout from base
|
||||||
|
GROWING = "growing" # accelerating, above MAs, volume expanding
|
||||||
|
LEADING = "leading" # at peak, driving sector, strongest momentum
|
||||||
|
EXHAUSTED = "exhausted" # extended from MA, volume fading
|
||||||
|
BREAKING = "breaking" # breaking below key MA (MA20)
|
||||||
|
DEAD = "dead" # below MA60, no recovery signal
|
||||||
|
RECOVERING = "recovering" # reclaiming MA20 after a break
|
||||||
|
UNKNOWN = "unknown" # insufficient data to classify
|
||||||
@@ -0,0 +1,82 @@
|
|||||||
|
"""Signal domain objects — the core of Signal Intelligence.
|
||||||
|
|
||||||
|
Signal × MarketState → Expectancy. This is the moat.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, date
|
||||||
|
from enum import Enum
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
|
||||||
|
class SignalType(str, Enum):
|
||||||
|
"""Known signal types. Each has its own detector and expectancy profile."""
|
||||||
|
# EMA52 family
|
||||||
|
EMA52_CROSS_UP = "ema52_cross_up"
|
||||||
|
EMA52_CROSS_DOWN = "ema52_cross_down"
|
||||||
|
EMA52_NEAR = "ema52_near"
|
||||||
|
# Vegas Channel family
|
||||||
|
VEGAS_LONG = "vegas_long" # price > EMA144, EMA12>EMA50>EMA144, pullback to EMA50
|
||||||
|
VEGAS_SHORT = "vegas_short" # price < EMA144, EMA12<EMA50<EMA144, rally to EMA50
|
||||||
|
# Chan Theory
|
||||||
|
CHAN_1BUY = "chan_1buy" # MACD bottom divergence (price↓ MACD↑)
|
||||||
|
CHAN_1SELL = "chan_1sell" # MACD top divergence (price↑ MACD↓)
|
||||||
|
# ORB (Opening Range Breakout — daily level)
|
||||||
|
ORB_UP = "orb_up" # close breaks above yesterday's high
|
||||||
|
ORB_DOWN = "orb_down" # close breaks below yesterday's low
|
||||||
|
# Gap signals
|
||||||
|
GAP_UP = "gap_up" # open > yesterday's high (breakaway gap)
|
||||||
|
GAP_DOWN = "gap_down" # open < yesterday's low (breakaway gap)
|
||||||
|
# NR7 (Narrow Range 7)
|
||||||
|
NR7 = "nr7" # narrowest range in 7 days (coiling)
|
||||||
|
# Inside Bar
|
||||||
|
IB_LONG = "ib_long" # today's range inside yesterday's, close > mid
|
||||||
|
IB_SHORT = "ib_short" # today's range inside yesterday's, close < mid
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def bullish_signals(cls) -> list["SignalType"]:
|
||||||
|
return [cls.EMA52_CROSS_UP, cls.VEGAS_LONG, cls.CHAN_1BUY, cls.ORB_UP]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def bearish_signals(cls) -> list["SignalType"]:
|
||||||
|
return [cls.EMA52_CROSS_DOWN, cls.VEGAS_SHORT, cls.CHAN_1SELL, cls.ORB_DOWN]
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class SignalInstance:
|
||||||
|
"""A single signal occurrence with its context and outcome.
|
||||||
|
|
||||||
|
Stored in signal_instance table. This is the training data
|
||||||
|
for the Expectancy Engine — every signal, its market state,
|
||||||
|
and what happened afterward.
|
||||||
|
"""
|
||||||
|
# Identity
|
||||||
|
signal_type: str # SignalType value
|
||||||
|
ts_code: str # stock code
|
||||||
|
trade_date: date # when signal fired
|
||||||
|
|
||||||
|
# Signal details
|
||||||
|
signal_price: float = 0.0 # close price at signal
|
||||||
|
|
||||||
|
# Market context at signal time (state vector snapshot)
|
||||||
|
state_trend: float = 0.0
|
||||||
|
state_fear: float = 0.0
|
||||||
|
state_liquidity: float = 0.0
|
||||||
|
state_rotation: float = 0.0
|
||||||
|
state_participation: float = 0.0
|
||||||
|
state_volatility: float = 0.0
|
||||||
|
state_breadth: float = 0.0
|
||||||
|
|
||||||
|
# Outcome (forward-looking)
|
||||||
|
return_5d: Optional[float] = None # return after 5 trading days
|
||||||
|
return_10d: Optional[float] = None
|
||||||
|
return_20d: Optional[float] = None
|
||||||
|
max_return_5d: Optional[float] = None # max favorable excursion
|
||||||
|
max_drawdown_5d: Optional[float] = None # max adverse excursion
|
||||||
|
holding_days: Optional[int] = None # days until signal invalidates
|
||||||
|
outcome_known: bool = False # True if outcome data is complete
|
||||||
|
|
||||||
|
# Metadata
|
||||||
|
created_at: datetime = field(default_factory=datetime.now)
|
||||||
@@ -0,0 +1,184 @@
|
|||||||
|
"""MarketState — the central domain object of the entire platform.
|
||||||
|
|
||||||
|
Every downstream consumer (Dashboard, AI Agent, Backtester, Signal Engine)
|
||||||
|
uses this same type. NOT a dict.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import date, datetime
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class MarketState:
|
||||||
|
"""Continuous market state vector.
|
||||||
|
|
||||||
|
Each dimension is 0.0–1.0, representing the strength/intensity
|
||||||
|
of that market property. NOT a discrete classification.
|
||||||
|
|
||||||
|
Consumers read the vector directly; the human-readable label
|
||||||
|
is derived by the presentation layer only.
|
||||||
|
"""
|
||||||
|
|
||||||
|
timestamp: datetime
|
||||||
|
trade_date: date
|
||||||
|
version: str = "1.0"
|
||||||
|
source: str = "ashare_dp"
|
||||||
|
|
||||||
|
# ── State dimensions (0.0–1.0 continuous) ──
|
||||||
|
trend: float = 0.0 # trend strength (breadth + MA alignment + resonance)
|
||||||
|
fear: float = 0.0 # fear/greed inverse (drawdown + range + decline breadth)
|
||||||
|
liquidity: float = 0.0 # liquidity health (volume vs avg + participation)
|
||||||
|
rotation: float = 0.0 # sector rotation intensity
|
||||||
|
participation: float = 0.0 # market participation breadth
|
||||||
|
volatility: float = 0.0 # normalized volatility
|
||||||
|
breadth: float = 0.0 # advance/decline health
|
||||||
|
|
||||||
|
# ── Quality metadata ──
|
||||||
|
confidence: float = 0.0 # how clearly the vector matches a regime (0–1)
|
||||||
|
quality: float = 0.0 # data quality / completeness score (0–1)
|
||||||
|
|
||||||
|
def to_dict(self) -> dict:
|
||||||
|
"""Serialize to dict for JSON API responses."""
|
||||||
|
return {
|
||||||
|
"version": self.version,
|
||||||
|
"source": self.source,
|
||||||
|
"timestamp": self.timestamp.isoformat(),
|
||||||
|
"trade_date": self.trade_date.isoformat(),
|
||||||
|
"dimensions": {
|
||||||
|
"trend": self.trend,
|
||||||
|
"fear": self.fear,
|
||||||
|
"liquidity": self.liquidity,
|
||||||
|
"rotation": self.rotation,
|
||||||
|
"participation": self.participation,
|
||||||
|
"volatility": self.volatility,
|
||||||
|
"breadth": self.breadth,
|
||||||
|
},
|
||||||
|
"confidence": self.confidence,
|
||||||
|
"quality": self.quality,
|
||||||
|
}
|
||||||
|
|
||||||
|
def to_vector(self) -> list[float]:
|
||||||
|
"""Return state as a plain float vector for ML / similarity search."""
|
||||||
|
return [
|
||||||
|
self.trend,
|
||||||
|
self.fear,
|
||||||
|
self.liquidity,
|
||||||
|
self.rotation,
|
||||||
|
self.participation,
|
||||||
|
self.volatility,
|
||||||
|
self.breadth,
|
||||||
|
]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def dimension_names(self) -> list[str]:
|
||||||
|
return [
|
||||||
|
"trend", "fear", "liquidity", "rotation",
|
||||||
|
"participation", "volatility", "breadth",
|
||||||
|
]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_features(
|
||||||
|
cls,
|
||||||
|
features: dict,
|
||||||
|
trade_date: date,
|
||||||
|
timestamp: Optional[datetime] = None,
|
||||||
|
) -> "MarketState":
|
||||||
|
"""Construct MarketState from computed feature vectors.
|
||||||
|
|
||||||
|
Each feature category contributes to one or more state dimensions.
|
||||||
|
"""
|
||||||
|
import time as _time
|
||||||
|
ts = timestamp or datetime.now()
|
||||||
|
|
||||||
|
# ── Breadth → breadth, participation ──
|
||||||
|
b = features.get("breadth_vector", {})
|
||||||
|
advance_ratio = b.get("advance_ratio", 0.5)
|
||||||
|
new_high_ratio = b.get("new_high_ratio", 0.0)
|
||||||
|
new_low_ratio = b.get("new_low_ratio", 0.0)
|
||||||
|
|
||||||
|
breadth_val = _clamp(advance_ratio, 0, 1)
|
||||||
|
|
||||||
|
# ── Volume → liquidity ──
|
||||||
|
v = features.get("volume_vector", {})
|
||||||
|
vs_5d = v.get("vs_5d", 1.0)
|
||||||
|
vs_20d = v.get("vs_20d", 1.0)
|
||||||
|
total_turnover = v.get("total_turnover", 0)
|
||||||
|
|
||||||
|
# Liquidity: volume relative to average, mapped to 0-1
|
||||||
|
# 1.0x avg → 0.5, 2.0x+ → 1.0, <0.5x → 0.0
|
||||||
|
liquidity_val = _clamp((min(vs_5d, vs_20d) - 0.5) / 1.5, 0, 1)
|
||||||
|
|
||||||
|
# ── Trend → trend ──
|
||||||
|
t = features.get("trend_vector", {})
|
||||||
|
above_ma20 = t.get("above_ma20_pct", 0.5)
|
||||||
|
resonance = t.get("resonance", 0.5)
|
||||||
|
divergence = t.get("divergence", 0.0)
|
||||||
|
|
||||||
|
# Trend: MA alignment + index resonance - divergence penalty
|
||||||
|
trend_val = _clamp(
|
||||||
|
above_ma20 * 0.5 + resonance * 0.4 - divergence * 0.3,
|
||||||
|
0, 1,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ── Fear → fear ──
|
||||||
|
f = features.get("fear_vector", {})
|
||||||
|
median_drawdown = abs(f.get("median_drawdown", 0.0))
|
||||||
|
pct_deep = f.get("pct_deep_drawdown", 0.0)
|
||||||
|
daily_range = f.get("daily_range_pct", 0.02)
|
||||||
|
|
||||||
|
# Fear: drawdown depth + breadth + intraday range
|
||||||
|
fear_val = _clamp(
|
||||||
|
median_drawdown * 2.0 + pct_deep * 1.5 + daily_range * 5.0,
|
||||||
|
0, 1,
|
||||||
|
)
|
||||||
|
|
||||||
|
# ── Rotation → rotation ──
|
||||||
|
r = features.get("rotation_vector", {})
|
||||||
|
speed = r.get("speed", 0.0)
|
||||||
|
persistence = r.get("persistence", 0.5)
|
||||||
|
|
||||||
|
rotation_val = _clamp(speed * 0.6 + persistence * 0.4, 0, 1)
|
||||||
|
|
||||||
|
# ── Participation → participation ──
|
||||||
|
p = features.get("participation_vector", {})
|
||||||
|
pct_above_ma20 = p.get("pct_above_ma20", 0.5)
|
||||||
|
pct_above_ma60 = p.get("pct_above_ma60", 0.5)
|
||||||
|
|
||||||
|
participation_val = _clamp((pct_above_ma20 + pct_above_ma60) / 2, 0, 1)
|
||||||
|
|
||||||
|
# ── Volatility ──
|
||||||
|
vol_val = _clamp(daily_range / 0.06, 0, 1) # 6% daily range → 1.0
|
||||||
|
|
||||||
|
# ── Confidence: how internally consistent the vector is ──
|
||||||
|
# High trend + low fear → high confidence
|
||||||
|
# High trend + high fear → conflicting, lower confidence
|
||||||
|
consistency = 1.0 - abs(trend_val - (1.0 - fear_val)) * 0.5
|
||||||
|
confidence = _clamp(consistency * breadth_val, 0, 1)
|
||||||
|
|
||||||
|
# ── Quality: data completeness ──
|
||||||
|
quality = 1.0 # Phase 1: assume full quality; future: check for gaps
|
||||||
|
|
||||||
|
return cls(
|
||||||
|
timestamp=ts,
|
||||||
|
trade_date=trade_date,
|
||||||
|
trend=round(trend_val, 4),
|
||||||
|
fear=round(fear_val, 4),
|
||||||
|
liquidity=round(liquidity_val, 4),
|
||||||
|
rotation=round(rotation_val, 4),
|
||||||
|
participation=round(participation_val, 4),
|
||||||
|
volatility=round(vol_val, 4),
|
||||||
|
breadth=round(breadth_val, 4),
|
||||||
|
confidence=round(confidence, 4),
|
||||||
|
quality=round(quality, 4),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _clamp(value: float, lo: float, hi: float) -> float:
|
||||||
|
"""Clamp value to [lo, hi], with NaN safety."""
|
||||||
|
import math
|
||||||
|
if value is None or (isinstance(value, float) and math.isnan(value)):
|
||||||
|
return (lo + hi) / 2
|
||||||
|
return max(lo, min(hi, value))
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
"""Trade domain — recording actions and outcomes for system learning."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import datetime, date
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class TradeRecord:
|
||||||
|
"""A single trade from entry to exit. The training data for system learning."""
|
||||||
|
# Identity
|
||||||
|
ts_code: str
|
||||||
|
trade_date: date
|
||||||
|
|
||||||
|
# Entry
|
||||||
|
entry_price: float
|
||||||
|
entry_time: datetime | None = None
|
||||||
|
signal_type: str = "" # which signal triggered the entry
|
||||||
|
position_pct: float = 0.0 # position size as % of portfolio
|
||||||
|
|
||||||
|
# Exit
|
||||||
|
exit_price: float | None = None
|
||||||
|
exit_time: datetime | None = None
|
||||||
|
exit_reason: str = "" # "target" | "stop" | "signal" | "time" | "manual"
|
||||||
|
|
||||||
|
# Outcome
|
||||||
|
return_pct: float | None = None # (exit - entry) / entry * 100
|
||||||
|
holding_days: int | None = None
|
||||||
|
max_favorable: float | None = None # best price during hold
|
||||||
|
max_adverse: float | None = None # worst price during hold
|
||||||
|
|
||||||
|
# Context (market state at entry)
|
||||||
|
state_trend: float = 0.0
|
||||||
|
state_fear: float = 0.0
|
||||||
|
state_liquidity: float = 0.0
|
||||||
|
state_breadth: float = 0.0
|
||||||
|
|
||||||
|
# Tags
|
||||||
|
tags: str = "" # comma-separated: "breakout,trend,momentum"
|
||||||
|
notes: str = ""
|
||||||
|
|
||||||
|
# Metadata
|
||||||
|
created_at: datetime = field(default_factory=datetime.now)
|
||||||
|
closed: bool = False
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
"""EXECUTION INTELLIGENCE — how to trade: playbook, risk, brief assembly."""
|
||||||
|
from ashare_dp.execution.playbook import build_playbook
|
||||||
|
from ashare_dp.execution.risk import evaluate_risks
|
||||||
|
from ashare_dp.execution.brief import build_brief
|
||||||
|
|
||||||
|
__all__ = ["build_playbook", "evaluate_risks", "build_brief"]
|
||||||
@@ -0,0 +1,190 @@
|
|||||||
|
"""TradingBrief Builder — assembles TradingContext + serializes for API.
|
||||||
|
|
||||||
|
NOT an engine. Pure assembly and serialization — no inference, no rules.
|
||||||
|
The single place where TradingContext becomes API JSON.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
from ashare_dp.domain.context import (
|
||||||
|
TradingContext, Playbook, Expectancy, Allocation,
|
||||||
|
)
|
||||||
|
from ashare_dp.domain.state import MarketState
|
||||||
|
|
||||||
|
|
||||||
|
def build_brief(
|
||||||
|
trade_date: date,
|
||||||
|
state: MarketState,
|
||||||
|
playbook: Playbook,
|
||||||
|
expectancy: Expectancy | None,
|
||||||
|
leaders: dict,
|
||||||
|
opportunities: list,
|
||||||
|
money_flow,
|
||||||
|
allocation: Allocation | None,
|
||||||
|
narrative: str,
|
||||||
|
timestamp: datetime | None = None,
|
||||||
|
) -> TradingContext:
|
||||||
|
"""Assemble TradingContext from all engine outputs. Pure assembly."""
|
||||||
|
return TradingContext(
|
||||||
|
timestamp=timestamp or datetime.now(),
|
||||||
|
trade_date=trade_date,
|
||||||
|
state=state,
|
||||||
|
leaders=leaders,
|
||||||
|
opportunities=opportunities,
|
||||||
|
money_flow=money_flow,
|
||||||
|
playbook=playbook,
|
||||||
|
expectancy=expectancy,
|
||||||
|
allocation=allocation,
|
||||||
|
risk_budget=playbook.risk_budget if playbook else "Medium",
|
||||||
|
narrative=narrative,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def generate_narrative(state, leaders, opportunities, flow_graph) -> str:
|
||||||
|
"""Rule-based narrative. Template from features — not LLM."""
|
||||||
|
parts = []
|
||||||
|
|
||||||
|
t = state.trend
|
||||||
|
f = state.fear
|
||||||
|
if t > 0.5:
|
||||||
|
parts.append("市场处于趋势行情")
|
||||||
|
elif t > 0.3:
|
||||||
|
parts.append("市场震荡")
|
||||||
|
else:
|
||||||
|
parts.append("市场偏弱")
|
||||||
|
|
||||||
|
if flow_graph and flow_graph.flows:
|
||||||
|
top_flow = flow_graph.flows[0]
|
||||||
|
parts.append(f"资金从{top_flow.source}流向{top_flow.target}")
|
||||||
|
|
||||||
|
if opportunities:
|
||||||
|
top_opps = [o.name for o in opportunities[:3]]
|
||||||
|
parts.append(f"热点集中在{'、'.join(top_opps)}")
|
||||||
|
|
||||||
|
alive = [ind for ind, ld in leaders.items()
|
||||||
|
if isinstance(ld, dict) and ld.get("state") in ("leading", "growing", "birth")]
|
||||||
|
broken = [ind for ind, ld in leaders.items()
|
||||||
|
if isinstance(ld, dict) and ld.get("state") in ("dead", "breaking")]
|
||||||
|
if alive:
|
||||||
|
parts.append(f"龙头板块{'、'.join(alive[:3])}保持强势")
|
||||||
|
if broken:
|
||||||
|
parts.append(f"{'、'.join(broken[:3])}龙头走弱需回避")
|
||||||
|
|
||||||
|
if f > 0.5:
|
||||||
|
parts.append("市场恐慌情绪较高,注意风险控制")
|
||||||
|
elif f < 0.3:
|
||||||
|
parts.append("风险情绪可控")
|
||||||
|
|
||||||
|
return "。".join(parts) + "。"
|
||||||
|
|
||||||
|
|
||||||
|
def brief_to_api_dict(
|
||||||
|
brief: TradingContext,
|
||||||
|
risk_events: list,
|
||||||
|
features: dict,
|
||||||
|
sentiment: dict,
|
||||||
|
all_expectancies: list | None = None,
|
||||||
|
theme_graph = None,
|
||||||
|
recommendations: list | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Serialize TradingContext to the versioned API response.
|
||||||
|
|
||||||
|
The SINGLE serialization point. Router calls this, never builds JSON by hand.
|
||||||
|
Response schema is stable (version 1.0) — consumers depend on it.
|
||||||
|
"""
|
||||||
|
pb = brief.playbook
|
||||||
|
exp = brief.expectancy
|
||||||
|
alloc = brief.allocation
|
||||||
|
mf = brief.money_flow
|
||||||
|
|
||||||
|
return {
|
||||||
|
"version": "1.0",
|
||||||
|
"timestamp": brief.timestamp.isoformat(),
|
||||||
|
"trade_date": brief.trade_date.isoformat(),
|
||||||
|
"playbook": {
|
||||||
|
"suitable_strategies": pb.suitable_strategies,
|
||||||
|
"unsuitable_strategies": pb.unsuitable_strategies,
|
||||||
|
"holding_time": pb.holding_time,
|
||||||
|
"risk_budget": pb.risk_budget,
|
||||||
|
"position_sizing": pb.position_sizing,
|
||||||
|
"bias": pb.bias,
|
||||||
|
"confidence": pb.confidence,
|
||||||
|
} if pb else {},
|
||||||
|
"expectancy": {
|
||||||
|
"signal_type": exp.signal_type,
|
||||||
|
"win_rate": exp.win_rate,
|
||||||
|
"avg_return": exp.avg_return,
|
||||||
|
"max_drawdown": exp.max_drawdown,
|
||||||
|
"avg_holding_days": exp.avg_holding_days,
|
||||||
|
"sample_count": exp.sample_count,
|
||||||
|
"similar_states": exp.similar_states,
|
||||||
|
"confidence": exp.confidence,
|
||||||
|
} if exp else {},
|
||||||
|
"allocation": {
|
||||||
|
"cash_pct": alloc.cash_pct,
|
||||||
|
"trend_pct": alloc.trend_pct,
|
||||||
|
"trial_pct": alloc.trial_pct,
|
||||||
|
"reasoning": alloc.reasoning,
|
||||||
|
} if alloc else {},
|
||||||
|
"opportunities": [
|
||||||
|
{
|
||||||
|
"name": o.name,
|
||||||
|
"score": o.score,
|
||||||
|
"lifecycle": o.lifecycle,
|
||||||
|
"persistence_days": o.persistence_days,
|
||||||
|
"leader_stock": o.leader_stock,
|
||||||
|
"top_stocks": o.top_stocks,
|
||||||
|
}
|
||||||
|
for o in (brief.opportunities or [])[:10]
|
||||||
|
],
|
||||||
|
"money_flow": {
|
||||||
|
"flows": [
|
||||||
|
{"source": e.source, "target": e.target,
|
||||||
|
"magnitude": e.magnitude, "strength": e.strength}
|
||||||
|
for e in mf.flows
|
||||||
|
] if mf else [],
|
||||||
|
"net_inflow": mf.net_inflow if mf else {},
|
||||||
|
"net_outflow": mf.net_outflow if mf else {},
|
||||||
|
},
|
||||||
|
"leaders": brief.leaders,
|
||||||
|
"risks": [e.to_dict() for e in risk_events],
|
||||||
|
"evidence": {
|
||||||
|
"state": brief.state.to_dict() if brief.state else {},
|
||||||
|
"features": features,
|
||||||
|
"sentiment": sentiment,
|
||||||
|
},
|
||||||
|
"theme_graph": {
|
||||||
|
"themes": [
|
||||||
|
{
|
||||||
|
"name": t.name,
|
||||||
|
"score": t.relevance_score,
|
||||||
|
"momentum": t.momentum,
|
||||||
|
"concepts": [
|
||||||
|
{"name": c.name, "score": c.relevance_score}
|
||||||
|
for c in (t.concepts or []) if c.relevance_score > 0
|
||||||
|
],
|
||||||
|
}
|
||||||
|
for t in (theme_graph.themes if theme_graph else [])
|
||||||
|
if t.relevance_score > 0
|
||||||
|
],
|
||||||
|
"edges": [
|
||||||
|
{"source": e.source, "target": e.target, "strength": e.strength}
|
||||||
|
for e in (theme_graph.edges if theme_graph else [])
|
||||||
|
],
|
||||||
|
} if theme_graph else {},
|
||||||
|
"recommendations": recommendations or [],
|
||||||
|
"narrative": brief.narrative,
|
||||||
|
"all_expectancies": [
|
||||||
|
{
|
||||||
|
"signal_type": e.signal_type,
|
||||||
|
"win_rate": e.win_rate,
|
||||||
|
"avg_return": e.avg_return,
|
||||||
|
"max_drawdown": e.max_drawdown,
|
||||||
|
"sample_count": e.sample_count,
|
||||||
|
"confidence": e.confidence,
|
||||||
|
}
|
||||||
|
for e in (all_expectancies or [])
|
||||||
|
],
|
||||||
|
}
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
"""Playbook Engine — maps MarketState + Leaders + Opportunities to a complete trading playbook.
|
||||||
|
|
||||||
|
Not a recommendation list. A complete playbook: what strategies work,
|
||||||
|
what to avoid, how long to hold, how much to risk.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from ashare_dp.domain.context import Playbook
|
||||||
|
from ashare_dp.domain.leadership import LeaderState
|
||||||
|
from ashare_dp.domain.state import MarketState
|
||||||
|
|
||||||
|
|
||||||
|
def build_playbook(
|
||||||
|
state: MarketState,
|
||||||
|
leaders: dict[str, dict],
|
||||||
|
opportunities: list,
|
||||||
|
) -> Playbook:
|
||||||
|
"""Build trading playbook from market state + leaders + opportunities.
|
||||||
|
|
||||||
|
Rules:
|
||||||
|
- High trend + low fear + high participation → Trend Following / Breakout
|
||||||
|
- Mid trend + high rotation → Sector Rotation
|
||||||
|
- Low trend + high fear → Defensive / Cash
|
||||||
|
- Leader state affects sector focus
|
||||||
|
"""
|
||||||
|
t = state.trend
|
||||||
|
f = state.fear
|
||||||
|
l = state.liquidity
|
||||||
|
r = state.rotation
|
||||||
|
p = state.participation
|
||||||
|
|
||||||
|
suitable = []
|
||||||
|
unsuitable = []
|
||||||
|
|
||||||
|
# ── Strategy selection ──
|
||||||
|
if t > 0.55 and f < 0.4 and l > 0.5:
|
||||||
|
suitable.extend(["Trend Following", "Breakout", "Momentum"])
|
||||||
|
unsuitable.extend(["Mean Reversion", "Bottom Fishing"])
|
||||||
|
elif t > 0.4 and r > 0.4:
|
||||||
|
suitable.extend(["Sector Rotation", "Breakout"])
|
||||||
|
unsuitable.extend(["Counter Trend"])
|
||||||
|
elif t < 0.3 and f > 0.5:
|
||||||
|
suitable.append("Cash")
|
||||||
|
unsuitable.extend(["Trend Following", "Breakout", "Momentum", "Bottom Fishing"])
|
||||||
|
else:
|
||||||
|
suitable.extend(["Range Trading"])
|
||||||
|
unsuitable.extend(["Trend Following"])
|
||||||
|
|
||||||
|
if p < 0.3:
|
||||||
|
unsuitable.append("Heavy Position")
|
||||||
|
|
||||||
|
if l < 0.3:
|
||||||
|
unsuitable.append("Large Cap Breakout")
|
||||||
|
|
||||||
|
# ── Holding time ──
|
||||||
|
if t > 0.6 and l > 0.6:
|
||||||
|
holding_time = "3~5 Days"
|
||||||
|
elif t > 0.4:
|
||||||
|
holding_time = "2~4 Days"
|
||||||
|
else:
|
||||||
|
holding_time = "1~2 Days"
|
||||||
|
|
||||||
|
# ── Risk budget ──
|
||||||
|
if f > 0.6:
|
||||||
|
risk_budget = "Low"
|
||||||
|
elif f > 0.35:
|
||||||
|
risk_budget = "Medium"
|
||||||
|
else:
|
||||||
|
risk_budget = "High"
|
||||||
|
|
||||||
|
# ── Position sizing ──
|
||||||
|
if state.confidence > 0.7 and f < 0.3:
|
||||||
|
position_sizing = "Standard"
|
||||||
|
elif f > 0.5:
|
||||||
|
position_sizing = "Reduced"
|
||||||
|
else:
|
||||||
|
position_sizing = "Standard"
|
||||||
|
|
||||||
|
# ── Focus sectors ──
|
||||||
|
focus_sectors = []
|
||||||
|
avoid_sectors = []
|
||||||
|
|
||||||
|
for opp in opportunities[:5]:
|
||||||
|
if opp.score > 60:
|
||||||
|
focus_sectors.append(opp.name)
|
||||||
|
|
||||||
|
for ind, ld in leaders.items():
|
||||||
|
if isinstance(ld, dict) and ld.get("state") in (
|
||||||
|
LeaderState.DEAD.value, LeaderState.BREAKING.value
|
||||||
|
):
|
||||||
|
avoid_sectors.append(ind)
|
||||||
|
|
||||||
|
# ── Bias ──
|
||||||
|
if t > 0.5 and f < 0.4:
|
||||||
|
bias = "LONG"
|
||||||
|
elif f > 0.6:
|
||||||
|
bias = "CASH"
|
||||||
|
else:
|
||||||
|
bias = "NEUTRAL"
|
||||||
|
|
||||||
|
return Playbook(
|
||||||
|
suitable_strategies=list(dict.fromkeys(suitable))[:4],
|
||||||
|
unsuitable_strategies=list(dict.fromkeys(unsuitable))[:4],
|
||||||
|
holding_time=holding_time,
|
||||||
|
risk_budget=risk_budget,
|
||||||
|
position_sizing=position_sizing,
|
||||||
|
focus_sectors=focus_sectors[:5],
|
||||||
|
avoid_sectors=avoid_sectors[:5],
|
||||||
|
bias=bias,
|
||||||
|
confidence=round(state.confidence, 4),
|
||||||
|
)
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
"""Risk Rule Engine — pluggable rules that emit RiskEvents.
|
||||||
|
|
||||||
|
Each rule evaluates features independently. Adding a new risk
|
||||||
|
means adding a new RiskRule — no engine code changes.
|
||||||
|
Dashboard renders, AI explains, Backtest counts.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
from ashare_dp.domain.events import RiskEvent
|
||||||
|
|
||||||
|
# Rule condition signature:
|
||||||
|
# (features: dict) -> (triggered: bool, severity: float, payload: dict)
|
||||||
|
|
||||||
|
|
||||||
|
class RiskRule:
|
||||||
|
"""A single risk detection rule."""
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
name: str,
|
||||||
|
event_type: str,
|
||||||
|
level: str, # "warning" | "danger"
|
||||||
|
affected: str,
|
||||||
|
condition: Callable[[dict], tuple[bool, float, dict]],
|
||||||
|
message_template: str,
|
||||||
|
):
|
||||||
|
self.name = name
|
||||||
|
self.event_type = event_type
|
||||||
|
self.level = level
|
||||||
|
self.affected = affected
|
||||||
|
self.condition = condition
|
||||||
|
self.message_template = message_template
|
||||||
|
|
||||||
|
|
||||||
|
# ── Rule definitions ──
|
||||||
|
|
||||||
|
def _check_drawdown_spread(features: dict) -> tuple[bool, float, dict]:
|
||||||
|
"""High-position stocks starting to drop broadly."""
|
||||||
|
fear = features.get("fear_vector", {})
|
||||||
|
pct_deep = fear.get("pct_deep_drawdown_20d", fear.get("pct_deep_drawdown", 0))
|
||||||
|
median_dd = abs(fear.get("median_drawdown_20d", fear.get("median_drawdown", 0)))
|
||||||
|
|
||||||
|
triggered = pct_deep > 0.30 or median_dd > 0.08
|
||||||
|
severity = min(max(pct_deep * 2 + median_dd * 5, 0), 1.0)
|
||||||
|
return triggered, severity, {
|
||||||
|
"pct_deep_drawdown": pct_deep,
|
||||||
|
"median_drawdown": median_dd,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _check_volume_drying(features: dict) -> tuple[bool, float, dict]:
|
||||||
|
"""Volume drying up significantly vs. recent average."""
|
||||||
|
vol = features.get("volume_vector", {})
|
||||||
|
vs_5d = vol.get("vs_5d", 1.0)
|
||||||
|
vs_20d = vol.get("vs_20d", 1.0)
|
||||||
|
|
||||||
|
triggered = vs_5d < 0.80 or vs_20d < 0.75
|
||||||
|
severity = min(max((1.0 - min(vs_5d, vs_20d)) * 2, 0), 1.0)
|
||||||
|
return triggered, severity, {
|
||||||
|
"vs_5d": vs_5d,
|
||||||
|
"vs_20d": vs_20d,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _check_breadth_collapse(features: dict) -> tuple[bool, float, dict]:
|
||||||
|
"""Market breadth collapsing — very few stocks advancing."""
|
||||||
|
breadth = features.get("breadth_vector", {})
|
||||||
|
advance_ratio = breadth.get("advance_ratio", 0.5)
|
||||||
|
down_prev = breadth.get("down_from_prev", 0)
|
||||||
|
up_prev = breadth.get("up_from_prev", 0)
|
||||||
|
total = breadth.get("advances", 0) + breadth.get("declines", 0) + breadth.get("unchanged", 0)
|
||||||
|
|
||||||
|
triggered = advance_ratio < 0.25
|
||||||
|
severity = min(max((0.5 - advance_ratio) * 2, 0), 1.0)
|
||||||
|
|
||||||
|
down_pct = round(down_prev / total * 100, 1) if total > 0 else 0
|
||||||
|
return triggered, severity, {
|
||||||
|
"advance_ratio": advance_ratio,
|
||||||
|
"down_pct": down_pct,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _check_high_volatility(features: dict) -> tuple[bool, float, dict]:
|
||||||
|
"""Market volatility spiking — unusual daily ranges."""
|
||||||
|
fear = features.get("fear_vector", {})
|
||||||
|
avg_range = fear.get("daily_range_pct", fear.get("avg_daily_range", 0.02))
|
||||||
|
|
||||||
|
triggered = avg_range > 0.05
|
||||||
|
severity = min(max((avg_range - 0.03) / 0.05, 0), 1.0)
|
||||||
|
return triggered, severity, {
|
||||||
|
"avg_daily_range": avg_range,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _check_divergence(features: dict) -> tuple[bool, float, dict]:
|
||||||
|
"""Indices diverging — some up, some down strongly."""
|
||||||
|
trend = features.get("trend_vector", {})
|
||||||
|
divergence = trend.get("divergence", 0.0)
|
||||||
|
|
||||||
|
triggered = divergence > 0.7
|
||||||
|
severity = min(divergence, 1.0)
|
||||||
|
return triggered, severity, {
|
||||||
|
"divergence": divergence,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
RISK_RULES: list[RiskRule] = [
|
||||||
|
RiskRule(
|
||||||
|
name="DrawdownSpread",
|
||||||
|
event_type="DrawdownSpread",
|
||||||
|
level="danger",
|
||||||
|
affected="All",
|
||||||
|
condition=_check_drawdown_spread,
|
||||||
|
message_template="高位股开始补跌:{pct_deep}% 的股票从高点回撤超 10%",
|
||||||
|
),
|
||||||
|
RiskRule(
|
||||||
|
name="BreadthCollapse",
|
||||||
|
event_type="BreadthCollapse",
|
||||||
|
level="danger",
|
||||||
|
affected="All",
|
||||||
|
condition=_check_breadth_collapse,
|
||||||
|
message_template="上涨家数仅 {advance_pct}%,{down_pct}% 的股票下跌",
|
||||||
|
),
|
||||||
|
RiskRule(
|
||||||
|
name="VolumeDrying",
|
||||||
|
event_type="VolumeDrying",
|
||||||
|
level="warning",
|
||||||
|
affected="All",
|
||||||
|
condition=_check_volume_drying,
|
||||||
|
message_template="成交额较5日均值下降 {vol_change}%",
|
||||||
|
),
|
||||||
|
RiskRule(
|
||||||
|
name="HighVolatility",
|
||||||
|
event_type="HighVolatility",
|
||||||
|
level="warning",
|
||||||
|
affected="All",
|
||||||
|
condition=_check_high_volatility,
|
||||||
|
message_template="市场波动加剧,日均振幅达 {range_pct}%",
|
||||||
|
),
|
||||||
|
RiskRule(
|
||||||
|
name="Divergence",
|
||||||
|
event_type="Divergence",
|
||||||
|
level="warning",
|
||||||
|
affected="All",
|
||||||
|
condition=_check_divergence,
|
||||||
|
message_template="指数走势分化,共振度低",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def evaluate_risks(
|
||||||
|
features: dict[str, dict[str, Any]],
|
||||||
|
timestamp: datetime | None = None,
|
||||||
|
) -> list[RiskEvent]:
|
||||||
|
"""Evaluate all risk rules against current features.
|
||||||
|
|
||||||
|
Returns triggered RiskEvents sorted by severity descending.
|
||||||
|
"""
|
||||||
|
if timestamp is None:
|
||||||
|
timestamp = datetime.now()
|
||||||
|
|
||||||
|
events: list[RiskEvent] = []
|
||||||
|
|
||||||
|
for rule in RISK_RULES:
|
||||||
|
try:
|
||||||
|
triggered, severity, payload = rule.condition(features)
|
||||||
|
except Exception:
|
||||||
|
continue # Skip rules that fail to evaluate
|
||||||
|
|
||||||
|
if not triggered:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Build human-readable message from template + payload
|
||||||
|
message = rule.message_template
|
||||||
|
|
||||||
|
# Build a formatting context with common derived values
|
||||||
|
fmt_ctx = {}
|
||||||
|
for k, v in payload.items():
|
||||||
|
if isinstance(v, float):
|
||||||
|
fmt_ctx[k] = round(v, 3)
|
||||||
|
else:
|
||||||
|
fmt_ctx[k] = v
|
||||||
|
|
||||||
|
# Add derived values for common templates
|
||||||
|
if "pct_deep_drawdown" in payload:
|
||||||
|
fmt_ctx["pct_deep"] = round(payload["pct_deep_drawdown"] * 100, 1)
|
||||||
|
if "pct_deep_drawdown_20d" in payload:
|
||||||
|
fmt_ctx["pct_deep"] = round(payload["pct_deep_drawdown_20d"] * 100, 1)
|
||||||
|
if "advance_ratio" in payload:
|
||||||
|
fmt_ctx["advance_pct"] = round(payload["advance_ratio"] * 100, 1)
|
||||||
|
if "vs_5d" in payload:
|
||||||
|
fmt_ctx["vol_change"] = round((1 - payload["vs_5d"]) * 100, 1)
|
||||||
|
if "avg_daily_range" in payload:
|
||||||
|
fmt_ctx["range_pct"] = round(payload["avg_daily_range"] * 100, 1)
|
||||||
|
|
||||||
|
try:
|
||||||
|
message = message.format(**fmt_ctx)
|
||||||
|
except (KeyError, ValueError):
|
||||||
|
pass # Keep template as-is if formatting fails
|
||||||
|
|
||||||
|
events.append(RiskEvent(
|
||||||
|
type=rule.event_type,
|
||||||
|
severity=round(severity, 4),
|
||||||
|
level=rule.level,
|
||||||
|
affected=rule.affected,
|
||||||
|
confidence=round(severity, 4), # Phase 1: severity ≈ confidence
|
||||||
|
message=message,
|
||||||
|
payload=payload,
|
||||||
|
timestamp=timestamp,
|
||||||
|
))
|
||||||
|
|
||||||
|
# Sort: danger first, then by severity desc
|
||||||
|
events.sort(key=lambda e: (0 if e.level == "danger" else 1, -e.severity))
|
||||||
|
return events
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
"""Feature Store — centralized market feature computation.
|
||||||
|
|
||||||
|
All feature modules register themselves with the FeatureRegistry on import.
|
||||||
|
Engine code imports only the registry, never individual feature functions.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
from ashare_dp.features import registry
|
||||||
|
features = registry.compute_all(db, trade_date)
|
||||||
|
"""
|
||||||
|
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
|
||||||
|
# Import feature modules to trigger registration
|
||||||
|
from ashare_dp.features import breadth # noqa: F401
|
||||||
|
from ashare_dp.features import volume # noqa: F401
|
||||||
|
from ashare_dp.features import trend # noqa: F401
|
||||||
|
from ashare_dp.features import fear # noqa: F401
|
||||||
|
from ashare_dp.features import rotation # noqa: F401
|
||||||
|
from ashare_dp.features import participation # noqa: F401
|
||||||
|
|
||||||
|
__all__ = ["registry"]
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
"""Breadth features: advance/decline, new highs/lows, up/down volume ratio."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_breadth_vector(db: Any, trade_date: date, _results: dict) -> dict[str, Any]:
|
||||||
|
"""Compute market breadth metrics for a single trading day.
|
||||||
|
|
||||||
|
Uses DuckDB to aggregate over ALL stocks' daily K-line in one query.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Single query: advance/decline, new highs/lows, up/down volume
|
||||||
|
sql = f"""
|
||||||
|
WITH latest AS (
|
||||||
|
SELECT *, ROW_NUMBER() OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time DESC
|
||||||
|
) as rn
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date <= $trade_date
|
||||||
|
),
|
||||||
|
today AS (
|
||||||
|
SELECT * FROM latest WHERE rn = 1
|
||||||
|
),
|
||||||
|
prev AS (
|
||||||
|
SELECT * FROM latest WHERE rn = 2
|
||||||
|
),
|
||||||
|
with_prev AS (
|
||||||
|
SELECT
|
||||||
|
t.ts_code,
|
||||||
|
t.close AS close_today,
|
||||||
|
t.open AS open_today,
|
||||||
|
t.volume AS vol_today,
|
||||||
|
t.amount AS amt_today,
|
||||||
|
t.high AS high_today,
|
||||||
|
t.low AS low_today,
|
||||||
|
p.close AS close_prev,
|
||||||
|
-- 20-day and 60-day rolling high
|
||||||
|
MAX(t.close) OVER (
|
||||||
|
PARTITION BY t.ts_code
|
||||||
|
ORDER BY t.trade_date
|
||||||
|
ROWS BETWEEN 19 PRECEDING AND CURRENT ROW
|
||||||
|
) AS high_20d,
|
||||||
|
MAX(t.close) OVER (
|
||||||
|
PARTITION BY t.ts_code
|
||||||
|
ORDER BY t.trade_date
|
||||||
|
ROWS BETWEEN 59 PRECEDING AND CURRENT ROW
|
||||||
|
) AS high_60d,
|
||||||
|
MIN(t.close) OVER (
|
||||||
|
PARTITION BY t.ts_code
|
||||||
|
ORDER BY t.trade_date
|
||||||
|
ROWS BETWEEN 19 PRECEDING AND CURRENT ROW
|
||||||
|
) AS low_20d,
|
||||||
|
MIN(t.close) OVER (
|
||||||
|
PARTITION BY t.ts_code
|
||||||
|
ORDER BY t.trade_date
|
||||||
|
ROWS BETWEEN 59 PRECEDING AND CURRENT ROW
|
||||||
|
) AS low_60d
|
||||||
|
FROM today t
|
||||||
|
LEFT JOIN prev p ON t.ts_code = p.ts_code
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
COUNT(*) AS total,
|
||||||
|
COUNT(*) FILTER (WHERE close_today > open_today) AS advances,
|
||||||
|
COUNT(*) FILTER (WHERE close_today < open_today) AS declines,
|
||||||
|
COUNT(*) FILTER (WHERE close_today = open_today) AS unchanged,
|
||||||
|
COUNT(*) FILTER (WHERE close_today > close_prev AND close_prev IS NOT NULL) AS up_from_prev,
|
||||||
|
COUNT(*) FILTER (WHERE close_today < close_prev AND close_prev IS NOT NULL) AS down_from_prev,
|
||||||
|
COUNT(*) FILTER (WHERE close_today >= high_20d * 0.98) AS near_20d_high,
|
||||||
|
COUNT(*) FILTER (WHERE close_today <= low_20d * 1.02) AS near_20d_low,
|
||||||
|
COUNT(*) FILTER (WHERE close_today >= high_60d * 0.98) AS near_60d_high,
|
||||||
|
COUNT(*) FILTER (WHERE close_today <= low_60d * 1.02) AS near_60d_low,
|
||||||
|
SUM(vol_today) AS total_volume,
|
||||||
|
SUM(amt_today) AS total_amount,
|
||||||
|
SUM(CASE WHEN close_today > open_today THEN vol_today ELSE 0 END) AS up_volume,
|
||||||
|
SUM(CASE WHEN close_today < open_today THEN vol_today ELSE 0 END) AS down_volume,
|
||||||
|
AVG((close_today - open_today) / NULLIF(open_today, 0) * 100) AS avg_pct_chg,
|
||||||
|
AVG((high_today - low_today) / NULLIF(close_today, 0)) AS avg_daily_range
|
||||||
|
FROM with_prev
|
||||||
|
"""
|
||||||
|
result = conn.execute(sql, {"trade_date": trade_date.isoformat()}).fetchone()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result is None or result[0] == 0:
|
||||||
|
return {
|
||||||
|
"advance_ratio": 0.5,
|
||||||
|
"advances": 0,
|
||||||
|
"declines": 0,
|
||||||
|
"unchanged": 0,
|
||||||
|
"up_from_prev": 0,
|
||||||
|
"down_from_prev": 0,
|
||||||
|
"new_high_ratio": 0.0,
|
||||||
|
"new_low_ratio": 0.0,
|
||||||
|
"up_vol_ratio": 0.5,
|
||||||
|
"total_volume": 0,
|
||||||
|
"total_amount": 0,
|
||||||
|
"avg_pct_chg": 0.0,
|
||||||
|
"avg_daily_range": 0.02,
|
||||||
|
}
|
||||||
|
|
||||||
|
total = result[0]
|
||||||
|
advances = result[1]
|
||||||
|
declines = result[2]
|
||||||
|
unchanged = result[3]
|
||||||
|
up_prev = result[4]
|
||||||
|
down_prev = result[5]
|
||||||
|
near_20h = result[6]
|
||||||
|
near_20l = result[7]
|
||||||
|
near_60h = result[8]
|
||||||
|
near_60l = result[9]
|
||||||
|
total_vol = result[10] or 0
|
||||||
|
total_amt = result[11] or 0
|
||||||
|
up_vol = result[12] or 0
|
||||||
|
down_vol = result[13] or 0
|
||||||
|
|
||||||
|
return {
|
||||||
|
"advance_ratio": round(advances / total, 4) if total else 0.5,
|
||||||
|
"advances": advances,
|
||||||
|
"declines": declines,
|
||||||
|
"unchanged": unchanged,
|
||||||
|
"up_from_prev": up_prev,
|
||||||
|
"down_from_prev": down_prev,
|
||||||
|
"new_high_ratio": round(max(near_20h, near_60h) / total, 4) if total else 0.0,
|
||||||
|
"new_low_ratio": round(max(near_20l, near_60l) / total, 4) if total else 0.0,
|
||||||
|
"up_vol_ratio": round(up_vol / (up_vol + down_vol), 4) if (up_vol + down_vol) > 0 else 0.5,
|
||||||
|
"total_volume": int(total_vol),
|
||||||
|
"total_amount": float(total_amt),
|
||||||
|
"avg_pct_chg": round(float(result[14] or 0), 4),
|
||||||
|
"avg_daily_range": round(float(result[15] or 0.02), 4),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
registry.register(FeatureDefinition(
|
||||||
|
name="breadth_vector",
|
||||||
|
category="breadth",
|
||||||
|
description="Advance/decline ratio, new high/low ratios, up/down volume ratio, total turnover",
|
||||||
|
dependencies=[],
|
||||||
|
compute=_compute_breadth_vector,
|
||||||
|
))
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
"""Fear features: drawdown depth, breadth of deep drawdowns, intraday range."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_fear_vector(db: Any, trade_date: date, _results: dict) -> dict[str, Any]:
|
||||||
|
"""Compute fear/greed metrics: drawdown distribution, intraday volatility.
|
||||||
|
|
||||||
|
Uses a single DuckDB window query to compute drawdown from 20d and 60d
|
||||||
|
highs for all stocks, then aggregates the distribution.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH ranked AS (
|
||||||
|
SELECT *,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time DESC) as rn_desc,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time ASC) as rn_asc
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $start_date
|
||||||
|
),
|
||||||
|
latest AS (
|
||||||
|
SELECT * FROM ranked WHERE rn_desc = 1
|
||||||
|
),
|
||||||
|
-- 20-day high per stock
|
||||||
|
high20 AS (
|
||||||
|
SELECT ts_code, MAX(high) as h20
|
||||||
|
FROM ranked
|
||||||
|
WHERE rn_desc <= 20
|
||||||
|
GROUP BY ts_code
|
||||||
|
),
|
||||||
|
-- 60-day high per stock
|
||||||
|
high60 AS (
|
||||||
|
SELECT ts_code, MAX(high) as h60
|
||||||
|
FROM ranked
|
||||||
|
WHERE rn_desc <= 60
|
||||||
|
GROUP BY ts_code
|
||||||
|
),
|
||||||
|
-- Join with latest
|
||||||
|
combined AS (
|
||||||
|
SELECT
|
||||||
|
l.ts_code, l.close, l.high, l.low, l.open,
|
||||||
|
h20.h20, h60.h60,
|
||||||
|
(l.high - l.low) / NULLIF(l.close, 0) AS daily_range
|
||||||
|
FROM latest l
|
||||||
|
JOIN high20 h20 ON l.ts_code = h20.ts_code
|
||||||
|
JOIN high60 h60 ON l.ts_code = h60.ts_code
|
||||||
|
),
|
||||||
|
drawdowns AS (
|
||||||
|
SELECT *,
|
||||||
|
(close - h20) / NULLIF(h20, 0) AS dd_20d,
|
||||||
|
(close - h60) / NULLIF(h60, 0) AS dd_60d
|
||||||
|
FROM combined
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY dd_20d) AS dd20_p25,
|
||||||
|
PERCENTILE_CONT(0.50) WITHIN GROUP (ORDER BY dd_20d) AS dd20_p50,
|
||||||
|
PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY dd_20d) AS dd20_p75,
|
||||||
|
PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY dd_60d) AS dd60_p25,
|
||||||
|
PERCENTILE_CONT(0.50) WITHIN GROUP (ORDER BY dd_60d) AS dd60_p50,
|
||||||
|
PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY dd_60d) AS dd60_p75,
|
||||||
|
COUNT(*) FILTER (WHERE dd_20d < -0.10) AS deep_dd_20_count,
|
||||||
|
COUNT(*) FILTER (WHERE dd_60d < -0.20) AS deep_dd_60_count,
|
||||||
|
COUNT(*) AS total,
|
||||||
|
PERCENTILE_CONT(0.25) WITHIN GROUP (ORDER BY daily_range) AS range_p25,
|
||||||
|
PERCENTILE_CONT(0.50) WITHIN GROUP (ORDER BY daily_range) AS range_p50,
|
||||||
|
PERCENTILE_CONT(0.75) WITHIN GROUP (ORDER BY daily_range) AS range_p75,
|
||||||
|
AVG(daily_range) AS avg_daily_range
|
||||||
|
FROM drawdowns
|
||||||
|
WHERE h20 IS NOT NULL
|
||||||
|
"""
|
||||||
|
# Look back ~60 trading days ≈ 90 calendar days
|
||||||
|
start_date = trade_date.strftime("%Y-%m-%d")
|
||||||
|
# Use a window that covers at least 60 trading days
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start_date", f"'{_date_sub_calendar(trade_date, 95)}'")
|
||||||
|
).fetchone()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result is None or result[8] == 0:
|
||||||
|
return _empty_fear()
|
||||||
|
|
||||||
|
total = int(result[8])
|
||||||
|
|
||||||
|
return {
|
||||||
|
"median_drawdown_20d": round(float(result[1] or 0), 4),
|
||||||
|
"median_drawdown_60d": round(float(result[4] or 0), 4),
|
||||||
|
"pct_deep_drawdown_20d": round(int(result[6]) / total, 4),
|
||||||
|
"pct_deep_drawdown_60d": round(int(result[7]) / total, 4),
|
||||||
|
# Legacy keys for backward compat with MarketState.from_features()
|
||||||
|
"median_drawdown": round(float(result[1] or 0), 4),
|
||||||
|
"pct_deep_drawdown": round(int(result[6]) / total, 4),
|
||||||
|
"daily_range_pct": round(float(result[12] or 0.02), 4),
|
||||||
|
"daily_range_p50": round(float(result[10] or 0.02), 4),
|
||||||
|
"total_stocks": total,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _date_sub_calendar(trade_date: date, days: int) -> str:
|
||||||
|
"""Approximate calendar date N days before trade_date."""
|
||||||
|
from datetime import timedelta
|
||||||
|
return (trade_date - timedelta(days=days)).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_fear() -> dict:
|
||||||
|
return {
|
||||||
|
"median_drawdown_20d": 0.0,
|
||||||
|
"median_drawdown_60d": 0.0,
|
||||||
|
"pct_deep_drawdown_20d": 0.0,
|
||||||
|
"pct_deep_drawdown_60d": 0.0,
|
||||||
|
"median_drawdown": 0.0,
|
||||||
|
"pct_deep_drawdown": 0.0,
|
||||||
|
"daily_range_pct": 0.02,
|
||||||
|
"daily_range_p50": 0.02,
|
||||||
|
"total_stocks": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
registry.register(FeatureDefinition(
|
||||||
|
name="fear_vector",
|
||||||
|
category="fear",
|
||||||
|
description="Drawdown distribution (20d/60d), deep drawdown breadth, intraday range",
|
||||||
|
dependencies=[],
|
||||||
|
compute=_compute_fear_vector,
|
||||||
|
))
|
||||||
@@ -0,0 +1,93 @@
|
|||||||
|
"""Participation features: breadth of market participation."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_participation_vector(db: Any, trade_date: date, _results: dict) -> dict[str, Any]:
|
||||||
|
"""Compute market participation: % stocks above key MAs, volume participation rate."""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH ranked AS (
|
||||||
|
SELECT *,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time DESC) as rn_desc
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date <= $trade_date
|
||||||
|
),
|
||||||
|
with_ma AS (
|
||||||
|
SELECT
|
||||||
|
ts_code,
|
||||||
|
close,
|
||||||
|
volume,
|
||||||
|
AVG(close) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 19 PRECEDING AND CURRENT ROW
|
||||||
|
) AS ma20,
|
||||||
|
AVG(close) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 59 PRECEDING AND CURRENT ROW
|
||||||
|
) AS ma60,
|
||||||
|
AVG(volume) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 19 PRECEDING AND CURRENT ROW
|
||||||
|
) AS vol_ma20,
|
||||||
|
rn_desc
|
||||||
|
FROM ranked
|
||||||
|
),
|
||||||
|
latest AS (
|
||||||
|
SELECT * FROM with_ma WHERE rn_desc = 1
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
COUNT(*) AS total,
|
||||||
|
COUNT(*) FILTER (WHERE close > ma20) AS above_ma20,
|
||||||
|
COUNT(*) FILTER (WHERE close > ma60) AS above_ma60,
|
||||||
|
COUNT(*) FILTER (WHERE volume > vol_ma20 AND vol_ma20 > 0) AS above_vol_ma20,
|
||||||
|
AVG(CASE WHEN vol_ma20 > 0 THEN volume / vol_ma20 ELSE NULL END) AS avg_vol_ratio
|
||||||
|
FROM latest
|
||||||
|
"""
|
||||||
|
result = conn.execute(sql, {"trade_date": trade_date.isoformat()}).fetchone()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result is None or result[0] == 0:
|
||||||
|
return _empty_participation()
|
||||||
|
|
||||||
|
total = int(result[0])
|
||||||
|
|
||||||
|
return {
|
||||||
|
"pct_above_ma20": round(int(result[1]) / total, 4),
|
||||||
|
"pct_above_ma60": round(int(result[2]) / total, 4),
|
||||||
|
"pct_above_vol_ma20": round(int(result[3]) / total, 4),
|
||||||
|
"avg_vol_ratio": round(float(result[4] or 1.0), 4),
|
||||||
|
"total_stocks": total,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_participation() -> dict:
|
||||||
|
return {
|
||||||
|
"pct_above_ma20": 0.5,
|
||||||
|
"pct_above_ma60": 0.5,
|
||||||
|
"pct_above_vol_ma20": 0.5,
|
||||||
|
"avg_vol_ratio": 1.0,
|
||||||
|
"total_stocks": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
registry.register(FeatureDefinition(
|
||||||
|
name="participation_vector",
|
||||||
|
category="participation",
|
||||||
|
description="% stocks above MA20/MA60, volume participation rate",
|
||||||
|
dependencies=[],
|
||||||
|
compute=_compute_participation_vector,
|
||||||
|
))
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
"""Feature Registry — centralized management of all market features.
|
||||||
|
|
||||||
|
Engine code only knows about the registry; individual feature functions
|
||||||
|
are looked up by name. Adding a new feature = register it, no engine changes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
|
||||||
|
|
||||||
|
class FeatureRegistry:
|
||||||
|
"""Central registry for all market features.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
|
||||||
|
# Register a feature
|
||||||
|
registry.register(FeatureDefinition(
|
||||||
|
name="breadth_vector",
|
||||||
|
category="breadth",
|
||||||
|
description="Advance/decline ratio and related breadth metrics",
|
||||||
|
dependencies=[],
|
||||||
|
compute=_compute_breadth_vector,
|
||||||
|
))
|
||||||
|
|
||||||
|
# Compute all features for a date
|
||||||
|
features = registry.compute_all(db, trade_date)
|
||||||
|
|
||||||
|
# Get features by category
|
||||||
|
breadth_features = registry.get_by_category("breadth")
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self._features: dict[str, FeatureDefinition] = {}
|
||||||
|
|
||||||
|
def register(self, feature: FeatureDefinition) -> None:
|
||||||
|
"""Register a feature definition. Overwrites if name exists."""
|
||||||
|
self._features[feature.name] = feature
|
||||||
|
|
||||||
|
def get(self, name: str) -> FeatureDefinition | None:
|
||||||
|
"""Get a feature definition by name."""
|
||||||
|
return self._features.get(name)
|
||||||
|
|
||||||
|
def get_by_category(self, category: str) -> list[FeatureDefinition]:
|
||||||
|
"""Get all features in a category."""
|
||||||
|
return [f for f in self._features.values() if f.category == category]
|
||||||
|
|
||||||
|
def compute_all(self, db: Any, trade_date: date) -> dict[str, dict[str, Any]]:
|
||||||
|
"""Compute all registered features for a given trading date.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
db: Database instance (read_only DuckDB connection).
|
||||||
|
trade_date: The trading date to compute features for.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Dict mapping feature name → feature value dict.
|
||||||
|
Features are computed in dependency order (simple topological sort).
|
||||||
|
"""
|
||||||
|
results: dict[str, dict[str, Any]] = {}
|
||||||
|
computed: set[str] = set()
|
||||||
|
pending: set[str] = set(self._features.keys())
|
||||||
|
|
||||||
|
while pending:
|
||||||
|
ready = [
|
||||||
|
name for name in pending
|
||||||
|
if all(dep in computed for dep in self._features[name].dependencies)
|
||||||
|
]
|
||||||
|
if not ready:
|
||||||
|
# Circular dependency or missing dependency
|
||||||
|
remaining = ", ".join(sorted(pending))
|
||||||
|
raise RuntimeError(
|
||||||
|
f"Cannot resolve feature dependencies. "
|
||||||
|
f"Remaining: {remaining}. Computed: {computed}"
|
||||||
|
)
|
||||||
|
|
||||||
|
for name in ready:
|
||||||
|
feature = self._features[name]
|
||||||
|
if feature.compute is not None:
|
||||||
|
results[name] = feature.compute(db, trade_date, results)
|
||||||
|
computed.add(name)
|
||||||
|
pending.discard(name)
|
||||||
|
|
||||||
|
return results
|
||||||
|
|
||||||
|
def compute_category(
|
||||||
|
self, db: Any, trade_date: date, category: str
|
||||||
|
) -> dict[str, dict[str, Any]]:
|
||||||
|
"""Compute all features in a specific category."""
|
||||||
|
features = self.get_by_category(category)
|
||||||
|
# Compute dependencies first
|
||||||
|
all_needed: set[str] = set()
|
||||||
|
for f in features:
|
||||||
|
all_needed.add(f.name)
|
||||||
|
all_needed.update(f.dependencies)
|
||||||
|
|
||||||
|
full_results = self.compute_all(db, trade_date)
|
||||||
|
return {k: v for k, v in full_results.items() if k in all_needed}
|
||||||
|
|
||||||
|
def list_categories(self) -> list[str]:
|
||||||
|
"""List all unique feature categories."""
|
||||||
|
return sorted({f.category for f in self._features.values()})
|
||||||
|
|
||||||
|
def list_features(self) -> list[dict]:
|
||||||
|
"""List all registered features with metadata."""
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"name": f.name,
|
||||||
|
"category": f.category,
|
||||||
|
"description": f.description,
|
||||||
|
"dependencies": f.dependencies,
|
||||||
|
"version": f.version,
|
||||||
|
}
|
||||||
|
for f in sorted(self._features.values(), key=lambda x: x.name)
|
||||||
|
]
|
||||||
|
|
||||||
|
def __len__(self) -> int:
|
||||||
|
return len(self._features)
|
||||||
|
|
||||||
|
def __contains__(self, name: str) -> bool:
|
||||||
|
return name in self._features
|
||||||
|
|
||||||
|
|
||||||
|
# Module-level singleton
|
||||||
|
registry = FeatureRegistry()
|
||||||
@@ -0,0 +1,316 @@
|
|||||||
|
"""Rotation features: sector rotation speed, persistence, concentration.
|
||||||
|
|
||||||
|
Also computes the RotationGraph (nodes + edges) from industry turnover changes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
|
||||||
|
from ashare_dp.domain.events import RotationEdge, RotationGraph
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_rotation_vector(db: Any, trade_date: date, _results: dict) -> dict[str, Any]:
|
||||||
|
"""Compute rotation intensity metrics.
|
||||||
|
|
||||||
|
Measures how fast money is rotating between industries and how
|
||||||
|
persistent the current leaders are.
|
||||||
|
"""
|
||||||
|
# Use the compute_rotation_graph result for rotation metrics
|
||||||
|
rg = compute_rotation_graph_from_db(db, trade_date, lookback=20)
|
||||||
|
momentum = compute_industry_momentum_from_db(db, trade_date)
|
||||||
|
|
||||||
|
if not rg.edges:
|
||||||
|
return _empty_rotation()
|
||||||
|
|
||||||
|
# Speed: average edge strength (higher = faster rotation)
|
||||||
|
speed = sum(e.strength for e in rg.edges) / len(rg.edges) if rg.edges else 0.0
|
||||||
|
|
||||||
|
# Persistence: fraction of top-5 industries today that were also top-5 5 days ago
|
||||||
|
# (simplified: use the strongest edge's persistence_days)
|
||||||
|
persistence = max((e.persistence_days / 10 for e in rg.edges), default=0.0)
|
||||||
|
persistence = min(persistence, 1.0)
|
||||||
|
|
||||||
|
# Concentration: how concentrated are flows in the top 3 edges?
|
||||||
|
if len(rg.edges) >= 3:
|
||||||
|
top3 = sum(e.strength for e in rg.edges[:3])
|
||||||
|
total = sum(e.strength for e in rg.edges)
|
||||||
|
concentration = top3 / total if total > 0 else 0.5
|
||||||
|
else:
|
||||||
|
concentration = 0.5
|
||||||
|
|
||||||
|
return {
|
||||||
|
"speed": round(speed, 4),
|
||||||
|
"persistence": round(persistence, 4),
|
||||||
|
"concentration": round(concentration, 4),
|
||||||
|
"edge_count": len(rg.edges),
|
||||||
|
"node_count": len(rg.nodes),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def compute_rotation_graph_from_db(
|
||||||
|
db: Any, trade_date: date, lookback: int = 20
|
||||||
|
) -> RotationGraph:
|
||||||
|
"""Build rotation graph from industry-level turnover changes.
|
||||||
|
|
||||||
|
Uses stock_industry table joined with daily K-line to compute
|
||||||
|
per-industry turnover trends, then generates directed edges
|
||||||
|
where money flows from declining industries to rising ones.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Get industries and their recent turnover trends
|
||||||
|
sql = f"""
|
||||||
|
WITH daily AS (
|
||||||
|
SELECT
|
||||||
|
k.ts_code AS ts_code,
|
||||||
|
k.trade_date,
|
||||||
|
k.amount,
|
||||||
|
si.industry_name
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true) k
|
||||||
|
JOIN stock_industry si ON k.ts_code = si.ts_code
|
||||||
|
WHERE k.trade_date >= $start_date AND k.trade_date <= $end_date
|
||||||
|
),
|
||||||
|
industry_daily AS (
|
||||||
|
SELECT
|
||||||
|
industry_name,
|
||||||
|
trade_date,
|
||||||
|
SUM(amount) AS total_amount,
|
||||||
|
COUNT(*) AS stock_count
|
||||||
|
FROM daily
|
||||||
|
GROUP BY industry_name, trade_date
|
||||||
|
),
|
||||||
|
-- Recent (last 5 trading days) vs prior (5 trading days before that)
|
||||||
|
ranked AS (
|
||||||
|
SELECT *,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY industry_name ORDER BY trade_date DESC) as rn
|
||||||
|
FROM industry_daily
|
||||||
|
),
|
||||||
|
recent AS (
|
||||||
|
SELECT industry_name, SUM(total_amount) as recent_amount,
|
||||||
|
SUM(stock_count) as recent_count
|
||||||
|
FROM ranked WHERE rn <= 5
|
||||||
|
GROUP BY industry_name
|
||||||
|
),
|
||||||
|
prior AS (
|
||||||
|
SELECT industry_name, SUM(total_amount) as prior_amount
|
||||||
|
FROM ranked WHERE rn > 5 AND rn <= 10
|
||||||
|
GROUP BY industry_name
|
||||||
|
),
|
||||||
|
changes AS (
|
||||||
|
SELECT
|
||||||
|
COALESCE(r.industry_name, p.industry_name) AS industry_name,
|
||||||
|
COALESCE(r.recent_amount, 0) AS recent_amount,
|
||||||
|
COALESCE(p.prior_amount, 0) AS prior_amount,
|
||||||
|
CASE
|
||||||
|
WHEN COALESCE(p.prior_amount, 0) > 0
|
||||||
|
THEN (COALESCE(r.recent_amount, 0) - p.prior_amount) / p.prior_amount
|
||||||
|
ELSE 0
|
||||||
|
END AS turnover_change,
|
||||||
|
COALESCE(r.recent_count, 0) AS stock_count
|
||||||
|
FROM recent r
|
||||||
|
FULL OUTER JOIN prior p ON r.industry_name = p.industry_name
|
||||||
|
WHERE COALESCE(r.recent_amount, 0) + COALESCE(p.prior_amount, 0) > 0
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
industry_name,
|
||||||
|
recent_amount,
|
||||||
|
prior_amount,
|
||||||
|
turnover_change,
|
||||||
|
stock_count
|
||||||
|
FROM changes
|
||||||
|
ORDER BY turnover_change DESC
|
||||||
|
"""
|
||||||
|
start_str = (trade_date - __timedelta(lookback + 10)).strftime("%Y-%m-%d")
|
||||||
|
end_str = trade_date.strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start_date", f"'{start_str}'").replace("$end_date", f"'{end_str}'")
|
||||||
|
).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return RotationGraph(nodes=[], edges=[])
|
||||||
|
|
||||||
|
industries = result["industry_name"].tolist()
|
||||||
|
changes = result.set_index("industry_name")["turnover_change"].to_dict()
|
||||||
|
|
||||||
|
# Generate edges: from declining industries to rising ones
|
||||||
|
edges = []
|
||||||
|
gainers = sorted(
|
||||||
|
[(k, v) for k, v in changes.items() if v > 0.05],
|
||||||
|
key=lambda x: x[1], reverse=True,
|
||||||
|
)
|
||||||
|
losers = sorted(
|
||||||
|
[(k, v) for k, v in changes.items() if v < -0.05],
|
||||||
|
key=lambda x: x[1],
|
||||||
|
)
|
||||||
|
|
||||||
|
# Create edges from top losers to top gainers
|
||||||
|
max_edges = min(len(gainers), len(losers), 5)
|
||||||
|
for i in range(max_edges):
|
||||||
|
los_name, los_change = losers[i]
|
||||||
|
win_name, win_change = gainers[i]
|
||||||
|
strength = min((win_change - los_change) / 2.0, 1.0)
|
||||||
|
edges.append(RotationEdge(
|
||||||
|
source=los_name,
|
||||||
|
target=win_name,
|
||||||
|
strength=round(strength, 4),
|
||||||
|
confidence=round(min(abs(win_change), abs(los_change)) * 2, 4),
|
||||||
|
source_change=round(los_change, 4),
|
||||||
|
target_change=round(win_change, 4),
|
||||||
|
persistence_days=1, # Phase 1: simplified
|
||||||
|
))
|
||||||
|
|
||||||
|
return RotationGraph(
|
||||||
|
nodes=sorted(industries),
|
||||||
|
edges=sorted(edges, key=lambda e: e.strength, reverse=True),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def compute_industry_momentum_from_db(
|
||||||
|
db: Any, trade_date: date, windows: list[int] | None = None
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Compute per-industry momentum across multiple time windows.
|
||||||
|
|
||||||
|
Returns list of dicts with turnover change % for 5d, 10d, 20d windows.
|
||||||
|
"""
|
||||||
|
if windows is None:
|
||||||
|
windows = [5, 10, 20]
|
||||||
|
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
code_list = "', '".join(
|
||||||
|
f"'{c}'" for c in ["主板", "创业板", "科创板", "北交所"]
|
||||||
|
)
|
||||||
|
# Phase 1: use derived market as industry proxy if stock_industry is empty
|
||||||
|
# Check if stock_industry has data
|
||||||
|
check_sql = "SELECT COUNT(*) FROM stock_industry"
|
||||||
|
has_industry = conn.execute(check_sql).fetchone()[0] > 0
|
||||||
|
|
||||||
|
if has_industry:
|
||||||
|
join_clause = "JOIN stock_industry si ON k.ts_code = si.ts_code"
|
||||||
|
group_col = "si.industry_name"
|
||||||
|
else:
|
||||||
|
# Fallback: derive market from ts_code
|
||||||
|
join_clause = ""
|
||||||
|
group_col = """
|
||||||
|
CASE
|
||||||
|
WHEN k.ts_code LIKE '6%SH' OR k.ts_code LIKE '0%SZ' THEN '主板'
|
||||||
|
WHEN k.ts_code LIKE '3%SZ' THEN '创业板'
|
||||||
|
WHEN k.ts_code LIKE '688%SH' THEN '科创板'
|
||||||
|
WHEN k.ts_code LIKE '4%BJ' OR k.ts_code LIKE '8%BJ' OR k.ts_code LIKE '92%BJ' THEN '北交所'
|
||||||
|
ELSE '其他'
|
||||||
|
END
|
||||||
|
"""
|
||||||
|
|
||||||
|
sql = f"""
|
||||||
|
WITH daily AS (
|
||||||
|
SELECT
|
||||||
|
k.ts_code AS ts_code,
|
||||||
|
k.trade_date,
|
||||||
|
k.amount,
|
||||||
|
{group_col} AS segment
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true) k
|
||||||
|
{join_clause}
|
||||||
|
WHERE k.trade_date >= $start_date AND k.trade_date <= $end_date
|
||||||
|
),
|
||||||
|
segment_daily AS (
|
||||||
|
SELECT
|
||||||
|
segment,
|
||||||
|
trade_date,
|
||||||
|
SUM(amount) AS total_amount,
|
||||||
|
COUNT(DISTINCT ts_code) AS stock_count
|
||||||
|
FROM daily
|
||||||
|
GROUP BY segment, trade_date
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
segment,
|
||||||
|
MAX(trade_date) AS latest_date,
|
||||||
|
AVG(CASE WHEN trade_date >= $recent_5 THEN total_amount END) AS avg_5d,
|
||||||
|
AVG(CASE WHEN trade_date < $recent_5 AND trade_date >= $prior_5 THEN total_amount END) AS avg_prior_5d,
|
||||||
|
AVG(CASE WHEN trade_date >= $recent_10 THEN total_amount END) AS avg_10d,
|
||||||
|
AVG(CASE WHEN trade_date < $recent_10 AND trade_date >= $prior_10 THEN total_amount END) AS avg_prior_10d,
|
||||||
|
MAX(stock_count) AS stock_count
|
||||||
|
FROM segment_daily
|
||||||
|
GROUP BY segment
|
||||||
|
ORDER BY avg_5d DESC
|
||||||
|
"""
|
||||||
|
end_str = trade_date.strftime("%Y-%m-%d")
|
||||||
|
# Approximate trading day offsets
|
||||||
|
from datetime import timedelta
|
||||||
|
recent_5 = (trade_date - timedelta(days=7)).strftime("%Y-%m-%d")
|
||||||
|
prior_5 = (trade_date - timedelta(days=14)).strftime("%Y-%m-%d")
|
||||||
|
recent_10 = (trade_date - timedelta(days=14)).strftime("%Y-%m-%d")
|
||||||
|
prior_10 = (trade_date - timedelta(days=28)).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
result = conn.execute(
|
||||||
|
sql
|
||||||
|
.replace("$start_date", f"'{prior_10}'")
|
||||||
|
.replace("$end_date", f"'{end_str}'")
|
||||||
|
.replace("$recent_5", f"'{recent_5}'")
|
||||||
|
.replace("$prior_5", f"'{prior_5}'")
|
||||||
|
.replace("$recent_10", f"'{recent_10}'")
|
||||||
|
.replace("$prior_10", f"'{prior_10}'")
|
||||||
|
).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
industries = []
|
||||||
|
for _, row in result.iterrows():
|
||||||
|
avg_5 = float(row["avg_5d"] or 0)
|
||||||
|
avg_p5 = float(row["avg_prior_5d"] or 0)
|
||||||
|
avg_10 = float(row["avg_10d"] or 0)
|
||||||
|
avg_p10 = float(row["avg_prior_10d"] or 0)
|
||||||
|
|
||||||
|
chg_5d = round((avg_5 - avg_p5) / avg_p5, 4) if avg_p5 > 0 else 0.0
|
||||||
|
chg_10d = round((avg_10 - avg_p10) / avg_p10, 4) if avg_p10 > 0 else 0.0
|
||||||
|
|
||||||
|
industries.append({
|
||||||
|
"segment": row["segment"],
|
||||||
|
"stock_count": int(row["stock_count"]),
|
||||||
|
"turnover_change_5d": chg_5d,
|
||||||
|
"turnover_change_10d": chg_10d,
|
||||||
|
})
|
||||||
|
|
||||||
|
return industries
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_rotation() -> dict:
|
||||||
|
return {
|
||||||
|
"speed": 0.0,
|
||||||
|
"persistence": 0.5,
|
||||||
|
"concentration": 0.5,
|
||||||
|
"edge_count": 0,
|
||||||
|
"node_count": 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def __timedelta(days: int):
|
||||||
|
from datetime import timedelta
|
||||||
|
return timedelta(days=days)
|
||||||
|
|
||||||
|
|
||||||
|
# Exported for use by router
|
||||||
|
compute_rotation_graph = compute_rotation_graph_from_db
|
||||||
|
compute_industry_momentum = compute_industry_momentum_from_db
|
||||||
|
|
||||||
|
|
||||||
|
registry.register(FeatureDefinition(
|
||||||
|
name="rotation_vector",
|
||||||
|
category="rotation",
|
||||||
|
description="Sector rotation speed, persistence, concentration",
|
||||||
|
dependencies=[],
|
||||||
|
compute=_compute_rotation_vector,
|
||||||
|
))
|
||||||
@@ -0,0 +1,179 @@
|
|||||||
|
"""Trend features: index MA positioning, resonance/divergence, trend strength."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
|
||||||
|
from ashare_dp.core.models import INDEX_CODES
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
# Sina symbol format for indices used in Parquet ts_code
|
||||||
|
# The index codes stored in Parquet use format like "000001.SH"
|
||||||
|
INDEX_TS_CODES = list(INDEX_CODES.keys())
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_trend_vector(db: Any, trade_date: date, _results: dict) -> dict[str, Any]:
|
||||||
|
"""Compute trend metrics from index daily K-line data.
|
||||||
|
|
||||||
|
Analyzes 7 major indices for MA positioning, slope, and mutual alignment.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
code_list = "', '".join(INDEX_TS_CODES)
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH idx_data AS (
|
||||||
|
SELECT
|
||||||
|
ts_code,
|
||||||
|
trade_date,
|
||||||
|
trade_time,
|
||||||
|
close,
|
||||||
|
volume,
|
||||||
|
AVG(close) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 4 PRECEDING AND CURRENT ROW
|
||||||
|
) AS ma5,
|
||||||
|
AVG(close) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 9 PRECEDING AND CURRENT ROW
|
||||||
|
) AS ma10,
|
||||||
|
AVG(close) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 19 PRECEDING AND CURRENT ROW
|
||||||
|
) AS ma20,
|
||||||
|
AVG(close) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 59 PRECEDING AND CURRENT ROW
|
||||||
|
) AS ma60,
|
||||||
|
AVG(volume) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 19 PRECEDING AND CURRENT ROW
|
||||||
|
) AS vol_ma20,
|
||||||
|
ROW_NUMBER() OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time DESC
|
||||||
|
) as rn
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE ts_code IN ('{code_list}')
|
||||||
|
),
|
||||||
|
latest AS (
|
||||||
|
SELECT * FROM idx_data WHERE rn = 1
|
||||||
|
),
|
||||||
|
prev AS (
|
||||||
|
SELECT * FROM idx_data WHERE rn = 2
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
l.ts_code,
|
||||||
|
l.close,
|
||||||
|
l.ma5, l.ma10, l.ma20, l.ma60,
|
||||||
|
l.volume,
|
||||||
|
l.vol_ma20,
|
||||||
|
p.close AS prev_close,
|
||||||
|
-- price vs MAs
|
||||||
|
CASE WHEN l.close > l.ma5 THEN 1 ELSE 0 END AS above_ma5,
|
||||||
|
CASE WHEN l.close > l.ma10 THEN 1 ELSE 0 END AS above_ma10,
|
||||||
|
CASE WHEN l.close > l.ma20 THEN 1 ELSE 0 END AS above_ma20,
|
||||||
|
CASE WHEN l.close > l.ma60 THEN 1 ELSE 0 END AS above_ma60,
|
||||||
|
-- MA alignment (bullish = ma5 > ma10 > ma20 > ma60)
|
||||||
|
CASE WHEN l.ma5 > l.ma10 THEN 1 ELSE 0 END AS ma5_gt_ma10,
|
||||||
|
CASE WHEN l.ma10 > l.ma20 THEN 1 ELSE 0 END AS ma10_gt_ma20,
|
||||||
|
CASE WHEN l.ma20 > l.ma60 THEN 1 ELSE 0 END AS ma20_gt_ma60,
|
||||||
|
-- slope (5d change / 5)
|
||||||
|
(l.ma5 - p.ma5) / NULLIF(p.ma5, 0) AS ma5_slope,
|
||||||
|
(l.close - p.close) / NULLIF(p.close, 0) AS pct_chg,
|
||||||
|
-- volume ratio
|
||||||
|
l.volume / NULLIF(l.vol_ma20, 0) AS vol_ratio
|
||||||
|
FROM latest l
|
||||||
|
LEFT JOIN prev p ON l.ts_code = p.ts_code
|
||||||
|
"""
|
||||||
|
result = conn.execute(sql).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return _empty_trend()
|
||||||
|
|
||||||
|
# ── Per-index stats ──
|
||||||
|
above_ma20_count = int(result["above_ma20"].sum())
|
||||||
|
above_ma60_count = int(result["above_ma60"].sum())
|
||||||
|
total = len(result)
|
||||||
|
|
||||||
|
# ── MA alignment score (bullish alignment) ──
|
||||||
|
alignment_cols = ["ma5_gt_ma10", "ma10_gt_ma20", "ma20_gt_ma60"]
|
||||||
|
avg_alignment = float(result[alignment_cols].mean().mean()) # fraction of alignments bullish
|
||||||
|
|
||||||
|
# ── Resonance: are indices moving in the same direction? ──
|
||||||
|
pct_chgs = result["pct_chg"].dropna()
|
||||||
|
if len(pct_chgs) >= 4:
|
||||||
|
positive = (pct_chgs > 0).sum()
|
||||||
|
negative = (pct_chgs < 0).sum()
|
||||||
|
resonance = max(positive, negative) / len(pct_chgs)
|
||||||
|
else:
|
||||||
|
resonance = 0.5
|
||||||
|
|
||||||
|
# ── Divergence: std of index returns (high = diverging) ──
|
||||||
|
divergence = float(pct_chgs.std()) if len(pct_chgs) > 1 else 0.0
|
||||||
|
# Normalize: 2% std → 1.0
|
||||||
|
divergence = min(divergence / 0.02, 1.0)
|
||||||
|
|
||||||
|
# ── Volume: what fraction of indices have above-avg volume? ──
|
||||||
|
vol_ratios = result["vol_ratio"].dropna()
|
||||||
|
vol_expanding = (vol_ratios > 1.0).sum() / len(vol_ratios) if len(vol_ratios) > 0 else 0.5
|
||||||
|
|
||||||
|
# ── Trend slope: average MA5 slope across indices ──
|
||||||
|
slopes = result["ma5_slope"].dropna()
|
||||||
|
avg_slope = float(slopes.mean()) if len(slopes) > 0 else 0.0
|
||||||
|
# Normalize: 2% slope → 1.0
|
||||||
|
slope_normalized = min(max(avg_slope / 0.02 + 0.5, 0.0), 1.0)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"above_ma20_pct": round(above_ma20_count / total, 4),
|
||||||
|
"above_ma60_pct": round(above_ma60_count / total, 4),
|
||||||
|
"resonance": round(resonance, 4),
|
||||||
|
"divergence": round(divergence, 4),
|
||||||
|
"ma_alignment": round(avg_alignment, 4),
|
||||||
|
"vol_expanding_pct": round(vol_expanding, 4),
|
||||||
|
"avg_slope": round(avg_slope, 6),
|
||||||
|
"index_count": total,
|
||||||
|
"index_details": [
|
||||||
|
{
|
||||||
|
"ts_code": row["ts_code"],
|
||||||
|
"name": INDEX_CODES.get(row["ts_code"], row["ts_code"]),
|
||||||
|
"close": round(float(row["close"]), 2),
|
||||||
|
"pct_chg": round(float(row["pct_chg"] or 0) * 100, 2),
|
||||||
|
"above_ma20": bool(row["above_ma20"]),
|
||||||
|
"above_ma60": bool(row["above_ma60"]),
|
||||||
|
"vol_ratio": round(float(row["vol_ratio"] or 1.0), 2),
|
||||||
|
}
|
||||||
|
for _, row in result.iterrows()
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_trend() -> dict:
|
||||||
|
return {
|
||||||
|
"above_ma20_pct": 0.5,
|
||||||
|
"above_ma60_pct": 0.5,
|
||||||
|
"resonance": 0.5,
|
||||||
|
"divergence": 0.5,
|
||||||
|
"ma_alignment": 0.5,
|
||||||
|
"vol_expanding_pct": 0.5,
|
||||||
|
"avg_slope": 0.0,
|
||||||
|
"index_count": 0,
|
||||||
|
"index_details": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
registry.register(FeatureDefinition(
|
||||||
|
name="trend_vector",
|
||||||
|
category="trend",
|
||||||
|
description="Index MA positioning, resonance/divergence, bullish alignment, volume expansion",
|
||||||
|
dependencies=[],
|
||||||
|
compute=_compute_trend_vector,
|
||||||
|
))
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
"""Volume features: total turnover, vs 5d/20d average, turnover tier."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, timedelta
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
|
||||||
|
from ashare_dp.domain.features import FeatureDefinition
|
||||||
|
from ashare_dp.features.registry import registry
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_volume_vector(db: Any, trade_date: date, _results: dict) -> dict[str, Any]:
|
||||||
|
"""Compute volume profile: total turnover, comparison to 5d/20d averages.
|
||||||
|
|
||||||
|
Also reads the trading_calendar to get prior trading days.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Get the last 20 trading days from calendar
|
||||||
|
cal_sql = """
|
||||||
|
SELECT trade_date FROM trading_calendar
|
||||||
|
WHERE trade_date <= $trade_date AND is_trading_day = true
|
||||||
|
ORDER BY trade_date DESC
|
||||||
|
LIMIT 22
|
||||||
|
"""
|
||||||
|
cal_dates = conn.execute(
|
||||||
|
cal_sql, {"trade_date": trade_date.isoformat()}
|
||||||
|
).fetchall()
|
||||||
|
|
||||||
|
if not cal_dates:
|
||||||
|
return _empty_volume()
|
||||||
|
|
||||||
|
cal_dates = [row[0] for row in cal_dates]
|
||||||
|
today_str = cal_dates[0]
|
||||||
|
|
||||||
|
# Compute daily turnover for each of the last 20+ trading days
|
||||||
|
# Use a single DuckDB scan instead of repeated queries
|
||||||
|
date_list = "', '".join(str(d) for d in cal_dates)
|
||||||
|
|
||||||
|
sql = f"""
|
||||||
|
SELECT
|
||||||
|
trade_date,
|
||||||
|
SUM(amount) as daily_amount,
|
||||||
|
SUM(volume) as daily_volume,
|
||||||
|
COUNT(*) as stock_count
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date IN ('{date_list}')
|
||||||
|
GROUP BY trade_date
|
||||||
|
ORDER BY trade_date DESC
|
||||||
|
"""
|
||||||
|
result = conn.execute(sql).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return _empty_volume()
|
||||||
|
|
||||||
|
today_row = result[result["trade_date"] == today_str]
|
||||||
|
if today_row.empty:
|
||||||
|
return _empty_volume()
|
||||||
|
|
||||||
|
today_amount = float(today_row["daily_amount"].iloc[0])
|
||||||
|
today_volume = float(today_row["daily_volume"].iloc[0])
|
||||||
|
today_count = int(today_row["stock_count"].iloc[0])
|
||||||
|
|
||||||
|
# 5d avg (today + prior 4 trading days)
|
||||||
|
recent_5 = result.head(5)
|
||||||
|
avg_5d_amount = float(recent_5["daily_amount"].mean())
|
||||||
|
avg_5d_volume = float(recent_5["daily_volume"].mean())
|
||||||
|
|
||||||
|
# 20d avg
|
||||||
|
recent_20 = result.head(min(20, len(result)))
|
||||||
|
avg_20d_amount = float(recent_20["daily_amount"].mean())
|
||||||
|
avg_20d_volume = float(recent_20["daily_volume"].mean())
|
||||||
|
|
||||||
|
vs_5d = round(today_amount / avg_5d_amount, 4) if avg_5d_amount > 0 else 1.0
|
||||||
|
vs_20d = round(today_amount / avg_20d_amount, 4) if avg_20d_amount > 0 else 1.0
|
||||||
|
|
||||||
|
# Turnover tier
|
||||||
|
amount_yi = today_amount / 1e8 # 亿
|
||||||
|
if amount_yi < 8000:
|
||||||
|
tier = "缩量"
|
||||||
|
elif amount_yi < 12000:
|
||||||
|
tier = "正常"
|
||||||
|
elif amount_yi < 15000:
|
||||||
|
tier = "放量"
|
||||||
|
else:
|
||||||
|
tier = "巨量"
|
||||||
|
|
||||||
|
return {
|
||||||
|
"total_turnover": today_amount,
|
||||||
|
"total_turnover_yi": round(amount_yi, 0),
|
||||||
|
"total_volume": int(today_volume),
|
||||||
|
"stock_count": today_count,
|
||||||
|
"vs_5d": vs_5d,
|
||||||
|
"vs_20d": vs_20d,
|
||||||
|
"avg_5d_amount": avg_5d_amount,
|
||||||
|
"avg_20d_amount": avg_20d_amount,
|
||||||
|
"tier": tier,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_volume() -> dict:
|
||||||
|
return {
|
||||||
|
"total_turnover": 0,
|
||||||
|
"total_turnover_yi": 0,
|
||||||
|
"total_volume": 0,
|
||||||
|
"stock_count": 0,
|
||||||
|
"vs_5d": 1.0,
|
||||||
|
"vs_20d": 1.0,
|
||||||
|
"avg_5d_amount": 0,
|
||||||
|
"avg_20d_amount": 0,
|
||||||
|
"tier": "未知",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
registry.register(FeatureDefinition(
|
||||||
|
name="volume_vector",
|
||||||
|
category="volume",
|
||||||
|
description="Total market turnover, vs 5d/20d average comparison, turnover tier classification",
|
||||||
|
dependencies=[],
|
||||||
|
compute=_compute_volume_vector,
|
||||||
|
))
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
"""MARKET INTELLIGENCE — market state inference and interpretation.
|
||||||
|
|
||||||
|
Knows nothing about signals or execution.
|
||||||
|
"""
|
||||||
|
from ashare_dp.market.state import infer_market_state
|
||||||
|
from ashare_dp.market.leadership import assess_leaders
|
||||||
|
from ashare_dp.market.opportunity import rank_opportunities
|
||||||
|
from ashare_dp.market.flow import compute_flow
|
||||||
|
from ashare_dp.market.sentiment import assess_sentiment
|
||||||
|
from ashare_dp.market.memory import StateStore, state_store
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"infer_market_state", "assess_leaders", "rank_opportunities",
|
||||||
|
"compute_flow", "assess_sentiment", "StateStore", "state_store",
|
||||||
|
]
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
"""Flow Engine — money flow graph from turnover changes.
|
||||||
|
|
||||||
|
Phase 1: uses turnover change as proxy for capital flow.
|
||||||
|
Phase 2: integrates 北向/ETF/主力/融资/龙虎榜 data.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, timedelta
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.domain.context import FlowEdge, FlowGraph
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def compute_flow(trade_date: date) -> FlowGraph:
|
||||||
|
"""Compute money flow graph from industry turnover changes.
|
||||||
|
|
||||||
|
Identifies industries with rising vs falling turnover,
|
||||||
|
creates directed edges from losers to gainers.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
end_str = trade_date.strftime("%Y-%m-%d")
|
||||||
|
start_str = (trade_date - timedelta(days=20)).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT
|
||||||
|
k.ts_code AS ts_code,
|
||||||
|
k.trade_date, k.amount
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true) k
|
||||||
|
WHERE k.trade_date >= $start_date AND k.trade_date <= $end_date
|
||||||
|
),
|
||||||
|
daily AS (
|
||||||
|
SELECT n.trade_date, n.amount, si.industry_name
|
||||||
|
FROM normalized n
|
||||||
|
JOIN stock_industry si ON n.ts_code = si.ts_code
|
||||||
|
),
|
||||||
|
industry_daily AS (
|
||||||
|
SELECT industry_name, trade_date, SUM(amount) AS total_amount
|
||||||
|
FROM daily GROUP BY industry_name, trade_date
|
||||||
|
),
|
||||||
|
ranked AS (
|
||||||
|
SELECT *,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY industry_name ORDER BY trade_date DESC) as rn
|
||||||
|
FROM industry_daily
|
||||||
|
),
|
||||||
|
recent AS (
|
||||||
|
SELECT industry_name, SUM(total_amount) AS recent_amount
|
||||||
|
FROM ranked WHERE rn <= 5 GROUP BY industry_name
|
||||||
|
),
|
||||||
|
prior AS (
|
||||||
|
SELECT industry_name, SUM(total_amount) AS prior_amount
|
||||||
|
FROM ranked WHERE rn > 5 AND rn <= 10 GROUP BY industry_name
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
COALESCE(r.industry_name, p.industry_name) AS industry_name,
|
||||||
|
COALESCE(r.recent_amount, 0) AS recent_amount,
|
||||||
|
COALESCE(p.prior_amount, 0) AS prior_amount,
|
||||||
|
CASE WHEN COALESCE(p.prior_amount, 0) > 0
|
||||||
|
THEN (r.recent_amount - p.prior_amount) / p.prior_amount
|
||||||
|
ELSE 0 END AS change
|
||||||
|
FROM recent r
|
||||||
|
FULL OUTER JOIN prior p ON r.industry_name = p.industry_name
|
||||||
|
WHERE COALESCE(r.recent_amount, 0) + COALESCE(p.prior_amount, 0) > 0
|
||||||
|
"""
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start_date", f"'{start_str}'").replace("$end_date", f"'{end_str}'")
|
||||||
|
).fetchdf()
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Flow computation failed: {e}")
|
||||||
|
conn.close()
|
||||||
|
return FlowGraph()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return FlowGraph()
|
||||||
|
|
||||||
|
# Build flow edges from losers to gainers
|
||||||
|
changes = {}
|
||||||
|
for _, row in result.iterrows():
|
||||||
|
changes[row["industry_name"]] = float(row["change"])
|
||||||
|
|
||||||
|
gainers = sorted([(k, v) for k, v in changes.items() if v > 0.05], key=lambda x: -x[1])
|
||||||
|
losers = sorted([(k, v) for k, v in changes.items() if v < -0.05], key=lambda x: x[1])
|
||||||
|
|
||||||
|
edges = []
|
||||||
|
for i in range(min(len(gainers), len(losers), 5)):
|
||||||
|
los_name, los_chg = losers[i]
|
||||||
|
win_name, win_chg = gainers[i]
|
||||||
|
|
||||||
|
# Magnitude string
|
||||||
|
los_mag = "↓↓" if los_chg < -0.15 else "↓"
|
||||||
|
win_mag = "↑↑" if win_chg > 0.15 else "↑"
|
||||||
|
magnitude = f"{los_mag}→{win_mag}"
|
||||||
|
|
||||||
|
strength = min((win_chg - los_chg) / 2, 1.0)
|
||||||
|
|
||||||
|
edges.append(FlowEdge(
|
||||||
|
source=los_name,
|
||||||
|
target=win_name,
|
||||||
|
magnitude=magnitude,
|
||||||
|
strength=round(strength, 4),
|
||||||
|
))
|
||||||
|
|
||||||
|
net_inflow = {k: round(v, 4) for k, v in changes.items() if v > 0}
|
||||||
|
net_outflow = {k: round(v, 4) for k, v in changes.items() if v < 0}
|
||||||
|
|
||||||
|
return FlowGraph(
|
||||||
|
flows=sorted(edges, key=lambda e: -e.strength),
|
||||||
|
net_inflow=net_inflow,
|
||||||
|
net_outflow=net_outflow,
|
||||||
|
)
|
||||||
@@ -0,0 +1,166 @@
|
|||||||
|
"""Knowledge Graph Engine — Theme activation, concept flow, narrative integration.
|
||||||
|
|
||||||
|
Maps daily market data (industry turnover, leader states, opportunities)
|
||||||
|
to the curated theme knowledge graph. Computes theme relevance scores
|
||||||
|
and detects theme-level rotation.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.domain.knowledge import (
|
||||||
|
Theme, Concept, ThemeEdge, ThemeGraph, THEME_KNOWLEDGE,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def activate_themes(
|
||||||
|
trade_date: date,
|
||||||
|
leaders: dict | None = None,
|
||||||
|
opportunities: list | None = None,
|
||||||
|
) -> ThemeGraph:
|
||||||
|
"""Compute theme relevance from today's market data.
|
||||||
|
|
||||||
|
For each theme in the knowledge graph:
|
||||||
|
- Match concepts to industries by keyword search on stock names
|
||||||
|
- Compute relevance from: leader health, opportunity score, turnover
|
||||||
|
- Build theme-level rotation edges from flow data
|
||||||
|
|
||||||
|
Returns a ThemeGraph with scored themes and edges.
|
||||||
|
"""
|
||||||
|
themes = []
|
||||||
|
leaders = leaders or {}
|
||||||
|
opps = opportunities or []
|
||||||
|
|
||||||
|
for theme_tmpl in THEME_KNOWLEDGE:
|
||||||
|
theme = Theme(name=theme_tmpl.name, concepts=[])
|
||||||
|
total_score = 0.0
|
||||||
|
concept_count = 0
|
||||||
|
|
||||||
|
for conc_tmpl in theme_tmpl.concepts:
|
||||||
|
concept = Concept(
|
||||||
|
name=conc_tmpl.name,
|
||||||
|
keywords=conc_tmpl.keywords,
|
||||||
|
leaders=conc_tmpl.leaders,
|
||||||
|
)
|
||||||
|
|
||||||
|
# Find matching opportunities for this concept
|
||||||
|
concept_score = 0.0
|
||||||
|
|
||||||
|
for opp in (opps or []):
|
||||||
|
# Check if opportunity name matches concept keywords
|
||||||
|
opp_name = opp.name if hasattr(opp, 'name') else str(opp)
|
||||||
|
for kw in conc_tmpl.keywords:
|
||||||
|
if kw in opp_name:
|
||||||
|
concept_score = max(concept_score, opp.score if hasattr(opp, 'score') else 50)
|
||||||
|
break
|
||||||
|
|
||||||
|
# Check leader health by matching concept keywords against industry names
|
||||||
|
for ind, ld_data in leaders.items():
|
||||||
|
if not isinstance(ld_data, dict):
|
||||||
|
continue
|
||||||
|
state = ld_data.get("state", "unknown")
|
||||||
|
# Match concept keywords against industry name
|
||||||
|
for kw in conc_tmpl.keywords:
|
||||||
|
if kw in ind or kw in str(ld_data.get("leader", "")):
|
||||||
|
if state in ("leading", "growing"):
|
||||||
|
concept_score = max(concept_score, 80)
|
||||||
|
elif state in ("birth", "recovering"):
|
||||||
|
concept_score = max(concept_score, 60)
|
||||||
|
elif state in ("exhausted",):
|
||||||
|
concept_score = max(concept_score, 40)
|
||||||
|
elif state in ("breaking", "dead"):
|
||||||
|
concept_score = max(concept_score, 10)
|
||||||
|
break
|
||||||
|
|
||||||
|
# Match concept keywords against opportunity names
|
||||||
|
for opp in (opps or []):
|
||||||
|
opp_name = opp.name if hasattr(opp, 'name') else str(opp)
|
||||||
|
top_stocks = getattr(opp, 'top_stocks', []) or []
|
||||||
|
for kw in conc_tmpl.keywords:
|
||||||
|
if kw in opp_name:
|
||||||
|
concept_score = max(concept_score, opp.score if hasattr(opp, 'score') else 50)
|
||||||
|
break
|
||||||
|
for s in top_stocks:
|
||||||
|
if kw in str(s):
|
||||||
|
concept_score = max(concept_score, opp.score if hasattr(opp, 'score') else 50)
|
||||||
|
break
|
||||||
|
|
||||||
|
concept.relevance_score = round(concept_score, 1)
|
||||||
|
if concept_score > 0:
|
||||||
|
theme.concepts.append(concept)
|
||||||
|
total_score += concept_score
|
||||||
|
concept_count += 1
|
||||||
|
|
||||||
|
if concept_count > 0:
|
||||||
|
theme.relevance_score = round(total_score / concept_count, 1)
|
||||||
|
else:
|
||||||
|
theme.relevance_score = 0.0
|
||||||
|
|
||||||
|
# Momentum label
|
||||||
|
if theme.relevance_score >= 70:
|
||||||
|
theme.momentum = "↑↑"
|
||||||
|
elif theme.relevance_score >= 50:
|
||||||
|
theme.momentum = "↑"
|
||||||
|
elif theme.relevance_score >= 30:
|
||||||
|
theme.momentum = "→"
|
||||||
|
elif theme.relevance_score > 0:
|
||||||
|
theme.momentum = "↓"
|
||||||
|
else:
|
||||||
|
theme.momentum = ""
|
||||||
|
|
||||||
|
themes.append(theme)
|
||||||
|
|
||||||
|
# Sort by relevance
|
||||||
|
themes.sort(key=lambda t: t.relevance_score, reverse=True)
|
||||||
|
|
||||||
|
# Build theme edges from top-to-bottom flow
|
||||||
|
edges = []
|
||||||
|
active = [t for t in themes if t.relevance_score > 30]
|
||||||
|
if len(active) >= 2:
|
||||||
|
for i in range(min(len(active) - 1, 3)):
|
||||||
|
edges.append(ThemeEdge(
|
||||||
|
source=active[-1 - i].name,
|
||||||
|
target=active[i].name,
|
||||||
|
strength=round(active[i].relevance_score / 100, 4),
|
||||||
|
))
|
||||||
|
|
||||||
|
return ThemeGraph(themes=themes, edges=edges)
|
||||||
|
|
||||||
|
|
||||||
|
def describe_theme_graph(tg: ThemeGraph) -> str:
|
||||||
|
"""Generate a human-readable description of the theme landscape."""
|
||||||
|
active = [t for t in tg.themes if t.relevance_score > 0]
|
||||||
|
if not active:
|
||||||
|
return "无明显主题热点"
|
||||||
|
|
||||||
|
parts = []
|
||||||
|
top3 = active[:3]
|
||||||
|
parts.append(f"主题集中在{'、'.join(t.name for t in top3)}")
|
||||||
|
|
||||||
|
# Describe top theme's concepts
|
||||||
|
if top3 and top3[0].concepts:
|
||||||
|
concepts = [c.name for c in top3[0].concepts if c.relevance_score > 0]
|
||||||
|
if concepts:
|
||||||
|
parts.append(f"{top3[0].name}主题下{'、'.join(concepts[:3])}最活跃")
|
||||||
|
|
||||||
|
# Describe rotation
|
||||||
|
if tg.edges:
|
||||||
|
e = tg.edges[0]
|
||||||
|
parts.append(f"资金从{e.source}流向{e.target}")
|
||||||
|
|
||||||
|
return "。".join(parts) + "。"
|
||||||
|
|
||||||
|
|
||||||
|
def get_theme_for_stock(stock_name: str, ts_code: str = "") -> list[str]:
|
||||||
|
"""Find which themes a stock belongs to."""
|
||||||
|
matched = []
|
||||||
|
for theme in THEME_KNOWLEDGE:
|
||||||
|
for concept in theme.concepts:
|
||||||
|
for kw in concept.keywords:
|
||||||
|
if kw in stock_name or kw in ts_code:
|
||||||
|
if theme.name not in matched:
|
||||||
|
matched.append(theme.name)
|
||||||
|
return matched
|
||||||
@@ -0,0 +1,137 @@
|
|||||||
|
"""Leadership Engine — assess leader lifecycle for each major industry.
|
||||||
|
|
||||||
|
Not just Alive/Dead. Full lifecycle: Birth → Growing → Leading →
|
||||||
|
Exhausted → Breaking → Dead → Recovering.
|
||||||
|
|
||||||
|
Uses daily K-line data to determine leader state from price/MA/volume.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.domain.leadership import LeaderState
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def assess_leaders(trade_date: date) -> dict[str, dict]:
|
||||||
|
"""For each major industry, identify the leader stock and its lifecycle state.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
{industry_name: {leader, state, close, above_ma20, above_ma60,
|
||||||
|
momentum, volume_ratio, days_above_ma20}}
|
||||||
|
|
||||||
|
Phase 1: uses top stock by amount per industry from stock_industry join.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT
|
||||||
|
k.ts_code AS ts_code,
|
||||||
|
k.trade_date, k.trade_time, k.close, k.volume, k.amount, k.high, k.low
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true) k
|
||||||
|
WHERE k.trade_date <= $trade_date
|
||||||
|
),
|
||||||
|
with_ma AS (
|
||||||
|
SELECT *,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 19 PRECEDING AND CURRENT ROW) AS ma20,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 59 PRECEDING AND CURRENT ROW) AS ma60,
|
||||||
|
AVG(volume) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 19 PRECEDING AND CURRENT ROW) AS vol_ma20,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time DESC) as rn
|
||||||
|
FROM normalized
|
||||||
|
),
|
||||||
|
latest AS (
|
||||||
|
SELECT * FROM with_ma WHERE rn = 1
|
||||||
|
),
|
||||||
|
prev AS (
|
||||||
|
SELECT * FROM with_ma WHERE rn = 21
|
||||||
|
),
|
||||||
|
ranked AS (
|
||||||
|
SELECT
|
||||||
|
l.ts_code, l.close, l.volume, l.amount,
|
||||||
|
l.ma20, l.ma60, l.vol_ma20,
|
||||||
|
p.close AS close_20d_ago,
|
||||||
|
(l.close - p.close) / NULLIF(p.close, 0) AS momentum_20d,
|
||||||
|
l.volume / NULLIF(l.vol_ma20, 0) AS vol_ratio,
|
||||||
|
si.industry_name,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY si.industry_name ORDER BY l.amount DESC) as amt_rank
|
||||||
|
FROM latest l
|
||||||
|
JOIN stock_industry si ON l.ts_code = si.ts_code
|
||||||
|
LEFT JOIN prev p ON l.ts_code = p.ts_code
|
||||||
|
WHERE l.amount > 1e8
|
||||||
|
)
|
||||||
|
SELECT * FROM ranked WHERE amt_rank = 1
|
||||||
|
ORDER BY industry_name
|
||||||
|
"""
|
||||||
|
result = conn.execute(sql, {"trade_date": trade_date.isoformat()}).fetchdf()
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Leadership assessment failed: {e}")
|
||||||
|
conn.close()
|
||||||
|
return {}
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
leaders = {}
|
||||||
|
for _, row in result.iterrows():
|
||||||
|
industry = row["industry_name"]
|
||||||
|
close = float(row["close"])
|
||||||
|
ma20 = float(row["ma20"] or close)
|
||||||
|
ma60 = float(row["ma60"] or close)
|
||||||
|
vol_ratio = float(row["vol_ratio"] or 1.0)
|
||||||
|
import math as _math
|
||||||
|
momentum_raw = row["momentum_20d"]
|
||||||
|
momentum = float(momentum_raw) if (momentum_raw is not None and not (isinstance(momentum_raw, float) and _math.isnan(momentum_raw))) else 0.0
|
||||||
|
|
||||||
|
above_ma20 = close > ma20
|
||||||
|
above_ma60 = close > ma60
|
||||||
|
|
||||||
|
state = _classify_leader_state(
|
||||||
|
above_ma20=above_ma20,
|
||||||
|
above_ma60=above_ma60,
|
||||||
|
vol_ratio=vol_ratio,
|
||||||
|
momentum=momentum,
|
||||||
|
)
|
||||||
|
|
||||||
|
leaders[industry] = {
|
||||||
|
"leader": row["ts_code"],
|
||||||
|
"state": state.value,
|
||||||
|
"close": round(close, 2),
|
||||||
|
"above_ma20": above_ma20,
|
||||||
|
"above_ma60": above_ma60,
|
||||||
|
"momentum_20d": round(momentum, 4),
|
||||||
|
"volume_ratio": round(vol_ratio, 2),
|
||||||
|
}
|
||||||
|
|
||||||
|
return leaders
|
||||||
|
|
||||||
|
|
||||||
|
def _classify_leader_state(
|
||||||
|
above_ma20: bool,
|
||||||
|
above_ma60: bool,
|
||||||
|
vol_ratio: float,
|
||||||
|
momentum: float,
|
||||||
|
) -> LeaderState:
|
||||||
|
"""Classify leader lifecycle from technical indicators."""
|
||||||
|
if not above_ma60:
|
||||||
|
return LeaderState.DEAD
|
||||||
|
if not above_ma20:
|
||||||
|
# Was it above MA20 recently? If so, breaking. If not, dead.
|
||||||
|
return LeaderState.BREAKING
|
||||||
|
|
||||||
|
# Above both MAs
|
||||||
|
if momentum > 0.05 and vol_ratio > 1.2:
|
||||||
|
return LeaderState.LEADING
|
||||||
|
elif momentum > 0.02:
|
||||||
|
return LeaderState.GROWING
|
||||||
|
elif momentum > 0:
|
||||||
|
return LeaderState.BIRTH if vol_ratio > 0.8 else LeaderState.EXHAUSTED
|
||||||
|
elif momentum > -0.02:
|
||||||
|
return LeaderState.EXHAUSTED
|
||||||
|
else:
|
||||||
|
return LeaderState.BREAKING # losing momentum fast
|
||||||
@@ -0,0 +1,154 @@
|
|||||||
|
"""Time Series State Store — persists MarketState snapshots for historical lookup.
|
||||||
|
|
||||||
|
NOT limited to daily. Schema: (entity, timeframe, timestamp, state).
|
||||||
|
Supports find_similar() for historical pattern matching (future Signal Engine).
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.domain.state import MarketState
|
||||||
|
|
||||||
|
|
||||||
|
class StateStore:
|
||||||
|
"""Persist and query MarketState snapshots.
|
||||||
|
|
||||||
|
Uses the Database class API: db.execute() for writes, db.query() for reads.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def save(
|
||||||
|
self,
|
||||||
|
db: Any,
|
||||||
|
entity: str,
|
||||||
|
timeframe: str,
|
||||||
|
timestamp: datetime,
|
||||||
|
state: MarketState,
|
||||||
|
) -> None:
|
||||||
|
"""Save a MarketState snapshot."""
|
||||||
|
try:
|
||||||
|
state_json = json.dumps(state.to_dict(), ensure_ascii=False)
|
||||||
|
db.execute(
|
||||||
|
"INSERT OR REPLACE INTO state_snapshot (entity, timeframe, timestamp, state) "
|
||||||
|
"VALUES (?, ?, ?, ?)",
|
||||||
|
(entity, timeframe, timestamp, state_json),
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"StateStore.save failed: {e}")
|
||||||
|
|
||||||
|
def get_latest(
|
||||||
|
self, db: Any, entity: str, timeframe: str
|
||||||
|
) -> MarketState | None:
|
||||||
|
"""Get the most recent MarketState for an entity/timeframe."""
|
||||||
|
try:
|
||||||
|
rows = db.query(
|
||||||
|
"SELECT state, timestamp FROM state_snapshot "
|
||||||
|
"WHERE entity = ? AND timeframe = ? "
|
||||||
|
"ORDER BY timestamp DESC LIMIT 1",
|
||||||
|
(entity, timeframe),
|
||||||
|
)
|
||||||
|
if rows:
|
||||||
|
state_dict = json.loads(rows[0][0])
|
||||||
|
return _dict_to_state(state_dict)
|
||||||
|
return None
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"StateStore.get_latest failed: {e}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
def query(
|
||||||
|
self,
|
||||||
|
db: Any,
|
||||||
|
entity: str,
|
||||||
|
timeframe: str,
|
||||||
|
start: datetime,
|
||||||
|
end: datetime,
|
||||||
|
) -> list[MarketState]:
|
||||||
|
"""Query MarketState snapshots in a time range."""
|
||||||
|
try:
|
||||||
|
rows = db.query(
|
||||||
|
"SELECT state, timestamp FROM state_snapshot "
|
||||||
|
"WHERE entity = ? AND timeframe = ? "
|
||||||
|
"AND timestamp >= ? AND timestamp <= ? "
|
||||||
|
"ORDER BY timestamp ASC",
|
||||||
|
(entity, timeframe, start, end),
|
||||||
|
)
|
||||||
|
return [_dict_to_state(json.loads(row[0])) for row in rows]
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"StateStore.query failed: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
def find_similar(
|
||||||
|
self, db: Any, state: MarketState, top_k: int = 10
|
||||||
|
) -> list[tuple[MarketState, float]]:
|
||||||
|
"""Find historically similar MarketStates by cosine similarity."""
|
||||||
|
try:
|
||||||
|
rows = db.query(
|
||||||
|
"SELECT state, timestamp FROM state_snapshot "
|
||||||
|
"WHERE entity = 'market' AND timeframe = '1d' "
|
||||||
|
"ORDER BY timestamp DESC LIMIT 1000"
|
||||||
|
)
|
||||||
|
target = state.to_vector()
|
||||||
|
scored = []
|
||||||
|
for row in rows:
|
||||||
|
hist_dict = json.loads(row[0])
|
||||||
|
hist = _dict_to_state(hist_dict)
|
||||||
|
hist_vec = hist.to_vector()
|
||||||
|
sim = _cosine_similarity(target, hist_vec)
|
||||||
|
scored.append((hist, sim))
|
||||||
|
scored.sort(key=lambda x: x[1], reverse=True)
|
||||||
|
return scored[:top_k]
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"StateStore.find_similar failed: {e}")
|
||||||
|
return []
|
||||||
|
|
||||||
|
def count(self, db: Any, entity: str, timeframe: str) -> int:
|
||||||
|
"""Count stored snapshots."""
|
||||||
|
try:
|
||||||
|
rows = db.query(
|
||||||
|
"SELECT COUNT(*) FROM state_snapshot WHERE entity = ? AND timeframe = ?",
|
||||||
|
(entity, timeframe),
|
||||||
|
)
|
||||||
|
return int(rows[0][0]) if rows else 0
|
||||||
|
except Exception:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def _dict_to_state(d: dict) -> MarketState:
|
||||||
|
"""Reconstruct MarketState from serialized dict."""
|
||||||
|
dims = d.get("dimensions", {})
|
||||||
|
return MarketState(
|
||||||
|
timestamp=datetime.fromisoformat(d["timestamp"]),
|
||||||
|
trade_date=datetime.fromisoformat(d["trade_date"]).date()
|
||||||
|
if "trade_date" in d
|
||||||
|
else datetime.fromisoformat(d["timestamp"]).date(),
|
||||||
|
version=d.get("version", "1.0"),
|
||||||
|
source=d.get("source", "ashare_dp"),
|
||||||
|
trend=dims.get("trend", 0.0),
|
||||||
|
fear=dims.get("fear", 0.0),
|
||||||
|
liquidity=dims.get("liquidity", 0.0),
|
||||||
|
rotation=dims.get("rotation", 0.0),
|
||||||
|
participation=dims.get("participation", 0.0),
|
||||||
|
volatility=dims.get("volatility", 0.0),
|
||||||
|
breadth=dims.get("breadth", 0.0),
|
||||||
|
confidence=d.get("confidence", 0.0),
|
||||||
|
quality=d.get("quality", 0.0),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _cosine_similarity(a: list[float], b: list[float]) -> float:
|
||||||
|
"""Compute cosine similarity between two vectors."""
|
||||||
|
import math
|
||||||
|
dot = sum(x * y for x, y in zip(a, b))
|
||||||
|
na = math.sqrt(sum(x * x for x in a))
|
||||||
|
nb = math.sqrt(sum(x * x for x in b))
|
||||||
|
if na == 0 or nb == 0:
|
||||||
|
return 0.0
|
||||||
|
return dot / (na * nb)
|
||||||
|
|
||||||
|
|
||||||
|
# Module-level singleton
|
||||||
|
state_store = StateStore()
|
||||||
@@ -0,0 +1,198 @@
|
|||||||
|
"""Opportunity Engine — ranks tradable opportunities by expected participation.
|
||||||
|
|
||||||
|
Not industry returns. Not leader rankings. Forward-looking scores for
|
||||||
|
"where can I make money in the next 1-3 days?"
|
||||||
|
|
||||||
|
Scoring: turnover trend + persistence + breadth within industry + leader state.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, timedelta
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.domain.context import Opportunity
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def rank_opportunities(trade_date: date, top_n: int = 10) -> list[Opportunity]:
|
||||||
|
"""Score industries by multi-factor opportunity model.
|
||||||
|
|
||||||
|
Factors:
|
||||||
|
- Turnover momentum (5d change vs prior 5d)
|
||||||
|
- Persistence (consecutive days as turnover leader)
|
||||||
|
- Internal breadth (% of stocks in industry advancing)
|
||||||
|
- Leader health (from Leadership Engine)
|
||||||
|
|
||||||
|
Returns sorted list of Opportunity objects.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Compute per-industry turnover trends + internal breadth
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT
|
||||||
|
k.ts_code AS ts_code,
|
||||||
|
k.trade_date, k.close, k.open, k.amount
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true) k
|
||||||
|
WHERE k.trade_date >= $start_date AND k.trade_date <= $end_date
|
||||||
|
),
|
||||||
|
daily AS (
|
||||||
|
SELECT
|
||||||
|
n.ts_code, n.trade_date, n.close, n.open, n.amount,
|
||||||
|
si.industry_name
|
||||||
|
FROM normalized n
|
||||||
|
JOIN stock_industry si ON n.ts_code = si.ts_code
|
||||||
|
),
|
||||||
|
industry_daily AS (
|
||||||
|
SELECT
|
||||||
|
industry_name,
|
||||||
|
trade_date,
|
||||||
|
SUM(amount) AS total_amount,
|
||||||
|
COUNT(*) AS stock_count,
|
||||||
|
COUNT(*) FILTER (WHERE close > open) AS advance_count
|
||||||
|
FROM daily
|
||||||
|
GROUP BY industry_name, trade_date
|
||||||
|
),
|
||||||
|
ranked AS (
|
||||||
|
SELECT *,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY industry_name ORDER BY trade_date DESC) as rn
|
||||||
|
FROM industry_daily
|
||||||
|
),
|
||||||
|
recent AS (
|
||||||
|
SELECT industry_name,
|
||||||
|
SUM(total_amount) AS recent_amount,
|
||||||
|
SUM(stock_count) AS recent_stocks,
|
||||||
|
SUM(advance_count) * 1.0 / NULLIF(SUM(stock_count), 0) AS advance_ratio
|
||||||
|
FROM ranked WHERE rn <= 5
|
||||||
|
GROUP BY industry_name
|
||||||
|
),
|
||||||
|
prior AS (
|
||||||
|
SELECT industry_name,
|
||||||
|
SUM(total_amount) AS prior_amount
|
||||||
|
FROM ranked WHERE rn > 5 AND rn <= 10
|
||||||
|
GROUP BY industry_name
|
||||||
|
),
|
||||||
|
persistence AS (
|
||||||
|
SELECT
|
||||||
|
industry_name,
|
||||||
|
COUNT(*) FILTER (WHERE daily_rank <= 3) AS lead_days
|
||||||
|
FROM (
|
||||||
|
SELECT *,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY trade_date ORDER BY total_amount DESC) as daily_rank
|
||||||
|
FROM industry_daily
|
||||||
|
WHERE trade_date >= $recent_10
|
||||||
|
) sub
|
||||||
|
WHERE daily_rank <= 3
|
||||||
|
GROUP BY industry_name
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
COALESCE(r.industry_name, p.industry_name) AS industry_name,
|
||||||
|
COALESCE(r.recent_amount, 0) AS recent_amount,
|
||||||
|
COALESCE(p.prior_amount, 0) AS prior_amount,
|
||||||
|
CASE WHEN COALESCE(p.prior_amount, 0) > 0
|
||||||
|
THEN (r.recent_amount - p.prior_amount) / p.prior_amount
|
||||||
|
ELSE 0 END AS turnover_change,
|
||||||
|
COALESCE(r.recent_stocks, 0) AS stock_count,
|
||||||
|
COALESCE(r.advance_ratio, 0.5) AS advance_ratio,
|
||||||
|
COALESCE(ps.lead_days, 0) AS persistence_days
|
||||||
|
FROM recent r
|
||||||
|
FULL OUTER JOIN prior p ON r.industry_name = p.industry_name
|
||||||
|
LEFT JOIN persistence ps ON r.industry_name = ps.industry_name
|
||||||
|
WHERE COALESCE(r.recent_amount, 0) + COALESCE(p.prior_amount, 0) > 0
|
||||||
|
"""
|
||||||
|
end_str = trade_date.strftime("%Y-%m-%d")
|
||||||
|
start_str = (trade_date - timedelta(days=30)).strftime("%Y-%m-%d")
|
||||||
|
recent_10 = (trade_date - timedelta(days=14)).strftime("%Y-%m-%d")
|
||||||
|
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start_date", f"'{start_str}'")
|
||||||
|
.replace("$end_date", f"'{end_str}'")
|
||||||
|
.replace("$recent_10", f"'{recent_10}'")
|
||||||
|
).fetchdf()
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Opportunity ranking failed: {e}")
|
||||||
|
conn.close()
|
||||||
|
return []
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return []
|
||||||
|
|
||||||
|
# Score each industry
|
||||||
|
opportunities = []
|
||||||
|
for _, row in result.iterrows():
|
||||||
|
chg = float(row["turnover_change"])
|
||||||
|
persistence = int(row["persistence_days"])
|
||||||
|
advance_ratio = float(row["advance_ratio"])
|
||||||
|
stock_count = int(row["stock_count"])
|
||||||
|
|
||||||
|
# Skip tiny industries
|
||||||
|
if stock_count < 3:
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Composite score (0-100)
|
||||||
|
momentum_score = min(max(chg * 100 + 50, 0), 100) # normalize to 0-100
|
||||||
|
persistence_score = min(persistence * 20, 40) # up to 40 points
|
||||||
|
breadth_score = advance_ratio * 30 # up to 30 points
|
||||||
|
|
||||||
|
score = momentum_score * 0.5 + persistence_score + breadth_score
|
||||||
|
|
||||||
|
# Lifecycle classification
|
||||||
|
if persistence >= 3 and chg > 0.1:
|
||||||
|
lifecycle = "Accelerating"
|
||||||
|
elif persistence >= 1 and chg > 0.05:
|
||||||
|
lifecycle = "Early"
|
||||||
|
elif chg > 0:
|
||||||
|
lifecycle = "Peak"
|
||||||
|
else:
|
||||||
|
lifecycle = "Declining"
|
||||||
|
|
||||||
|
opportunities.append(Opportunity(
|
||||||
|
name=row["industry_name"],
|
||||||
|
score=round(min(score, 100), 1),
|
||||||
|
lifecycle=lifecycle,
|
||||||
|
persistence_days=persistence,
|
||||||
|
catalyst="资金轮动" if chg > 0.1 else "",
|
||||||
|
top_stocks=[], # filled by router
|
||||||
|
))
|
||||||
|
|
||||||
|
# Sort by score desc
|
||||||
|
opportunities.sort(key=lambda o: o.score, reverse=True)
|
||||||
|
|
||||||
|
# Get top stocks for top opportunities (fresh connection needed)
|
||||||
|
if opportunities:
|
||||||
|
import duckdb
|
||||||
|
fresh_conn = duckdb.connect("data/duckdb/ashare.db")
|
||||||
|
try:
|
||||||
|
_fill_top_stocks(fresh_conn, trade_date, opportunities[:top_n])
|
||||||
|
finally:
|
||||||
|
fresh_conn.close()
|
||||||
|
|
||||||
|
return opportunities[:top_n]
|
||||||
|
|
||||||
|
|
||||||
|
def _fill_top_stocks(conn, trade_date: date, opportunities: list[Opportunity]):
|
||||||
|
"""For each opportunity, find the top 3 stocks by amount."""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
|
||||||
|
for opp in opportunities:
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
SELECT
|
||||||
|
k.ts_code AS ts_code
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true) k
|
||||||
|
JOIN stock_industry si ON k.ts_code = si.ts_code
|
||||||
|
WHERE k.trade_date = ? AND si.industry_name = ?
|
||||||
|
ORDER BY k.amount DESC
|
||||||
|
LIMIT 3
|
||||||
|
"""
|
||||||
|
rows = conn.execute(sql, [trade_date.isoformat(), opp.name]).fetchall()
|
||||||
|
opp.top_stocks = [r[0] for r in rows]
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
@@ -0,0 +1,246 @@
|
|||||||
|
"""Stock Recommendation Engine — money flow driven stock picks.
|
||||||
|
|
||||||
|
Identifies top stocks in sectors with strong capital inflow,
|
||||||
|
ranks by momentum/volume/trend, generates actionable trade plans
|
||||||
|
with entry/stop/target levels.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
|
||||||
|
|
||||||
|
def get_recommendations(
|
||||||
|
trade_date: date,
|
||||||
|
top_n: int = 10,
|
||||||
|
min_amount_yi: float = 2.0, # 最低日成交额(亿)
|
||||||
|
) -> list[dict]:
|
||||||
|
"""Generate ranked stock recommendations based on money flow.
|
||||||
|
|
||||||
|
1. Find sectors with strongest capital inflow
|
||||||
|
2. Within each sector, rank stocks by momentum + volume + trend
|
||||||
|
3. Generate trade plans (entry, stop, targets)
|
||||||
|
|
||||||
|
Returns list of recommendation dicts sorted by composite score.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
# ═══ Step 1: Find top inflow sectors ═══
|
||||||
|
try:
|
||||||
|
flow_sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT ts_code, trade_date, amount
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $start_date AND trade_date <= $end_date
|
||||||
|
),
|
||||||
|
daily AS (
|
||||||
|
SELECT n.trade_date, n.amount, si.industry_name
|
||||||
|
FROM normalized n
|
||||||
|
JOIN stock_industry si ON n.ts_code = si.ts_code
|
||||||
|
),
|
||||||
|
industry_daily AS (
|
||||||
|
SELECT industry_name, trade_date, SUM(amount) AS total_amount
|
||||||
|
FROM daily GROUP BY industry_name, trade_date
|
||||||
|
),
|
||||||
|
ranked AS (
|
||||||
|
SELECT *, ROW_NUMBER() OVER (PARTITION BY industry_name ORDER BY trade_date DESC) AS rn
|
||||||
|
FROM industry_daily
|
||||||
|
),
|
||||||
|
recent AS (
|
||||||
|
SELECT industry_name, SUM(total_amount) AS recent
|
||||||
|
FROM ranked WHERE rn <= 5 GROUP BY industry_name
|
||||||
|
),
|
||||||
|
prior AS (
|
||||||
|
SELECT industry_name, SUM(total_amount) AS prior
|
||||||
|
FROM ranked WHERE rn > 5 AND rn <= 10 GROUP BY industry_name
|
||||||
|
)
|
||||||
|
SELECT COALESCE(r.industry_name, p.industry_name) AS industry_name,
|
||||||
|
CASE WHEN COALESCE(p.prior,0) > 0 THEN (COALESCE(r.recent,0) - p.prior) / p.prior ELSE 0 END AS flow_change
|
||||||
|
FROM recent r FULL OUTER JOIN prior p ON r.industry_name = p.industry_name
|
||||||
|
WHERE COALESCE(r.recent, 0) + COALESCE(p.prior, 0) > 0
|
||||||
|
ORDER BY flow_change DESC
|
||||||
|
LIMIT 5
|
||||||
|
"""
|
||||||
|
end_str = trade_date.strftime("%Y-%m-%d")
|
||||||
|
from datetime import timedelta
|
||||||
|
start_str = (trade_date - timedelta(days=20)).strftime("%Y-%m-%d")
|
||||||
|
flow_df = conn.execute(
|
||||||
|
flow_sql.replace("$start_date", f"'{start_str}'").replace("$end_date", f"'{end_str}'")
|
||||||
|
).fetchdf()
|
||||||
|
top_sectors = flow_df["industry_name"].tolist() if not flow_df.empty else []
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Flow sector query failed: {e}")
|
||||||
|
top_sectors = []
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if not top_sectors:
|
||||||
|
return []
|
||||||
|
|
||||||
|
# ═══ Step 2: Find best stocks in top sectors ═══
|
||||||
|
conn = analytics_conn()
|
||||||
|
recommendations = []
|
||||||
|
|
||||||
|
for sector in top_sectors[:3]: # top 3 inflow sectors
|
||||||
|
try:
|
||||||
|
sector_list = "', '".join(top_sectors)
|
||||||
|
stock_sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT ts_code, trade_date, trade_time, close, volume, amount, high, low, open
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date <= $trade_date
|
||||||
|
),
|
||||||
|
with_ma AS (
|
||||||
|
SELECT *,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 19 PRECEDING AND CURRENT ROW) AS ma20,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 59 PRECEDING AND CURRENT ROW) AS ma60,
|
||||||
|
AVG(volume) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 19 PRECEDING AND CURRENT ROW) AS vol_ma20,
|
||||||
|
(high - low) AS day_range,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time DESC) AS rn
|
||||||
|
FROM normalized
|
||||||
|
),
|
||||||
|
with_atr AS (
|
||||||
|
SELECT *,
|
||||||
|
AVG(day_range) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 19 PRECEDING AND CURRENT ROW) AS atr20
|
||||||
|
FROM with_ma
|
||||||
|
),
|
||||||
|
latest AS (
|
||||||
|
SELECT w.*, si.industry_name
|
||||||
|
FROM with_atr w
|
||||||
|
JOIN stock_industry si ON w.ts_code = si.ts_code
|
||||||
|
WHERE w.rn = 1 AND si.industry_name = $sector
|
||||||
|
),
|
||||||
|
prev AS (
|
||||||
|
SELECT ts_code, close AS close_prev
|
||||||
|
FROM with_ma WHERE rn = 6
|
||||||
|
),
|
||||||
|
scored AS (
|
||||||
|
SELECT
|
||||||
|
l.ts_code, l.industry_name, l.close, l.volume, l.amount,
|
||||||
|
l.ma20, l.ma60, l.vol_ma20, l.atr20,
|
||||||
|
l.high, l.low, l.open,
|
||||||
|
p.close_prev,
|
||||||
|
-- Momentum: 5d return
|
||||||
|
(l.close - p.close_prev) / NULLIF(p.close_prev, 0) AS ret_5d,
|
||||||
|
-- Volume expansion
|
||||||
|
l.volume / NULLIF(l.vol_ma20, 0) AS vol_ratio,
|
||||||
|
-- Trend: above MAs
|
||||||
|
CASE WHEN l.close > l.ma20 THEN 1 ELSE 0 END + CASE WHEN l.close > l.ma60 THEN 1 ELSE 0 END AS trend_score
|
||||||
|
FROM latest l
|
||||||
|
LEFT JOIN prev p ON l.ts_code = p.ts_code
|
||||||
|
WHERE l.amount > $min_amt
|
||||||
|
)
|
||||||
|
SELECT *,
|
||||||
|
(COALESCE(ret_5d, 0) * 40 + LEAST(vol_ratio, 3.0) / 3.0 * 30 + trend_score * 15) AS composite
|
||||||
|
FROM scored
|
||||||
|
ORDER BY composite DESC
|
||||||
|
LIMIT 4
|
||||||
|
"""
|
||||||
|
min_amt = min_amount_yi * 1e8
|
||||||
|
stock_df = conn.execute(
|
||||||
|
stock_sql,
|
||||||
|
{"trade_date": trade_date.isoformat(), "sector": sector, "min_amt": min_amt},
|
||||||
|
).fetchdf()
|
||||||
|
|
||||||
|
# Look up stock names
|
||||||
|
stock_names = {}
|
||||||
|
try:
|
||||||
|
nc = duckdb.connect("data/duckdb/ashare.db", read_only=True)
|
||||||
|
for c in stock_df["ts_code"].tolist():
|
||||||
|
r = nc.execute("SELECT name FROM stock_info WHERE ts_code=?", [c]).fetchone()
|
||||||
|
stock_names[c] = r[0] if r else c
|
||||||
|
nc.close()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
for _, row in stock_df.iterrows():
|
||||||
|
import math
|
||||||
|
entry = float(row["close"])
|
||||||
|
atr_raw = row.get("atr20")
|
||||||
|
atr = float(atr_raw) if atr_raw and not (isinstance(atr_raw, float) and math.isnan(atr_raw)) else entry * 0.03
|
||||||
|
|
||||||
|
# Stop: MA20 or 2 ATR below entry
|
||||||
|
ma20 = float(row["ma20"] or entry)
|
||||||
|
stop = round(min(ma20 * 0.97, entry - 2 * atr), 2)
|
||||||
|
|
||||||
|
# Targets
|
||||||
|
target1 = round(entry + 1.5 * atr, 2)
|
||||||
|
target2 = round(entry + 3.0 * atr, 2)
|
||||||
|
|
||||||
|
# Risk/reward
|
||||||
|
risk = entry - stop
|
||||||
|
reward = target1 - entry
|
||||||
|
rr_ratio = round(reward / risk, 1) if risk > 0 else 0
|
||||||
|
|
||||||
|
# Position sizing guidance
|
||||||
|
position_advice = "Standard"
|
||||||
|
if float(row["trend_score"]) >= 2:
|
||||||
|
position_advice = "Aggressive"
|
||||||
|
elif float(row.get("vol_ratio", 1)) < 0.8:
|
||||||
|
position_advice = "Reduced"
|
||||||
|
|
||||||
|
import math as _m
|
||||||
|
def _safe(v, d):
|
||||||
|
try:
|
||||||
|
f = float(v)
|
||||||
|
return d if _m.isnan(f) else f
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
return d
|
||||||
|
_ret5 = _safe(row.get("ret_5d"), 0.0)
|
||||||
|
_volr = _safe(row.get("vol_ratio"), 1.0)
|
||||||
|
_trnd = _safe(row.get("trend_score"), 0.0)
|
||||||
|
_comp = _safe(row.get("composite"), 0.0)
|
||||||
|
if _m.isnan(_ret5): _ret5 = 0.0
|
||||||
|
if _m.isnan(_volr): _volr = 1.0
|
||||||
|
if _m.isnan(_trnd): _trnd = 0.0
|
||||||
|
if _m.isnan(_comp): _comp = 0.0
|
||||||
|
|
||||||
|
ts = row["ts_code"]
|
||||||
|
recommendations.append({
|
||||||
|
"ts_code": ts,
|
||||||
|
"name": stock_names.get(ts, ts),
|
||||||
|
"sector": sector,
|
||||||
|
"entry": round(entry, 2),
|
||||||
|
"stop": stop,
|
||||||
|
"target1": target1,
|
||||||
|
"target2": target2,
|
||||||
|
"rr_ratio": round(float(rr_ratio), 1) if not _m.isnan(rr_ratio) else 0.0,
|
||||||
|
"position": position_advice,
|
||||||
|
"score": round(_comp, 1),
|
||||||
|
"metrics": {
|
||||||
|
"ret_5d": round(_ret5 * 100, 1),
|
||||||
|
"vol_ratio": round(_volr, 2),
|
||||||
|
"trend": "Strong" if _trnd >= 2 else "Weak",
|
||||||
|
"amount_yi": round(float(row["amount"]) / 1e8, 1),
|
||||||
|
},
|
||||||
|
"action": _recommend_action(_ret5, _volr, _trnd),
|
||||||
|
})
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Stock scoring failed for sector {sector}: {e}")
|
||||||
|
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
# Sort by composite score
|
||||||
|
recommendations.sort(key=lambda r: r["score"], reverse=True)
|
||||||
|
return recommendations[:top_n]
|
||||||
|
|
||||||
|
|
||||||
|
def _recommend_action(ret_5d: float, vol_ratio: float, trend_score: float) -> str:
|
||||||
|
"""Generate trading action recommendation."""
|
||||||
|
if trend_score >= 2 and ret_5d > 0.03 and vol_ratio > 1.2:
|
||||||
|
return "买入 — 趋势强势,放量上涨"
|
||||||
|
elif trend_score >= 2 and ret_5d > 0:
|
||||||
|
return "关注 — 趋势健康,等待回踩"
|
||||||
|
elif trend_score >= 1 and vol_ratio > 1.0:
|
||||||
|
return "观察 — 趋势形成中,可轻仓试"
|
||||||
|
elif ret_5d < -0.03:
|
||||||
|
return "回避 — 短期偏弱,等待企稳"
|
||||||
|
else:
|
||||||
|
return "观望 — 方向不明,暂不参与"
|
||||||
@@ -0,0 +1,195 @@
|
|||||||
|
"""Sentiment Engine — real limit-up/down data from akshare.
|
||||||
|
|
||||||
|
Phase 2: fetches 涨停/跌停/炸板 data via EM API,
|
||||||
|
stores daily snapshots, computes profit effect score.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_daily_sentiment(trade_date: date | None = None) -> dict:
|
||||||
|
"""Fetch today's sentiment data from akshare.
|
||||||
|
|
||||||
|
Uses East Money APIs (accessible outside China — different from stock data).
|
||||||
|
Returns structured sentiment dict for API/dashboard consumption.
|
||||||
|
"""
|
||||||
|
import akshare as ak
|
||||||
|
|
||||||
|
if trade_date is None:
|
||||||
|
trade_date = date.today()
|
||||||
|
|
||||||
|
date_str = trade_date.strftime("%Y%m%d")
|
||||||
|
|
||||||
|
# ── Fetch limit-up pool ──
|
||||||
|
try:
|
||||||
|
zt_df = ak.stock_zt_pool_em(date=date_str)
|
||||||
|
limit_up_count = len(zt_df)
|
||||||
|
# Extract board stats
|
||||||
|
board_stats = _extract_board_stats(zt_df)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Limit-up fetch failed: {e}")
|
||||||
|
limit_up_count = 0
|
||||||
|
board_stats = {}
|
||||||
|
|
||||||
|
# ── Fetch limit-down pool ──
|
||||||
|
try:
|
||||||
|
dt_df = ak.stock_zt_pool_dtgc_em(date=date_str)
|
||||||
|
limit_down_count = len(dt_df)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Limit-down fetch failed: {e}")
|
||||||
|
limit_down_count = 0
|
||||||
|
|
||||||
|
# ── Fetch broken-board pool (炸板) ──
|
||||||
|
try:
|
||||||
|
zb_df = ak.stock_zt_pool_zbgc_em(date=date_str)
|
||||||
|
broken_board_count = len(zb_df)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Broken-board fetch failed: {e}")
|
||||||
|
broken_board_count = 0
|
||||||
|
|
||||||
|
# ── Compute derived metrics ──
|
||||||
|
board_total = limit_up_count + broken_board_count
|
||||||
|
broken_board_rate = round(broken_board_count / board_total, 4) if board_total > 0 else 0
|
||||||
|
|
||||||
|
# Board progression rate: 连板数>=2 / total limit-up
|
||||||
|
consecutive_count = board_stats.get("consecutive_count", 0)
|
||||||
|
consecutive_board_rate = round(consecutive_count / limit_up_count, 4) if limit_up_count > 0 else 0
|
||||||
|
|
||||||
|
# First-board count
|
||||||
|
first_board_count = board_stats.get("first_board_count", 0)
|
||||||
|
|
||||||
|
# Profit effect score (0-100)
|
||||||
|
profit_score = _compute_profit_score(
|
||||||
|
limit_up_count=limit_up_count,
|
||||||
|
limit_down_count=limit_down_count,
|
||||||
|
broken_board_rate=broken_board_rate,
|
||||||
|
consecutive_board_rate=consecutive_board_rate,
|
||||||
|
total_stocks=board_stats.get("total_analyzed", 5000),
|
||||||
|
)
|
||||||
|
|
||||||
|
# Profit effect label
|
||||||
|
if profit_score >= 80:
|
||||||
|
profit_effect = "Strong"
|
||||||
|
elif profit_score >= 60:
|
||||||
|
profit_effect = "Average"
|
||||||
|
elif profit_score >= 40:
|
||||||
|
profit_effect = "Weak"
|
||||||
|
else:
|
||||||
|
profit_effect = "Collapse"
|
||||||
|
|
||||||
|
return {
|
||||||
|
"status": "live",
|
||||||
|
"trade_date": trade_date.isoformat(),
|
||||||
|
"profit_effect": profit_effect,
|
||||||
|
"profit_effect_score": profit_score,
|
||||||
|
"limit_up_count": limit_up_count,
|
||||||
|
"limit_down_count": limit_down_count,
|
||||||
|
"broken_board_count": broken_board_count,
|
||||||
|
"broken_board_rate": broken_board_rate,
|
||||||
|
"first_board_count": first_board_count,
|
||||||
|
"consecutive_count": consecutive_count,
|
||||||
|
"consecutive_board_rate": consecutive_board_rate,
|
||||||
|
"max_consecutive": board_stats.get("max_consecutive", 0),
|
||||||
|
"top_industries": board_stats.get("top_industries", []),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _extract_board_stats(zt_df) -> dict:
|
||||||
|
"""Extract board statistics from limit-up DataFrame."""
|
||||||
|
total = len(zt_df)
|
||||||
|
if total == 0:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
consecutive_count = 0
|
||||||
|
first_board_count = 0
|
||||||
|
max_consecutive = 0
|
||||||
|
industry_counts = {}
|
||||||
|
|
||||||
|
for _, row in zt_df.iterrows():
|
||||||
|
# Parse 连板数
|
||||||
|
lb = row.get("连板数", 0)
|
||||||
|
try:
|
||||||
|
lb = int(lb)
|
||||||
|
except (ValueError, TypeError):
|
||||||
|
lb = 1
|
||||||
|
|
||||||
|
if lb >= 2:
|
||||||
|
consecutive_count += 1
|
||||||
|
if lb == 1:
|
||||||
|
first_board_count += 1
|
||||||
|
max_consecutive = max(max_consecutive, lb)
|
||||||
|
|
||||||
|
# Industry
|
||||||
|
ind = row.get("所属行业", "其他")
|
||||||
|
industry_counts[ind] = industry_counts.get(ind, 0) + 1
|
||||||
|
|
||||||
|
top_industries = sorted(industry_counts.items(), key=lambda x: -x[1])[:5]
|
||||||
|
|
||||||
|
return {
|
||||||
|
"consecutive_count": consecutive_count,
|
||||||
|
"first_board_count": first_board_count,
|
||||||
|
"max_consecutive": max_consecutive,
|
||||||
|
"top_industries": [{"name": n, "count": c} for n, c in top_industries],
|
||||||
|
"total_analyzed": total,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _compute_profit_score(
|
||||||
|
limit_up_count: int,
|
||||||
|
limit_down_count: int,
|
||||||
|
broken_board_rate: float,
|
||||||
|
consecutive_board_rate: float,
|
||||||
|
total_stocks: int = 5000,
|
||||||
|
) -> float:
|
||||||
|
"""Compute profit effect score (0-100).
|
||||||
|
|
||||||
|
Factors:
|
||||||
|
- Limit-up ratio vs total stocks (more = better, up to 50 points)
|
||||||
|
- Limit-up/down ratio (more ups than downs = better, up to 25 points)
|
||||||
|
- Broken board rate penalty (higher = worse, up to -15 points)
|
||||||
|
- Consecutive board rate (higher = stronger sentiment, up to 10 points)
|
||||||
|
"""
|
||||||
|
score = 50.0
|
||||||
|
|
||||||
|
# Limit-up breadth
|
||||||
|
zt_pct = limit_up_count / max(total_stocks, 1) * 100
|
||||||
|
score += min(zt_pct * 3, 20) # 0-20 bonus
|
||||||
|
|
||||||
|
# Up/down ratio
|
||||||
|
if limit_down_count > 0:
|
||||||
|
ratio = limit_up_count / limit_down_count
|
||||||
|
score += min(ratio * 2, 15) # 0-15 bonus
|
||||||
|
else:
|
||||||
|
score += 15
|
||||||
|
|
||||||
|
# Broken board penalty
|
||||||
|
score -= broken_board_rate * 20 # 0 to -20
|
||||||
|
|
||||||
|
# Consecutive board bonus
|
||||||
|
score += consecutive_board_rate * 15 # 0 to 15
|
||||||
|
|
||||||
|
return round(max(0, min(100, score)), 1)
|
||||||
|
|
||||||
|
|
||||||
|
def assess_sentiment(trade_date: date | None = None) -> dict:
|
||||||
|
"""Main entry point — fetch sentiment for a trading date.
|
||||||
|
|
||||||
|
Tries akshare first. Falls back to stored data if API unavailable.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
return fetch_daily_sentiment(trade_date)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Sentiment assessment failed, using fallback: {e}")
|
||||||
|
return {
|
||||||
|
"status": "error",
|
||||||
|
"profit_effect": None,
|
||||||
|
"limit_up_count": None,
|
||||||
|
"limit_down_count": None,
|
||||||
|
"broken_board_rate": None,
|
||||||
|
"consecutive_board_rate": None,
|
||||||
|
"profit_effect_score": None,
|
||||||
|
}
|
||||||
@@ -0,0 +1,89 @@
|
|||||||
|
"""Market State Engine — infers MarketState from feature vectors.
|
||||||
|
|
||||||
|
Takes computed features from the Feature Registry, produces a
|
||||||
|
continuous MarketState vector. Does NOT import individual features.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, datetime
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.domain.state import MarketState
|
||||||
|
from ashare_dp.features.registry import FeatureRegistry
|
||||||
|
|
||||||
|
|
||||||
|
def infer_market_state(
|
||||||
|
features: dict[str, dict[str, Any]],
|
||||||
|
trade_date: date,
|
||||||
|
timestamp: datetime | None = None,
|
||||||
|
) -> MarketState:
|
||||||
|
"""Infer MarketState from feature vectors.
|
||||||
|
|
||||||
|
The Feature Registry computes raw features. This engine
|
||||||
|
transforms them into the unified MarketState domain object.
|
||||||
|
|
||||||
|
Args:
|
||||||
|
features: Dict mapping feature name → feature value dict,
|
||||||
|
as returned by FeatureRegistry.compute_all().
|
||||||
|
trade_date: The trading date these features describe.
|
||||||
|
timestamp: Optional override for the state timestamp.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
MarketState domain object with all dimensions populated.
|
||||||
|
"""
|
||||||
|
if timestamp is None:
|
||||||
|
timestamp = datetime.now()
|
||||||
|
|
||||||
|
try:
|
||||||
|
state = MarketState.from_features(features, trade_date, timestamp)
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"MarketState inference failed: {e}")
|
||||||
|
state = MarketState(
|
||||||
|
timestamp=timestamp,
|
||||||
|
trade_date=trade_date,
|
||||||
|
confidence=0.0,
|
||||||
|
quality=0.0,
|
||||||
|
)
|
||||||
|
|
||||||
|
return state
|
||||||
|
|
||||||
|
|
||||||
|
def state_to_label(state: MarketState) -> str:
|
||||||
|
"""Derive a human-readable regime label from the state vector.
|
||||||
|
|
||||||
|
This is for DISPLAY ONLY. Downstream engines use the vector
|
||||||
|
directly, never the label.
|
||||||
|
|
||||||
|
Classification logic:
|
||||||
|
- trend > 0.65, fear < 0.35 → "趋势"
|
||||||
|
- trend > 0.65, fear >= 0.35 → "波动上升"
|
||||||
|
- trend 0.35–0.65 → "震荡"
|
||||||
|
- trend < 0.35, fear > 0.65 → "恐慌"
|
||||||
|
- trend low, fear low → "低迷"
|
||||||
|
- trend improving, fear decreasing → "复苏"
|
||||||
|
"""
|
||||||
|
t = state.trend
|
||||||
|
f = state.fear
|
||||||
|
l = state.liquidity
|
||||||
|
|
||||||
|
if t > 0.65 and f < 0.35:
|
||||||
|
if l > 0.7:
|
||||||
|
return "趋势"
|
||||||
|
return "趋势(缩量)"
|
||||||
|
elif t > 0.65 and f >= 0.35:
|
||||||
|
return "波动上升"
|
||||||
|
elif t > 0.45:
|
||||||
|
if f < 0.4:
|
||||||
|
return "震荡偏强"
|
||||||
|
return "震荡"
|
||||||
|
elif t > 0.25:
|
||||||
|
return "震荡偏弱"
|
||||||
|
elif f > 0.65:
|
||||||
|
return "恐慌"
|
||||||
|
elif t < 0.25 and f < 0.35:
|
||||||
|
return "低迷"
|
||||||
|
else:
|
||||||
|
return "复苏"
|
||||||
@@ -1,221 +0,0 @@
|
|||||||
"""Scheduler job implementations."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from datetime import date, datetime
|
|
||||||
|
|
||||||
from loguru import logger
|
|
||||||
|
|
||||||
from ashare_dp.core.calendar import BEIJING_TZ
|
|
||||||
from ashare_dp.data.eod import EODPipeline
|
|
||||||
from ashare_dp.storage.repository import KLineRepository
|
|
||||||
|
|
||||||
|
|
||||||
async def eod_pull_job():
|
|
||||||
"""End-of-day data pull job.
|
|
||||||
|
|
||||||
Triggered at 15:05 Beijing time on trading days.
|
|
||||||
Pulls daily, minute, weekly, and monthly data for all active stocks.
|
|
||||||
"""
|
|
||||||
now = datetime.now(BEIJING_TZ)
|
|
||||||
today = now.date()
|
|
||||||
|
|
||||||
# Verify today is a weekday (trading day check done by scheduler)
|
|
||||||
if now.weekday() >= 5:
|
|
||||||
logger.info("EOD: Weekend, skipping")
|
|
||||||
return
|
|
||||||
|
|
||||||
logger.info(f"EOD job starting for {today.isoformat()}...")
|
|
||||||
|
|
||||||
pipeline = EODPipeline()
|
|
||||||
|
|
||||||
try:
|
|
||||||
symbols = pipeline.get_active_symbols()
|
|
||||||
logger.info(f"EOD: {len(symbols)} active stocks")
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EOD: Failed to get stock list: {e}")
|
|
||||||
return
|
|
||||||
|
|
||||||
# Pull daily data
|
|
||||||
try:
|
|
||||||
pipeline.pull_daily(symbols, today)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EOD daily pull failed: {e}")
|
|
||||||
|
|
||||||
# Pull minute data (1m, 5m, 15m, 30m, 1h)
|
|
||||||
try:
|
|
||||||
pipeline.pull_minute(symbols, today)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EOD minute pull failed: {e}")
|
|
||||||
|
|
||||||
# Pull weekly/monthly
|
|
||||||
try:
|
|
||||||
pipeline.pull_weekly_monthly(symbols, today)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EOD weekly/monthly pull failed: {e}")
|
|
||||||
|
|
||||||
logger.info(f"EOD job completed for {today.isoformat()}")
|
|
||||||
|
|
||||||
|
|
||||||
async def health_check_job():
|
|
||||||
"""Daily health check: report database statistics."""
|
|
||||||
logger.info("Health check running...")
|
|
||||||
repo = KLineRepository()
|
|
||||||
try:
|
|
||||||
for freq_res in [repo.get_date_range(f) for f in [Freq.d1, Freq.h1, Freq.m5]]:
|
|
||||||
pass
|
|
||||||
logger.info(
|
|
||||||
f"Health check: DB OK, "
|
|
||||||
f"1d records={repo.count_records(Freq.d1)}, "
|
|
||||||
f"1h records={repo.count_records(Freq.h1)}"
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Health check failed: {e}")
|
|
||||||
|
|
||||||
|
|
||||||
# Import at bottom to avoid circular
|
|
||||||
from ashare_dp.core.models import Freq
|
|
||||||
|
|
||||||
|
|
||||||
async def ema52_screening_job():
|
|
||||||
"""EOD EMA52 screening: find stocks near EMA52 for 1d and 1w.
|
|
||||||
|
|
||||||
Triggered after EOD data pull. Scans all stocks, computes EMA52
|
|
||||||
from close prices, and saves results to ema52_screening table.
|
|
||||||
"""
|
|
||||||
from ashare_dp.config import Settings
|
|
||||||
|
|
||||||
import numpy as np
|
|
||||||
import pandas as pd
|
|
||||||
from ashare_dp.storage.database import get_db
|
|
||||||
|
|
||||||
settings = Settings()
|
|
||||||
threshold = settings.ema52_threshold
|
|
||||||
db = get_db()
|
|
||||||
|
|
||||||
# Get all ts_codes and names
|
|
||||||
stocks = db.query(
|
|
||||||
"SELECT ts_code, name FROM stock_info ORDER BY ts_code"
|
|
||||||
)
|
|
||||||
symbols = [row[0] for row in stocks]
|
|
||||||
symbol_names = {row[0]: row[1] for row in stocks}
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
f"EMA52 screening: {len(symbols)} stocks, "
|
|
||||||
f"threshold=±{threshold * 100:.1f}%"
|
|
||||||
)
|
|
||||||
|
|
||||||
for freq, freq_label, bars_needed in [
|
|
||||||
(Freq.d1, "1d", 100),
|
|
||||||
(Freq.w1, "1w", 60),
|
|
||||||
]:
|
|
||||||
try:
|
|
||||||
from ashare_dp.storage.partitioning import partition_glob
|
|
||||||
glob = partition_glob(freq)
|
|
||||||
# Get last N bars per stock using ROW_NUMBER()
|
|
||||||
sql = f"""
|
|
||||||
WITH ranked AS (
|
|
||||||
SELECT *, ROW_NUMBER() OVER (
|
|
||||||
PARTITION BY ts_code ORDER BY trade_time DESC
|
|
||||||
) as rn
|
|
||||||
FROM read_parquet('{glob}', hive_partitioning=true, union_by_name=true)
|
|
||||||
)
|
|
||||||
SELECT * FROM ranked WHERE rn <= {bars_needed}
|
|
||||||
ORDER BY ts_code, trade_time ASC
|
|
||||||
"""
|
|
||||||
df = db.conn.execute(sql).fetchdf()
|
|
||||||
if df.empty:
|
|
||||||
logger.warning(f"EMA52: no data for {freq_label}")
|
|
||||||
continue
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EMA52: failed to read {freq_label}: {e}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
df["trade_time"] = pd.to_datetime(df["trade_time"])
|
|
||||||
df = df.sort_values(["ts_code", "trade_time"])
|
|
||||||
|
|
||||||
# Get last N bars per stock, compute EMA52
|
|
||||||
results = []
|
|
||||||
for ts_code, group in df.groupby("ts_code"):
|
|
||||||
if ts_code not in symbol_names:
|
|
||||||
continue
|
|
||||||
group = group.tail(bars_needed)
|
|
||||||
if len(group) < 26:
|
|
||||||
continue
|
|
||||||
|
|
||||||
closes = group["close"].astype(float).values
|
|
||||||
# Compute EMA52
|
|
||||||
ema_values = _compute_ema(closes, 52)
|
|
||||||
if len(ema_values) == 0:
|
|
||||||
continue
|
|
||||||
|
|
||||||
latest_close = closes[-1]
|
|
||||||
latest_ema = ema_values[-1]
|
|
||||||
if latest_ema <= 0:
|
|
||||||
continue
|
|
||||||
distance = (latest_close - latest_ema) / latest_ema
|
|
||||||
|
|
||||||
# Compute daily amount in 亿 CNY
|
|
||||||
latest_amount = float(group["amount"].iloc[-1])
|
|
||||||
amount_yi = latest_amount / 100_000_000 # 元 → 亿
|
|
||||||
|
|
||||||
if abs(distance) <= threshold:
|
|
||||||
results.append({
|
|
||||||
"trade_date": group["trade_date"].iloc[-1],
|
|
||||||
"ts_code": ts_code,
|
|
||||||
"name": symbol_names[ts_code],
|
|
||||||
"freq": freq_label,
|
|
||||||
"close_price": round(latest_close, 2),
|
|
||||||
"ema52": round(latest_ema, 2),
|
|
||||||
"distance_pct": round(distance * 100, 2),
|
|
||||||
"amount": round(amount_yi, 2),
|
|
||||||
})
|
|
||||||
|
|
||||||
if results:
|
|
||||||
# Sort by amount DESC (largest turnover first)
|
|
||||||
results.sort(key=lambda r: r.get("amount", 0) or 0, reverse=True)
|
|
||||||
# Upsert: delete old then insert
|
|
||||||
try:
|
|
||||||
db.conn.execute(
|
|
||||||
"DELETE FROM ema52_screening WHERE freq = ?",
|
|
||||||
[freq_label],
|
|
||||||
)
|
|
||||||
db.conn.executemany(
|
|
||||||
"""INSERT INTO ema52_screening
|
|
||||||
(trade_date, ts_code, name, freq, close_price, ema52, distance_pct, amount)
|
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
|
||||||
ON CONFLICT (trade_date, ts_code, freq) DO UPDATE SET
|
|
||||||
close_price=excluded.close_price,
|
|
||||||
ema52=excluded.ema52,
|
|
||||||
distance_pct=excluded.distance_pct,
|
|
||||||
amount=excluded.amount,
|
|
||||||
name=excluded.name,
|
|
||||||
updated_at=now()""",
|
|
||||||
[(r["trade_date"], r["ts_code"], r["name"],
|
|
||||||
r["freq"], r["close_price"], r["ema52"], r["distance_pct"], r["amount"])
|
|
||||||
for r in results],
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"EMA52: failed to save {freq_label}: {e}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
near = [r for r in results if abs(r["distance_pct"]) <= threshold * 100]
|
|
||||||
logger.info(
|
|
||||||
f"EMA52 {freq_label}: {len(results)} near "
|
|
||||||
f"(±{threshold * 100:.1f}%), {len(near)} within threshold"
|
|
||||||
)
|
|
||||||
|
|
||||||
logger.info("EMA52 screening complete")
|
|
||||||
|
|
||||||
|
|
||||||
def _compute_ema(values: "np.ndarray", period: int) -> "np.ndarray":
|
|
||||||
"""Compute EMA for a 1D array of values."""
|
|
||||||
import numpy as np
|
|
||||||
if len(values) < period:
|
|
||||||
return np.array([])
|
|
||||||
alpha = 2.0 / (period + 1)
|
|
||||||
ema = np.zeros(len(values))
|
|
||||||
ema[period - 1] = np.mean(values[:period])
|
|
||||||
for i in range(period, len(values)):
|
|
||||||
ema[i] = alpha * values[i] + (1 - alpha) * ema[i - 1]
|
|
||||||
return ema
|
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
"""SIGNAL INTELLIGENCE — Signal × MarketState → Expectancy. The moat.
|
||||||
|
|
||||||
|
Signal detection, storage, and historical expectancy queries.
|
||||||
|
"""
|
||||||
|
from ashare_dp.signals.detectors import (
|
||||||
|
detect_ema52_signals, detect_vegas_signals,
|
||||||
|
detect_chan_signals, detect_orb_signals,
|
||||||
|
detect_gap_signals, detect_nr7_signals, detect_ib_signals,
|
||||||
|
run_ema52_screening,
|
||||||
|
)
|
||||||
|
from ashare_dp.signals.store import store_signals, count_signals
|
||||||
|
from ashare_dp.signals.expectancy import get_expectancy
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"detect_ema52_signals", "detect_vegas_signals",
|
||||||
|
"detect_chan_signals", "detect_orb_signals",
|
||||||
|
"run_ema52_screening",
|
||||||
|
"store_signals", "count_signals", "get_expectancy",
|
||||||
|
]
|
||||||
@@ -0,0 +1,696 @@
|
|||||||
|
"""Signal Detector — finds trading signals in historical K-line data.
|
||||||
|
|
||||||
|
Phase 1: EMA52 crossover detection with forward outcome computation.
|
||||||
|
Single DuckDB scan — detects, captures context, computes outcomes.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import date, timedelta
|
||||||
|
|
||||||
|
import duckdb
|
||||||
|
import pandas as pd
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.data.store.database import analytics_conn, kline_glob
|
||||||
|
from ashare_dp.domain.signal import SignalType
|
||||||
|
|
||||||
|
|
||||||
|
def detect_ema52_signals(
|
||||||
|
end_date: date | None = None,
|
||||||
|
lookback_days: int = 500,
|
||||||
|
) -> pd.DataFrame:
|
||||||
|
"""Detect EMA52 cross signals from daily K-line data.
|
||||||
|
|
||||||
|
Single DuckDB query that:
|
||||||
|
1. Computes EMA52 per stock
|
||||||
|
2. Detects cross-up and cross-down points
|
||||||
|
3. Computes forward 5d/10d/20d returns
|
||||||
|
4. Computes max favorable/adverse excursion
|
||||||
|
5. Captures daily market breadth at signal time
|
||||||
|
|
||||||
|
Args:
|
||||||
|
end_date: Latest date to scan (default: latest available).
|
||||||
|
lookback_days: Calendar days to look back.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
DataFrame with columns matching SignalInstance fields.
|
||||||
|
"""
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
if end_date is None:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT MAX(trade_date) FROM read_parquet('{parquet_glob}', "
|
||||||
|
f"hive_partitioning=true, union_by_name=true)"
|
||||||
|
).fetchone()
|
||||||
|
if not row or not row[0]:
|
||||||
|
conn.close()
|
||||||
|
return pd.DataFrame()
|
||||||
|
end_date = date.fromisoformat(str(row[0])[:10])
|
||||||
|
|
||||||
|
start_date = end_date - timedelta(days=lookback_days)
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT
|
||||||
|
ts_code,
|
||||||
|
trade_date, trade_time, close, volume, open, high, low
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $start_date AND trade_date <= $end_date
|
||||||
|
),
|
||||||
|
-- EMA52 per stock
|
||||||
|
with_ema AS (
|
||||||
|
SELECT *,
|
||||||
|
-- EMA52: alpha = 2/(52+1) ≈ 0.03774
|
||||||
|
-- Use recursive approximation via window
|
||||||
|
CASE
|
||||||
|
WHEN ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time) = 1
|
||||||
|
THEN close
|
||||||
|
ELSE 0.03774 * close + (1 - 0.03774) * LAG(close) OVER (PARTITION BY ts_code ORDER BY trade_time)
|
||||||
|
END AS ema52_seed
|
||||||
|
FROM normalized
|
||||||
|
),
|
||||||
|
-- Better EMA52: pandas-style using cumulative product isn't easy in SQL.
|
||||||
|
-- Simplified: use a 52-day SMA as close-enough proxy for Phase 1.
|
||||||
|
with_sma AS (
|
||||||
|
SELECT *,
|
||||||
|
AVG(close) OVER (
|
||||||
|
PARTITION BY ts_code ORDER BY trade_time
|
||||||
|
ROWS BETWEEN 51 PRECEDING AND CURRENT ROW
|
||||||
|
) AS sma52,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time) as rn
|
||||||
|
FROM normalized
|
||||||
|
),
|
||||||
|
-- Cross detection: signal when close crosses SMA52
|
||||||
|
with_cross AS (
|
||||||
|
SELECT *,
|
||||||
|
LAG(close) OVER (PARTITION BY ts_code ORDER BY trade_time) AS prev_close,
|
||||||
|
LAG(sma52) OVER (PARTITION BY ts_code ORDER BY trade_time) AS prev_sma52,
|
||||||
|
-- Daily market breadth at this date
|
||||||
|
AVG(CASE WHEN close > open THEN 1.0 ELSE 0.0 END) OVER (PARTITION BY trade_date) AS daily_advance_ratio,
|
||||||
|
SUM(volume) OVER (PARTITION BY trade_date) AS daily_total_volume,
|
||||||
|
AVG((high - low) / NULLIF(close, 0)) OVER (PARTITION BY trade_date) AS daily_range
|
||||||
|
FROM with_sma
|
||||||
|
WHERE rn > 52 -- need SMA to be valid
|
||||||
|
),
|
||||||
|
signals AS (
|
||||||
|
SELECT
|
||||||
|
ts_code,
|
||||||
|
trade_date,
|
||||||
|
trade_time,
|
||||||
|
close AS signal_price,
|
||||||
|
sma52,
|
||||||
|
daily_advance_ratio,
|
||||||
|
daily_range,
|
||||||
|
-- Cross detection
|
||||||
|
CASE
|
||||||
|
WHEN prev_close <= prev_sma52 AND close > sma52
|
||||||
|
THEN 'ema52_cross_up'
|
||||||
|
WHEN prev_close >= prev_sma52 AND close < sma52
|
||||||
|
THEN 'ema52_cross_down'
|
||||||
|
ELSE NULL
|
||||||
|
END AS signal_type,
|
||||||
|
rn
|
||||||
|
FROM with_cross
|
||||||
|
),
|
||||||
|
-- Forward returns (join with future data)
|
||||||
|
signal_with_future AS (
|
||||||
|
SELECT
|
||||||
|
s.ts_code, s.trade_date, s.signal_price, s.sma52,
|
||||||
|
s.daily_advance_ratio, s.daily_range, s.signal_type, s.rn,
|
||||||
|
-- 5d forward
|
||||||
|
f5.close AS close_5d,
|
||||||
|
(f5.close - s.signal_price) / NULLIF(s.signal_price, 0) AS ret_5d,
|
||||||
|
-- 10d forward
|
||||||
|
f10.close AS close_10d,
|
||||||
|
(f10.close - s.signal_price) / NULLIF(s.signal_price, 0) AS ret_10d,
|
||||||
|
-- 20d forward
|
||||||
|
f20.close AS close_20d,
|
||||||
|
(f20.close - s.signal_price) / NULLIF(s.signal_price, 0) AS ret_20d
|
||||||
|
FROM signals s
|
||||||
|
LEFT JOIN with_sma f5 ON s.ts_code = f5.ts_code AND f5.rn = s.rn + 5
|
||||||
|
LEFT JOIN with_sma f10 ON s.ts_code = f10.ts_code AND f10.rn = s.rn + 10
|
||||||
|
LEFT JOIN with_sma f20 ON s.ts_code = f20.ts_code AND f20.rn = s.rn + 20
|
||||||
|
WHERE s.signal_type IS NOT NULL
|
||||||
|
),
|
||||||
|
-- Max drawdown within 5d window
|
||||||
|
with_mdd AS (
|
||||||
|
SELECT sf.*,
|
||||||
|
-- 5d max drawdown: need to check each day between signal and 5d later
|
||||||
|
-- Simplified: use the min close in the window
|
||||||
|
MIN(w.close) OVER (
|
||||||
|
PARTITION BY sf.ts_code, sf.trade_date
|
||||||
|
ORDER BY w.trade_time
|
||||||
|
ROWS BETWEEN CURRENT ROW AND 4 FOLLOWING
|
||||||
|
) AS min_close_5d
|
||||||
|
FROM signal_with_future sf
|
||||||
|
LEFT JOIN with_sma w
|
||||||
|
ON sf.ts_code = w.ts_code
|
||||||
|
AND w.rn BETWEEN sf.rn AND sf.rn + 5
|
||||||
|
)
|
||||||
|
SELECT DISTINCT
|
||||||
|
signal_type,
|
||||||
|
ts_code,
|
||||||
|
trade_date,
|
||||||
|
signal_price,
|
||||||
|
daily_advance_ratio,
|
||||||
|
daily_range,
|
||||||
|
ret_5d,
|
||||||
|
ret_10d,
|
||||||
|
ret_20d,
|
||||||
|
-- Max drawdown within 5d
|
||||||
|
CASE WHEN min_close_5d IS NOT NULL AND signal_price > 0
|
||||||
|
THEN (min_close_5d - signal_price) / signal_price
|
||||||
|
ELSE NULL END AS max_dd_5d,
|
||||||
|
-- Max return within 5d (approximate)
|
||||||
|
CASE WHEN close_5d IS NOT NULL AND signal_price > 0
|
||||||
|
THEN (close_5d - signal_price) / signal_price
|
||||||
|
ELSE NULL END AS max_return_5d
|
||||||
|
FROM with_mdd
|
||||||
|
WHERE signal_type IS NOT NULL
|
||||||
|
ORDER BY trade_date, ts_code
|
||||||
|
"""
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start_date", f"'{start_date.strftime('%Y-%m-%d')}'")
|
||||||
|
.replace("$end_date", f"'{end_date.strftime('%Y-%m-%d')}'")
|
||||||
|
).fetchdf()
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Signal detection failed: {e}")
|
||||||
|
conn.close()
|
||||||
|
return pd.DataFrame()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return result
|
||||||
|
|
||||||
|
# Clean up
|
||||||
|
result = result.drop_duplicates(subset=["signal_type", "ts_code", "trade_date"])
|
||||||
|
logger.info(
|
||||||
|
f"Detected {len(result)} EMA52 signals "
|
||||||
|
f"({len(result[result['signal_type']=='ema52_cross_up'])} cross-up, "
|
||||||
|
f"{len(result[result['signal_type']=='ema52_cross_down'])} cross-down)"
|
||||||
|
)
|
||||||
|
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════
|
||||||
|
# Vegas Channel Detection
|
||||||
|
# ═══════════════════════════════════════════════════
|
||||||
|
|
||||||
|
def detect_vegas_signals(end_date=None, lookback_days=500) -> "pd.DataFrame":
|
||||||
|
"""Detect Vegas Channel signals from daily K-line.
|
||||||
|
|
||||||
|
Vegas Channel: EMA12, EMA50, EMA144.
|
||||||
|
LONG signal: price > EMA144, EMA12 > EMA50 > EMA144 (bullish tunnel),
|
||||||
|
price pulls back to near EMA50.
|
||||||
|
SHORT signal: inverse.
|
||||||
|
"""
|
||||||
|
import math as _math
|
||||||
|
from datetime import date as _date, timedelta as _td
|
||||||
|
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
if end_date is None:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT MAX(trade_date) FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)"
|
||||||
|
).fetchone()
|
||||||
|
if not row or not row[0]:
|
||||||
|
conn.close()
|
||||||
|
return pd.DataFrame()
|
||||||
|
end_date = _date.fromisoformat(str(row[0])[:10])
|
||||||
|
|
||||||
|
start_date = end_date - _td(days=lookback_days)
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT ts_code, trade_date, trade_time, close, volume, high, low, open
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $start AND trade_date <= $end
|
||||||
|
),
|
||||||
|
with_ema AS (
|
||||||
|
SELECT *,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 11 PRECEDING AND CURRENT ROW) AS ema12,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 49 PRECEDING AND CURRENT ROW) AS ema50,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 143 PRECEDING AND CURRENT ROW) AS ema144,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time) as rn,
|
||||||
|
LAG(close) OVER (PARTITION BY ts_code ORDER BY trade_time) AS prev_close
|
||||||
|
FROM normalized
|
||||||
|
),
|
||||||
|
with_signal AS (
|
||||||
|
SELECT *,
|
||||||
|
-- Vegas LONG: bullish tunnel (ema12>ema50>ema144) + price near EMA50 (within 3%)
|
||||||
|
CASE WHEN rn > 144
|
||||||
|
AND ema12 > ema50 AND ema50 > ema144
|
||||||
|
AND close > ema144
|
||||||
|
AND ABS(close - ema50) / NULLIF(ema50, 0) < 0.03
|
||||||
|
AND prev_close <= ema50
|
||||||
|
THEN 'vegas_long' ELSE NULL END AS signal_type_up,
|
||||||
|
-- Vegas SHORT
|
||||||
|
CASE WHEN rn > 144
|
||||||
|
AND ema12 < ema50 AND ema50 < ema144
|
||||||
|
AND close < ema144
|
||||||
|
AND ABS(close - ema50) / NULLIF(ema50, 0) < 0.03
|
||||||
|
AND prev_close >= ema50
|
||||||
|
THEN 'vegas_short' ELSE NULL END AS signal_type_down
|
||||||
|
FROM with_ema
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
COALESCE(signal_type_up, signal_type_down) AS signal_type,
|
||||||
|
ts_code, trade_date, close AS signal_price,
|
||||||
|
AVG(CASE WHEN close > open THEN 1.0 ELSE 0.0 END) OVER (PARTITION BY trade_date) AS daily_advance_ratio,
|
||||||
|
AVG((high - low) / NULLIF(close, 0)) OVER (PARTITION BY trade_date) AS daily_range,
|
||||||
|
rn
|
||||||
|
FROM with_signal
|
||||||
|
WHERE signal_type_up IS NOT NULL OR signal_type_down IS NOT NULL
|
||||||
|
"""
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start", f"'{start_date.strftime('%Y-%m-%d')}'")
|
||||||
|
.replace("$end", f"'{end_date.strftime('%Y-%m-%d')}'")
|
||||||
|
).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return result
|
||||||
|
|
||||||
|
result = result.drop_duplicates(subset=["signal_type", "ts_code", "trade_date"])
|
||||||
|
_compute_forward_returns(result, parquet_glob)
|
||||||
|
logger.info(f"Detected {len(result)} Vegas signals")
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════
|
||||||
|
# Chan Theory Detection (MACD divergence)
|
||||||
|
# ═══════════════════════════════════════════════════
|
||||||
|
|
||||||
|
def detect_chan_signals(end_date=None, lookback_days=500) -> "pd.DataFrame":
|
||||||
|
"""Detect Chan Theory 1buy/1sell signals via MACD divergence.
|
||||||
|
|
||||||
|
1buy: price makes lower low but MACD makes higher low (bottom divergence).
|
||||||
|
1sell: price makes higher high but MACD makes lower high (top divergence).
|
||||||
|
"""
|
||||||
|
from datetime import date as _date, timedelta as _td
|
||||||
|
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
if end_date is None:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT MAX(trade_date) FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)"
|
||||||
|
).fetchone()
|
||||||
|
if not row or not row[0]:
|
||||||
|
conn.close()
|
||||||
|
return pd.DataFrame()
|
||||||
|
end_date = _date.fromisoformat(str(row[0])[:10])
|
||||||
|
|
||||||
|
start_date = end_date - _td(days=lookback_days)
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT ts_code, trade_date, trade_time, close, volume, high, low, open
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $start AND trade_date <= $end
|
||||||
|
),
|
||||||
|
with_macd AS (
|
||||||
|
SELECT *,
|
||||||
|
AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 11 PRECEDING AND CURRENT ROW)
|
||||||
|
- AVG(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 25 PRECEDING AND CURRENT ROW) AS macd,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time) as rn
|
||||||
|
FROM normalized
|
||||||
|
),
|
||||||
|
with_signal AS (
|
||||||
|
SELECT *,
|
||||||
|
AVG(macd) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 8 PRECEDING AND CURRENT ROW) AS macd_signal
|
||||||
|
FROM with_macd
|
||||||
|
),
|
||||||
|
with_divergence AS (
|
||||||
|
SELECT *,
|
||||||
|
MIN(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 59 PRECEDING AND CURRENT ROW) AS price_60low,
|
||||||
|
MAX(close) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 59 PRECEDING AND CURRENT ROW) AS price_60high,
|
||||||
|
MIN(macd) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 59 PRECEDING AND CURRENT ROW) AS macd_60low,
|
||||||
|
MAX(macd) OVER (PARTITION BY ts_code ORDER BY trade_time ROWS BETWEEN 59 PRECEDING AND CURRENT ROW) AS macd_60high,
|
||||||
|
LAG(macd, 20) OVER (PARTITION BY ts_code ORDER BY trade_time) AS macd_20ago
|
||||||
|
FROM with_signal
|
||||||
|
WHERE rn > 60
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
CASE
|
||||||
|
WHEN close <= price_60low * 1.02 AND macd > macd_60low * 1.1 AND macd > macd_20ago
|
||||||
|
THEN 'chan_1buy'
|
||||||
|
WHEN close >= price_60high * 0.98 AND macd < macd_60high * 0.9 AND macd < macd_20ago
|
||||||
|
THEN 'chan_1sell'
|
||||||
|
END AS signal_type,
|
||||||
|
ts_code, trade_date, close AS signal_price,
|
||||||
|
AVG(CASE WHEN close > open THEN 1.0 ELSE 0.0 END) OVER (PARTITION BY trade_date) AS daily_advance_ratio,
|
||||||
|
AVG((high - low) / NULLIF(close, 0)) OVER (PARTITION BY trade_date) AS daily_range
|
||||||
|
FROM with_divergence
|
||||||
|
WHERE (close <= price_60low * 1.02 AND macd > macd_60low * 1.1 AND macd > macd_20ago)
|
||||||
|
OR (close >= price_60high * 0.98 AND macd < macd_60high * 0.9 AND macd < macd_20ago)
|
||||||
|
"""
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start", f"'{start_date.strftime('%Y-%m-%d')}'")
|
||||||
|
.replace("$end", f"'{end_date.strftime('%Y-%m-%d')}'")
|
||||||
|
).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return result
|
||||||
|
|
||||||
|
result = result.drop_duplicates(subset=["signal_type", "ts_code", "trade_date"])
|
||||||
|
_compute_forward_returns(result, parquet_glob)
|
||||||
|
logger.info(f"Detected {len(result)} Chan signals")
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ═══════════════════════════════════════════════════
|
||||||
|
# ORB Detection (Opening Range Breakout — daily)
|
||||||
|
# ═══════════════════════════════════════════════════
|
||||||
|
|
||||||
|
def detect_orb_signals(end_date=None, lookback_days=500) -> "pd.DataFrame":
|
||||||
|
"""Detect daily ORB signals: close breaks above yesterday's high or below low.
|
||||||
|
|
||||||
|
Simple but effective — breakouts often lead to trend days.
|
||||||
|
"""
|
||||||
|
from datetime import date as _date, timedelta as _td
|
||||||
|
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
|
||||||
|
if end_date is None:
|
||||||
|
row = conn.execute(
|
||||||
|
f"SELECT MAX(trade_date) FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)"
|
||||||
|
).fetchone()
|
||||||
|
if not row or not row[0]:
|
||||||
|
conn.close()
|
||||||
|
return pd.DataFrame()
|
||||||
|
end_date = _date.fromisoformat(str(row[0])[:10])
|
||||||
|
|
||||||
|
start_date = end_date - _td(days=lookback_days)
|
||||||
|
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT ts_code, trade_date, trade_time, close, volume, high, low, open
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $start AND trade_date <= $end
|
||||||
|
),
|
||||||
|
with_prev AS (
|
||||||
|
SELECT *,
|
||||||
|
LAG(high) OVER (PARTITION BY ts_code ORDER BY trade_time) AS prev_high,
|
||||||
|
LAG(low) OVER (PARTITION BY ts_code ORDER BY trade_time) AS prev_low,
|
||||||
|
LAG(close) OVER (PARTITION BY ts_code ORDER BY trade_time) AS prev_close
|
||||||
|
FROM normalized
|
||||||
|
)
|
||||||
|
SELECT
|
||||||
|
CASE
|
||||||
|
WHEN close > prev_high AND prev_close <= prev_high THEN 'orb_up'
|
||||||
|
WHEN close < prev_low AND prev_close >= prev_low THEN 'orb_down'
|
||||||
|
END AS signal_type,
|
||||||
|
ts_code, trade_date, close AS signal_price,
|
||||||
|
AVG(CASE WHEN close > open THEN 1.0 ELSE 0.0 END) OVER (PARTITION BY trade_date) AS daily_advance_ratio,
|
||||||
|
AVG((high - low) / NULLIF(close, 0)) OVER (PARTITION BY trade_date) AS daily_range
|
||||||
|
FROM with_prev
|
||||||
|
WHERE (close > prev_high AND prev_close <= prev_high)
|
||||||
|
OR (close < prev_low AND prev_close >= prev_low)
|
||||||
|
"""
|
||||||
|
result = conn.execute(
|
||||||
|
sql.replace("$start", f"'{start_date.strftime('%Y-%m-%d')}'")
|
||||||
|
.replace("$end", f"'{end_date.strftime('%Y-%m-%d')}'")
|
||||||
|
).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
if result.empty:
|
||||||
|
return result
|
||||||
|
|
||||||
|
result = result.drop_duplicates(subset=["signal_type", "ts_code", "trade_date"])
|
||||||
|
_compute_forward_returns(result, parquet_glob)
|
||||||
|
logger.info(f"Detected {len(result)} ORB signals")
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ══════════════════════════════════════════
|
||||||
|
# Shared Forward Returns Computation
|
||||||
|
# ══════════════════════════════════════════
|
||||||
|
|
||||||
|
def _compute_forward_returns(df: "pd.DataFrame", parquet_glob: str):
|
||||||
|
"""Join forward returns (5d, 10d, 20d) onto a signal DataFrame. Mutates in place."""
|
||||||
|
conn = analytics_conn()
|
||||||
|
try:
|
||||||
|
for days, col_name in [(5, "ret_5d"), (10, "ret_10d"), (20, "ret_20d")]:
|
||||||
|
sql = f"""
|
||||||
|
WITH future AS (
|
||||||
|
SELECT ts_code, trade_date, close,
|
||||||
|
LEAD(close, {days}) OVER (PARTITION BY ts_code ORDER BY trade_date) AS future_close,
|
||||||
|
MIN(close) OVER (PARTITION BY ts_code ORDER BY trade_date ROWS BETWEEN CURRENT ROW AND {days} FOLLOWING) AS min_close
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
)
|
||||||
|
SELECT ts_code, trade_date,
|
||||||
|
(future_close - close) / NULLIF(close, 0) AS ret,
|
||||||
|
(min_close - close) / NULLIF(close, 0) AS mdd
|
||||||
|
FROM future
|
||||||
|
"""
|
||||||
|
fwd = conn.execute(sql).fetchdf()
|
||||||
|
fwd["trade_date"] = pd.to_datetime(fwd["trade_date"])
|
||||||
|
df["trade_date"] = pd.to_datetime(df["trade_date"])
|
||||||
|
# Merge forward returns
|
||||||
|
df_temp = df.merge(fwd, on=["ts_code", "trade_date"], how="left")
|
||||||
|
if col_name not in df.columns:
|
||||||
|
df[col_name] = df_temp["ret"] if "ret" in df_temp.columns else None
|
||||||
|
if col_name == "ret_5d":
|
||||||
|
if "max_dd_5d" not in df.columns:
|
||||||
|
df["max_dd_5d"] = df_temp["mdd"] if "mdd" in df_temp.columns else None
|
||||||
|
if "max_return_5d" not in df.columns:
|
||||||
|
df["max_return_5d"] = df_temp["ret"] if "ret" in df_temp.columns else None
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
# ══════════════════════════════════════════
|
||||||
|
# EMA52 Screening — CLI + scheduler shared
|
||||||
|
# ══════════════════════════════════════════
|
||||||
|
|
||||||
|
def run_ema52_screening(
|
||||||
|
threshold: float | None = None,
|
||||||
|
freqs: list | None = None,
|
||||||
|
) -> dict:
|
||||||
|
"""Run EMA52 screening. Shared by CLI and scheduler — no duplicated logic."""
|
||||||
|
import numpy as np
|
||||||
|
import pandas as pd
|
||||||
|
from ashare_dp.config import Settings
|
||||||
|
from ashare_dp.core.models import Freq
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
from ashare_dp.data.store.partitioning import partition_glob
|
||||||
|
|
||||||
|
settings = Settings()
|
||||||
|
if threshold is None:
|
||||||
|
threshold = settings.ema52_threshold
|
||||||
|
if freqs is None:
|
||||||
|
freqs = [(Freq.d1, "1d", 100), (Freq.w1, "1w", 60)]
|
||||||
|
|
||||||
|
with get_db(read_only=True) as db:
|
||||||
|
stocks = db.query("SELECT ts_code, name FROM stock_info ORDER BY ts_code")
|
||||||
|
symbol_names = {row[0]: row[1] for row in stocks}
|
||||||
|
|
||||||
|
logger.info(f"EMA52 screening: {len(symbol_names)} stocks, threshold=±{threshold*100:.1f}%")
|
||||||
|
results_by_freq = {}
|
||||||
|
|
||||||
|
for freq, freq_label, bars_needed in freqs:
|
||||||
|
try:
|
||||||
|
glob = partition_glob(freq)
|
||||||
|
sql = f"""
|
||||||
|
WITH ranked AS (
|
||||||
|
SELECT *, ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_time DESC) as rn
|
||||||
|
FROM read_parquet('{glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
)
|
||||||
|
SELECT * FROM ranked WHERE rn <= {bars_needed} ORDER BY ts_code, trade_time ASC
|
||||||
|
"""
|
||||||
|
import duckdb
|
||||||
|
c = duckdb.connect()
|
||||||
|
df = c.execute(sql).fetchdf() if True else None
|
||||||
|
c.close()
|
||||||
|
if df.empty:
|
||||||
|
logger.warning(f"EMA52: no data for {freq_label}")
|
||||||
|
continue
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"EMA52: read failed {freq_label}: {e}")
|
||||||
|
continue
|
||||||
|
|
||||||
|
df["trade_time"] = pd.to_datetime(df["trade_time"])
|
||||||
|
df = df.sort_values(["ts_code", "trade_time"])
|
||||||
|
found = []
|
||||||
|
|
||||||
|
for ts_code, group in df.groupby("ts_code"):
|
||||||
|
if ts_code not in symbol_names:
|
||||||
|
continue
|
||||||
|
group = group.tail(bars_needed)
|
||||||
|
if len(group) < 26:
|
||||||
|
continue
|
||||||
|
closes = group["close"].astype(float).values
|
||||||
|
ema = _ema(closes, 52)
|
||||||
|
if len(ema) == 0:
|
||||||
|
continue
|
||||||
|
latest_close, latest_ema = closes[-1], ema[-1]
|
||||||
|
if latest_ema <= 0:
|
||||||
|
continue
|
||||||
|
dist = (latest_close - latest_ema) / latest_ema
|
||||||
|
if abs(dist) <= threshold:
|
||||||
|
found.append({
|
||||||
|
"trade_date": group["trade_date"].iloc[-1],
|
||||||
|
"ts_code": ts_code, "name": symbol_names[ts_code],
|
||||||
|
"freq": freq_label,
|
||||||
|
"close_price": round(latest_close, 2),
|
||||||
|
"ema52": round(latest_ema, 2),
|
||||||
|
"distance_pct": round(dist * 100, 2),
|
||||||
|
"amount": round(float(group["amount"].iloc[-1]) / 1e8, 2),
|
||||||
|
})
|
||||||
|
|
||||||
|
if found:
|
||||||
|
found.sort(key=lambda r: r.get("amount", 0) or 0, reverse=True)
|
||||||
|
with get_db(read_only=False) as wdb:
|
||||||
|
wdb.conn.execute("DELETE FROM ema52_screening WHERE freq = ?", [freq_label])
|
||||||
|
wdb.conn.executemany(
|
||||||
|
"""INSERT INTO ema52_screening
|
||||||
|
(trade_date, ts_code, name, freq, close_price, ema52, distance_pct, amount)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
ON CONFLICT (trade_date, ts_code, freq) DO UPDATE SET
|
||||||
|
close_price=excluded.close_price, ema52=excluded.ema52,
|
||||||
|
distance_pct=excluded.distance_pct, amount=excluded.amount,
|
||||||
|
name=excluded.name, updated_at=now()""",
|
||||||
|
[(r["trade_date"], r["ts_code"], r["name"], r["freq"], r["close_price"],
|
||||||
|
r["ema52"], r["distance_pct"], r["amount"]) for r in found],
|
||||||
|
)
|
||||||
|
logger.info(f"EMA52 {freq_label}: {len(found)} near (±{threshold*100:.1f}%)")
|
||||||
|
results_by_freq[freq_label] = len(found)
|
||||||
|
|
||||||
|
logger.info("EMA52 screening complete")
|
||||||
|
return results_by_freq
|
||||||
|
|
||||||
|
|
||||||
|
# ══════════════════════════════════════════
|
||||||
|
# Gap / NR7 / Inside Bar Detection
|
||||||
|
# ══════════════════════════════════════════
|
||||||
|
|
||||||
|
def detect_gap_signals(end_date=None, lookback_days=500) -> "pd.DataFrame":
|
||||||
|
"""Detect gap-up and gap-down signals."""
|
||||||
|
from datetime import date as _date, timedelta as _td
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
if end_date is None:
|
||||||
|
r = conn.execute(f"SELECT MAX(trade_date) FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)").fetchone()
|
||||||
|
end_date = _date.fromisoformat(str(r[0])[:10]) if r and r[0] else _date.today()
|
||||||
|
start_date = end_date - _td(days=lookback_days)
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT ts_code, trade_date, close, open, high, low
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $s AND trade_date <= $e
|
||||||
|
),
|
||||||
|
prev AS (
|
||||||
|
SELECT *, LAG(high) OVER (PARTITION BY ts_code ORDER BY trade_date) AS prev_high,
|
||||||
|
LAG(low) OVER (PARTITION BY ts_code ORDER BY trade_date) AS prev_low
|
||||||
|
FROM normalized
|
||||||
|
)
|
||||||
|
SELECT CASE WHEN open > prev_high THEN 'gap_up' WHEN open < prev_low THEN 'gap_down' END AS signal_type,
|
||||||
|
ts_code, trade_date, close AS signal_price
|
||||||
|
FROM prev WHERE (open > prev_high) OR (open < prev_low)
|
||||||
|
"""
|
||||||
|
df = conn.execute(sql.replace("$s", f"'{start_date}'").replace("$e", f"'{end_date}'")).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
if not df.empty:
|
||||||
|
df = df.drop_duplicates(subset=["signal_type", "ts_code", "trade_date"])
|
||||||
|
logger.info(f"Detected {len(df)} Gap signals")
|
||||||
|
return df
|
||||||
|
|
||||||
|
|
||||||
|
def detect_nr7_signals(end_date=None, lookback_days=500) -> "pd.DataFrame":
|
||||||
|
"""Detect NR7: narrowest range in 7 days."""
|
||||||
|
from datetime import date as _date, timedelta as _td
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
if end_date is None:
|
||||||
|
r = conn.execute(f"SELECT MAX(trade_date) FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)").fetchone()
|
||||||
|
end_date = _date.fromisoformat(str(r[0])[:10]) if r and r[0] else _date.today()
|
||||||
|
start_date = end_date - _td(days=lookback_days)
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH ranges AS (
|
||||||
|
SELECT ts_code, trade_date, close, (high - low) AS day_range,
|
||||||
|
ROW_NUMBER() OVER (PARTITION BY ts_code ORDER BY trade_date) AS rn
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $s AND trade_date <= $e
|
||||||
|
),
|
||||||
|
window_min AS (
|
||||||
|
SELECT *,
|
||||||
|
MIN(day_range) OVER (PARTITION BY ts_code ORDER BY trade_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS min_range_7,
|
||||||
|
COUNT(*) OVER (PARTITION BY ts_code ORDER BY trade_date ROWS BETWEEN 6 PRECEDING AND CURRENT ROW) AS window_count
|
||||||
|
FROM ranges
|
||||||
|
)
|
||||||
|
SELECT 'nr7' AS signal_type, ts_code, trade_date, close AS signal_price
|
||||||
|
FROM window_min WHERE day_range = min_range_7 AND window_count = 7 AND rn >= 7
|
||||||
|
"""
|
||||||
|
df = conn.execute(sql.replace("$s", f"'{start_date}'").replace("$e", f"'{end_date}'")).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
if not df.empty:
|
||||||
|
df = df.drop_duplicates(subset=["ts_code", "trade_date"])
|
||||||
|
logger.info(f"Detected {len(df)} NR7 signals")
|
||||||
|
return df
|
||||||
|
|
||||||
|
|
||||||
|
def detect_ib_signals(end_date=None, lookback_days=500) -> "pd.DataFrame":
|
||||||
|
"""Detect Inside Bar: today's range inside yesterday's."""
|
||||||
|
from datetime import date as _date, timedelta as _td
|
||||||
|
parquet_glob = kline_glob()
|
||||||
|
conn = analytics_conn()
|
||||||
|
if end_date is None:
|
||||||
|
r = conn.execute(f"SELECT MAX(trade_date) FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)").fetchone()
|
||||||
|
end_date = _date.fromisoformat(str(r[0])[:10]) if r and r[0] else _date.today()
|
||||||
|
start_date = end_date - _td(days=lookback_days)
|
||||||
|
try:
|
||||||
|
sql = f"""
|
||||||
|
WITH normalized AS (
|
||||||
|
SELECT ts_code, trade_date, close, open, high, low
|
||||||
|
FROM read_parquet('{parquet_glob}', hive_partitioning=true, union_by_name=true)
|
||||||
|
WHERE trade_date >= $s AND trade_date <= $e
|
||||||
|
),
|
||||||
|
prev AS (
|
||||||
|
SELECT *, LAG(high) OVER (PARTITION BY ts_code ORDER BY trade_date) AS prev_high,
|
||||||
|
LAG(low) OVER (PARTITION BY ts_code ORDER BY trade_date) AS prev_low,
|
||||||
|
LAG(close) OVER (PARTITION BY ts_code ORDER BY trade_date) AS prev_close
|
||||||
|
FROM normalized
|
||||||
|
)
|
||||||
|
SELECT CASE WHEN close > (high+low)/2.0 THEN 'ib_long' ELSE 'ib_short' END AS signal_type,
|
||||||
|
ts_code, trade_date, close AS signal_price
|
||||||
|
FROM prev WHERE high < prev_high AND low > prev_low AND prev_high IS NOT NULL
|
||||||
|
"""
|
||||||
|
df = conn.execute(sql.replace("$s", f"'{start_date}'").replace("$e", f"'{end_date}'")).fetchdf()
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
if not df.empty:
|
||||||
|
df = df.drop_duplicates(subset=["ts_code", "trade_date"])
|
||||||
|
logger.info(f"Detected {len(df)} Inside Bar signals")
|
||||||
|
return df
|
||||||
|
|
||||||
|
|
||||||
|
def _ema(values, period: int):
|
||||||
|
"""EMA for 1D array."""
|
||||||
|
import numpy as np
|
||||||
|
if len(values) < period:
|
||||||
|
return np.array([])
|
||||||
|
a = 2.0 / (period + 1)
|
||||||
|
ema = np.zeros(len(values))
|
||||||
|
ema[period - 1] = np.mean(values[:period])
|
||||||
|
for i in range(period, len(values)):
|
||||||
|
ema[i] = a * values[i] + (1 - a) * ema[i - 1]
|
||||||
|
return ema
|
||||||
@@ -0,0 +1,176 @@
|
|||||||
|
"""Expectancy Engine — Signal × MarketState → historical expectancy.
|
||||||
|
|
||||||
|
Queries signal_instance table for signals in similar market contexts.
|
||||||
|
This is the moat. Phase 1: working with EMA52 signals.
|
||||||
|
Phase 2: all signal types.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import math
|
||||||
|
from datetime import date
|
||||||
|
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.domain.context import Expectancy
|
||||||
|
from ashare_dp.domain.state import MarketState
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
|
||||||
|
# Signal types to try in priority order when querying general expectancy
|
||||||
|
_SIGNAL_PRIORITY = [
|
||||||
|
"ema52_cross_up", "ema52_cross_down",
|
||||||
|
"vegas_long", "chan_1buy", "orb_up",
|
||||||
|
"vegas_short", "chan_1sell", "orb_down",
|
||||||
|
"gap_up", "nr7", "ib_long",
|
||||||
|
"gap_down", "ib_short",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def get_expectancy(state: MarketState, signal_type: str | None = None) -> Expectancy:
|
||||||
|
"""The single entry point for expectancy queries.
|
||||||
|
|
||||||
|
If signal_type given: query that signal's historical stats.
|
||||||
|
If None: try signals in priority order, return first with real samples.
|
||||||
|
Falls back to heuristic estimate when no historical data matches.
|
||||||
|
All fallback logic lives HERE — callers never chain fallbacks themselves.
|
||||||
|
"""
|
||||||
|
if signal_type:
|
||||||
|
return query_expectancy(state, signal_type=signal_type)
|
||||||
|
|
||||||
|
for st in _SIGNAL_PRIORITY:
|
||||||
|
exp = query_expectancy(state, signal_type=st)
|
||||||
|
if exp.sample_count > 0:
|
||||||
|
return exp
|
||||||
|
|
||||||
|
return estimate_expectancy(state, signal_type="generic")
|
||||||
|
|
||||||
|
|
||||||
|
def get_all_expectancies(state: MarketState) -> list[Expectancy]:
|
||||||
|
"""Query ALL signal types and return ranked by win_rate desc.
|
||||||
|
|
||||||
|
Used by the dashboard Expectancy comparison table.
|
||||||
|
Signal types with 0 samples get heuristic estimates (marked by sample_count=0).
|
||||||
|
"""
|
||||||
|
results = []
|
||||||
|
for st in _SIGNAL_PRIORITY:
|
||||||
|
exp = query_expectancy(state, signal_type=st, min_samples=1)
|
||||||
|
results.append(exp)
|
||||||
|
|
||||||
|
# Sort: real samples first (by win_rate desc), then heuristics last
|
||||||
|
results.sort(key=lambda e: (e.sample_count > 0, e.win_rate), reverse=True)
|
||||||
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def query_expectancy(
|
||||||
|
state: MarketState,
|
||||||
|
signal_type: str = "ema52_cross_up",
|
||||||
|
min_samples: int = 5,
|
||||||
|
) -> Expectancy:
|
||||||
|
"""Query historical expectancy for a signal type in current market state.
|
||||||
|
|
||||||
|
Matches signals with similar market context (state vector cosine similarity
|
||||||
|
or simple threshold matching in Phase 1).
|
||||||
|
|
||||||
|
Args:
|
||||||
|
state: Current MarketState vector.
|
||||||
|
signal_type: Signal type to query (e.g., "ema52_cross_up").
|
||||||
|
min_samples: Minimum historical samples required for valid result.
|
||||||
|
|
||||||
|
Returns:
|
||||||
|
Expectancy with real historical stats, or heuristic fallback if
|
||||||
|
insufficient samples.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
with get_db(read_only=True) as db:
|
||||||
|
# Query all signals of this type with outcomes
|
||||||
|
rows = db.query(
|
||||||
|
"""
|
||||||
|
SELECT
|
||||||
|
signal_type, ts_code, trade_date, signal_price,
|
||||||
|
state_breadth, state_volatility,
|
||||||
|
return_5d, return_10d, return_20d,
|
||||||
|
max_return_5d, max_drawdown_5d
|
||||||
|
FROM signal_instance
|
||||||
|
WHERE signal_type = ?
|
||||||
|
AND outcome_known = TRUE
|
||||||
|
AND return_5d IS NOT NULL
|
||||||
|
ORDER BY trade_date DESC
|
||||||
|
LIMIT 5000
|
||||||
|
""",
|
||||||
|
(signal_type,),
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning(f"Expectancy query failed: {e}")
|
||||||
|
return estimate_expectancy(state, signal_type)
|
||||||
|
|
||||||
|
if len(rows) < min_samples:
|
||||||
|
logger.info(f"Only {len(rows)} samples for {signal_type}, using heuristic")
|
||||||
|
return estimate_expectancy(state, signal_type)
|
||||||
|
|
||||||
|
# Filter by similar market state (simple threshold match in Phase 1)
|
||||||
|
target_breadth = state.breadth
|
||||||
|
similar = []
|
||||||
|
for row in rows:
|
||||||
|
hist_breadth = float(row[4] or 0.5)
|
||||||
|
# Simple similarity: breadth within ±0.15
|
||||||
|
if abs(hist_breadth - target_breadth) < 0.15:
|
||||||
|
similar.append(row)
|
||||||
|
|
||||||
|
if len(similar) < max(min_samples, 1):
|
||||||
|
# Not enough similar contexts — use all available samples
|
||||||
|
similar = rows
|
||||||
|
|
||||||
|
# Compute expectancy stats
|
||||||
|
# Columns: 0=signal_type, 1=ts_code, 2=trade_date, 3=signal_price,
|
||||||
|
# 4=state_breadth, 5=state_volatility, 6=return_5d, 7=return_10d,
|
||||||
|
# 8=return_20d, 9=max_return_5d, 10=max_drawdown_5d
|
||||||
|
returns_5d = [float(r[6]) for r in similar if r[6] is not None]
|
||||||
|
returns_10d = [float(r[7]) for r in similar if r[7] is not None]
|
||||||
|
returns_20d = [float(r[8]) for r in similar if r[8] is not None]
|
||||||
|
drawdowns = [float(r[10]) for r in similar if r[10] is not None]
|
||||||
|
|
||||||
|
if not returns_5d:
|
||||||
|
return estimate_expectancy(state, signal_type)
|
||||||
|
|
||||||
|
wins = sum(1 for r in returns_5d if r > 0)
|
||||||
|
win_rate = wins / len(returns_5d)
|
||||||
|
avg_return = sum(returns_5d) / len(returns_5d)
|
||||||
|
max_dd = min(drawdowns) if drawdowns else avg_return * -0.5
|
||||||
|
avg_hold = 5.0 # Phase 1: fixed 5d horizon
|
||||||
|
|
||||||
|
confidence = min(len(similar) / 200, 1.0) # scale with sample size
|
||||||
|
|
||||||
|
return Expectancy(
|
||||||
|
signal_type=signal_type,
|
||||||
|
win_rate=round(win_rate, 4),
|
||||||
|
avg_return=round(avg_return, 4),
|
||||||
|
max_drawdown=round(max_dd, 4),
|
||||||
|
avg_holding_days=avg_hold,
|
||||||
|
sample_count=len(similar),
|
||||||
|
confidence=round(confidence, 4),
|
||||||
|
similar_states=len(similar),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def estimate_expectancy(state: MarketState, signal_type: str = "generic") -> Expectancy:
|
||||||
|
"""Heuristic fallback when insufficient historical data."""
|
||||||
|
t = state.trend
|
||||||
|
f = state.fear
|
||||||
|
l = state.liquidity
|
||||||
|
|
||||||
|
base_win = 0.50 + max(t - 0.5, 0) * 0.20 - f * 0.15 + max(l - 0.5, 0) * 0.10
|
||||||
|
win_rate = min(max(base_win, 0.25), 0.80)
|
||||||
|
avg_return = t * 0.08 - f * 0.04
|
||||||
|
max_dd = -(f * 0.04 + (1 - l) * 0.02)
|
||||||
|
avg_hold = 2.0 + t * 4.0 - f * 2.0
|
||||||
|
|
||||||
|
return Expectancy(
|
||||||
|
signal_type=signal_type,
|
||||||
|
win_rate=round(win_rate, 4),
|
||||||
|
avg_return=round(avg_return, 4),
|
||||||
|
max_drawdown=round(max_dd, 4),
|
||||||
|
avg_holding_days=round(avg_hold, 1),
|
||||||
|
sample_count=0,
|
||||||
|
confidence=0.0,
|
||||||
|
similar_states=0,
|
||||||
|
)
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
"""Signal Instance Store — CRUD for signal_instance table.
|
||||||
|
|
||||||
|
Separate from detection logic. Detection finds signals,
|
||||||
|
store persists them. Both used by backfill pipeline.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pandas as pd
|
||||||
|
from loguru import logger
|
||||||
|
|
||||||
|
from ashare_dp.data.store.database import get_db
|
||||||
|
|
||||||
|
|
||||||
|
def store_signals(df: pd.DataFrame) -> int:
|
||||||
|
"""Store detected signals in signal_instance table.
|
||||||
|
|
||||||
|
Uses ON CONFLICT DO NOTHING to safely handle re-runs.
|
||||||
|
|
||||||
|
Returns number of rows stored.
|
||||||
|
"""
|
||||||
|
if df.empty:
|
||||||
|
return 0
|
||||||
|
|
||||||
|
with get_db(read_only=False) as db:
|
||||||
|
stored = 0
|
||||||
|
for _, row in df.iterrows():
|
||||||
|
try:
|
||||||
|
db.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO signal_instance
|
||||||
|
(signal_type, ts_code, trade_date, signal_price,
|
||||||
|
state_breadth, state_volatility,
|
||||||
|
return_5d, return_10d, return_20d,
|
||||||
|
max_return_5d, max_drawdown_5d,
|
||||||
|
outcome_known)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, TRUE)
|
||||||
|
ON CONFLICT (signal_type, ts_code, trade_date) DO NOTHING
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
row["signal_type"],
|
||||||
|
row["ts_code"],
|
||||||
|
row["trade_date"],
|
||||||
|
float(row["signal_price"]),
|
||||||
|
float(row["daily_advance_ratio"]) if pd.notna(row.get("daily_advance_ratio")) else 0.5,
|
||||||
|
float(row["daily_range"]) if pd.notna(row.get("daily_range")) else 0.02,
|
||||||
|
float(row["ret_5d"]) if pd.notna(row.get("ret_5d")) else None,
|
||||||
|
float(row["ret_10d"]) if pd.notna(row.get("ret_10d")) else None,
|
||||||
|
float(row["ret_20d"]) if pd.notna(row.get("ret_20d")) else None,
|
||||||
|
float(row["max_return_5d"]) if pd.notna(row.get("max_return_5d")) else None,
|
||||||
|
float(row["max_dd_5d"]) if pd.notna(row.get("max_dd_5d")) else None,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
stored += 1
|
||||||
|
except Exception as e:
|
||||||
|
logger.debug(f"Signal store failed for {row.get('ts_code')}: {e}")
|
||||||
|
|
||||||
|
logger.info(f"Stored {stored} signal instances")
|
||||||
|
return stored
|
||||||
|
|
||||||
|
|
||||||
|
def count_signals(signal_type: str | None = None) -> int:
|
||||||
|
"""Count stored signal instances, optionally filtered by type."""
|
||||||
|
with get_db(read_only=True) as db:
|
||||||
|
if signal_type:
|
||||||
|
rows = db.query(
|
||||||
|
"SELECT COUNT(*) FROM signal_instance WHERE signal_type = ?",
|
||||||
|
(signal_type,),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
rows = db.query("SELECT COUNT(*) FROM signal_instance")
|
||||||
|
return int(rows[0][0]) if rows else 0
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
"""Database schema: DDL statements for DuckDB tables and views."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
# DDL for persistent tables
|
|
||||||
DDL_STATEMENTS = [
|
|
||||||
"""
|
|
||||||
CREATE TABLE IF NOT EXISTS stock_info (
|
|
||||||
ts_code VARCHAR(9) PRIMARY KEY,
|
|
||||||
symbol VARCHAR(6) NOT NULL,
|
|
||||||
name VARCHAR(40) NOT NULL,
|
|
||||||
exchange VARCHAR(2) NOT NULL,
|
|
||||||
area VARCHAR(20),
|
|
||||||
industry VARCHAR(40),
|
|
||||||
list_date DATE,
|
|
||||||
delist_date DATE,
|
|
||||||
market VARCHAR(10),
|
|
||||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
||||||
)
|
|
||||||
""",
|
|
||||||
"""
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_stock_symbol ON stock_info(symbol)
|
|
||||||
""",
|
|
||||||
"""
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_stock_exchange ON stock_info(exchange)
|
|
||||||
""",
|
|
||||||
"""
|
|
||||||
CREATE TABLE IF NOT EXISTS trading_calendar (
|
|
||||||
trade_date DATE PRIMARY KEY,
|
|
||||||
is_trading_day BOOLEAN NOT NULL DEFAULT TRUE,
|
|
||||||
week_day TINYINT NOT NULL,
|
|
||||||
year SMALLINT NOT NULL,
|
|
||||||
month TINYINT NOT NULL
|
|
||||||
)
|
|
||||||
""",
|
|
||||||
"""
|
|
||||||
CREATE SEQUENCE IF NOT EXISTS seq_ema52_id
|
|
||||||
""",
|
|
||||||
"""
|
|
||||||
CREATE TABLE IF NOT EXISTS ema52_screening (
|
|
||||||
id BIGINT PRIMARY KEY DEFAULT nextval('seq_ema52_id'),
|
|
||||||
trade_date DATE NOT NULL,
|
|
||||||
ts_code VARCHAR(9) NOT NULL,
|
|
||||||
name VARCHAR(40),
|
|
||||||
freq VARCHAR(3) NOT NULL,
|
|
||||||
close_price DOUBLE NOT NULL,
|
|
||||||
ema52 DOUBLE NOT NULL,
|
|
||||||
distance_pct DOUBLE NOT NULL,
|
|
||||||
amount DOUBLE,
|
|
||||||
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
UNIQUE(trade_date, ts_code, freq)
|
|
||||||
)
|
|
||||||
""",
|
|
||||||
"""
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_ema52_date ON ema52_screening(trade_date)
|
|
||||||
""",
|
|
||||||
# Migration: add amount column if upgrading from older schema
|
|
||||||
"""
|
|
||||||
ALTER TABLE ema52_screening ADD COLUMN IF NOT EXISTS amount DOUBLE
|
|
||||||
""",
|
|
||||||
]
|
|
||||||
Reference in New Issue
Block a user