chore: 将 data_provider 拆出为独立仓库。

数据服务已迁移至 jack/data_provider,不再随 chan 维护。

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
jackyu66git
2026-08-05 18:10:29 +08:00
co-authored by Cursor
parent b31215057e
commit f2e77e1bdb
9 changed files with 0 additions and 3766 deletions
-20
View File
@@ -1,20 +0,0 @@
FROM python:3.11-slim
ENV PYTHONUNBUFFERED=1 \
PIP_NO_CACHE_DIR=1
WORKDIR /app
COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
COPY . /app
ENV CONFIG_PATH=/app/config.json \
UVICORN_HOST=0.0.0.0 \
UVICORN_PORT=80
EXPOSE 80
CMD ["python", "-m", "main"]
-82
View File
@@ -1,82 +0,0 @@
# Chan 数据提供商 (Chan Data Provider)
**Binance 期货** 交易所拉取加密货币 K 线数据,提供 HTTP + WebSocket 数据服务。
## 功能
- **多交易对**:支持 BTC, ETH, SOL, DOGE 等 9 个交易对
- **多时间周期**:基础周期 1m/1h/1d/1w,可合成 30+ 种衍生周期(如 5m, 15m, 4h 等)
- **本地缓存**:CSV 持久化到磁盘,重启快速加载
- **断线恢复**:交易所连接中断时记录断点,自动补拉缺失数据
- **实时推送**WebSocket 订阅最新 K 线更新
- **内存服务**:启动即加载本地数据,不阻塞服务
## 启动
```bash
# Docker
docker compose up -d
# 直接运行
python main.py
# 或指定配置
CONFIG_PATH=./config.json python main.py
```
服务默认监听 `http://0.0.0.0:9009`
## 配置
编辑 `config.json`
```json
{
"exchange": "binance",
"symbols": ["BTC/USDT:USDT", "ETH/USDT:USDT"],
"start_time": "2024-01-01T00:00:00Z",
"timeframes": ["1m", "1h", "1d", "1w"],
"data_dir": "./data"
}
```
| 字段 | 说明 |
|------|------|
| `exchange` | 交易所名称(ccxt 支持即可) |
| `symbols` | 交易对列表 |
| `start_time` | 历史数据起始时间 |
| `timeframes` | 基础周期(从交易所直接拉取) |
| `data_dir` | CSV 数据存储目录 |
## 可用周期
### 基础周期(交易所直接拉取)
`1m`, `1h`, `1d`, `1w`
### 衍生周期(内存中合成)
| 基础周期 | 可合成的衍生周期 |
|----------|----------------|
| 1m | 2m, 3m, 4m, 5m, 10m, 15m, 20m, 25m, 30m, 45m |
| 1h | 2h, 3h, 4h, 5h, 6h, 7h, 8h, 9h, 10h, 11h, 12h, 16h, 20h |
| 1d | 2d, 3d, 4d, 5d, 6d |
| 1w | 2w, 3w |
## 数据存储
数据以 CSV 格式存储,按时间周期分目录:
```
./data/
1m/
binance_BTC_USDT_USDT_1m.csv
binance_ETH_USDT_USDT_1m.csv
...
1h/
...
```
每根 K 线包含:`timestamp`, `datetime`, `open`, `high`, `low`, `close`, `volume`
---
API 文档请访问 `http://<host>:9009/api/docs`
-576
View File
@@ -1,576 +0,0 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-LVVXH3TL04"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-LVVXH3TL04');
</script>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Chan 数据提供商 - API 文档</title>
<style>
:root {
--bg: #0d1117;
--surface: #161b22;
--border: #30363d;
--text: #e6edf3;
--text-secondary: #8b949e;
--accent: #58a6ff;
--green: #3fb950;
--orange: #d29922;
--red: #f85149;
--purple: #bc8cff;
--font: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif;
--mono: "SF Mono", "Fira Code", "Consolas", monospace;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: var(--font);
background: var(--bg);
color: var(--text);
line-height: 1.6;
padding: 0;
}
.container { max-width: 960px; margin: 0 auto; padding: 24px 20px; }
/* Header */
header {
border-bottom: 1px solid var(--border);
padding: 32px 0 24px;
margin-bottom: 32px;
}
header h1 { font-size: 28px; font-weight: 600; margin-bottom: 8px; }
header h1 span { color: var(--accent); }
header .subtitle { color: var(--text-secondary); font-size: 15px; }
header .badge {
display: inline-block;
background: var(--surface);
border: 1px solid var(--border);
border-radius: 6px;
padding: 2px 10px;
font-size: 13px;
font-family: var(--mono);
color: var(--text-secondary);
margin-top: 12px;
}
header .badge span { color: var(--green); }
/* Section */
section { margin-bottom: 40px; }
section h2 {
font-size: 20px;
font-weight: 600;
margin-bottom: 16px;
padding-bottom: 8px;
border-bottom: 1px solid var(--border);
}
section h3 {
font-size: 16px;
font-weight: 600;
margin: 20px 0 8px;
color: var(--accent);
}
/* Endpoint card */
.endpoint {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
margin-bottom: 16px;
overflow: hidden;
}
.endpoint-header {
display: flex;
align-items: center;
gap: 12px;
padding: 14px 16px;
cursor: pointer;
user-select: none;
}
.endpoint-header:hover { background: rgba(255,255,255,0.03); }
.method {
display: inline-block;
font-family: var(--mono);
font-size: 13px;
font-weight: 700;
padding: 2px 8px;
border-radius: 4px;
min-width: 56px;
text-align: center;
}
.method.get { background: #1c3d5a; color: var(--accent); }
.method.ws { background: #2d1b5e; color: var(--purple); }
.endpoint-path {
font-family: var(--mono);
font-size: 14px;
font-weight: 500;
color: var(--text);
}
.endpoint-desc {
font-size: 13px;
color: var(--text-secondary);
margin-left: auto;
text-align: right;
}
.endpoint-body {
padding: 0 16px 16px;
border-top: 1px solid var(--border);
display: none;
}
.endpoint.open .endpoint-body { display: block; }
.endpoint-body > div { margin-top: 12px; }
/* Table */
table {
width: 100%;
border-collapse: collapse;
font-size: 14px;
}
th, td {
text-align: left;
padding: 8px 12px;
border-bottom: 1px solid var(--border);
}
th { color: var(--text-secondary); font-weight: 500; font-size: 12px; text-transform: uppercase; }
td { font-family: var(--mono); font-size: 13px; }
td.optional { color: var(--text-secondary); font-size: 12px; }
td .type { color: var(--orange); }
td .type-num { color: var(--accent); }
/* Code block */
pre {
background: #010409;
border: 1px solid var(--border);
border-radius: 6px;
padding: 12px 16px;
overflow-x: auto;
font-family: var(--mono);
font-size: 13px;
line-height: 1.5;
margin: 8px 0;
}
code { font-family: var(--mono); font-size: 13px; }
pre .comment { color: #8b949e; }
pre .string { color: #a5d6ff; }
pre .key { color: #79c0ff; }
pre .num { color: #79c0ff; }
pre .null { color: #d2a8ff; }
pre .bool { color: #d2a8ff; }
/* WS message box */
.ws-box {
background: #010409;
border: 1px solid var(--border);
border-radius: 6px;
padding: 12px 16px;
margin: 8px 0;
}
.ws-box .label {
font-size: 12px;
font-weight: 600;
color: var(--text-secondary);
text-transform: uppercase;
margin-bottom: 6px;
}
p { font-size: 14px; color: var(--text-secondary); margin-bottom: 8px; }
ul { padding-left: 20px; font-size: 14px; color: var(--text-secondary); }
li { margin-bottom: 4px; }
a { color: var(--accent); text-decoration: none; }
a:hover { text-decoration: underline; }
.note {
background: rgba(210, 153, 34, 0.1);
border: 1px solid rgba(210, 153, 34, 0.3);
border-radius: 6px;
padding: 10px 14px;
font-size: 13px;
color: var(--orange);
margin: 8px 0;
}
.toc { margin-bottom: 32px; }
.toc a {
display: inline-block;
padding: 4px 12px;
margin: 2px 0;
font-size: 14px;
color: var(--accent);
}
footer {
border-top: 1px solid var(--border);
padding: 20px 0;
text-align: center;
color: var(--text-secondary);
font-size: 13px;
}
</style>
</head>
<body>
<div class="container">
<header>
<h1><span>Chan</span> 数据提供商</h1>
<p class="subtitle">加密货币 K 线 + 衍生品数据 HTTP + WebSocket API</p>
<div class="badge">v1.0.0 &nbsp;|&nbsp; <span>binance</span> &nbsp;|&nbsp; port 9009</div>
</header>
<nav class="toc">
<a href="#root">GET /</a>
<a href="#health">GET /health</a>
<a href="#timeframes">GET /timeframes</a>
<a href="#candles">GET /api/candles</a>
<a href="#derivatives">GET /api/derivatives</a>
<a href="#websocket">WebSocket /ws</a>
<a href="#timeframes-ref">时间周期参考</a>
</nav>
<!-- ============ GET / ============ -->
<section id="root">
<h2>服务信息</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/</span>
<span class="endpoint-desc">服务基本信息</span>
</div>
<div class="endpoint-body">
<p>返回服务名称、交易所、交易对列表、可用周期及就绪状态。</p>
<h3>响应</h3>
<pre>{
<span class="key">"service"</span>: <span class="string">"Data Provider"</span>,
<span class="key">"exchange"</span>: <span class="string">"binance"</span>,
<span class="key">"symbols"</span>: [<span class="string">"BTC/USDT:USDT"</span>, <span class="string">"ETH/USDT:USDT"</span>, ...],
<span class="key">"base_timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, <span class="string">"1d"</span>, <span class="string">"1w"</span>],
<span class="key">"derived_timeframes"</span>: [<span class="string">"5m"</span>, <span class="string">"15m"</span>, <span class="string">"4h"</span>, ...],
<span class="key">"timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, ..., <span class="string">"5m"</span>, <span class="string">"15m"</span>, ...],
<span class="key">"ready"</span>: <span class="bool">true</span>
}</pre>
</div>
</div>
</section>
<!-- ============ GET /health ============ -->
<section id="health">
<h2>健康检查</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/health</span>
<span class="endpoint-desc">存活检查</span>
</div>
<div class="endpoint-body">
<p>返回服务健康状态,与 <code>/</code> 相同结构,适合负载均衡探测器。</p>
<h3>响应</h3>
<pre>{
<span class="key">"status"</span>: <span class="string">"ok"</span>,
<span class="key">"exchange"</span>: <span class="string">"binance"</span>,
<span class="key">"symbols"</span>: [<span class="string">"BTC/USDT:USDT"</span>, ...],
<span class="key">"ready"</span>: <span class="bool">true</span>,
...
}</pre>
</div>
</div>
</section>
<!-- ============ GET /timeframes ============ -->
<section id="timeframes">
<h2>可用周期</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/timeframes</span>
<span class="endpoint-desc">列出所有时间周期</span>
</div>
<div class="endpoint-body">
<p>返回基础周期(交易所直接拉取)和衍生周期(合成生成)的完整列表。</p>
<h3>响应</h3>
<pre>{
<span class="key">"base_timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, <span class="string">"1d"</span>, <span class="string">"1w"</span>],
<span class="key">"derived_timeframes"</span>: [<span class="string">"5m"</span>, <span class="string">"15m"</span>, <span class="string">"4h"</span>, ...],
<span class="key">"timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, ..., <span class="string">"5m"</span>, <span class="string">"15m"</span>, ...]
}</pre>
</div>
</div>
</section>
<!-- ============ GET /api/candles ============ -->
<section id="candles">
<h2>查询 K 线</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/api/candles</span>
<span class="endpoint-desc">获取 OHLCV K 线数据</span>
</div>
<div class="endpoint-body">
<table>
<tr><th>参数</th><th>类型</th><th>必填</th><th>说明</th></tr>
<tr>
<td>symbol</td>
<td><span class="type">string</span></td>
<td></td>
<td>交易对,如 <code>BTC/USDT:USDT</code></td>
</tr>
<tr>
<td>tf</td>
<td><span class="type">string</span></td>
<td></td>
<td>时间周期,默认 <code>1m</code>。支持基础及衍生周期</td>
</tr>
<tr>
<td>start</td>
<td><span class="type-num">int</span></td>
<td class="optional">可选</td>
<td>开始时间戳(毫秒)</td>
</tr>
<tr>
<td>end</td>
<td><span class="type-num">int</span></td>
<td class="optional">可选</td>
<td>结束时间戳(毫秒)</td>
</tr>
<tr>
<td>limit</td>
<td><span class="type-num">int</span></td>
<td class="optional">可选</td>
<td>限制返回的 K 线数量(返回最后 N 根)</td>
</tr>
</table>
<div class="note">若不传 start/end,返回内存中全部数据(可能很多),建议搭配 limit 使用。</div>
<h3>请求示例</h3>
<pre><span class="comment"># 获取 BTC 最近 100 根 5 分钟 K 线</span>
GET /api/candles?symbol=BTC/USDT:USDT&tf=5m&limit=100
<span class="comment"># 指定时间范围</span>
GET /api/candles?symbol=ETH/USDT:USDT&tf=1h&start=1704067200000&end=1704153600000
<span class="comment"># 获取 4 小时周期(衍生周期)</span>
GET /api/candles?symbol=SOL/USDT:USDT&tf=4h&limit=50</pre>
<h3>响应</h3>
<p>返回 OHLCV 对象数组:</p>
<pre>[
{
<span class="key">"timestamp"</span>: <span class="num">1704067200000</span>,
<span class="key">"datetime"</span>: <span class="string">"2024-01-01T00:00:00Z"</span>,
<span class="key">"open"</span>: <span class="num">42850.12</span>,
<span class="key">"high"</span>: <span class="num">43100.00</span>,
<span class="key">"low"</span>: <span class="num">42780.50</span>,
<span class="key">"close"</span>: <span class="num">43050.80</span>,
<span class="key">"volume"</span>: <span class="num">125.34</span>
},
...
]</pre>
<h3>字段说明</h3>
<table>
<tr><th>字段</th><th>类型</th><th>说明</th></tr>
<tr><td>timestamp</td><td><span class="type-num">int</span></td><td>UTC 毫秒时间戳</td></tr>
<tr><td>datetime</td><td><span class="type">string</span></td><td>ISO 8601 格式(末尾 Z</td></tr>
<tr><td>open</td><td><span class="type-num">float</span></td><td>开盘价</td></tr>
<tr><td>high</td><td><span class="type-num">float</span></td><td>最高价</td></tr>
<tr><td>low</td><td><span class="type-num">float</span></td><td>最低价</td></tr>
<tr><td>close</td><td><span class="type-num">float</span></td><td>收盘价</td></tr>
<tr><td>volume</td><td><span class="type-num">float</span></td><td>成交量</td></tr>
</table>
</div>
</div>
</section>
<!-- ============ GET /api/derivatives ============ -->
<section id="derivatives">
<h2>查询衍生品数据</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/api/derivatives</span>
<span class="endpoint-desc">获取资金费率、持仓量、基差</span>
</div>
<div class="endpoint-body">
<table>
<tr><th>参数</th><th>类型</th><th>必填</th><th>说明</th></tr>
<tr>
<td>symbol</td>
<td><span class="type">string</span></td>
<td></td>
<td>交易对,默认 <code>BTC/USDT:USDT</code></td>
</tr>
</table>
<div class="note">数据每 60 秒自动刷新,落盘到 <code>data/derivatives/</code> 目录。</div>
<h3>请求示例</h3>
<pre><span class="comment"># 获取 BTC 衍生品数据</span>
GET /api/derivatives?symbol=BTC/USDT:USDT</pre>
<h3>响应</h3>
<pre>{
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timestamp"</span>: <span class="num">1719705600000</span>,
<span class="key">"datetime"</span>: <span class="string">"2024-06-30T00:00:00Z"</span>,
<span class="key">"funding_rate"</span>: <span class="num">0.0001</span>,
<span class="key">"open_interest"</span>: <span class="num">35120000000.0</span>,
<span class="key">"oi_change_pct"</span>: <span class="num">3.52</span>,
<span class="key">"basis"</span>: <span class="num">8.5</span>
}</pre>
<h3>字段说明</h3>
<table>
<tr><th>字段</th><th>类型</th><th>说明</th></tr>
<tr><td>funding_rate</td><td>float</td><td>当前资金费率(每 8 小时)</td></tr>
<tr><td>open_interest</td><td>float</td><td>当前持仓量(USD</td></tr>
<tr><td>oi_change_pct</td><td>float</td><td>24 小时持仓量变化百分比</td></tr>
<tr><td>basis</td><td>float</td><td>期货-现货年化基差(%</td></tr>
</table>
</div>
</div>
</section>
<!-- ============ WebSocket ============ -->
<section id="websocket">
<h2>WebSocket 实时推送</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method ws">WS</span>
<span class="endpoint-path">/ws</span>
<span class="endpoint-desc">实时 K 线订阅</span>
</div>
<div class="endpoint-body">
<p>连接 WebSocket 后,通过 JSON 消息进行订阅管理。服务端在数据更新时主动推送最新 K 线。</p>
<h3>客户端 → 服务端</h3>
<div class="ws-box">
<div class="label">订阅 K 线</div>
<pre>{
<span class="key">"action"</span>: <span class="string">"subscribe"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>
}</pre>
</div>
<div class="ws-box">
<div class="label">取消订阅</div>
<pre>{
<span class="key">"action"</span>: <span class="string">"unsubscribe"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>
}</pre>
</div>
<div class="ws-box">
<div class="label">心跳 Ping</div>
<pre>{ <span class="key">"action"</span>: <span class="string">"ping"</span> }</pre>
</div>
<h3>服务端 → 客户端</h3>
<div class="ws-box">
<div class="label">订阅确认</div>
<pre>{
<span class="key">"type"</span>: <span class="string">"subscribed"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>
}</pre>
</div>
<div class="ws-box">
<div class="label">初始快照(订阅后立即推送最近 500 根 K 线)</div>
<pre>{
<span class="key">"type"</span>: <span class="string">"snapshot"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>,
<span class="key">"data"</span>: [ ... ]
}</pre>
</div>
<div class="ws-box">
<div class="label">K 线更新(增量推送最近 2 根)</div>
<pre>{
<span class="key">"type"</span>: <span class="string">"kline"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>,
<span class="key">"data"</span>: [ ... ]
}</pre>
</div>
<div class="ws-box">
<div class="label">Pong 响应</div>
<pre>{ <span class="key">"type"</span>: <span class="string">"pong"</span> }</pre>
</div>
<div class="ws-box">
<div class="label">错误消息</div>
<pre>{ <span class="key">"type"</span>: <span class="string">"error"</span>, <span class="key">"message"</span>: <span class="string">"..."</span> }</pre>
</div>
<h3>JavaScript 示例</h3>
<pre><span class="comment">// 连接</span>
<span class="key">const</span> ws = <span class="string">new WebSocket("ws://localhost:9009/ws")</span>;
ws.<span class="key">onopen</span> = () => {
<span class="comment">// 订阅 BTC 1m K 线</span>
ws.send(JSON.stringify({
action: <span class="string">"subscribe"</span>,
symbol: <span class="string">"BTC/USDT:USDT"</span>,
timeframe: <span class="string">"1m"</span>
}));
};
ws.<span class="key">onmessage</span> = (event) => {
<span class="key">const</span> msg = JSON.parse(event.data);
<span class="key">if</span> (msg.type === <span class="string">"kline"</span>) {
console.log(msg.data); <span class="comment">// 最新 K 线数组</span>
}
};</pre>
</div>
</div>
</section>
<!-- ============ 时间周期参考 ============ -->
<section id="timeframes-ref">
<h2>时间周期参考</h2>
<p>以下是完整的周期对照表:</p>
<table>
<tr><th>基础周期</th><th>合成衍生周期</th></tr>
<tr><td><code>1m</code></td><td><code>2m, 3m, 4m, 5m, 10m, 15m, 20m, 25m, 30m, 45m</code></td></tr>
<tr><td><code>1h</code></td><td><code>2h, 3h, 4h, 5h, 6h, 7h, 8h, 9h, 10h, 11h, 12h, 16h, 20h</code></td></tr>
<tr><td><code>1d</code></td><td><code>2d, 3d, 4d, 5d, 6d</code></td></tr>
<tr><td><code>1w</code></td><td><code>2w, 3w</code></td></tr>
</table>
<p>衍生周期由对应基础周期的 K 线通过 OHLCV 聚合合成,查询方式与基础周期完全一致。</p>
</section>
<footer>
Chan Data Provider &mdash; Built with FastAPI + ccxt + pandas
</footer>
</div>
<script>
<span class="comment">// 展开/折叠端点详情</span>
document.querySelectorAll('.endpoint-header').forEach(el => {
el.addEventListener('click', () => {
el.parentElement.classList.toggle('open');
});
});
<span class="comment">// 默认展开所有端点</span>
document.querySelectorAll('.endpoint').forEach(el => el.classList.add('open'));
</script>
</body>
</html>
-31
View File
@@ -1,31 +0,0 @@
{
"exchange": "binance",
"symbols": [
"BTC/USDT:USDT",
"ETH/USDT:USDT",
"SOL/USDT:USDT",
"XAU/USDT:USDT",
"XAG/USDT:USDT",
"SAGA/USDT:USDT",
"CL/USDT:USDT",
"ZEC/USDT:USDT",
"XRP/USDT:USDT",
"DOGE/USDT:USDT",
"BNB/USDT:USDT",
"SUI/USDT:USDT",
"BILL/USDT:USDT",
"BZ/USDT:USDT",
"LAB/USDT:USDT",
"TON/USDT:USDT",
"CRCL/USDT:USDT",
"SNDK/USDT:USDT",
"1000PEPE/USDT:USDT",
"CHIP/USDT:USDT"
],
"start_time": "2024-01-01T00:00:00Z",
"start_time_per_tf": {
"1m": "2026-01-01T00:00:00Z"
},
"timeframes": ["1m", "1h", "1d", "1w"],
"data_dir": "./data"
}
-15
View File
@@ -1,15 +0,0 @@
services:
data_provider:
build: .
container_name: data-provider
restart: unless-stopped
environment:
CONFIG_PATH: /app/config.json
UVICORN_HOST: 0.0.0.0
UVICORN_PORT: "80"
volumes:
- ./config.json:/app/config.json:ro
- ./data:/app/data
ports:
- "80:80"
-425
View File
@@ -1,425 +0,0 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-LVVXH3TL04"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-LVVXH3TL04');
</script>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Chan 数据提供商</title>
<style>
:root {
--bg: #0d1117;
--surface: #161b22;
--border: #30363d;
--text: #e6edf3;
--text-secondary: #8b949e;
--accent: #58a6ff;
--green: #3fb950;
--orange: #d29922;
--red: #f85149;
--purple: #bc8cff;
--font: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif;
--mono: "SF Mono", "Fira Code", "Consolas", monospace;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: var(--font);
background: var(--bg);
color: var(--text);
line-height: 1.6;
min-height: 100vh;
}
.container { max-width: 1000px; margin: 0 auto; padding: 32px 24px; }
/* Header */
header {
display: flex;
align-items: center;
justify-content: space-between;
padding-bottom: 20px;
border-bottom: 1px solid var(--border);
margin-bottom: 28px;
}
header .brand {
display: flex;
align-items: center;
gap: 14px;
}
header .logo {
display: flex;
align-items: center;
justify-content: center;
width: 44px; height: 44px;
border-radius: 10px;
background: linear-gradient(135deg, #1c3d5a 0%, #2d1b5e 100%);
border: 1px solid var(--border);
font-size: 20px; font-weight: 700;
color: var(--accent);
}
header .brand h1 { font-size: 20px; font-weight: 600; }
header .brand h1 span { color: var(--accent); }
header .brand .sub { font-size: 12px; color: var(--text-secondary); }
.header-time {
font-family: var(--mono);
font-size: 13px;
color: var(--text-secondary);
}
/* Overview cards */
.overview {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(140px, 1fr));
gap: 10px;
margin-bottom: 24px;
}
.ov-card {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
padding: 14px 16px;
text-align: center;
}
.ov-card .ov-label { font-size: 12px; color: var(--text-secondary); margin-bottom: 4px; }
.ov-card .ov-value {
font-family: var(--mono);
font-size: 18px;
font-weight: 600;
}
.ov-card .ov-value.green { color: var(--green); }
.ov-card .ov-value.orange { color: var(--orange); }
.ov-card .ov-value.accent { color: var(--accent); }
.ov-card .ov-value.purple { color: var(--purple); }
/* Section */
.section {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
margin-bottom: 12px;
overflow: hidden;
}
.section-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 12px 16px;
border-bottom: 1px solid var(--border);
cursor: pointer;
user-select: none;
}
.section-header:hover { background: rgba(255,255,255,0.02); }
.section-header h2 { font-size: 14px; font-weight: 600; }
.section-header .count-badge {
font-size: 12px;
font-family: var(--mono);
color: var(--text-secondary);
background: var(--bg);
padding: 2px 8px;
border-radius: 10px;
}
.section-body { padding: 8px 16px 12px; }
/* Status dot */
.dot {
display: inline-block;
width: 8px; height: 8px;
border-radius: 50%;
flex-shrink: 0;
}
.dot.green { background: var(--green); box-shadow: 0 0 6px #3fb95060; }
.dot.orange { background: var(--orange); box-shadow: 0 0 6px #d2992260; }
.dot.red { background: var(--red); box-shadow: 0 0 6px #f8514960; }
.dot.gray { background: var(--text-secondary); }
/* Symbol row */
.symbol-row {
display: flex;
align-items: center;
gap: 12px;
padding: 8px 0;
border-bottom: 1px solid rgba(48,54,61,0.5);
font-size: 14px;
}
.symbol-row:last-child { border-bottom: none; }
.symbol-row .name { font-family: var(--mono); font-size: 13px; min-width: 140px; }
.symbol-row .tag {
font-size: 11px;
font-family: var(--mono);
padding: 1px 6px;
border-radius: 4px;
background: rgba(88,166,255,0.1);
color: var(--accent);
}
/* TF chips */
.tf-list { display: flex; flex-wrap: wrap; gap: 4px; }
.tf-chip {
font-family: var(--mono);
font-size: 12px;
padding: 2px 8px;
border-radius: 4px;
background: var(--bg);
border: 1px solid var(--border);
color: var(--text-secondary);
}
.tf-chip.base { border-color: #58a6ff40; color: var(--accent); }
.tf-chip.derived { border-color: #bc8cff40; color: var(--purple); }
/* Links bar */
.links-bar {
display: flex;
gap: 8px;
flex-wrap: wrap;
margin: 16px 0 24px;
}
.links-bar a {
display: flex;
align-items: center;
gap: 6px;
padding: 6px 14px;
border-radius: 6px;
font-size: 13px;
text-decoration: none;
background: var(--surface);
border: 1px solid var(--border);
color: var(--text);
transition: border-color 0.15s;
}
.links-bar a:hover { border-color: var(--accent); }
/* Footer */
footer {
border-top: 1px solid var(--border);
padding: 16px 0;
margin-top: 32px;
text-align: center;
color: var(--text-secondary);
font-size: 13px;
}
.fade-in { animation: fadeIn 0.3s ease-in; }
@keyframes fadeIn { from { opacity: 0; } to { opacity: 1; } }
</style>
</head>
<body>
<div class="container">
<header>
<div class="brand">
<div class="logo">C</div>
<div>
<h1><span>Chan</span> 数据提供商</h1>
<div class="sub">加密货币 K 线数据服务</div>
</div>
</div>
<div class="header-time" id="header-time"></div>
</header>
<!-- Overview -->
<div class="overview" id="overview">
<div class="ov-card"><div class="ov-label">服务状态</div><div class="ov-value" id="ov-status">加载中...</div></div>
<div class="ov-card"><div class="ov-label">交易所</div><div class="ov-value accent" id="ov-exchange"></div></div>
<div class="ov-card"><div class="ov-label">交易对</div><div class="ov-value" id="ov-symbols"></div></div>
<div class="ov-card"><div class="ov-label">基础周期</div><div class="ov-value accent" id="ov-base-tf"></div></div>
<div class="ov-card"><div class="ov-label">衍生周期</div><div class="ov-value purple" id="ov-derived-tf"></div></div>
<div class="ov-card"><div class="ov-label">数据就绪</div><div class="ov-value" id="ov-ready"></div></div>
</div>
<!-- Links -->
<div class="links-bar">
<a href="/api/docs">📖 API 文档</a>
<a href="/docs">📋 Swagger UI</a>
<a href="/redoc">📄 ReDoc</a>
<a href="/api/candles?symbol=BTC/USDT:USDT&tf=1m&limit=5" target="_blank">📊 BTC 1m 示例</a>
<a href="/api/candles?symbol=ETH/USDT:USDT&tf=4h&limit=10" target="_blank">📊 ETH 4h 示例</a>
</div>
<!-- Symbols -->
<div class="section" id="section-symbols">
<div class="section-header" onclick="this.parentElement.classList.toggle('collapsed')">
<h2>📈 交易对</h2>
<span class="count-badge" id="sym-count">0</span>
</div>
<div class="section-body" id="symbol-list">
<div style="color:var(--text-secondary);font-size:13px;">加载中...</div>
</div>
</div>
<!-- Timeframes -->
<div class="section">
<div class="section-header" onclick="this.parentElement.classList.toggle('collapsed')">
<h2>⏱ 时间周期</h2>
<span class="count-badge" id="tf-total">0</span>
</div>
<div class="section-body">
<div style="margin-bottom:8px;font-size:13px;color:var(--text-secondary);">基础周期(交易所直拉)</div>
<div class="tf-list" id="base-tf-list"></div>
<div style="margin:10px 0 8px;font-size:13px;color:var(--text-secondary);">衍生周期(内存合成)</div>
<div class="tf-list" id="derived-tf-list"></div>
</div>
</div>
<!-- Quick query -->
<div class="section">
<div class="section-header">
<h2>⚡ 快速查询</h2>
</div>
<div class="section-body">
<div style="display:flex;gap:8px;flex-wrap:wrap;align-items:end;">
<div>
<div style="font-size:12px;color:var(--text-secondary);margin-bottom:4px;">交易对</div>
<select id="q-symbol" style="background:var(--bg);border:1px solid var(--border);color:var(--text);padding:6px 10px;border-radius:6px;font-family:var(--mono);font-size:13px;"></select>
</div>
<div>
<div style="font-size:12px;color:var(--text-secondary);margin-bottom:4px;">周期</div>
<select id="q-tf" style="background:var(--bg);border:1px solid var(--border);color:var(--text);padding:6px 10px;border-radius:6px;font-family:var(--mono);font-size:13px;"></select>
</div>
<div>
<div style="font-size:12px;color:var(--text-secondary);margin-bottom:4px;">数量</div>
<select id="q-limit" style="background:var(--bg);border:1px solid var(--border);color:var(--text);padding:6px 10px;border-radius:6px;font-family:var(--mono);font-size:13px;">
<option>5</option><option selected>10</option><option>20</option><option>50</option>
</select>
</div>
<button onclick="quickQuery()" style="background:var(--accent);color:#fff;border:none;padding:6px 18px;border-radius:6px;cursor:pointer;font-size:13px;font-weight:600;">查询</button>
</div>
<pre id="q-result" style="margin-top:10px;display:none;"></pre>
</div>
</div>
</div>
<footer>
Chan Data Provider &nbsp;·&nbsp; Built with FastAPI + ccxt + pandas &nbsp;·&nbsp;
更新于 <span id="footer-time"></span>
</footer>
<script>
let healthData = null;
function fmtTime(ts) {
return new Date(ts).toLocaleString('zh-CN', { timeZone: 'UTC', hour12: false }) + ' UTC';
}
async function loadHealth() {
try {
const res = await fetch('/health');
healthData = await res.json();
renderHealth(healthData);
} catch {
document.getElementById('ov-status').textContent = '无法连接';
document.getElementById('ov-status').style.color = 'var(--red)';
document.getElementById('ov-ready').textContent = '断开';
document.getElementById('ov-ready').style.color = 'var(--red)';
}
}
function renderHealth(d) {
const now = Date.now();
document.getElementById('header-time').textContent = fmtTime(now);
document.getElementById('footer-time').textContent = fmtTime(now);
// Overview
const statusEl = document.getElementById('ov-status');
statusEl.textContent = '运行中';
statusEl.style.color = 'var(--green)';
document.getElementById('ov-exchange').textContent = d.exchange || '—';
const symCount = (d.symbols || []).length;
const symEl = document.getElementById('ov-symbols');
symEl.textContent = symCount + ' 个';
symEl.style.color = 'var(--accent)';
document.getElementById('ov-base-tf').textContent = (d.base_timeframes || []).length + ' 个';
document.getElementById('ov-derived-tf').textContent = (d.derived_timeframes || []).length + ' 个';
const readyEl = document.getElementById('ov-ready');
if (d.ready) {
readyEl.textContent = '已就绪';
readyEl.style.color = 'var(--green)';
} else {
readyEl.textContent = '同步中...';
readyEl.style.color = 'var(--orange)';
setTimeout(loadHealth, 2000);
}
// Symbols
const symList = document.getElementById('symbol-list');
const symCountEl = document.getElementById('sym-count');
symCountEl.textContent = symCount;
if (d.symbols && d.symbols.length > 0) {
symList.innerHTML = d.symbols.map(s => `
<div class="symbol-row">
<span class="dot green"></span>
<span class="name">${s}</span>
<span class="tag">${d.exchange || '—'}</span>
<a href="/api/candles?symbol=${encodeURIComponent(s)}&tf=1m&limit=5" target="_blank" style="margin-left:auto;font-size:12px;color:var(--accent);text-decoration:none;">1m →</a>
</div>
`).join('');
} else {
symList.innerHTML = '<div style="color:var(--text-secondary);font-size:13px;">暂无交易对</div>';
}
// Timeframes
document.getElementById('tf-total').textContent = (d.timeframes || []).length;
const baseList = document.getElementById('base-tf-list');
if (d.base_timeframes) {
baseList.innerHTML = d.base_timeframes.map(t => `<span class="tf-chip base">${t}</span>`).join('');
}
const derivedList = document.getElementById('derived-tf-list');
if (d.derived_timeframes) {
derivedList.innerHTML = d.derived_timeframes.map(t => `<span class="tf-chip derived">${t}</span>`).join('');
}
// Populate query selects
const symSelect = document.getElementById('q-symbol');
if (d.symbols && symSelect.options.length === 0) {
d.symbols.forEach(s => {
const opt = document.createElement('option');
opt.value = s;
opt.textContent = s;
symSelect.appendChild(opt);
});
}
const tfSelect = document.getElementById('q-tf');
if (d.timeframes && tfSelect.options.length === 0) {
d.timeframes.forEach(t => {
const opt = document.createElement('option');
opt.value = t;
opt.textContent = t;
tfSelect.appendChild(opt);
});
}
}
async function quickQuery() {
const symbol = document.getElementById('q-symbol').value;
const tf = document.getElementById('q-tf').value;
const limit = document.getElementById('q-limit').value;
const url = `/api/candles?symbol=${encodeURIComponent(symbol)}&tf=${tf}&limit=${limit}`;
const pre = document.getElementById('q-result');
pre.style.display = 'block';
pre.textContent = '查询中...';
try {
const res = await fetch(url);
const data = await res.json();
pre.textContent = JSON.stringify(data, null, 2);
} catch {
pre.textContent = '查询失败';
}
}
loadHealth();
setInterval(loadHealth, 10000);
</script>
</body>
</html>
File diff suppressed because it is too large Load Diff
-597
View File
@@ -1,597 +0,0 @@
"""
链上指标数据模块:拉取 BTC 交易所净流量、稳定币供应、ETF 净流入、MVRV Z-Score、SOPR。
本地 CSV 落盘 + 内存缓存,参照 derivatives 模式。
免费数据源:
- CoinMetrics Community API: 交易所流入/流出(USD)、MVRV Ratio
- CoinGecko: 稳定币市值
需要 API key 的指标(可配置):
- ETF 净流入: Coinglass / Farside / Glassnode
- SOPR: Glassnode / CoinMetrics Pro
"""
import csv
import json
import logging
import os
import re
import threading
import time
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Dict, List, Optional
import requests
from fastapi import APIRouter, HTTPException, Query
logger = logging.getLogger("onchain_metrics")
# ── 常量 ─────────────────────────────────────────────────────────────
METRIC_NAMES = [
"btc_netflow",
"stablecoin_supply",
"etf_flow",
"mvrv_zscore",
"sopr",
]
REFRESH_INTERVAL = 300 # 后台刷新间隔(秒),免费 API 限频较严
COINMETRICS_BASE = "https://community-api.coinmetrics.io/v4"
COINGECKO_BASE = "https://api.coingecko.com/api/v3"
REQUEST_TIMEOUT = 30
HTTP_HEADERS = {"User-Agent": "Mozilla/5.0 (compatible; chan-data-provider/1.0)"}
def to_utc_iso(ts_ms: int) -> str:
dt = datetime.fromtimestamp(ts_ms / 1000, tz=timezone.utc)
return dt.isoformat().replace("+00:00", "Z")
# ── OnchainMetricsManager ────────────────────────────────────────────
class OnchainMetricsManager:
"""管理链上指标的拉取、缓存和持久化。每个指标独立存储。"""
def __init__(self, data_dir: Path, api_keys: Optional[Dict[str, str]] = None) -> None:
self.data_dir = data_dir / "onchain"
self.data_dir.mkdir(parents=True, exist_ok=True)
self.api_keys = api_keys or {}
# 内存缓存: metric_name -> List[Dict]
self._metrics: Dict[str, List[Dict]] = {name: [] for name in METRIC_NAMES}
self._lock = threading.RLock()
self._stop_event = threading.Event()
self._thread: Optional[threading.Thread] = None
# 加载本地历史
for name in METRIC_NAMES:
local = self._load_local(name)
if local:
self._metrics[name] = local
logger.info("链上指标 %s 加载本地记录: %d", name, len(local))
# ── CSV 路径 ─────────────────────────────────────────────────────
def _path(self, metric: str) -> Path:
return self.data_dir / f"{metric}.csv"
# ── CSV 读写 ─────────────────────────────────────────────────────
def _load_local(self, metric: str) -> List[Dict]:
path = self._path(metric)
if not path.exists():
return []
records = []
with path.open("r", encoding="utf-8", newline="") as fp:
for row in csv.DictReader(fp):
try:
records.append({
"timestamp": int(row["timestamp"]),
"datetime": row.get("datetime", ""),
"value": float(row.get("value", 0)),
"sub_value": float(row.get("sub_value", 0)) if row.get("sub_value") else None,
"extra": json.loads(row.get("extra", "{}")) if row.get("extra") else {},
})
except (KeyError, ValueError):
continue
records.sort(key=lambda r: r["timestamp"])
return records
def _write_local(self, metric: str, records: List[Dict]) -> None:
path = self._path(metric)
fieldnames = ["timestamp", "datetime", "value", "sub_value", "extra"]
tmp = path.with_suffix(".tmp")
try:
with tmp.open("w", encoding="utf-8", newline="") as fp:
w = csv.DictWriter(fp, fieldnames=fieldnames, extrasaction="ignore")
w.writeheader()
for r in records:
row = {
"timestamp": r.get("timestamp", 0),
"datetime": r.get("datetime", ""),
"value": r.get("value", 0),
"sub_value": "" if r.get("sub_value") is None else r["sub_value"],
"extra": json.dumps(r.get("extra", {}), ensure_ascii=False),
}
w.writerow(row)
tmp.replace(path)
logger.info("链上指标 %s 已写入磁盘: %d", metric, len(records))
except Exception as e:
logger.error("链上指标 %s 落盘失败: %s", metric, e)
if tmp.exists():
tmp.unlink()
def _merge(self, base: List[Dict], new: List[Dict]) -> List[Dict]:
"""按 timestamp 去重合并,新覆盖旧。"""
merged = {r["timestamp"]: r for r in base}
for r in new:
merged[r["timestamp"]] = r
return sorted(merged.values(), key=lambda r: r["timestamp"])
# ── 数据拉取 ─────────────────────────────────────────────────────
def _fetch_cm_metrics(
self, metrics: str, days: int = 365
) -> Optional[List[Dict]]:
"""从 CoinMetrics Community API 拉取指标。返回 [{time, metric: value}, ...]"""
page_size = min(days, 10000)
url = (
f"{COINMETRICS_BASE}/timeseries/asset-metrics"
f"?assets=btc&metrics={metrics}&frequency=1d&page_size={page_size}"
)
try:
r = requests.get(url, timeout=REQUEST_TIMEOUT, headers=HTTP_HEADERS)
if r.status_code != 200:
logger.warning("CoinMetrics %s 返回 %d: %s", metrics, r.status_code, r.text[:200])
return None
data = r.json().get("data", [])
if not data:
return None
# 翻页取更多历史
all_data = list(data)
next_url = r.json().get("next_page_url")
pages = 0
while next_url and pages < 10:
pages += 1
time.sleep(0.5)
r2 = requests.get(next_url, timeout=REQUEST_TIMEOUT, headers=HTTP_HEADERS)
if r2.status_code != 200:
break
batch = r2.json()
all_data.extend(batch.get("data", []))
next_url = batch.get("next_page_url")
if not next_url or next_url == r.json().get("next_page_url"):
break
return all_data
except Exception as e:
logger.error("CoinMetrics %s 拉取失败: %s", metrics, e)
return None
def _fetch_btc_netflow(self) -> List[Dict]:
"""BTC 交易所净流量 (USD) = 流入 - 流出。"""
data = self._fetch_cm_metrics("FlowInExUSD,FlowOutExUSD", days=365)
if not data:
return []
records = []
for d in data:
ts_str = d.get("time", "")
inflow = float(d.get("FlowInExUSD") or 0)
outflow = float(d.get("FlowOutExUSD") or 0)
netflow = inflow - outflow
try:
ts = int(datetime.fromisoformat(ts_str.replace("Z", "+00:00")).timestamp() * 1000)
except Exception:
continue
records.append({
"timestamp": ts,
"datetime": to_utc_iso(ts),
"value": round(netflow, 2),
"sub_value": round(inflow, 2),
"extra": {"inflow_usd": round(inflow, 2), "outflow_usd": round(outflow, 2)},
})
return sorted(records, key=lambda r: r["timestamp"])
def _fetch_stablecoin_supply(self) -> List[Dict]:
"""稳定币总供应(USDT + USDC + DAI + FDUSD + TUSD 市值之和)。"""
try:
url = (
f"{COINGECKO_BASE}/coins/markets"
f"?vs_currency=usd&category=stablecoins&order=market_cap_desc"
f"&per_page=5&page=1&sparkline=false"
)
r = requests.get(url, timeout=REQUEST_TIMEOUT, headers=HTTP_HEADERS)
if r.status_code != 200:
logger.warning("CoinGecko stablecoins 返回 %d", r.status_code)
return []
coins = r.json()
if not isinstance(coins, list):
return []
total_mcap = sum(c.get("market_cap", 0) or 0 for c in coins)
now_ms = int(time.time() * 1000)
breakdown = {
c.get("symbol", "?").upper(): c.get("market_cap", 0) or 0
for c in coins[:5]
}
return [{
"timestamp": now_ms,
"datetime": to_utc_iso(now_ms),
"value": round(total_mcap, 2),
"sub_value": round(total_mcap / 1e9, 2), # Billions
"extra": breakdown,
}]
except Exception as e:
logger.error("稳定币供应拉取失败: %s", e)
return []
def _fetch_etf_flow(self) -> List[Dict]:
"""BTC ETF 净流入/流出。优先从 Farside 免费爬取,Coinglass 为备用。
数据源优先级:
1. Farside (免费, HTML 爬取)
2. Coinglass (需 coinglass_key)
3. Glassnode (需 glassnode_key)
"""
# ── 优先:Farside 免费爬取 ──
try:
records = self._fetch_etf_farside()
if records:
logger.info("ETF 数据从 Farside 拉取: %d", len(records))
return records
except Exception as e:
logger.warning("Farside ETF 爬取失败: %s", e)
# ── 备用:Coinglass ──
cg_key = self.api_keys.get("coinglass_key")
if cg_key:
try:
url = "https://open-api-v3.coinglass.com/api/bitcoin/etf/net-inflow?interval=30"
r = requests.get(url, timeout=REQUEST_TIMEOUT, headers={
**HTTP_HEADERS, "coinglassSecret": cg_key,
})
if r.status_code == 200:
data = r.json()
records = []
if isinstance(data, dict) and "data" in data:
for item in data["data"]:
ts = int(item.get("date", 0)) * 1000 if item.get("date") else 0
if ts:
records.append({
"timestamp": ts,
"datetime": to_utc_iso(ts),
"value": float(item.get("netInflow", 0)),
"sub_value": None,
"extra": item,
})
return records
logger.warning("Coinglass ETF 返回 %d", r.status_code)
except Exception as e:
logger.error("Coinglass ETF 拉取失败: %s", e)
logger.warning("ETF 数据源均不可用(Farside/Coinglass/Glassnode")
return []
def _fetch_etf_farside(self) -> List[Dict]:
"""从 Farside 网站爬取 BTC ETF 每日净流量。
https://farside.co.uk/btc/ 页面包含一个 HTML 表格,
每行包含日期和各 ETF 的当日流量(百万美元),最后一列为总计。
"""
url = "https://farside.co.uk/btc/"
r = requests.get(url, timeout=REQUEST_TIMEOUT, headers={
**HTTP_HEADERS,
"Accept": "text/html,application/xhtml+xml,*/*",
})
if r.status_code != 200:
logger.warning("Farside 返回 %d", r.status_code)
return []
html = r.text
# 查找表格
table_match = re.search(r'<table[^>]*>(.*?)</table>', html, re.DOTALL)
if not table_match:
logger.warning("Farside 页面未找到表格")
return []
rows_html = re.findall(r'<tr[^>]*>(.*?)</tr>', table_match.group(1), re.DOTALL)
month_map = {
"jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6,
"jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12,
}
records = []
for row_html in rows_html:
cells = re.findall(r'<t[dh][^>]*>(.*?)</t[dh]>', row_html, re.DOTALL)
# 清理 HTML 标签和空白
clean = []
for c in cells:
t = re.sub(r'<[^>]+>', '', c).strip()
t = t.replace('\xa0', ' ').replace('&nbsp;', ' ').strip()
clean.append(t)
if not clean:
continue
# 第一列应为日期格式 "15 Jun 2026"
date_str = clean[0]
parts = date_str.split()
if len(parts) != 3:
continue
day_str, mon_str, year_str = parts
mon = month_map.get(mon_str.lower()[:3])
if mon is None:
continue
try:
day = int(day_str)
year = int(year_str)
except ValueError:
continue
# 最后一列为总计(可能带括号表示负值)
total_str = clean[-1] if len(clean) > 1 else ""
if not total_str or total_str in ("", "Total", "-"):
continue
# 解析 "(123.4)" → -123.4, "123.4" → 123.4
total_str = total_str.replace(",", "")
is_negative = total_str.startswith("(") and total_str.endswith(")")
if is_negative:
total_str = total_str[1:-1]
try:
total_m = float(total_str)
except ValueError:
continue
if is_negative:
total_m = -total_m
# 构建时间戳(UTC 午夜)
from datetime import datetime, timezone as tz
dt = datetime(year, mon, day, tzinfo=tz.utc)
ts = int(dt.timestamp() * 1000)
# 分解各 ETF 明细
etf_breakdown = {}
if len(clean) > 2:
etf_names = ["IBIT", "FBTC", "BITB", "ARKB", "BTCO", "EZBC",
"BRRR", "HODL", "BTCW", "MSBT", "GBTC", "BTC"]
for i, name in enumerate(etf_names):
idx = i + 1
if idx < len(clean) - 1:
val_str = clean[idx].replace(",", "").replace("(", "").replace(")", "")
neg = clean[idx].startswith("(")
try:
val = float(val_str)
etf_breakdown[name] = -val if neg else val
except ValueError:
pass
records.append({
"timestamp": ts,
"datetime": dt.isoformat().replace("+00:00", "Z"),
"value": round(total_m, 2), # 净流入 (百万 USD)
"sub_value": round(total_m * 1_000_000, 2), # 净流入 (USD)
"extra": {"total_million_usd": round(total_m, 2), "breakdown": etf_breakdown},
})
return sorted(records, key=lambda r: r["timestamp"])
def _fetch_mvrv_zscore(self) -> List[Dict]:
"""MVRV Z-Score = (当前 MVRV - 滚动均值) / 滚动标准差。
从 CoinMetrics 拉取 CapMVRVCur(免费),计算 365 天滚动 Z-Score。
"""
data = self._fetch_cm_metrics("CapMVRVCur", days=400)
if not data:
return []
# 解析时间序列
mvrv_series = []
for d in data:
ts_str = d.get("time", "")
mvrv_val = d.get("CapMVRVCur")
if mvrv_val is None:
continue
try:
ts = int(datetime.fromisoformat(ts_str.replace("Z", "+00:00")).timestamp() * 1000)
except Exception:
continue
mvrv_series.append((ts, float(mvrv_val)))
mvrv_series.sort(key=lambda x: x[0])
# 计算滚动 Z-Score (365 天窗口,即约 365 个数据点)
window = 365
records = []
values = []
timestamps = []
for ts, val in mvrv_series:
values.append(val)
timestamps.append(ts)
if len(values) < window:
continue
window_vals = values[-window:]
mean = sum(window_vals) / window
variance = sum((v - mean) ** 2 for v in window_vals) / window
stddev = variance ** 0.5
zscore = (val - mean) / stddev if stddev > 0 else 0.0
records.append({
"timestamp": ts,
"datetime": to_utc_iso(ts),
"value": round(zscore, 4),
"sub_value": round(val, 4),
"extra": {"mvrv_ratio": round(val, 4), "rolling_mean": round(mean, 4), "rolling_stddev": round(stddev, 4)},
})
return records
def _fetch_sopr(self) -> List[Dict]:
"""SOPR (Spent Output Profit Ratio)。需要 API key。
支持:
- glassnode_key: Glassnode v1/metrics/indicators/sopr
"""
gn_key = self.api_keys.get("glassnode_key")
if gn_key:
try:
url = (
f"https://api.glassnode.com/v1/metrics/indicators/sopr"
f"?a=btc&f=json&api_key={gn_key}"
)
r = requests.get(url, timeout=REQUEST_TIMEOUT, headers=HTTP_HEADERS)
if r.status_code == 200:
data = r.json()
records = []
for item in data if isinstance(data, list) else []:
ts = int(item.get("t", 0)) * 1000
if ts:
records.append({
"timestamp": ts,
"datetime": to_utc_iso(ts),
"value": float(item.get("v", 0)),
"sub_value": None,
"extra": {"raw": item},
})
return records
logger.warning("Glassnode SOPR 返回 %d", r.status_code)
except Exception as e:
logger.error("Glassnode SOPR 拉取失败: %s", e)
logger.warning(
"SOPR 数据未配置 API key。请设置环境变量 GLASSNODE_API_KEY"
)
return []
# ── 刷新入口 ─────────────────────────────────────────────────────
def refresh_all(self) -> None:
"""拉取全部 5 个指标,合并到内存并写盘。"""
fetchers = {
"btc_netflow": self._fetch_btc_netflow,
"stablecoin_supply": self._fetch_stablecoin_supply,
"etf_flow": self._fetch_etf_flow,
"mvrv_zscore": self._fetch_mvrv_zscore,
"sopr": self._fetch_sopr,
}
for name, fetcher in fetchers.items():
try:
new_records = fetcher()
if not new_records:
continue
with self._lock:
base = list(self._metrics.get(name, []))
merged = self._merge(base, new_records)
self._metrics[name] = merged
self._write_local(name, merged)
logger.info("链上指标 %s 刷新: +%d 条, 总计 %d",
name, len(new_records), len(merged))
except Exception:
logger.exception("链上指标 %s 刷新异常", name)
# ── 后台线程 ─────────────────────────────────────────────────────
def _refresh_loop(self) -> None:
"""后台线程:启动立即拉取一次,之后每 REFRESH_INTERVAL 秒刷新。"""
logger.info("链上指标后台线程启动,间隔 %ds", REFRESH_INTERVAL)
# 启动立即拉取
self.refresh_all()
while not self._stop_event.wait(REFRESH_INTERVAL):
self.refresh_all()
logger.info("链上指标后台线程已退出")
def start(self) -> None:
self._stop_event.clear()
self._thread = threading.Thread(
target=self._refresh_loop, name="onchain-loop", daemon=True,
)
self._thread.start()
logger.info("链上指标模块已启动")
def stop(self) -> None:
self._stop_event.set()
if self._thread:
self._thread.join(timeout=5)
# ── 查询接口 ─────────────────────────────────────────────────────
def get_metric(
self,
metric: str,
limit: int = 100,
start_ms: Optional[int] = None,
end_ms: Optional[int] = None,
) -> List[Dict]:
if metric not in METRIC_NAMES:
raise HTTPException(
status_code=400,
detail=f"不支持的指标: {metric}。可选: {METRIC_NAMES}",
)
with self._lock:
records = list(self._metrics.get(metric, []))
if start_ms is not None:
records = [r for r in records if r["timestamp"] >= start_ms]
if end_ms is not None:
records = [r for r in records if r["timestamp"] <= end_ms]
return records[-limit:] if limit else records
def get_latest(self) -> Dict[str, Any]:
"""获取所有指标的最新值。"""
result = {"timestamp": int(time.time() * 1000)}
for name in METRIC_NAMES:
with self._lock:
records = self._metrics.get(name, [])
if records:
latest = records[-1]
result[name] = {
"timestamp": latest["timestamp"],
"datetime": latest["datetime"],
"value": latest["value"],
"sub_value": latest.get("sub_value"),
"extra": latest.get("extra", {}),
}
else:
result[name] = None
return result
# ── FastAPI Router ────────────────────────────────────────────────────
def create_onchain_router(manager: OnchainMetricsManager) -> APIRouter:
router = APIRouter(prefix="/api/onchain", tags=["onchain"])
@router.get("/metrics")
async def get_metrics(
metric: str = Query(..., description=f"指标名称: {', '.join(METRIC_NAMES)}"),
limit: int = Query(100, ge=1, le=5000, description="返回条数上限"),
start: Optional[int] = Query(None, description="起始时间戳(ms)"),
end: Optional[int] = Query(None, description="结束时间戳(ms)"),
):
"""获取单个链上指标的时间序列。"""
data = manager.get_metric(metric, limit=limit, start_ms=start, end_ms=end)
return {"metric": metric, "count": len(data), "data": data}
@router.get("/latest")
async def get_latest():
"""获取全部 5 个指标的最新快照。"""
return manager.get_latest()
@router.get("/available")
async def get_available():
"""列出可用指标及当前数据量。"""
info = {}
for name in METRIC_NAMES:
records = manager.get_metric(name, limit=0)
info[name] = {
"count": len(records),
"has_data": len(records) > 0,
"first_ts": records[0]["datetime"] if records else None,
"last_ts": records[-1]["datetime"] if records else None,
}
return info
return router
# ── 环境变量辅助 ─────────────────────────────────────────────────────
def api_keys_from_env() -> Dict[str, str]:
"""从环境变量读取 API keys。"""
keys = {}
for env_var, key_name in [
("COINGLASS_API_KEY", "coinglass_key"),
("GLASSNODE_API_KEY", "glassnode_key"),
("FARSIDE_API_KEY", "farside_key"),
("COINMETRICS_API_KEY", "coinmetrics_key"),
]:
val = os.getenv(env_var, "").strip()
if val and "***" not in val: # 忽略占位符
keys[key_name] = val
return keys
-6
View File
@@ -1,6 +0,0 @@
ccxt>=4.0.0,<5.0.0
fastapi>=0.110.0,<1.0.0
uvicorn[standard]>=0.23.0,<1.0.0
pandas>=2.0.0,<3.0.0
technical==1.5.0
requests>=2.28.0