更新data_provider逻辑,能够更快开始提供服务,添加说明

This commit is contained in:
jackyu66git
2026-05-01 17:13:24 +08:00
parent 425d513a37
commit 5ab69c2a64
4 changed files with 1057 additions and 17 deletions
+82
View File
@@ -0,0 +1,82 @@
# Chan 数据提供商 (Chan Data Provider)
**Binance 期货** 交易所拉取加密货币 K 线数据,提供 HTTP + WebSocket 数据服务。
## 功能
- **多交易对**:支持 BTC, ETH, SOL, DOGE 等 9 个交易对
- **多时间周期**:基础周期 1m/1h/1d/1w,可合成 30+ 种衍生周期(如 5m, 15m, 4h 等)
- **本地缓存**:CSV 持久化到磁盘,重启快速加载
- **断线恢复**:交易所连接中断时记录断点,自动补拉缺失数据
- **实时推送**WebSocket 订阅最新 K 线更新
- **内存服务**:启动即加载本地数据,不阻塞服务
## 启动
```bash
# Docker
docker compose up -d
# 直接运行
python main.py
# 或指定配置
CONFIG_PATH=./config.json python main.py
```
服务默认监听 `http://0.0.0.0:9009`
## 配置
编辑 `config.json`
```json
{
"exchange": "binance",
"symbols": ["BTC/USDT:USDT", "ETH/USDT:USDT"],
"start_time": "2024-01-01T00:00:00Z",
"timeframes": ["1m", "1h", "1d", "1w"],
"data_dir": "./data"
}
```
| 字段 | 说明 |
|------|------|
| `exchange` | 交易所名称(ccxt 支持即可) |
| `symbols` | 交易对列表 |
| `start_time` | 历史数据起始时间 |
| `timeframes` | 基础周期(从交易所直接拉取) |
| `data_dir` | CSV 数据存储目录 |
## 可用周期
### 基础周期(交易所直接拉取)
`1m`, `1h`, `1d`, `1w`
### 衍生周期(内存中合成)
| 基础周期 | 可合成的衍生周期 |
|----------|----------------|
| 1m | 2m, 3m, 4m, 5m, 10m, 15m, 20m, 25m, 30m, 45m |
| 1h | 2h, 3h, 4h, 5h, 6h, 7h, 8h, 9h, 10h, 11h, 12h, 16h, 20h |
| 1d | 2d, 3d, 4d, 5d, 6d |
| 1w | 2w, 3w |
## 数据存储
数据以 CSV 格式存储,按时间周期分目录:
```
./data/
1m/
binance_BTC_USDT_USDT_1m.csv
binance_ETH_USDT_USDT_1m.csv
...
1h/
...
```
每根 K 线包含:`timestamp`, `datetime`, `open`, `high`, `low`, `close`, `volume`
---
API 文档请访问 `http://<host>:9009/api/docs`
+517
View File
@@ -0,0 +1,517 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Chan 数据提供商 - API 文档</title>
<style>
:root {
--bg: #0d1117;
--surface: #161b22;
--border: #30363d;
--text: #e6edf3;
--text-secondary: #8b949e;
--accent: #58a6ff;
--green: #3fb950;
--orange: #d29922;
--red: #f85149;
--purple: #bc8cff;
--font: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif;
--mono: "SF Mono", "Fira Code", "Consolas", monospace;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: var(--font);
background: var(--bg);
color: var(--text);
line-height: 1.6;
padding: 0;
}
.container { max-width: 960px; margin: 0 auto; padding: 24px 20px; }
/* Header */
header {
border-bottom: 1px solid var(--border);
padding: 32px 0 24px;
margin-bottom: 32px;
}
header h1 { font-size: 28px; font-weight: 600; margin-bottom: 8px; }
header h1 span { color: var(--accent); }
header .subtitle { color: var(--text-secondary); font-size: 15px; }
header .badge {
display: inline-block;
background: var(--surface);
border: 1px solid var(--border);
border-radius: 6px;
padding: 2px 10px;
font-size: 13px;
font-family: var(--mono);
color: var(--text-secondary);
margin-top: 12px;
}
header .badge span { color: var(--green); }
/* Section */
section { margin-bottom: 40px; }
section h2 {
font-size: 20px;
font-weight: 600;
margin-bottom: 16px;
padding-bottom: 8px;
border-bottom: 1px solid var(--border);
}
section h3 {
font-size: 16px;
font-weight: 600;
margin: 20px 0 8px;
color: var(--accent);
}
/* Endpoint card */
.endpoint {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
margin-bottom: 16px;
overflow: hidden;
}
.endpoint-header {
display: flex;
align-items: center;
gap: 12px;
padding: 14px 16px;
cursor: pointer;
user-select: none;
}
.endpoint-header:hover { background: rgba(255,255,255,0.03); }
.method {
display: inline-block;
font-family: var(--mono);
font-size: 13px;
font-weight: 700;
padding: 2px 8px;
border-radius: 4px;
min-width: 56px;
text-align: center;
}
.method.get { background: #1c3d5a; color: var(--accent); }
.method.ws { background: #2d1b5e; color: var(--purple); }
.endpoint-path {
font-family: var(--mono);
font-size: 14px;
font-weight: 500;
color: var(--text);
}
.endpoint-desc {
font-size: 13px;
color: var(--text-secondary);
margin-left: auto;
text-align: right;
}
.endpoint-body {
padding: 0 16px 16px;
border-top: 1px solid var(--border);
display: none;
}
.endpoint.open .endpoint-body { display: block; }
.endpoint-body > div { margin-top: 12px; }
/* Table */
table {
width: 100%;
border-collapse: collapse;
font-size: 14px;
}
th, td {
text-align: left;
padding: 8px 12px;
border-bottom: 1px solid var(--border);
}
th { color: var(--text-secondary); font-weight: 500; font-size: 12px; text-transform: uppercase; }
td { font-family: var(--mono); font-size: 13px; }
td.optional { color: var(--text-secondary); font-size: 12px; }
td .type { color: var(--orange); }
td .type-num { color: var(--accent); }
/* Code block */
pre {
background: #010409;
border: 1px solid var(--border);
border-radius: 6px;
padding: 12px 16px;
overflow-x: auto;
font-family: var(--mono);
font-size: 13px;
line-height: 1.5;
margin: 8px 0;
}
code { font-family: var(--mono); font-size: 13px; }
pre .comment { color: #8b949e; }
pre .string { color: #a5d6ff; }
pre .key { color: #79c0ff; }
pre .num { color: #79c0ff; }
pre .null { color: #d2a8ff; }
pre .bool { color: #d2a8ff; }
/* WS message box */
.ws-box {
background: #010409;
border: 1px solid var(--border);
border-radius: 6px;
padding: 12px 16px;
margin: 8px 0;
}
.ws-box .label {
font-size: 12px;
font-weight: 600;
color: var(--text-secondary);
text-transform: uppercase;
margin-bottom: 6px;
}
p { font-size: 14px; color: var(--text-secondary); margin-bottom: 8px; }
ul { padding-left: 20px; font-size: 14px; color: var(--text-secondary); }
li { margin-bottom: 4px; }
a { color: var(--accent); text-decoration: none; }
a:hover { text-decoration: underline; }
.note {
background: rgba(210, 153, 34, 0.1);
border: 1px solid rgba(210, 153, 34, 0.3);
border-radius: 6px;
padding: 10px 14px;
font-size: 13px;
color: var(--orange);
margin: 8px 0;
}
.toc { margin-bottom: 32px; }
.toc a {
display: inline-block;
padding: 4px 12px;
margin: 2px 0;
font-size: 14px;
color: var(--accent);
}
footer {
border-top: 1px solid var(--border);
padding: 20px 0;
text-align: center;
color: var(--text-secondary);
font-size: 13px;
}
</style>
</head>
<body>
<div class="container">
<header>
<h1><span>Chan</span> 数据提供商</h1>
<p class="subtitle">加密货币 K 线数据 HTTP + WebSocket API</p>
<div class="badge">v1.0.0 &nbsp;|&nbsp; <span>binance</span> &nbsp;|&nbsp; port 9009</div>
</header>
<nav class="toc">
<a href="#root">GET /</a>
<a href="#health">GET /health</a>
<a href="#timeframes">GET /timeframes</a>
<a href="#candles">GET /api/candles</a>
<a href="#websocket">WebSocket /ws</a>
<a href="#timeframes-ref">时间周期参考</a>
</nav>
<!-- ============ GET / ============ -->
<section id="root">
<h2>服务信息</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/</span>
<span class="endpoint-desc">服务基本信息</span>
</div>
<div class="endpoint-body">
<p>返回服务名称、交易所、交易对列表、可用周期及就绪状态。</p>
<h3>响应</h3>
<pre>{
<span class="key">"service"</span>: <span class="string">"Data Provider"</span>,
<span class="key">"exchange"</span>: <span class="string">"binance"</span>,
<span class="key">"symbols"</span>: [<span class="string">"BTC/USDT:USDT"</span>, <span class="string">"ETH/USDT:USDT"</span>, ...],
<span class="key">"base_timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, <span class="string">"1d"</span>, <span class="string">"1w"</span>],
<span class="key">"derived_timeframes"</span>: [<span class="string">"5m"</span>, <span class="string">"15m"</span>, <span class="string">"4h"</span>, ...],
<span class="key">"timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, ..., <span class="string">"5m"</span>, <span class="string">"15m"</span>, ...],
<span class="key">"ready"</span>: <span class="bool">true</span>
}</pre>
</div>
</div>
</section>
<!-- ============ GET /health ============ -->
<section id="health">
<h2>健康检查</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/health</span>
<span class="endpoint-desc">存活检查</span>
</div>
<div class="endpoint-body">
<p>返回服务健康状态,与 <code>/</code> 相同结构,适合负载均衡探测器。</p>
<h3>响应</h3>
<pre>{
<span class="key">"status"</span>: <span class="string">"ok"</span>,
<span class="key">"exchange"</span>: <span class="string">"binance"</span>,
<span class="key">"symbols"</span>: [<span class="string">"BTC/USDT:USDT"</span>, ...],
<span class="key">"ready"</span>: <span class="bool">true</span>,
...
}</pre>
</div>
</div>
</section>
<!-- ============ GET /timeframes ============ -->
<section id="timeframes">
<h2>可用周期</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/timeframes</span>
<span class="endpoint-desc">列出所有时间周期</span>
</div>
<div class="endpoint-body">
<p>返回基础周期(交易所直接拉取)和衍生周期(合成生成)的完整列表。</p>
<h3>响应</h3>
<pre>{
<span class="key">"base_timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, <span class="string">"1d"</span>, <span class="string">"1w"</span>],
<span class="key">"derived_timeframes"</span>: [<span class="string">"5m"</span>, <span class="string">"15m"</span>, <span class="string">"4h"</span>, ...],
<span class="key">"timeframes"</span>: [<span class="string">"1m"</span>, <span class="string">"1h"</span>, ..., <span class="string">"5m"</span>, <span class="string">"15m"</span>, ...]
}</pre>
</div>
</div>
</section>
<!-- ============ GET /api/candles ============ -->
<section id="candles">
<h2>查询 K 线</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method get">GET</span>
<span class="endpoint-path">/api/candles</span>
<span class="endpoint-desc">获取 OHLCV K 线数据</span>
</div>
<div class="endpoint-body">
<table>
<tr><th>参数</th><th>类型</th><th>必填</th><th>说明</th></tr>
<tr>
<td>symbol</td>
<td><span class="type">string</span></td>
<td></td>
<td>交易对,如 <code>BTC/USDT:USDT</code></td>
</tr>
<tr>
<td>tf</td>
<td><span class="type">string</span></td>
<td></td>
<td>时间周期,默认 <code>1m</code>。支持基础及衍生周期</td>
</tr>
<tr>
<td>start</td>
<td><span class="type-num">int</span></td>
<td class="optional">可选</td>
<td>开始时间戳(毫秒)</td>
</tr>
<tr>
<td>end</td>
<td><span class="type-num">int</span></td>
<td class="optional">可选</td>
<td>结束时间戳(毫秒)</td>
</tr>
<tr>
<td>limit</td>
<td><span class="type-num">int</span></td>
<td class="optional">可选</td>
<td>限制返回的 K 线数量(返回最后 N 根)</td>
</tr>
</table>
<div class="note">若不传 start/end,返回内存中全部数据(可能很多),建议搭配 limit 使用。</div>
<h3>请求示例</h3>
<pre><span class="comment"># 获取 BTC 最近 100 根 5 分钟 K 线</span>
GET /api/candles?symbol=BTC/USDT:USDT&tf=5m&limit=100
<span class="comment"># 指定时间范围</span>
GET /api/candles?symbol=ETH/USDT:USDT&tf=1h&start=1704067200000&end=1704153600000
<span class="comment"># 获取 4 小时周期(衍生周期)</span>
GET /api/candles?symbol=SOL/USDT:USDT&tf=4h&limit=50</pre>
<h3>响应</h3>
<p>返回 OHLCV 对象数组:</p>
<pre>[
{
<span class="key">"timestamp"</span>: <span class="num">1704067200000</span>,
<span class="key">"datetime"</span>: <span class="string">"2024-01-01T00:00:00Z"</span>,
<span class="key">"open"</span>: <span class="num">42850.12</span>,
<span class="key">"high"</span>: <span class="num">43100.00</span>,
<span class="key">"low"</span>: <span class="num">42780.50</span>,
<span class="key">"close"</span>: <span class="num">43050.80</span>,
<span class="key">"volume"</span>: <span class="num">125.34</span>
},
...
]</pre>
<h3>字段说明</h3>
<table>
<tr><th>字段</th><th>类型</th><th>说明</th></tr>
<tr><td>timestamp</td><td><span class="type-num">int</span></td><td>UTC 毫秒时间戳</td></tr>
<tr><td>datetime</td><td><span class="type">string</span></td><td>ISO 8601 格式(末尾 Z</td></tr>
<tr><td>open</td><td><span class="type-num">float</span></td><td>开盘价</td></tr>
<tr><td>high</td><td><span class="type-num">float</span></td><td>最高价</td></tr>
<tr><td>low</td><td><span class="type-num">float</span></td><td>最低价</td></tr>
<tr><td>close</td><td><span class="type-num">float</span></td><td>收盘价</td></tr>
<tr><td>volume</td><td><span class="type-num">float</span></td><td>成交量</td></tr>
</table>
</div>
</div>
</section>
<!-- ============ WebSocket ============ -->
<section id="websocket">
<h2>WebSocket 实时推送</h2>
<div class="endpoint open">
<div class="endpoint-header" onclick="this.parentElement.classList.toggle('open')">
<span class="method ws">WS</span>
<span class="endpoint-path">/ws</span>
<span class="endpoint-desc">实时 K 线订阅</span>
</div>
<div class="endpoint-body">
<p>连接 WebSocket 后,通过 JSON 消息进行订阅管理。服务端在数据更新时主动推送最新 K 线。</p>
<h3>客户端 → 服务端</h3>
<div class="ws-box">
<div class="label">订阅 K 线</div>
<pre>{
<span class="key">"action"</span>: <span class="string">"subscribe"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>
}</pre>
</div>
<div class="ws-box">
<div class="label">取消订阅</div>
<pre>{
<span class="key">"action"</span>: <span class="string">"unsubscribe"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>
}</pre>
</div>
<div class="ws-box">
<div class="label">心跳 Ping</div>
<pre>{ <span class="key">"action"</span>: <span class="string">"ping"</span> }</pre>
</div>
<h3>服务端 → 客户端</h3>
<div class="ws-box">
<div class="label">订阅确认</div>
<pre>{
<span class="key">"type"</span>: <span class="string">"subscribed"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>
}</pre>
</div>
<div class="ws-box">
<div class="label">初始快照(订阅后立即推送最近 500 根 K 线)</div>
<pre>{
<span class="key">"type"</span>: <span class="string">"snapshot"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>,
<span class="key">"data"</span>: [ ... ]
}</pre>
</div>
<div class="ws-box">
<div class="label">K 线更新(增量推送最近 2 根)</div>
<pre>{
<span class="key">"type"</span>: <span class="string">"kline"</span>,
<span class="key">"symbol"</span>: <span class="string">"BTC/USDT:USDT"</span>,
<span class="key">"timeframe"</span>: <span class="string">"1m"</span>,
<span class="key">"data"</span>: [ ... ]
}</pre>
</div>
<div class="ws-box">
<div class="label">Pong 响应</div>
<pre>{ <span class="key">"type"</span>: <span class="string">"pong"</span> }</pre>
</div>
<div class="ws-box">
<div class="label">错误消息</div>
<pre>{ <span class="key">"type"</span>: <span class="string">"error"</span>, <span class="key">"message"</span>: <span class="string">"..."</span> }</pre>
</div>
<h3>JavaScript 示例</h3>
<pre><span class="comment">// 连接</span>
<span class="key">const</span> ws = <span class="string">new WebSocket("ws://localhost:9009/ws")</span>;
ws.<span class="key">onopen</span> = () => {
<span class="comment">// 订阅 BTC 1m K 线</span>
ws.send(JSON.stringify({
action: <span class="string">"subscribe"</span>,
symbol: <span class="string">"BTC/USDT:USDT"</span>,
timeframe: <span class="string">"1m"</span>
}));
};
ws.<span class="key">onmessage</span> = (event) => {
<span class="key">const</span> msg = JSON.parse(event.data);
<span class="key">if</span> (msg.type === <span class="string">"kline"</span>) {
console.log(msg.data); <span class="comment">// 最新 K 线数组</span>
}
};</pre>
</div>
</div>
</section>
<!-- ============ 时间周期参考 ============ -->
<section id="timeframes-ref">
<h2>时间周期参考</h2>
<p>以下是完整的周期对照表:</p>
<table>
<tr><th>基础周期</th><th>合成衍生周期</th></tr>
<tr><td><code>1m</code></td><td><code>2m, 3m, 4m, 5m, 10m, 15m, 20m, 25m, 30m, 45m</code></td></tr>
<tr><td><code>1h</code></td><td><code>2h, 3h, 4h, 5h, 6h, 7h, 8h, 9h, 10h, 11h, 12h, 16h, 20h</code></td></tr>
<tr><td><code>1d</code></td><td><code>2d, 3d, 4d, 5d, 6d</code></td></tr>
<tr><td><code>1w</code></td><td><code>2w, 3w</code></td></tr>
</table>
<p>衍生周期由对应基础周期的 K 线通过 OHLCV 聚合合成,查询方式与基础周期完全一致。</p>
</section>
<footer>
Chan Data Provider &mdash; Built with FastAPI + ccxt + pandas
</footer>
</div>
<script>
<span class="comment">// 展开/折叠端点详情</span>
document.querySelectorAll('.endpoint-header').forEach(el => {
el.addEventListener('click', () => {
el.parentElement.classList.toggle('open');
});
});
<span class="comment">// 默认展开所有端点</span>
document.querySelectorAll('.endpoint').forEach(el => el.classList.add('open'));
</script>
</body>
</html>
+417
View File
@@ -0,0 +1,417 @@
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Chan 数据提供商</title>
<style>
:root {
--bg: #0d1117;
--surface: #161b22;
--border: #30363d;
--text: #e6edf3;
--text-secondary: #8b949e;
--accent: #58a6ff;
--green: #3fb950;
--orange: #d29922;
--red: #f85149;
--purple: #bc8cff;
--font: -apple-system, BlinkMacSystemFont, "Segoe UI", Helvetica, Arial, sans-serif;
--mono: "SF Mono", "Fira Code", "Consolas", monospace;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: var(--font);
background: var(--bg);
color: var(--text);
line-height: 1.6;
min-height: 100vh;
}
.container { max-width: 1000px; margin: 0 auto; padding: 32px 24px; }
/* Header */
header {
display: flex;
align-items: center;
justify-content: space-between;
padding-bottom: 20px;
border-bottom: 1px solid var(--border);
margin-bottom: 28px;
}
header .brand {
display: flex;
align-items: center;
gap: 14px;
}
header .logo {
display: flex;
align-items: center;
justify-content: center;
width: 44px; height: 44px;
border-radius: 10px;
background: linear-gradient(135deg, #1c3d5a 0%, #2d1b5e 100%);
border: 1px solid var(--border);
font-size: 20px; font-weight: 700;
color: var(--accent);
}
header .brand h1 { font-size: 20px; font-weight: 600; }
header .brand h1 span { color: var(--accent); }
header .brand .sub { font-size: 12px; color: var(--text-secondary); }
.header-time {
font-family: var(--mono);
font-size: 13px;
color: var(--text-secondary);
}
/* Overview cards */
.overview {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(140px, 1fr));
gap: 10px;
margin-bottom: 24px;
}
.ov-card {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
padding: 14px 16px;
text-align: center;
}
.ov-card .ov-label { font-size: 12px; color: var(--text-secondary); margin-bottom: 4px; }
.ov-card .ov-value {
font-family: var(--mono);
font-size: 18px;
font-weight: 600;
}
.ov-card .ov-value.green { color: var(--green); }
.ov-card .ov-value.orange { color: var(--orange); }
.ov-card .ov-value.accent { color: var(--accent); }
.ov-card .ov-value.purple { color: var(--purple); }
/* Section */
.section {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 8px;
margin-bottom: 12px;
overflow: hidden;
}
.section-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 12px 16px;
border-bottom: 1px solid var(--border);
cursor: pointer;
user-select: none;
}
.section-header:hover { background: rgba(255,255,255,0.02); }
.section-header h2 { font-size: 14px; font-weight: 600; }
.section-header .count-badge {
font-size: 12px;
font-family: var(--mono);
color: var(--text-secondary);
background: var(--bg);
padding: 2px 8px;
border-radius: 10px;
}
.section-body { padding: 8px 16px 12px; }
/* Status dot */
.dot {
display: inline-block;
width: 8px; height: 8px;
border-radius: 50%;
flex-shrink: 0;
}
.dot.green { background: var(--green); box-shadow: 0 0 6px #3fb95060; }
.dot.orange { background: var(--orange); box-shadow: 0 0 6px #d2992260; }
.dot.red { background: var(--red); box-shadow: 0 0 6px #f8514960; }
.dot.gray { background: var(--text-secondary); }
/* Symbol row */
.symbol-row {
display: flex;
align-items: center;
gap: 12px;
padding: 8px 0;
border-bottom: 1px solid rgba(48,54,61,0.5);
font-size: 14px;
}
.symbol-row:last-child { border-bottom: none; }
.symbol-row .name { font-family: var(--mono); font-size: 13px; min-width: 140px; }
.symbol-row .tag {
font-size: 11px;
font-family: var(--mono);
padding: 1px 6px;
border-radius: 4px;
background: rgba(88,166,255,0.1);
color: var(--accent);
}
/* TF chips */
.tf-list { display: flex; flex-wrap: wrap; gap: 4px; }
.tf-chip {
font-family: var(--mono);
font-size: 12px;
padding: 2px 8px;
border-radius: 4px;
background: var(--bg);
border: 1px solid var(--border);
color: var(--text-secondary);
}
.tf-chip.base { border-color: #58a6ff40; color: var(--accent); }
.tf-chip.derived { border-color: #bc8cff40; color: var(--purple); }
/* Links bar */
.links-bar {
display: flex;
gap: 8px;
flex-wrap: wrap;
margin: 16px 0 24px;
}
.links-bar a {
display: flex;
align-items: center;
gap: 6px;
padding: 6px 14px;
border-radius: 6px;
font-size: 13px;
text-decoration: none;
background: var(--surface);
border: 1px solid var(--border);
color: var(--text);
transition: border-color 0.15s;
}
.links-bar a:hover { border-color: var(--accent); }
/* Footer */
footer {
border-top: 1px solid var(--border);
padding: 16px 0;
margin-top: 32px;
text-align: center;
color: var(--text-secondary);
font-size: 13px;
}
.fade-in { animation: fadeIn 0.3s ease-in; }
@keyframes fadeIn { from { opacity: 0; } to { opacity: 1; } }
</style>
</head>
<body>
<div class="container">
<header>
<div class="brand">
<div class="logo">C</div>
<div>
<h1><span>Chan</span> 数据提供商</h1>
<div class="sub">加密货币 K 线数据服务</div>
</div>
</div>
<div class="header-time" id="header-time"></div>
</header>
<!-- Overview -->
<div class="overview" id="overview">
<div class="ov-card"><div class="ov-label">服务状态</div><div class="ov-value" id="ov-status">加载中...</div></div>
<div class="ov-card"><div class="ov-label">交易所</div><div class="ov-value accent" id="ov-exchange"></div></div>
<div class="ov-card"><div class="ov-label">交易对</div><div class="ov-value" id="ov-symbols"></div></div>
<div class="ov-card"><div class="ov-label">基础周期</div><div class="ov-value accent" id="ov-base-tf"></div></div>
<div class="ov-card"><div class="ov-label">衍生周期</div><div class="ov-value purple" id="ov-derived-tf"></div></div>
<div class="ov-card"><div class="ov-label">数据就绪</div><div class="ov-value" id="ov-ready"></div></div>
</div>
<!-- Links -->
<div class="links-bar">
<a href="/api/docs">📖 API 文档</a>
<a href="/docs">📋 Swagger UI</a>
<a href="/redoc">📄 ReDoc</a>
<a href="/api/candles?symbol=BTC/USDT:USDT&tf=1m&limit=5" target="_blank">📊 BTC 1m 示例</a>
<a href="/api/candles?symbol=ETH/USDT:USDT&tf=4h&limit=10" target="_blank">📊 ETH 4h 示例</a>
</div>
<!-- Symbols -->
<div class="section" id="section-symbols">
<div class="section-header" onclick="this.parentElement.classList.toggle('collapsed')">
<h2>📈 交易对</h2>
<span class="count-badge" id="sym-count">0</span>
</div>
<div class="section-body" id="symbol-list">
<div style="color:var(--text-secondary);font-size:13px;">加载中...</div>
</div>
</div>
<!-- Timeframes -->
<div class="section">
<div class="section-header" onclick="this.parentElement.classList.toggle('collapsed')">
<h2>⏱ 时间周期</h2>
<span class="count-badge" id="tf-total">0</span>
</div>
<div class="section-body">
<div style="margin-bottom:8px;font-size:13px;color:var(--text-secondary);">基础周期(交易所直拉)</div>
<div class="tf-list" id="base-tf-list"></div>
<div style="margin:10px 0 8px;font-size:13px;color:var(--text-secondary);">衍生周期(内存合成)</div>
<div class="tf-list" id="derived-tf-list"></div>
</div>
</div>
<!-- Quick query -->
<div class="section">
<div class="section-header">
<h2>⚡ 快速查询</h2>
</div>
<div class="section-body">
<div style="display:flex;gap:8px;flex-wrap:wrap;align-items:end;">
<div>
<div style="font-size:12px;color:var(--text-secondary);margin-bottom:4px;">交易对</div>
<select id="q-symbol" style="background:var(--bg);border:1px solid var(--border);color:var(--text);padding:6px 10px;border-radius:6px;font-family:var(--mono);font-size:13px;"></select>
</div>
<div>
<div style="font-size:12px;color:var(--text-secondary);margin-bottom:4px;">周期</div>
<select id="q-tf" style="background:var(--bg);border:1px solid var(--border);color:var(--text);padding:6px 10px;border-radius:6px;font-family:var(--mono);font-size:13px;"></select>
</div>
<div>
<div style="font-size:12px;color:var(--text-secondary);margin-bottom:4px;">数量</div>
<select id="q-limit" style="background:var(--bg);border:1px solid var(--border);color:var(--text);padding:6px 10px;border-radius:6px;font-family:var(--mono);font-size:13px;">
<option>5</option><option selected>10</option><option>20</option><option>50</option>
</select>
</div>
<button onclick="quickQuery()" style="background:var(--accent);color:#fff;border:none;padding:6px 18px;border-radius:6px;cursor:pointer;font-size:13px;font-weight:600;">查询</button>
</div>
<pre id="q-result" style="margin-top:10px;display:none;"></pre>
</div>
</div>
</div>
<footer>
Chan Data Provider &nbsp;·&nbsp; Built with FastAPI + ccxt + pandas &nbsp;·&nbsp;
更新于 <span id="footer-time"></span>
</footer>
<script>
let healthData = null;
function fmtTime(ts) {
return new Date(ts).toLocaleString('zh-CN', { timeZone: 'UTC', hour12: false }) + ' UTC';
}
async function loadHealth() {
try {
const res = await fetch('/health');
healthData = await res.json();
renderHealth(healthData);
} catch {
document.getElementById('ov-status').textContent = '无法连接';
document.getElementById('ov-status').style.color = 'var(--red)';
document.getElementById('ov-ready').textContent = '断开';
document.getElementById('ov-ready').style.color = 'var(--red)';
}
}
function renderHealth(d) {
const now = Date.now();
document.getElementById('header-time').textContent = fmtTime(now);
document.getElementById('footer-time').textContent = fmtTime(now);
// Overview
const statusEl = document.getElementById('ov-status');
statusEl.textContent = '运行中';
statusEl.style.color = 'var(--green)';
document.getElementById('ov-exchange').textContent = d.exchange || '—';
const symCount = (d.symbols || []).length;
const symEl = document.getElementById('ov-symbols');
symEl.textContent = symCount + ' 个';
symEl.style.color = 'var(--accent)';
document.getElementById('ov-base-tf').textContent = (d.base_timeframes || []).length + ' 个';
document.getElementById('ov-derived-tf').textContent = (d.derived_timeframes || []).length + ' 个';
const readyEl = document.getElementById('ov-ready');
if (d.ready) {
readyEl.textContent = '已就绪';
readyEl.style.color = 'var(--green)';
} else {
readyEl.textContent = '同步中...';
readyEl.style.color = 'var(--orange)';
setTimeout(loadHealth, 2000);
}
// Symbols
const symList = document.getElementById('symbol-list');
const symCountEl = document.getElementById('sym-count');
symCountEl.textContent = symCount;
if (d.symbols && d.symbols.length > 0) {
symList.innerHTML = d.symbols.map(s => `
<div class="symbol-row">
<span class="dot green"></span>
<span class="name">${s}</span>
<span class="tag">${d.exchange || '—'}</span>
<a href="/api/candles?symbol=${encodeURIComponent(s)}&tf=1m&limit=5" target="_blank" style="margin-left:auto;font-size:12px;color:var(--accent);text-decoration:none;">1m →</a>
</div>
`).join('');
} else {
symList.innerHTML = '<div style="color:var(--text-secondary);font-size:13px;">暂无交易对</div>';
}
// Timeframes
document.getElementById('tf-total').textContent = (d.timeframes || []).length;
const baseList = document.getElementById('base-tf-list');
if (d.base_timeframes) {
baseList.innerHTML = d.base_timeframes.map(t => `<span class="tf-chip base">${t}</span>`).join('');
}
const derivedList = document.getElementById('derived-tf-list');
if (d.derived_timeframes) {
derivedList.innerHTML = d.derived_timeframes.map(t => `<span class="tf-chip derived">${t}</span>`).join('');
}
// Populate query selects
const symSelect = document.getElementById('q-symbol');
if (d.symbols && symSelect.options.length === 0) {
d.symbols.forEach(s => {
const opt = document.createElement('option');
opt.value = s;
opt.textContent = s;
symSelect.appendChild(opt);
});
}
const tfSelect = document.getElementById('q-tf');
if (d.timeframes && tfSelect.options.length === 0) {
d.timeframes.forEach(t => {
const opt = document.createElement('option');
opt.value = t;
opt.textContent = t;
tfSelect.appendChild(opt);
});
}
}
async function quickQuery() {
const symbol = document.getElementById('q-symbol').value;
const tf = document.getElementById('q-tf').value;
const limit = document.getElementById('q-limit').value;
const url = `/api/candles?symbol=${encodeURIComponent(symbol)}&tf=${tf}&limit=${limit}`;
const pre = document.getElementById('q-result');
pre.style.display = 'block';
pre.textContent = '查询中...';
try {
const res = await fetch(url);
const data = await res.json();
pre.textContent = JSON.stringify(data, null, 2);
} catch {
pre.textContent = '查询失败';
}
}
loadHealth();
setInterval(loadHealth, 10000);
</script>
</body>
</html>
+41 -17
View File
@@ -19,6 +19,7 @@ import ccxt # type: ignore
import pandas as pd # type: ignore import pandas as pd # type: ignore
from fastapi import FastAPI, HTTPException, Query, WebSocket, WebSocketDisconnect from fastapi import FastAPI, HTTPException, Query, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import HTMLResponse
import uvicorn import uvicorn
from technical.util import resample_to_interval from technical.util import resample_to_interval
@@ -405,11 +406,23 @@ class DataProvider:
return results return results
def initialize(self) -> None: def initialize(self) -> None:
"""阻塞式启动:加载本地、从倒数第二根或配置起点补历史、写盘并 set _ready""" """快速启动:加载本地磁盘已有数据到内存,然后立即标记就绪,不阻塞服务"""
logger.info("开始初始化数据提供商") logger.info("开始加载本地数据")
for symbol in self.symbols: for symbol in self.symbols:
for timeframe in self.timeframes: for timeframe in self.timeframes:
existing = self._load_local(symbol, timeframe) existing = self._load_local(symbol, timeframe)
with self._lock:
self.data.setdefault(symbol, {})[timeframe] = existing
self._ready.set()
logger.info("本地数据加载完成,服务已就绪")
def run_initial_history_fetch(self) -> None:
"""后台一次性拉取所有 symbol/tf 的历史数据(从本地末根或配置起点到当前),然后落盘。"""
logger.info("开始后台历史数据拉取")
for symbol in self.symbols:
for timeframe in self.timeframes:
with self._lock:
existing = list(self.data.get(symbol, {}).get(timeframe, []))
tf_ms = TIMEFRAME_TO_MS[timeframe] tf_ms = TIMEFRAME_TO_MS[timeframe]
last_ts = existing[-1]["timestamp"] if existing else None last_ts = existing[-1]["timestamp"] if existing else None
if last_ts is not None: if last_ts is not None:
@@ -433,10 +446,10 @@ class DataProvider:
history = self._fetch_history(symbol, timeframe, fetch_since) history = self._fetch_history(symbol, timeframe, fetch_since)
merged = self._merge_candles(timeframe, existing, history) merged = self._merge_candles(timeframe, existing, history)
with self._lock: with self._lock:
self.data.setdefault(symbol, {})[timeframe] = merged self.data[symbol][timeframe] = merged
self._write_to_disk(symbol, timeframe, merged) self._write_to_disk(symbol, timeframe, merged)
self._ready.set() self._notify_update(symbol, timeframe)
logger.info("数据初始化完成") logger.info("后台历史数据拉取完成")
def resample_df(self, df: pd.DataFrame, interval: int) -> pd.DataFrame: def resample_df(self, df: pd.DataFrame, interval: int) -> pd.DataFrame:
"""将基础周期 DataFrame 聚合为 interval 分钟周期(freqtrade technical.util)。""" """将基础周期 DataFrame 聚合为 interval 分钟周期(freqtrade technical.util)。"""
@@ -788,8 +801,15 @@ def create_app(provider: DataProvider) -> FastAPI:
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
ws_manager.set_loop(loop) ws_manager.set_loop(loop)
provider.on_update(_on_data_update) provider.on_update(_on_data_update)
# 快速加载本地数据后立即就绪,不阻塞服务启动
await loop.run_in_executor(None, provider.initialize) await loop.run_in_executor(None, provider.initialize)
provider.start_background_workers() provider.start_background_workers()
# 后台拉取历史数据补齐(不阻塞 HTTP/WS 服务)
threading.Thread(
target=provider.run_initial_history_fetch,
name="initial-history-fetch",
daemon=True,
).start()
try: try:
yield yield
finally: finally:
@@ -840,18 +860,15 @@ def create_app(provider: DataProvider) -> FastAPI:
data = provider.get_klines(symbol=symbol, timeframe=tf, start_time=start, end_time=end, limit=limit) data = provider.get_klines(symbol=symbol, timeframe=tf, start_time=start, end_time=end, limit=limit)
return data return data
@app.get("/") homepage_path = Path(__file__).resolve().parent / "homepage.html"
async def root() -> Dict[str, object]: docs_path = Path(__file__).resolve().parent / "api_docs.html"
"""根路径:服务名、交易所、交易对与可用周期(含 ready 标志)。"""
return { @app.get("/", response_class=HTMLResponse)
"service": "Data Provider", async def root():
"exchange": provider.exchange_name, """服务主页。"""
"symbols": provider.symbols, if homepage_path.exists():
"base_timeframes": provider.timeframes, return HTMLResponse(content=homepage_path.read_text(encoding="utf-8"))
"derived_timeframes": provider.get_derived_timeframes(), return HTMLResponse(content="<h1>主页页面未找到</h1>", status_code=404)
"timeframes": provider.get_available_timeframes(),
"ready": provider.is_ready(),
}
@app.websocket("/ws") @app.websocket("/ws")
async def websocket_endpoint(ws: WebSocket): async def websocket_endpoint(ws: WebSocket):
@@ -921,6 +938,13 @@ def create_app(provider: DataProvider) -> FastAPI:
finally: finally:
await ws_manager.disconnect(ws) await ws_manager.disconnect(ws)
@app.get("/api/docs", response_class=HTMLResponse, include_in_schema=False)
async def api_docs():
"""返回自定义 API 文档页面。"""
if docs_path.exists():
return HTMLResponse(content=docs_path.read_text(encoding="utf-8"))
return HTMLResponse(content="<h1>API 文档页面未找到</h1>", status_code=404)
return app return app