research: 影子交易器落在 Hummingbot 上,并修掉 Bitget 连接器的换根延迟

1m 腿的滑点余量只有几个 bp,所以要测的必须是生产路径的滑点——换个运行时
测出来的数就不作数。框架因此从「滑点已知后再定」提前到测量阶段就定为
Hummingbot(Spot/Perp 连接器均 v2.0,Bitget 是 Foundation Partner)。

新增 research/live/。前置测量:

- bench_compute.py 本机算力,1m 单币 0.318s、三币串行 1.38s
- venue_parity.py Binance 与 Bitget 同根信号重合仅 14.6~42.6%
- signal_sensitivity.py 0.25bp 扰动就换掉一半信号
- aggregate_robustness.py 但总体期望不降——脆的是信号身份,不是 alpha
- bitget_baseline.py 因此改用 Bitget 原生基线定预算:余量 BTC -0.13bp、
  ETH +4.02bp、SOL +2.92bp。BTC 本就为负,只作延迟测量的参照物

运行时选型:

- parity_env.py 容器与本机信号逐一相同(下标、中枢数、checksum 全等),
  容器内 0.26s/币反而更快。故 chanlun 直接挂载进容器,不必另起信号服务。
  装进现有 .venv 那条路走不通:Hummingbot 要 numba>=0.61.2 与
  aiohttp<3.14,与本机 Python 3.14 冲突
- latency_ccxt.py / latency_hummingbot.py / latency_compare.py 初测显示
  Hummingbot 比 ccxt.pro 慢约 1030ms,90 根逐根配对里 80~97% 更慢
- probe_ws_action.py 否掉「丢弃 snapshot」的猜测:换根首条就是 update
- probe_hb_vs_raw.py 与 latency_attribute.py 四路归因——容器网络 2~18ms、
  Hummingbot 处理 -10~-30ms,1350~1480ms 全落在解析方式上
- probe_ws_payload.py 定位根因:Bitget 换根会推一条带两根的消息
  [上一根, 新一根],而上游取 data["data"][0] 拿到的是上一根,新一根要等
  下一条单元素消息

修复:

- patched_candles.py 处理消息里的全部元素。不能简单改成 [-1]——那样上一根
  的收盘价会永远停在换根前约 1 秒的那次推送上,而信号对 0.25bp 都敏感
- verify_patch.py 60 根配对验证:拿回 1060~1090ms,与原始 WS 只差 5~14ms
  已贴理论下限,19 根已收盘 K 线 OHLCV 逐根未变。折算 ETH 省 0.54bp、
  SOL 省 0.42bp。此 bug 值得向上游反馈

影子交易器:

- shadow_hb.py 不下单,读连接器真实盘口按仓位吃单深度算成交价,与次根开盘价
  (回测 entry_delay=1 的口径)相减,分解成延迟漂移、盘口价差、深度冲击。
  盘口 10Hz 滚动缓冲 30 秒,把延迟变成自变量:每个信号记 0.5/1/2/5s 与实际
  算完时刻各一个滑点值,本机算得慢也不影响能读出的曲线
- shadow_signal.py 信号计算隔离到子进程。0.26s 是纯 CPU 且 chanlun 受 GIL
  限制,放进 asyncio 循环会把行情处理一起卡住
- shadow_report.py 首日延迟门槛与滑点曲线报表

不用 paper trade 测滑点:它的成交由 Hummingbot 自己的撮合模型模拟,
测出来是模型行为而非市场行为。

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Ubuntu
2026-08-27 23:51:57 +08:00
co-authored by Cursor
parent 66061f79a1
commit 7e339d2a54
44 changed files with 11946 additions and 0 deletions
+186
View File
@@ -0,0 +1,186 @@
"""定位 Hummingbot 那 1030ms 究竟出在处理链路还是容器网络。
前一轮对照有两个变量同时在变:运行时(Hummingbot vs ccxt.pro)和位置
(容器内 vs 宿主机)。所以 1030ms 无法归因。原始 WS 探针已否掉「丢弃
snapshot」这个猜测——换根首条推送就是 update,Hummingbot 确实收到了。
本脚本在**容器内同一个进程**里并行跑两条路径,共用一个时钟、一条网络:
A. Hummingbot 的 BitgetPerpetualCandles,轮询 _candles 尾部时间戳
B. 一条原始 aiohttp WS,直接订阅 candle1m
两条路径对同一根 K 线各记一个到达时刻,差值即 Hummingbot 处理链路的净开销。
若 B 也慢,则是容器网络,与框架无关。
在容器内运行:
docker run --rm -v $PWD:/repo:ro -v $PWD/research/out:/out \
-w /home/hummingbot -e PYTHONPATH=/home/hummingbot \
--entrypoint /opt/conda/envs/hummingbot/bin/python \
hummingbot/hummingbot:latest /repo/research/live/probe_hb_vs_raw.py --minutes 20
"""
from __future__ import annotations
import argparse
import asyncio
import csv
import json
import time
from pathlib import Path
SYMS = ("BTC", "ETH", "SOL")
WSS = "wss://ws.bitget.com/v2/ws/public"
def out_dir() -> Path:
p = Path("/out")
return p if p.is_dir() else Path(__file__).resolve().parents[1] / "out"
async def raw_ws(rec: dict, stop: asyncio.Event) -> None:
"""B 路径:原始 WS,收到第一条带新时间戳的推送就记时刻。"""
import aiohttp
payload = {"op": "subscribe",
"args": [{"instType": "USDT-FUTURES", "channel": "candle1m",
"instId": f"{s}USDT"} for s in SYMS]}
while not stop.is_set():
try:
async with aiohttp.ClientSession() as sess, \
sess.ws_connect(WSS, heartbeat=20) as ws:
await ws.send_str(json.dumps(payload))
print(" [raw] 已订阅", flush=True)
last: dict[str, int] = {}
while not stop.is_set():
msg = await ws.receive()
if msg.type is not aiohttp.WSMsgType.TEXT:
# 关闭类消息必须跳出重连,否则 receive() 会立刻返回造成空转
print(f" [raw] 非文本消息 {msg.type},重连", flush=True)
break
if msg.data == "pong":
continue
d = json.loads(msg.data)
if "data" not in d or "arg" not in d:
continue
sym = d["arg"]["instId"].replace("USDT", "")
kts = int(d["data"][0][0])
if last.get(sym) is not None and kts > last[sym]:
rec.setdefault((sym, kts), {})["raw_ms"] = \
int(time.time() * 1000)
last[sym] = max(kts, last.get(sym, 0))
except asyncio.CancelledError:
raise
except Exception as e:
print(f" [raw] 异常 {type(e).__name__}: {e}", flush=True)
# 无论是正常跳出还是异常,重连前都歇一下,避免服务端持续拒绝时打成风暴
if not stop.is_set():
await asyncio.sleep(1)
async def hb_feed(rec: dict, stop: asyncio.Event) -> None:
"""A 路径:Hummingbot candles feed10ms 轮询 deque 尾部。"""
from hummingbot.data_feed.candles_feed.candles_factory import CandlesFactory
from hummingbot.data_feed.candles_feed.data_types import CandlesConfig
feeds = {}
for s in SYMS:
cfg = CandlesConfig(connector="bitget_perpetual",
trading_pair=f"{s}-USDT", interval="1m",
max_records=20)
f = CandlesFactory.get_candle(cfg)
f.start()
feeds[s] = f
print(" [hb] 已订阅", flush=True)
t0 = time.time()
while time.time() - t0 < 120 and not all(f.ready for f in feeds.values()):
await asyncio.sleep(0.5)
print(f" [hb] 回填完成 {time.time() - t0:.1f}s", flush=True)
last = {s: (int(f._candles[-1][0]) if len(f._candles) else None)
for s, f in feeds.items()}
while not stop.is_set():
for s, f in feeds.items():
if not len(f._candles):
continue
newest = int(f._candles[-1][0])
if last[s] is not None and newest > last[s]:
# HB 的时间戳是秒,统一成毫秒好与原始 WS 的 kline_ts 对齐
kts = newest * 1000 if newest < 1e12 else newest
rec.setdefault((s, kts), {})["hb_ms"] = int(time.time() * 1000)
last[s] = newest
await asyncio.sleep(0.01)
for f in feeds.values():
f.stop()
async def main_async(minutes: int, raw_only: bool = False) -> None:
rec: dict = {}
stop = asyncio.Event()
tasks = [asyncio.create_task(raw_ws(rec, stop))]
if not raw_only:
tasks.append(asyncio.create_task(hb_feed(rec, stop)))
where = "宿主机 raw 单跑" if raw_only else "容器内同进程对照"
print(f"[{where}] {SYMS} · 跑 {minutes} 分钟", flush=True)
deadline = time.time() + minutes * 60
reported = set()
while time.time() < deadline:
await asyncio.sleep(2)
for k, v in rec.items():
if k in reported or "raw_ms" not in v or "hb_ms" not in v:
continue
reported.add(k)
print(f" {k[0]} @{k[1]}: raw->hb 延后 {v['hb_ms'] - v['raw_ms']}ms",
flush=True)
stop.set()
for t in tasks:
t.cancel()
await asyncio.gather(*tasks, return_exceptions=True)
if raw_only:
# 只落盘 raw 到达时刻,供与容器侧按 kline_ts 对齐
f = out_dir() / "probe_raw_host.csv"
with f.open("w", newline="") as fh:
w = csv.writer(fh)
w.writerow(["sym", "kline_ts", "raw_ms"])
for (s, k), v in sorted(rec.items(), key=lambda x: x[0][1]):
if "raw_ms" in v:
w.writerow([s, k, v["raw_ms"]])
print(f"\n宿主机 raw 样本 {sum('raw_ms' in v for v in rec.values())}")
print(f"产物写入 {f}")
return
f = out_dir() / "probe_hb_vs_raw.csv"
both = [(s, k, v) for (s, k), v in rec.items()
if "raw_ms" in v and "hb_ms" in v]
with f.open("w", newline="") as fh:
w = csv.writer(fh)
w.writerow(["sym", "kline_ts", "raw_ms", "hb_ms", "delta_ms"])
for s, k, v in sorted(both, key=lambda x: x[1]):
w.writerow([s, k, v["raw_ms"], v["hb_ms"],
v["hb_ms"] - v["raw_ms"]])
print(f"\n########## 同进程内 raw WS 与 Hummingbot 的到达差 ##########")
print(f"(正数 = Hummingbot 更慢;配对 {len(both)} 根)")
for s in SYMS:
d = sorted(v["hb_ms"] - v["raw_ms"] for ss, _, v in both if ss == s)
if not d:
print(f" {s}: 无配对样本")
continue
print(f" {s}: n={len(d)} 中位 {d[len(d) // 2]}ms "
f"P90 {d[int(len(d) * .9)]}ms 最小 {d[0]}ms 最大 {d[-1]}ms")
print(f"\n判读:中位接近 0 说明 1030ms 来自容器网络或宿主机对照本身;"
f"中位接近 1000ms 说明是 Hummingbot 处理链路的净开销。")
print(f"产物写入 {f}")
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("--minutes", type=int, default=20)
ap.add_argument("--raw-only", action="store_true",
help="不加载 hummingbot,只跑原始 WS(供宿主机对照)")
a = ap.parse_args()
asyncio.run(main_async(a.minutes, a.raw_only))
if __name__ == "__main__":
main()