引擎四处「算了没人要的东西」，append_bar 13.9ms → 6.6ms
服务端把增量上线后回传两个热点：add_indicators 为加一根重算全表（占 34%）、 cal_bi_list 整表重扫（51%）。顺着查下来四处都不是算法慢，是算了没人读的结果。 1. cal_trend 挂到 lean 下。它不是笔的依赖（bi.py:221 在它自己的循环里读自身 序列状态），服务端 verify_incr_parity 三币 1800 根已对拍定论。web 走非 lean，klc_trend 图层不受影响。 2. check_fx_pattern 删掉拼完就丢的字符串。它把 klu.to_string() 拼成 p 只为 一行注释掉的 print——2000 根上近 3 万次 f-string 加 6 万次 enum 格式化， 而且在 cal_bi_list 内层。klu.pattern 只被 cal_klu_pattern 自己的双K/三K 判定读，不出模块不进 web，所以整个调用在 lean 下也跳过。 3. ChanBI.add_klc 去二次方。去重原本线性扫 klc_list，且每加一根就把整笔所有 KLU 的 macdhist 重累一遍，往一笔加 k 根是 O(k²)。改成下标集合加 macd_hist/macd_div 惰性求值。这两个值只有背驰判定（bsp.py）读，lean 下 bsp 根本不算。 4. add_indicators 批量挂列。2001 行上 TA 计算合计只有 2.5ms，而 30 多次 df['x']= 要 3.6ms——开销大头是 BlockManager 逐列插入不是计算，改为一次 concat。cal_volume_ratio 里为算一列 rolling 而 copy() 整张 40 列表，一并去掉。 实测（本机，2001 根窗口。服务端基线 21.8ms 是另一台机器，别直接比绝对值）： append_bar 13.9 → 6.6ms └ rebuild_bi_zs 8.7 → 2.8ms └ add_indicators 4.4 → 3.5ms TF_DF lean 49.8 → 32.9ms TF_DF full 72.7 → 64.9ms 对拍用 git worktree 检出改动前的提交，同一份 BTC 1m 4000 根跑 38 项指纹： full 模式 19 项全部一致（web 那条路没动）；lean 模式差 2 项，正是设计要它差 的 klc.trend 和 klu.pattern，而 lean 下 bi/zs/seg/bsp/dataframe 全部一致—— 这就是「这两个字段没人读」的实测证据：打空它们，下游一位不变。 瓶颈已经换位置了。新增 probe_inner.py 拆 inner_ms 分档：本机 TF_DF 两条腿占 70%、build_htf_zones 13%、htf_fx_timeline 6%，而服务端报的是 chan 构建 22ms / 信号链 86ms，机器差解释不了这个四倍差距。曾怀疑是 payload 反序列化，实测 _rebuild 只有 1.0ms，假设不成立。两边跑同一探针对分档表才能定位。 HANDOFF 顺带修掉一处 5.6 重号（增量落地那节改为 5.7，本节挂 5.71）。 Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
jackyu66git
+52
-21
@@ -15,10 +15,12 @@ class ChanBI():
|
||||
self.sure_time = None
|
||||
self.klc_list = []
|
||||
self.klc_list.append(klc)
|
||||
self._klc_idx = {klc.index}
|
||||
self.end_time = klc.end_time
|
||||
self.start_time = klc.start_time
|
||||
self.macd_hist = 0
|
||||
self.macd_div = 0
|
||||
self._macd_hist = 0
|
||||
self._macd_div = 0
|
||||
self._macd_dirty = False
|
||||
self.seg = None
|
||||
self.height = 0
|
||||
self.width = 0
|
||||
@@ -33,26 +35,57 @@ class ChanBI():
|
||||
def set_seg(self, seg):
|
||||
self.seg = seg
|
||||
self.seg_index = len(seg.bi_list)-1
|
||||
# macd_hist / macd_div 改为惰性。原来 add_klc 每加一根 KLC 就把整笔的
|
||||
# 所有 KLU 重新累加一遍，是 O(k²)；而这两个值只有背驰判定（bsp.py）在读，
|
||||
# lean 模式下 bsp 根本不算，等于全程白算。这里只标脏，取值时才算。
|
||||
# 语义不变：klc_list 只增不减（set_start_klc 会重置并同时清脏），
|
||||
# dir 在 klc 累加期间固定，所以延后到读取时算与逐次重算结果相同。
|
||||
@property
|
||||
def macd_hist(self):
|
||||
if self._macd_dirty:
|
||||
self.cal_macdhist()
|
||||
return self._macd_hist
|
||||
|
||||
@macd_hist.setter
|
||||
def macd_hist(self, v):
|
||||
self._macd_hist = v
|
||||
self._macd_dirty = False
|
||||
|
||||
@property
|
||||
def macd_div(self):
|
||||
self.cal_macd_div()
|
||||
return self._macd_div
|
||||
|
||||
@macd_div.setter
|
||||
def macd_div(self, v):
|
||||
self._macd_div = v
|
||||
|
||||
def set_macdhist(self, macd_hist):
|
||||
self.macd_hist = macd_hist
|
||||
def set_macd_div(self, macd_div):
|
||||
self.macd_div = macd_div
|
||||
def cal_macd_div(self):
|
||||
self.macd_div = 0.0
|
||||
self._macd_div = 0.0
|
||||
if self.pre and self.pre.pre:
|
||||
if self.pre.pre.macd_hist == 0:
|
||||
self.macd_div = 0.0
|
||||
self._macd_div = 0.0
|
||||
else:
|
||||
self.macd_div = self.macd_hist / self.pre.pre.macd_hist
|
||||
self._macd_div = self.macd_hist / self.pre.pre.macd_hist
|
||||
#print(self.start_time, self.end_time, self.macd_hist, self.pre.pre.macd_hist, self.macd_div)
|
||||
def cal_macdhist(self):
|
||||
self.macd_hist = 0
|
||||
self._macd_dirty = False
|
||||
self._macd_hist = 0
|
||||
up = self.dir == Chan_BI_DIR.UP
|
||||
acc = 0
|
||||
for klc in self.klc_list:
|
||||
for klu in klc.klu_list:
|
||||
if self.dir == Chan_BI_DIR.UP and klu.macdhist > 0:
|
||||
self.macd_hist += klu.macdhist
|
||||
if self.dir == Chan_BI_DIR.DOWN and klu.macdhist < 0:
|
||||
self.macd_hist -= klu.macdhist
|
||||
h = klu.macdhist
|
||||
if up:
|
||||
if h > 0:
|
||||
acc += h
|
||||
elif h < 0:
|
||||
acc -= h
|
||||
self._macd_hist = acc
|
||||
def check_bi_zs_overlap(self):
|
||||
if self.next and self.next.next:
|
||||
if self.dir == Chan_BI_DIR.UP:
|
||||
@@ -95,6 +128,10 @@ class ChanBI():
|
||||
self.start_klc = klc
|
||||
self.klc_list = []
|
||||
self.klc_list.append(klc)
|
||||
# klc_list 被整个换掉，去重集合与惰性缓存都要跟着重置，
|
||||
# 否则后续 add_klc 会以为旧下标还在里面而漏加
|
||||
self._klc_idx = {klc.index}
|
||||
self._macd_dirty = True
|
||||
self.high = klc.high
|
||||
self.low = klc.low
|
||||
self.dir = ddir
|
||||
@@ -103,20 +140,14 @@ class ChanBI():
|
||||
def set_next(self, bi):
|
||||
self.next = bi
|
||||
def add_klc(self, klc):
|
||||
added = False
|
||||
if len(self.klc_list) > 0:
|
||||
for index in range(0, len(self.klc_list)):
|
||||
if self.klc_list[index].index == klc.index:
|
||||
added = True
|
||||
break
|
||||
if not added:
|
||||
# 去重原本是对 klc_list 线性扫描，配合下面每次全量重算的 macdhist，
|
||||
# 让「往一笔里加 k 根 KLC」变成 O(k²)。改用下标集合，O(1)。
|
||||
if klc.index not in self._klc_idx:
|
||||
self._klc_idx.add(klc.index)
|
||||
self.klc_list.append(klc)
|
||||
#print(self.start_time, klc.start_time)
|
||||
#print(klc.end_time, klc.index)
|
||||
self.end_klc = klc
|
||||
self.end_time = klc.klu_list[-1].time
|
||||
self.cal_macdhist()
|
||||
self.cal_macd_div()
|
||||
self._macd_dirty = True
|
||||
def append_klc_list(self, klc_list):
|
||||
self.klc_list.append(klc_list)
|
||||
def get_decimal(self, value):
|
||||
|
||||
@@ -54,6 +54,12 @@ class IndicatorsBuilderMixin:
|
||||
return None
|
||||
|
||||
def add_indicators(self, df):
|
||||
"""算指标并一次性挂到 df 上。
|
||||
|
||||
这里不逐列 `df['x'] = ...`：那样每一列都触发一次 BlockManager 插入，
|
||||
30 多列的开销比全部 TA 计算本身还大（2001 行实测 TA 合计 2.5ms，
|
||||
逐列赋值 3.6ms）。增量路径每根都要走一遍，这笔开销是白付的。
|
||||
"""
|
||||
fast = 26
|
||||
slow = 52
|
||||
period = 9
|
||||
@@ -75,40 +81,43 @@ class IndicatorsBuilderMixin:
|
||||
bbp30 = (df['close'] - bb30['lowerband']) / (bb30['upperband'] - bb30['lowerband'])
|
||||
bbp302 = (df['close'] - bb302['lowerband']) / (bb302['upperband'] - bb302['lowerband'])
|
||||
bbp2633 = (df['close'] - bb2633['lowerband']) / (bb2633['upperband'] - bb2633['lowerband'])
|
||||
df['bb2633upper'] = bb2633['upperband']
|
||||
df['bb2633lower'] = bb2633['lowerband']
|
||||
df['bbp2633'] = bbp2633
|
||||
df['bb2633middle'] = bb2633['middleband']
|
||||
df['atr'] = ta.ATR(df, timeperiod=14)
|
||||
df['bbup365'] = bb365['upperband']
|
||||
df['bblow365'] = bb365['lowerband']
|
||||
df['bbp365'] = bbp365
|
||||
df['bbup120'] = bb120['upperband']
|
||||
df['bblow120'] = bb120['lowerband']
|
||||
df['bbp120'] = bbp120
|
||||
df['bbup30'] = bb30['upperband']
|
||||
df['bblow30'] = bb30['lowerband']
|
||||
df['bbmiddle30'] = bb30_middle # 添加bb30中轨
|
||||
df['bbp30'] = bbp30
|
||||
df['bbup302'] = bb302['upperband']
|
||||
df['bblow302'] = bb302['lowerband']
|
||||
df['bbp302'] = bbp302
|
||||
df['macd'] = macd['macd']
|
||||
df['macdsignal'] = macd['macdsignal']
|
||||
df['macdhist'] = macd['macdhist']
|
||||
df['ema5'] = ta.EMA(df, timeperiod=5)
|
||||
df['ema10'] = ta.EMA(df, timeperiod=10)
|
||||
df['ema24'] = ta.EMA(df, timeperiod=24)
|
||||
df['ema52'] = ta.EMA(df, timeperiod=52)
|
||||
df['ema104'] = ta.EMA(df, timeperiod=104)
|
||||
df['ema156'] = ta.EMA(df, timeperiod=156)
|
||||
df['ema208'] = ta.EMA(df, timeperiod=208)
|
||||
df['ema26'] = ta.EMA(df, timeperiod=26)
|
||||
df['ema13'] = ta.EMA(df, timeperiod=13)
|
||||
df['ema7'] = ta.EMA(df, timeperiod=7)
|
||||
df['rsi'] = ta.RSI(df, timeperiod=14)
|
||||
df['volume_ratio'] = self.cal_volume_ratio(df)
|
||||
return df
|
||||
cols = {
|
||||
'bb2633upper': bb2633['upperband'],
|
||||
'bb2633lower': bb2633['lowerband'],
|
||||
'bbp2633': bbp2633,
|
||||
'bb2633middle': bb2633['middleband'],
|
||||
'atr': ta.ATR(df, timeperiod=14),
|
||||
'bbup365': bb365['upperband'],
|
||||
'bblow365': bb365['lowerband'],
|
||||
'bbp365': bbp365,
|
||||
'bbup120': bb120['upperband'],
|
||||
'bblow120': bb120['lowerband'],
|
||||
'bbp120': bbp120,
|
||||
'bbup30': bb30['upperband'],
|
||||
'bblow30': bb30['lowerband'],
|
||||
'bbmiddle30': bb30_middle,
|
||||
'bbp30': bbp30,
|
||||
'bbup302': bb302['upperband'],
|
||||
'bblow302': bb302['lowerband'],
|
||||
'bbp302': bbp302,
|
||||
'macd': macd['macd'],
|
||||
'macdsignal': macd['macdsignal'],
|
||||
'macdhist': macd['macdhist'],
|
||||
}
|
||||
for _p, _n in ((5, 'ema5'), (10, 'ema10'), (24, 'ema24'), (52, 'ema52'),
|
||||
(104, 'ema104'), (156, 'ema156'), (208, 'ema208'),
|
||||
(26, 'ema26'), (13, 'ema13'), (7, 'ema7')):
|
||||
cols[_n] = ta.EMA(df, timeperiod=_p)
|
||||
cols['rsi'] = ta.RSI(df, timeperiod=14)
|
||||
cols['volume_ratio'] = self.cal_volume_ratio(df)
|
||||
|
||||
# 重复调用（增量路径每根都会）时先摘掉旧列，否则 concat 出重名列。
|
||||
# 摘掉再接回末尾，列序与逐列覆盖的结果一致。
|
||||
new = pd.DataFrame(cols, index=df.index)
|
||||
dup = [c for c in new.columns if c in df.columns]
|
||||
if dup:
|
||||
df = df.drop(columns=dup)
|
||||
return pd.concat([df, new], axis=1)
|
||||
|
||||
def get_ema_state(self, dataframe):
|
||||
klu_list = self.get_klu_list(dataframe)
|
||||
|
||||
@@ -128,23 +128,28 @@ class KlineBuilderMixin:
|
||||
return Chan_FX_TYPE.UNKNOWN
|
||||
|
||||
def check_fx_pattern(self, klc):
|
||||
"""给分型前后三根 KLC 的裸 K 打 pattern 标记。
|
||||
|
||||
原本还会把 `klu.to_string()` 拼成一个字符串——那是给下面那行注释掉的
|
||||
print 用的，拼完就丢。它在 cal_bi_list 的内层，2000 根上要跑近三万次
|
||||
f-string + 六万次 enum 格式化，是纯废动作，已删。
|
||||
|
||||
`klu.pattern` 只被 cal_klu_pattern 自己的双 K / 三 K 判定读，
|
||||
不出这个模块，也不进 web 序列化。所以 lean 下整个调用可跳。
|
||||
"""
|
||||
if getattr(self, 'lean', False):
|
||||
return
|
||||
klu_list = klc.pre.klu_list + klc.klu_list + klc.next.klu_list
|
||||
|
||||
self.cal_klu_pattern(klu_list)
|
||||
p = ""
|
||||
for klu in klu_list:
|
||||
p += klu.to_string()
|
||||
#print(p)
|
||||
|
||||
def cal_volume_ratio(self, dataframe, window=10):
|
||||
df = dataframe.copy()
|
||||
# 计算过去N根K线的平均成交量
|
||||
df['avg_volume'] = df['volume'].rolling(window=window).mean()
|
||||
# 计算量比
|
||||
df['volume_ratio'] = df['volume'] / df['avg_volume']
|
||||
# 填充缺失值（前N根K线）
|
||||
df['volume_ratio'] = df['volume_ratio'].fillna(1.0)
|
||||
return df['volume_ratio']
|
||||
"""当根量 / 前 window 根均量。前 window-1 根无基准，填 1.0。
|
||||
|
||||
原写法先 `dataframe.copy()` 再挂两列——为算一列 rolling 复制了整张
|
||||
四十列的表。直接在 Series 上算，结果逐值相同。
|
||||
"""
|
||||
vol = dataframe['volume']
|
||||
return (vol / vol.rolling(window=window).mean()).fillna(1.0).rename('volume_ratio')
|
||||
|
||||
def cal_kl_data(self, dataframe:DataFrame):
|
||||
"""按行构造 KLU 链。
|
||||
@@ -249,7 +254,12 @@ class KlineBuilderMixin:
|
||||
for klu in klu_list:
|
||||
self._push_klu_into_klc_list(klc_list, klu, last_klu)
|
||||
last_klu = klu
|
||||
klc_list = self.cal_trend(klc_list)
|
||||
# cal_trend 只产出 klc.trend，而全仓只有它自己（经 prev_klcs 读自身序列
|
||||
# 状态）、一个 __str__ 和 web 的 klc_trend 图层消费它——笔与中枢不读。
|
||||
# 增量路径的 klc 其 trend 恒为 UNKNOWN 却与批量构建逐字段相同，是实证。
|
||||
# 所以 lean 下可跳；web 走非 lean，图层不受影响。
|
||||
if not getattr(self, 'lean', False):
|
||||
klc_list = self.cal_trend(klc_list)
|
||||
return klc_list
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user