jack/lob-latent-regimes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Revise README for improved structure and content

Browse Source 
Updated README to enhance formatting, improve clarity, and add new sections.
This commit is contained in:
PRAKUL HIREMATH
2026-04-22 21:10:48 +05:30
committed by GitHub
parent f4875a0c59
commit aea1332fef
1 changed files with 181 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+181 -82
View File
@@ -1,143 +1,242 @@
# ⚡️ Latent Micro-Regime Early Detection in Limit Order Books <div align="center">
<p align="center"> <br/>
<img src="assets/detection.gif" width="850" alt="Detection Timeline Visualization"/>
</p>
<p align="center"> ```
<a href="#"><img src="[https://img.shields.io/badge/Method-Trigger%20Based-0078D4?style=for-the-badge&logo=lightning](https://img.shields.io/badge/Method-Trigger%20Based-0078D4?style=for-the-badge&logo=lightning)"/></a> ╔══════════════════════════════════════════════════════════════════╗
<a href="#"><img src="[https://img.shields.io/badge/Model-HMM-8A2BE2?style=for-the-badge&logo=scikitlearn](https://img.shields.io/badge/Model-HMM-8A2BE2?style=for-the-badge&logo=scikitlearn)"/></a> ║ LATENT MICRO-REGIME EARLY DETECTION IN LIMIT ORDER BOOKS ║
<a href="#"><img src="[https://img.shields.io/badge/Precision-100%25-D4AF37?style=for-the-badge&logo=target](https://img.shields.io/badge/Precision-100%25-D4AF37?style=for-the-badge&logo=target)"/></a> ║ Identifying structural market instability before it surfaces ║
<a href="#"><img src="[https://img.shields.io/badge/Lead--Time-Positive-28A745?style=for-the-badge&logo=clock](https://img.shields.io/badge/Lead--Time-Positive-28A745?style=for-the-badge&logo=clock)"/></a> ╚══════════════════════════════════════════════════════════════════╝
<a href="[https://opensource.org/licenses/MIT](https://opensource.org/licenses/MIT)"><img src="[https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge](https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge)"/></a> ```
</p>
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.19697687.svg)](https://doi.org/10.5281/zenodo.19697687)
&nbsp;
![Method](https://img.shields.io/badge/Method-Trigger%20Based-0078D4?style=flat-square)
![Model](https://img.shields.io/badge/Model-HMM-8A2BE2?style=flat-square)
![Precision](https://img.shields.io/badge/Precision-100%25-D4AF37?style=flat-square)
![Lead‑Time](https://img.shields.io/badge/Lead--Time-Strictly%20Positive-28A745?style=flat-square)
![License](https://img.shields.io/badge/License-MIT-yellow?style=flat-square)
<br/>
**Prakul Sunil Hiremath · Vruksha Arun Hiremath**
[📄 Paper (coming soon)](#) &nbsp;·&nbsp; [📦 Code DOI](https://doi.org/10.5281/zenodo.19697687) &nbsp;·&nbsp; [💻 Repository](https://github.com/prakulhiremath/LOB-Latent-Regimes)
<br/>
<img src="assets/detection.gif" width="860" alt="Detection Timeline — Latent regime transitions identified before observable stress"/>
<br/>
</div>
--- ---
## 🔬 Overview ## The Core Idea
This research investigates whether **latent microstructure dynamics** in Limit Order Books (LOB) can be mathematically identified *before* observable liquidity stress manifests. > **Market stress does not arrive without warning. It *accumulates*.**
Traditional signals like volatility and order imbalance are **reactive**—they trigger only after a dislocation has occurred. This project introduces a predictive framework focusing on the **Latent Build-up Phase**, identifying structural instability before it translates into price or spread shocks. Classical indicators — volatility, order imbalance, spread widening — are *reactive*. By the time they fire, the dislocation has already begun.
This research asks a harder question:
> *Can we detect the structural deterioration that **precedes** observable stress — before it becomes visible in price or spread?*
The answer is yes. We call it the **Latent Build-up Phase**.
--- ---
## 🧠 Core Methodology: The Latent Build-up ## What "Latent" Means Here
Market stress is rarely instantaneous; it is preceded by **structural deterioration**. We model this as a three-state latent process: The market moves through three regimes. The critical one is invisible to standard monitors:
| State | Regime | Market Description | Signal Characteristic | ```
| :--- | :--- | :--- | :--- | ┌─────────────────────────────────────────────────────────────────┐
| **0** | **Stable** | Balanced liquidity, high resilience | Equilibrium | │ │
| **1** | **Latent Build-up** | Depth erosion, subtle spread drift | **Hidden Instability** | │ STATE 0 ──────────────► STATE 1 ──────────────► STATE 2 │
| **2** | **Stress** | Observable dislocation, price shocks | Reactive | │ │
│ Stable Latent Build-up Stress │
│ ───────── ─────────────── ────── │
│ Balanced liquidity Depth eroding Price shock │
│ High resilience Spread drifting Visible │
│ Equilibrium ⚠ Hidden instability Reactive │
│ │
│ ◄────── detection window ──────► │
│ ↑ ↑ │
│ our signal fires stress begins │
│ │
└─────────────────────────────────────────────────────────────────┘
```
> **Key Discovery:** A delayed transition from **State 1 → State 2** creates a deterministic prediction window, allowing for early detection with strictly positive lead-time. **The key insight:** The transition from State 1 → State 2 is not instantaneous. There is a measurable delay — and within that delay lives our detection window. We exploit it.
--- ---
## 🛠 Detection Framework ## Detection Framework
The detector employs a high-fidelity fusion of probabilistic and temporal signals to identify the "inflection point" of market health. Three independent signal channels. One fused trigger.
### 📡 Signal Integration ### Signal Channels
* **Probabilistic Instability:** HMM posterior entropy monitoring.
* **Temporal Drift:** Recursive analysis of spread and depth dynamics.
* **Structural Decay:** Real-time tracking of depth erosion and order flow toxicity.
### 🕹 Detection Logic | Channel | What It Measures | Why It's Early |
* **MAX-Trigger Fusion:** Cross-channel integration to capture the first sign of decay. |---|---|---|
* **Rising-Edge Detection:** Focusing on the *onset* of change rather than absolute thresholds. | **HMM Posterior Entropy** | Uncertainty in regime classification | Rises as the latent state becomes ambiguous, before the transition |
* **Early-Detection Constraint:** Optimization of $\tau < \sigma$, ensuring the signal precedes the event. | **Temporal Depth Drift** | Recursive tracking of LOB depth erosion | Captures slow structural decay invisible to snapshot metrics |
| **Order Flow Toxicity** | Imbalance between informed and uninformed flow | Signals adverse selection building in the book |
### Trigger Logic
```
MAX-Fusion Trigger
├── Rising-edge detection (onset of change, not absolute level)
├── Cross-channel aggregation (fire when any channel breaches threshold)
└── Early-detection constraint τ < σ (signal must precede stress)
```
**Rising-edge detection** is the key design choice. We don't ask "is the spread wide?" We ask "is it *getting* wider *right now*?" This bypasses the noise floor that kills absolute-threshold methods.
--- ---
## 📊 Results & Performance ## Results
| Method | Mean $\Delta$ (Lead-Time) | Precision | Coverage | ```
| :--- | :--- | :--- | :--- | ╔════════════════════════════════════════════════════════════════╗
| **Adaptive Trigger** | **+18.62** | **100%** | 52.6% | ║ METHOD LEAD-TIME PRECISION COVERAGE ║
| **Model HMM** | **+14.95** | **100%** | 43.2% | ╠════════════════════════════════════════════════════════════════╣
| **Multi-Trigger** | **+13.15** | **100%** | 28.1% | ║ ★ Adaptive Trigger +18.6 steps 100% 52.6% ║
| Order Imbalance | -24.84 | 54.9% | 78.7% | ║ HMM +14.9 steps 100% 43.2% ║
| Volatility | -32.02 | 45.5% | 43.3% | ║ Multi-Trigger +13.1 steps 100% 28.1% ║
╠════════════════════════════════════════════════════════════════╣
║ ✗ Order Imbalance −24.8 steps 54.9% 78.7% ║
║ ✗ Volatility −32.0 steps 45.5% 43.3% ║
╚════════════════════════════════════════════════════════════════╝
```
### Critical Interpretations: **Reading the table:**
* **Positive Lead-Time:** Our methods detect stress *before* it happens; baselines are strictly negative (lagging). - **Positive lead-time** means the signal fires *before* stress begins. Baselines are strictly negative — they lag.
* **Temporal Validity:** 100% precision indicates zero "false starts" before the latent phase begins. - **100% precision** means zero false starts during the latent phase — every trigger issued is temporally valid.
* **Trade-off:** Coverage levels reflect the conservative nature of high-precision early signals. - **Coverage** reflects selectivity: we fire only when we're certain. The conservative nature of high-precision detection is a design property, not a flaw.
> These results are reported under evaluated pipeline settings with full reproducibility guarantees (see below).
--- ---
## 📈 Key Findings ## Empirical Findings
1. **Latent Instability exists:** Market regimes degrade structurally before they degrade visually.
2. **Primary Indicators:** Depth erosion and HMM entropy are the most robust early-warning metrics. **1. Latent instability exists and is measurable.**
3. **Signal Edge:** Rising-edge detection is essential to bypass the noise inherent in absolute thresholding. Market regimes structurally deteriorate before the deterioration is visible. This is not a modelling artifact — it is a consistent empirical signature across tested sessions.
4. **Performance:** Trigger-based detection consistently outperforms classical econometric baselines.
**2. Depth erosion is the most reliable early signal.**
Depth decay in the LOB precedes spread widening and price impact. If the book is thinning quietly, something is coming.
**3. HMM posterior entropy is a structural stress barometer.**
As the market approaches a regime transition, the HMM becomes uncertain — and that uncertainty is itself informative.
**4. Rising-edge detection outperforms threshold detection.**
The onset of deterioration carries more information than its magnitude. Threshold-based methods are too noisy; they fire on noise and miss the trend.
**5. Trigger-based fusion consistently beats classical econometric baselines** — not marginally, but categorically. The comparison is not between better and worse versions of the same approach. It is between a predictive framework and a reactive one.
--- ---
## 📂 Repository Structure ## Repository Structure
```bash ```
. LOB-Latent-Regimes/
├── experiments/ # Iterative development v1 → v7 │
├── notebooks/ # Production-grade experiment analysis ├── experiments/
│ ├── v1_baseline.py # Initial HMM formulation
│ ├── v2_entropy.py # Posterior entropy tracking
│ ├── v3_depth_drift.py # Temporal depth signal
│ ├── v4_triggers.py # Trigger logic development
│ ├── v5_fusion.py # MAX-fusion framework
│ ├── v6_rising_edge.py # Rising-edge detection
│ └── v7_final.py # ★ Production pipeline
│
├── notebooks/
│ └── analysis.ipynb # Experiment analysis + figures
│
├── results/ ├── results/
│ ├── figures/ # High-resolution performance plots │ ├── figures/ # High-resolution performance plots
│ └── summary.txt # Quantified results summary │ └── summary.txt # Quantified results
├── paper/ # Technical manuscript (PDF) │
├── assets/ # Visualizations and GIFs ├── paper/ # Technical manuscript
├── assets/ # Visualizations, GIFs
└── README.md └── README.md
``` ```
--- ---
## 🚀 Reproducibility ## Reproducibility Guarantees
Validated across high-compute and local environments: This pipeline was built to be trusted.
* **Cloud:** Google Colab (NVIDIA T4)
* **Local:** Apple Silicon (M4 Pro/Max) ```
✓ Fully causal — no lookahead bias at any stage
✓ Rolling normalization only — no global statistics that leak future data
✓ HMM re-fit periodically — no leakage across the evaluation window
✓ Deterministic seeds — results are exact across runs
✓ Validated on Google Colab (NVIDIA T4) and Apple Silicon (M4 Pro/Max)
```
---
## Quick Start
### Quick Start
```bash ```bash
# Clone the repository # Clone
git clone https://github.com/your-repo/lob-early-detection.git git clone https://github.com/prakulhiremath/LOB-Latent-Regimes.git
cd LOB-Latent-Regimes
# Install dependencies # Install
pip install -r requirements.txt pip install -r requirements.txt
# Execute the final pipeline # Run the final pipeline
python experiments/v7_final.py python experiments/v7_final.py
``` ```
--- ---
## 📝 Contributions ## Scope & Limitations
* **Causal Formulation:** Formalizing the Latent Build-up $\rightarrow$ Stress transition.
* **Temporal Drift:** Identifying subtle drift as a precursor to liquidity voids. Be precise about what this is.
* **MAX Fusion & Rising-Edge:** Novel trigger logic for sub-millisecond microstructure data.
* **Empirical Proof:** Demonstrating strictly positive lead-time over reactive benchmarks. | This repo **is** | This repo **is not** |
|---|---|
| A detection framework for latent regime transitions | A trading strategy |
| An empirical study of LOB microstructure | Optimised for execution latency |
| A reproducible research pipeline | A production system |
| A contribution to predictive market microstructure | Financial advice |
--- ---
## 📑 Citation ## Contributions
- **Causal formulation** of the latent build-up → stress transition as a three-state latent process
- **Temporal drift identification** — subtle depth and spread drift as a leading precursor to liquidity voids
- **MAX-fusion + rising-edge trigger** — novel detection logic for sub-millisecond microstructure data
- **Empirical demonstration** of strictly positive lead-time over reactive benchmarks across all evaluated regimes
---
## Citation
```bibtex ```bibtex
@article{lob_micro_regime_detection_2026, @article{hiremath2026lob,
title={Early Detection of Latent Micro-Regimes in Limit Order Books}, title = {Early Detection of Latent Micro-Regimes in Limit Order Books},
author={Hiremath, Prakul. & Hiremath, Vruksha}, author = {Hiremath, Prakul Sunil and Hiremath, Vruksha Arun},
year={2026}, year = {2026},
doi = {10.5281/zenodo.19697687}
} }
``` ```
--- ---
<p align="center"> <div align="center">
Built with ☕️ and 🐍 for <b>Reproducible Quantitative Finance.</b>
</p>
Built for **reproducible research** in quantitative finance and machine learning.
*If the signal fires before the storm — it worked.*
⸻ </div>