jack/lob-latent-regimes
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Revise README for clarity and updated content

Browse Source 
Updated README to enhance clarity and accuracy of project description, methodology, and quick start instructions.
This commit is contained in:
PRAKUL HIREMATH
2026-04-10 21:02:53 +05:30
committed by GitHub
parent a6b85195c7
commit 57108ec4a1
1 changed files with 86 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+86 -52
View File
@@ -1,110 +1,144 @@
# ⚡️ Latent Micro-Regime Early Detection in LOBs
# ⚡️ Latent Micro-Regime Early Detection in Limit Order Books
<p align="center">
<img src="assets/detection.gif" width="800" alt="Latent Detection Demo"/>
<img src="assets/detection.gif" width="850" alt="Detection Timeline Visualization"/>
</p>
<p align="center">
<img src="[https://img.shields.io/badge/Status-Cutting--Edge-brightgreen?style=for-the-badge](https://img.shields.io/badge/Status-Cutting--Edge-brightgreen?style=for-the-badge)" alt="Status"/>
<img src="[https://img.shields.io/badge/Framework-HMM--Trigger-blueviolet?style=for-the-badge](https://img.shields.io/badge/Framework-HMM--Trigger-blueviolet?style=for-the-badge)" alt="Framework"/>
<img src="[https://img.shields.io/badge/Hardware-NVIDIA%20T4%20%7C%20Apple%20M4-black?style=for-the-badge](https://img.shields.io/badge/Hardware-NVIDIA%20T4%20%7C%20Apple%20M4-black?style=for-the-badge)" alt="Hardware"/>
<img src="[https://img.shields.io/badge/Precision-100%25-orange?style=for-the-badge](https://img.shields.io/badge/Precision-100%25-orange?style=for-the-badge)" alt="Precision"/>
<a href="#"><img src="[https://img.shields.io/badge/Method-Trigger%20Based-0078D4?style=for-the-badge&logo=lightning](https://img.shields.io/badge/Method-Trigger%20Based-0078D4?style=for-the-badge&logo=lightning)"/></a>
<a href="#"><img src="[https://img.shields.io/badge/Model-HMM-8A2BE2?style=for-the-badge&logo=scikitlearn](https://img.shields.io/badge/Model-HMM-8A2BE2?style=for-the-badge&logo=scikitlearn)"/></a>
<a href="#"><img src="[https://img.shields.io/badge/Precision-100%25-D4AF37?style=for-the-badge&logo=target](https://img.shields.io/badge/Precision-100%25-D4AF37?style=for-the-badge&logo=target)"/></a>
<a href="#"><img src="[https://img.shields.io/badge/Lead--Time-Positive-28A745?style=for-the-badge&logo=clock](https://img.shields.io/badge/Lead--Time-Positive-28A745?style=for-the-badge&logo=clock)"/></a>
<a href="[https://opensource.org/licenses/MIT](https://opensource.org/licenses/MIT)"><img src="[https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge](https://img.shields.io/badge/License-MIT-yellow.svg?style=for-the-badge)"/></a>
</p>
---
## 🌪 The Thesis: Stress is not a Bolt from the Blue
Standard market signals like **Volatility** and **Imbalance** are *post-mortem* indicators—they tell you the ship is sinking while you're already underwater.
## 🔬 Overview
This project proves that **liquidity instability has a "incubation period."** We capture the latent structural decay *before* the price cracks.
This research investigates whether **latent microstructure dynamics** in Limit Order Books (LOB) can be mathematically identified *before* observable liquidity stress manifests.
> **"If you wait for the volatility spike, you've already lost. We detect the silence before the scream."**
Traditional signals like volatility and order imbalance are **reactive**—they trigger only after a dislocation has occurred. This project introduces a predictive framework focusing on the **Latent Build-up Phase**, identifying structural instability before it translates into price or spread shocks.
---
## 🧠 The Architecture: Causal Delayed Transition
We move beyond simple binary states. Our model architecture assumes a three-stage causal evolution of market failure:
## 🧠 Core Methodology: The Latent Build-up
| State | Regime | Market Reality | Observability |
Market stress is rarely instantaneous; it is preceded by **structural deterioration**. We model this as a three-state latent process:
| State | Regime | Market Description | Signal Characteristic |
| :--- | :--- | :--- | :--- |
| **🟢 0** | **Stable** | Equilibrium liquidity | High |
| **🟡 1** | **Latent Build-up** | **Hidden deterioration / Depth erosion** | **Invisible (The "Ghost" Phase)** |
| **🔴 2** | **Stress** | Observable Dislocation / Crash | High (Reactive) |
| **0** | **Stable** | Balanced liquidity, high resilience | Equilibrium |
| **1** | **Latent Build-up** | Depth erosion, subtle spread drift | **Hidden Instability** |
| **2** | **Stress** | Observable dislocation, price shocks | Reactive |
### 🛠 The "God-Mode" Detection Logic
Our detector operates on a **MAX-trigger fusion** across high-dimensional channels:
* **Probabilistic Instability:** HMM posterior entropy spikes.
* **Temporal Drift:** Recursive analysis of spread & depth velocity.
* **Structural Erosion:** Real-time order flow decay signals.
* **Rising-Edge Trigger:** Precise identification of the *onset* of instability.
> **Key Discovery:** A delayed transition from **State 1 → State 2** creates a deterministic prediction window, allowing for early detection with strictly positive lead-time.
---
## 📊 Performance Matrix (The "Kill-The-Baselines" Table)
## 🛠 Detection Framework
While classical signals lag by **20+ units**, our **Adaptive Trigger** provides an average lead-time of **+18.62**, catching the regime shift before it manifests in price.
The detector employs a high-fidelity fusion of probabilistic and temporal signals to identify the "inflection point" of market health.
### 📡 Signal Integration
* **Probabilistic Instability:** HMM posterior entropy monitoring.
* **Temporal Drift:** Recursive analysis of spread and depth dynamics.
* **Structural Decay:** Real-time tracking of depth erosion and order flow toxicity.
### 🕹 Detection Logic
* **MAX-Trigger Fusion:** Cross-channel integration to capture the first sign of decay.
* **Rising-Edge Detection:** Focusing on the *onset* of change rather than absolute thresholds.
* **Early-Detection Constraint:** Optimization of $\tau < \sigma$, ensuring the signal precedes the event.
---
## 📊 Results & Performance
| Method | Mean $\Delta$ (Lead-Time) | Precision | Coverage |
| :--- | :--- | :--- | :--- |
| **🔥 Adaptive Trigger** | **+18.62** | **100%** | **52.6%** |
| 🧬 Model-Only | +14.95 | 100% | 43.2% |
| 📡 Multi-Trigger | +13.15 | 100% | 28.1% |
| 📉 Imbalance | -24.84 | 54.9% | 78.7% |
| 📉 Volatility | -32.02 | 45.5% | 43.3% |
| **Adaptive Trigger** | **+18.62** | **100%** | 52.6% |
| **Model HMM** | **+14.95** | **100%** | 43.2% |
| **Multi-Trigger** | **+13.15** | **100%** | 28.1% |
| Order Imbalance | -24.84 | 54.9% | 78.7% |
| Volatility | -32.02 | 45.5% | 43.3% |
### Critical Interpretations:
* **Positive Lead-Time:** Our methods detect stress *before* it happens; baselines are strictly negative (lagging).
* **Temporal Validity:** 100% precision indicates zero "false starts" before the latent phase begins.
* **Trade-off:** Coverage levels reflect the conservative nature of high-precision early signals.
---
## 📂 Project Anatomy
## 📈 Key Findings
1. **Latent Instability exists:** Market regimes degrade structurally before they degrade visually.
2. **Primary Indicators:** Depth erosion and HMM entropy are the most robust early-warning metrics.
3. **Signal Edge:** Rising-edge detection is essential to bypass the noise inherent in absolute thresholding.
4. **Performance:** Trigger-based detection consistently outperforms classical econometric baselines.
---
## 📂 Repository Structure
```bash
.
├── 🧪 experiments/ # Evolutionary stages v1 → v7 (The Lab)
├── 📓 notebooks/ # Final production-grade experiment notebook
├── 📈 results/ # High-fidelity figures & summary.txt
├── 📄 paper/ # The theoretical foundation (PDF)
├── 🎨 assets/ # Dynamic visualization & GIFs
└── 📜 README.md # You are here.
├── experiments/ # Iterative development v1 → v7
├── notebooks/ # Production-grade experiment analysis
├── results/
│ ├── figures/ # High-resolution performance plots
│ └── summary.txt # Quantified results summary
├── paper/ # Technical manuscript (PDF)
├── assets/ # Visualizations and GIFs
└── README.md
```
---
## ⚡ Quick Start (Reproduction Pipeline)
## 🚀 Reproducibility
Tested on **NVIDIA T4 (Colab)** and **Apple Silicon M4**. Results are 100% deterministic via seeded initialization.
Validated across high-compute and local environments:
* **Cloud:** Google Colab (NVIDIA T4)
* **Local:** Apple Silicon (M4 Pro/Max)
### Quick Start
```bash
# Clone the madness
git clone https://github.com/your-repo/latent-lob-detection.git
cd latent-lob-detection
# Clone the repository
git clone https://github.com/your-repo/lob-early-detection.git
# Arm the environment
# Install dependencies
pip install -r requirements.txt
# Run the flagship pipeline
# Execute the final pipeline
python experiments/v7_final.py
```
---
## 🏆 Key Contributions
1. **Causal Latent Theory:** Formalized the $Regime 1 \rightarrow Regime 2$ transition delay.
2. **Zero-False-Positive Logic:** 100% precision thresholding in the early-detection window.
3. **The Trigger Fusion:** Combined entropy, drift, and flow into a single "Early-Warning" signal.
4. **Positive Lead-Time:** Mathematically demonstrated dominance over reactive baselines.
## 📝 Contributions
* **Causal Formulation:** Formalizing the Latent Build-up $\rightarrow$ Stress transition.
* **Temporal Drift:** Identifying subtle drift as a precursor to liquidity voids.
* **MAX Fusion & Rising-Edge:** Novel trigger logic for sub-millisecond microstructure data.
* **Empirical Proof:** Demonstrating strictly positive lead-time over reactive benchmarks.
---
## 📝 Citation
## 📑 Citation
```bibtex
@article{lob_micro_regime_detection_2026,
title={Early Detection of Latent Micro-Regimes in Limit Order Books},
author={Hiremath, Prakul},
year={2026}
year={2026},
journal={Reproducible Research in Market Microstructure}
}
```
---
<p align="center">
Built with ☕️, 🐍, and a relentless pursuit of Alpha. <br/>
<b>License: MIT</b>
Built with ☕️ and 🐍 for <b>Reproducible Quantitative Finance.</b>
</p>
⸻