Fintech · Market Data Platform
SpaceTime
High-throughput market data platform processing 10,000+ ticks/sec with real-time Space-Time Reversal Indicators and multi-chart visualization.
Status
Completed
Throughput
10,000+ ticks/sec
Reversal versions
v1 · v2 · v3 · v4
Uptime
Production deployed
Compute
NumPy vectorized
The Problem
Processing 10,000+ ticks/sec from a live market feed, computing complex indicators (SMA/EMA/ZLMA smoothing, intensity, reversal detection) in real time, and simultaneously distributing data across multiple chart instances — without data loss, lag, or performance degradation.
The Solution
A Redis Pub/Sub pipeline decouples ingestion from distribution. NumPy vectorized arrays replace pandas for 10x computation speed. InfluxDB handles time-series storage for historical and spike data. An API Gateway aggregates and batches WebSocket messages (TradingView-style) to handle peak market throughput. The frontend uses an event bus to update charts independently.
Features
Real-Time Data Pipeline
- DTN live market feed ingestion at 10,000+ ticks/sec
- Redis Pub/Sub for zero-latency inter-microservice messaging
- 10,000-entry Redis buffer queue to eliminate data loss under peak load
- NumPy vectorized sliding windows replacing pandas for 10x speed
- InfluxDB v3 for time-series spike and intensity historical storage
- PostgreSQL for relational data and OHLCV candle storage
Space-Time Indicators
- Intensity formula with SMA, EMA, and ZLMA smoothing types
- Spike detection with historical bulk-load and live event streaming
- Space-Time Reversal Indicator in 4 versions — live market tested
- v1: intensity + adaptive buffer with CV scaling
- v2: price + intensity with momentum confirmation
- v3: intensity + high/low timeframe trend + volume gate
- v4: EMA crossover + choppy market filter + volume spike confirmation
- Adaptive buffer with 80:20 ratio and 1000-tick window for noise reduction
Multi-Chart Frontend
- TradingView Lightweight Charts: smoothing line, candle, marker, volume, spike
- Event bus architecture — charts update independently without re-renders
- OHLCV candle chart with multi-timeframe support (1s, 10s, 1m and more)
- Live in-progress candle updates with price tags and reversal markers
- ZLMA volume series (purple line) overlay on candle chart
- Intensity bars with threshold-based logic and animation
- Historical range queries with initial 10-minute bulk load on connect
- Auto-scaling, dynamic chart resize on window change
- Dark/light theme with centralized CSS variable system
- Fully responsive mobile layout with hamburger sidebar
Production Infrastructure
- Message batching at API Gateway — TradingView-style grouped messages
- Backpressure handling to prevent overload during peak market hours
- WebSocket auto-reconnect with exponential backoff
- Keycloak authentication with JWT token refresh handling
- Race condition resolution with session locking for concurrency safety
- GitHub Actions CI/CD — automated Docker deploy on push
- Nginx reverse proxy + Certbot SSL on Linux (AWS)
- InfluxDB v3 with SQL queries and optimized config for production
Architecture
“DTN feed → Live Service (NumPy processing) → Redis Pub/Sub → API Gateway (message batching, backpressure) → WebSocket → Next.js (event bus → independent chart state managers). Historical Service (separate microservice) → InfluxDB v3 for spike/intensity history. All services containerized via Docker Compose. GitHub Actions deploys to AWS Linux server on every push to main.”