boki/stock-bot
1
0
Fork 0
Code Issues 1 Pull requests Projects Releases Packages Wiki Activity Actions
stock-bot/ARCHITECTURE.md

18 KiB
Raw Blame History

🏗️ Stock Bot Trading System - Architecture Documentation

📋 Table of Contents

🎯 System Overview

The Stock Bot Trading System is a microservice-based, event-driven trading platform built for real-time market analysis, strategy execution, and risk management. The system follows a service-oriented architecture (SOA) with clear separation of concerns and horizontal scalability.

Core Principles

  • Microservices Architecture: Independent, deployable services
  • Event-Driven Communication: WebSocket and Redis pub/sub
  • Real-Time Processing: Sub-second latency requirements
  • Scalable Design: Horizontal scaling capabilities
  • Fault Tolerance: Circuit breakers and graceful degradation
  • Type Safety: Full TypeScript implementation

🏗️ Current Architecture

graph TB
    subgraph "Frontend Layer"
        UI[Angular Trading Dashboard]
    end
    
    subgraph "API Gateway Layer"
        GW[API Gateway - Future]
    end
    
    subgraph "Core Services"
        MDG[Market Data Gateway<br/>Port 3001]
        RG[Risk Guardian<br/>Port 3002]
        SO[Strategy Orchestrator<br/>Port 4001]
    end
    
    subgraph "Data Layer"
        Redis[(Redis Cache)]
        PG[(PostgreSQL)]
        QDB[(QuestDB)]
        Mongo[(MongoDB)]
    end
    
    subgraph "External APIs"
        Alpha[Alpha Vantage]
        IEX[IEX Cloud]
        Yahoo[Yahoo Finance]
    end
    
    UI -->|WebSocket/HTTP| MDG
    UI -->|WebSocket/HTTP| RG
    UI -->|WebSocket/HTTP| SO
    
    MDG --> Redis
    MDG --> QDB
    MDG -->|Fetch Data| Alpha
    MDG -->|Fetch Data| IEX
    MDG -->|Fetch Data| Yahoo
    
    RG --> Redis
    RG --> PG
    
    SO --> Redis
    SO --> PG
    SO --> Mongo

🔧 Service Breakdown

1. Interface Services

Trading Dashboard (apps/interface-services/trading-dashboard)

  • Framework: Angular 20 + Angular Material + Tailwind CSS
  • Port: 4200 (development)
  • Purpose: Real-time trading interface and strategy management
  • Key Features:
    • Real-time market data visualization
    • Strategy creation and backtesting UI
    • Risk management dashboard
    • Portfolio monitoring
    • WebSocket integration for live updates

Current Structure:

trading-dashboard/
├── src/
│   ├── app/
│   │   ├── components/          # Reusable UI components
│   │   │   ├── sidebar/         # Navigation sidebar
│   │   │   └── notifications/   # Alert system
│   │   ├── pages/              # Route-based pages
│   │   │   ├── dashboard/      # Main trading dashboard
│   │   │   ├── market-data/    # Market data visualization
│   │   │   ├── portfolio/      # Portfolio management
│   │   │   ├── strategies/     # Strategy management
│   │   │   └── risk-management/ # Risk controls
│   │   ├── services/           # Angular services
│   │   │   ├── api.service.ts  # HTTP API communication
│   │   │   ├── websocket.service.ts # WebSocket management
│   │   │   └── strategy.service.ts  # Strategy operations
│   │   └── shared/             # Shared utilities
│   └── styles.css              # Global styles

2. Core Services

Market Data Gateway (apps/core-services/market-data-gateway)

  • Framework: Hono + Bun
  • Port: 3001
  • Purpose: Market data aggregation and real-time distribution
  • Database: QuestDB (time-series), Redis (caching)

Responsibilities:

  • Aggregate data from multiple market data providers
  • Real-time WebSocket streaming to clients
  • Historical data storage and retrieval
  • Rate limiting and API management
  • Data normalization and validation

API Endpoints:

GET  /health                    # Health check
GET  /api/market-data/:symbol   # Get latest market data
GET  /api/historical/:symbol    # Get historical data
WS   /ws                        # WebSocket for real-time data

Risk Guardian (apps/core-services/risk-guardian)

  • Framework: Hono + Bun
  • Port: 3002
  • Purpose: Real-time risk monitoring and controls
  • Database: PostgreSQL (persistent), Redis (real-time)

Responsibilities:

  • Position size monitoring
  • Daily loss tracking
  • Portfolio risk assessment
  • Volatility monitoring
  • Real-time risk alerts
  • Risk threshold management

API Endpoints:

GET  /health                    # Health check
GET  /api/risk/thresholds       # Get risk thresholds
PUT  /api/risk/thresholds       # Update risk thresholds
POST /api/risk/evaluate         # Evaluate position risk
GET  /api/risk/history          # Risk evaluation history
WS   /ws                        # WebSocket for risk alerts

Strategy Orchestrator (apps/intelligence-services/strategy-orchestrator)

  • Framework: Hono + Bun
  • Port: 4001
  • Purpose: Strategy lifecycle management and execution
  • Database: MongoDB (strategies), PostgreSQL (trades), Redis (signals)

Responsibilities:

  • Strategy creation and management
  • Backtesting engine (vectorized & event-based)
  • Real-time strategy execution
  • Signal generation and broadcasting
  • Performance analytics
  • Strategy optimization

Current Structure:

strategy-orchestrator/
├── src/
│   ├── core/
│   │   ├── backtesting/
│   │   │   ├── BacktestEngine.ts      # Main backtesting engine
│   │   │   ├── BacktestService.ts     # Backtesting service layer
│   │   │   ├── MarketDataFeed.ts      # Historical data provider
│   │   │   └── PerformanceAnalytics.ts # Performance metrics
│   │   ├── execution/
│   │   │   └── StrategyExecutionService.ts # Real-time execution
│   │   ├── strategies/
│   │   │   ├── Strategy.ts            # Base strategy interface
│   │   │   ├── StrategyRegistry.ts    # Strategy management
│   │   │   ├── BaseStrategy.ts        # Abstract base class
│   │   │   ├── VectorizedStrategy.ts  # Vectorized base class
│   │   │   ├── MovingAverageCrossover.ts # MA strategy
│   │   │   └── MeanReversionStrategy.ts  # Mean reversion
│   │   └── indicators/
│   │       └── TechnicalIndicators.ts # Technical analysis
│   ├── controllers/
│   │   └── StrategyController.ts      # API endpoints
│   └── index.ts                       # Main entry point

API Endpoints:

GET  /health                           # Health check
GET  /api/strategies                   # List strategies
POST /api/strategies                   # Create strategy
PUT  /api/strategies/:id               # Update strategy
POST /api/strategies/:id/:action       # Start/stop/pause strategy
GET  /api/strategies/:id/signals       # Get strategy signals
POST /api/strategies/:id/backtest      # Run backtest
GET  /api/strategies/:id/performance   # Get performance metrics
WS   /ws                              # WebSocket for strategy updates

3. Shared Packages

Shared Types (packages/shared-types)

export interface MarketData {
  symbol: string;
  price: number;
  volume: number;
  timestamp: Date;
  bid: number;
  ask: number;
}

export interface Strategy {
  id: string;
  name: string;
  symbols: string[];
  parameters: Record<string, any>;
  status: 'ACTIVE' | 'PAUSED' | 'STOPPED';
}

export interface BacktestResult {
  totalReturn: number;
  sharpeRatio: number;
  maxDrawdown: number;
  winRate: number;
  totalTrades: number;
}

Configuration (packages/config)

export const config = {
  redis: {
    host: process.env.REDIS_HOST || 'localhost',
    port: parseInt(process.env.REDIS_PORT || '6379')
  },
  database: {
    postgres: process.env.POSTGRES_URL,
    questdb: process.env.QUESTDB_URL,
    mongodb: process.env.MONGODB_URL
  },
  marketData: {
    alphaVantageKey: process.env.ALPHA_VANTAGE_KEY,
    iexKey: process.env.IEX_KEY
  }
};

🔄 Data Flow

Real-Time Market Data Flow

sequenceDiagram
    participant EXT as External APIs
    participant MDG as Market Data Gateway
    participant Redis as Redis Cache
    participant QDB as QuestDB
    participant UI as Trading Dashboard
    
    EXT->>MDG: Market data feed
    MDG->>Redis: Cache latest prices
    MDG->>QDB: Store historical data
    MDG->>UI: WebSocket broadcast
    UI->>UI: Update charts/tables

Strategy Execution Flow

sequenceDiagram
    participant UI as Trading Dashboard
    participant SO as Strategy Orchestrator
    participant MDG as Market Data Gateway
    participant RG as Risk Guardian
    participant Redis as Redis
    
    UI->>SO: Start strategy
    SO->>MDG: Subscribe to market data
    MDG->>SO: Real-time price updates
    SO->>SO: Generate trading signals
    SO->>RG: Risk evaluation
    RG->>SO: Risk approval/rejection
    SO->>Redis: Store signals
    SO->>UI: WebSocket signal broadcast

💻 Technology Stack

Backend

  • Runtime: Bun (ultra-fast JavaScript runtime)
  • Web Framework: Hono (lightweight, fast web framework)
  • Language: TypeScript (type safety)
  • Build Tool: Turbo (monorepo management)

Frontend

  • Framework: Angular 20 (latest stable)
  • UI Library: Angular Material + Tailwind CSS
  • State Management: Angular Signals (reactive programming)
  • WebSocket: Native WebSocket API

Databases

  • Time-Series: QuestDB (market data storage)
  • Relational: PostgreSQL (structured data)
  • Document: MongoDB (strategy configurations)
  • Cache/Pub-Sub: Redis (real-time data)

Infrastructure

  • Containerization: Docker + Docker Compose
  • Process Management: PM2 (production)
  • Monitoring: Built-in health checks
  • Development: Hot reload, TypeScript compilation

🚀 Future Architecture

Phase 1: Enhanced Microservices (Q2 2025)

graph TB
    subgraph "API Gateway Layer"
        GW[Kong/Envoy API Gateway]
        LB[Load Balancer]
    end
    
    subgraph "Authentication"
        AUTH[Auth Service<br/>JWT + OAuth]
    end
    
    subgraph "Core Services"
        MDG[Market Data Gateway]
        RG[Risk Guardian]
        SO[Strategy Orchestrator]
        OE[Order Execution Engine]
        NS[Notification Service]
    end
    
    subgraph "Analytics Services"
        BA[Backtest Analytics]
        PA[Performance Analytics]
        ML[ML Prediction Service]
    end
    
    subgraph "Message Queue"
        NATS[NATS/Apache Kafka]
    end

Phase 2: Machine Learning Integration (Q3 2025)

  • ML Pipeline: Python-based ML services
  • Feature Engineering: Real-time feature computation
  • Model Training: Automated model retraining
  • Prediction API: Real-time predictions

Phase 3: Multi-Asset Support (Q4 2025)

  • Crypto Trading: Binance, Coinbase integration
  • Forex Trading: OANDA, FXCM integration
  • Options Trading: Interactive Brokers integration
  • Futures Trading: CME, ICE integration

📈 Improvement Recommendations

1. High Priority Improvements

API Gateway Implementation

// Implement Kong or Envoy for:
- Rate limiting per service
- Authentication/authorization
- Request/response transformation
- Circuit breaker patterns
- Load balancing

Enhanced Error Handling

// Implement structured error handling:
interface ServiceError {
  code: string;
  message: string;
  service: string;
  timestamp: Date;
  correlationId: string;
}

Comprehensive Logging

// Implement structured logging:
interface LogEntry {
  level: 'debug' | 'info' | 'warn' | 'error';
  service: string;
  message: string;
  metadata: Record<string, any>;
  timestamp: Date;
  correlationId: string;
}

2. Medium Priority Improvements

Database Optimization

-- QuestDB optimizations for market data:
CREATE TABLE market_data (
  symbol SYMBOL,
  timestamp TIMESTAMP,
  price DOUBLE,
  volume DOUBLE,
  bid DOUBLE,
  ask DOUBLE
) timestamp(timestamp) PARTITION BY DAY;

-- Add indexes for fast queries:
CREATE INDEX idx_symbol_timestamp ON market_data (symbol, timestamp);

Caching Strategy

// Implement multi-layer caching:
interface CacheStrategy {
  L1: Map<string, any>;        // In-memory cache
  L2: Redis;                   // Distributed cache
  L3: Database;                // Persistent storage
}

WebSocket Optimization

// Implement WebSocket connection pooling:
interface WSConnectionPool {
  connections: Map<string, WebSocket[]>;
  balancer: RoundRobinBalancer;
  heartbeat: HeartbeatManager;
}

3. Low Priority Improvements

Code Quality

  • Implement comprehensive unit tests (>90% coverage)
  • Add integration tests for all services
  • Implement E2E tests for critical user flows
  • Add performance benchmarks

Documentation

  • API documentation with OpenAPI/Swagger
  • Developer onboarding guide
  • Deployment runbooks
  • Architecture decision records (ADRs)

🏗️ Deployment Architecture

Development Environment

# docker-compose.dev.yml
version: '3.8'
services:
  # Databases
  postgres:
    image: postgres:15
    ports: ["5432:5432"]
  
  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]
  
  questdb:
    image: questdb/questdb:latest
    ports: ["9000:9000", "8812:8812"]
  
  mongodb:
    image: mongo:6
    ports: ["27017:27017"]
  
  # Services
  market-data-gateway:
    build: ./apps/core-services/market-data-gateway
    ports: ["3001:3001"]
    depends_on: [redis, questdb]
  
  risk-guardian:
    build: ./apps/core-services/risk-guardian
    ports: ["3002:3002"]
    depends_on: [postgres, redis]
  
  strategy-orchestrator:
    build: ./apps/intelligence-services/strategy-orchestrator
    ports: ["4001:4001"]
    depends_on: [mongodb, postgres, redis]
  
  trading-dashboard:
    build: ./apps/interface-services/trading-dashboard
    ports: ["4200:4200"]

Production Environment

# kubernetes deployment example
apiVersion: apps/v1
kind: Deployment
metadata:
  name: market-data-gateway
spec:
  replicas: 3
  selector:
    matchLabels:
      app: market-data-gateway
  template:
    metadata:
      labels:
        app: market-data-gateway
    spec:
      containers:
      - name: market-data-gateway
        image: stockbot/market-data-gateway:latest
        ports:
        - containerPort: 3001
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"

🔒 Security Architecture

Authentication & Authorization

// JWT-based authentication
interface AuthToken {
  userId: string;
  roles: string[];
  permissions: string[];
  expiresAt: Date;
}

// Role-based access control
enum UserRole {
  TRADER = 'TRADER',
  ADMIN = 'ADMIN',
  VIEWER = 'VIEWER'
}

enum Permission {
  READ_MARKET_DATA = 'READ_MARKET_DATA',
  EXECUTE_TRADES = 'EXECUTE_TRADES',
  MANAGE_STRATEGIES = 'MANAGE_STRATEGIES',
  CONFIGURE_RISK = 'CONFIGURE_RISK'
}

API Security

// Rate limiting configuration
interface RateLimit {
  windowMs: number;      // 15 minutes
  maxRequests: number;   // 100 requests per window
  skipIf: (req: Request) => boolean;
}

// Input validation
interface ApiValidation {
  schema: JSONSchema;
  sanitization: SanitizationRules;
  authentication: AuthenticationMiddleware;
}

Data Security

  • Encryption at Rest: AES-256 for sensitive data
  • Encryption in Transit: TLS 1.3 for all communications
  • Secrets Management: Kubernetes secrets or HashiCorp Vault
  • Network Security: VPC, security groups, firewalls

📊 Monitoring & Observability

Metrics Collection

interface ServiceMetrics {
  // Performance metrics
  requestLatency: Histogram;
  requestRate: Counter;
  errorRate: Counter;
  
  // Business metrics
  tradesExecuted: Counter;
  strategiesActive: Gauge;
  portfolioValue: Gauge;
  
  // System metrics
  memoryUsage: Gauge;
  cpuUsage: Gauge;
  dbConnections: Gauge;
}

Health Checks

interface HealthCheck {
  service: string;
  status: 'healthy' | 'degraded' | 'unhealthy';
  checks: {
    database: boolean;
    redis: boolean;
    externalApis: boolean;
    webSocket: boolean;
  };
  uptime: number;
  version: string;
}

Alerting Rules

# Prometheus alerting rules
groups:
- name: stockbot
  rules:
  - alert: HighErrorRate
    expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.1
    for: 2m
    
  - alert: HighLatency
    expr: http_request_duration_seconds{quantile="0.95"} > 1
    for: 2m
    
  - alert: ServiceDown
    expr: up{job="stockbot"} == 0
    for: 30s

📋 Migration Plan

Phase 1: Current → Enhanced (1-2 months)

  1. Week 1-2: Implement API Gateway and authentication
  2. Week 3-4: Add comprehensive logging and monitoring
  3. Week 5-6: Enhance error handling and resilience
  4. Week 7-8: Performance optimization and testing

Phase 2: Enhanced → ML-Ready (2-3 months)

  1. Month 1: Implement ML pipeline infrastructure
  2. Month 2: Develop feature engineering services
  3. Month 3: Integrate ML predictions into strategies

Phase 3: ML-Ready → Multi-Asset (3-4 months)

  1. Month 1: Abstract market data interfaces
  2. Month 2: Implement crypto trading support
  3. Month 3: Add forex and options trading
  4. Month 4: Performance optimization and testing

🎯 Success Metrics

Technical KPIs

  • Latency: < 100ms for market data updates
  • Throughput: > 10,000 requests/second
  • Availability: 99.9% uptime
  • Error Rate: < 0.1% of requests

Business KPIs

  • Strategy Performance: Sharpe ratio > 1.5
  • Risk Management: Max drawdown < 5%
  • Execution Quality: Slippage < 0.01%
  • System Adoption: > 90% user satisfaction

This architecture document serves as a living blueprint for the Stock Bot Trading System, providing clear guidance for current development and future scaling decisions.