stock-bot/ARCHITECTURE.md

# 🏗️ Stock Bot Trading System - Architecture Documentation

## 📋 Table of Contents
- [System Overview](#system-overview)
- [Current Architecture](#current-architecture)
- [Service Breakdown](#service-breakdown)
- [Data Flow](#data-flow)
- [Technology Stack](#technology-stack)
- [Future Architecture](#future-architecture)
- [Improvement Recommendations](#improvement-recommendations)
- [Deployment Architecture](#deployment-architecture)
- [Security Architecture](#security-architecture)
- [Monitoring & Observability](#monitoring--observability)

---

## 🎯 System Overview

The Stock Bot Trading System is a **microservice-based**, **event-driven** trading platform built for **real-time market analysis**, **strategy execution**, and **risk management**. The system follows a **service-oriented architecture (SOA)** with **clear separation of concerns** and **horizontal scalability**.

### Core Principles
- **Microservices Architecture**: Independent, deployable services
- **Event-Driven Communication**: WebSocket and Redis pub/sub
- **Real-Time Processing**: Sub-second latency requirements
- **Scalable Design**: Horizontal scaling capabilities
- **Fault Tolerance**: Circuit breakers and graceful degradation
- **Type Safety**: Full TypeScript implementation

---

## 🏗️ Current Architecture

```mermaid
graph TB
    subgraph "Frontend Layer"
        UI[Angular Trading Dashboard]
    end
    
    subgraph "API Gateway Layer"
        GW[API Gateway - Future]
    end
    
    subgraph "Core Services"
        MDG[Market Data Gateway<br/>Port 3001]
        RG[Risk Guardian<br/>Port 3002]
        SO[Strategy Orchestrator<br/>Port 4001]
    end
    
    subgraph "Data Layer"
        Redis[(Redis Cache)]
        PG[(PostgreSQL)]
        QDB[(QuestDB)]
        Mongo[(MongoDB)]
    end
    
    subgraph "External APIs"
        Alpha[Alpha Vantage]
        IEX[IEX Cloud]
        Yahoo[Yahoo Finance]
    end
    
    UI -->|WebSocket/HTTP| MDG
    UI -->|WebSocket/HTTP| RG
    UI -->|WebSocket/HTTP| SO
    
    MDG --> Redis
    MDG --> QDB
    MDG -->|Fetch Data| Alpha
    MDG -->|Fetch Data| IEX
    MDG -->|Fetch Data| Yahoo
    
    RG --> Redis
    RG --> PG
    
    SO --> Redis
    SO --> PG
    SO --> Mongo
```

---

## 🔧 Service Breakdown

### **1. Interface Services**

#### **Trading Dashboard** (`apps/interface-services/trading-dashboard`)
- **Framework**: Angular 20 + Angular Material + Tailwind CSS
- **Port**: 4200 (development)
- **Purpose**: Real-time trading interface and strategy management
- **Key Features**:
  - Real-time market data visualization
  - Strategy creation and backtesting UI
  - Risk management dashboard
  - Portfolio monitoring
  - WebSocket integration for live updates

**Current Structure:**
```
trading-dashboard/
├── src/
│   ├── app/
│   │   ├── components/          # Reusable UI components
│   │   │   ├── sidebar/         # Navigation sidebar
│   │   │   └── notifications/   # Alert system
│   │   ├── pages/              # Route-based pages
│   │   │   ├── dashboard/      # Main trading dashboard
│   │   │   ├── market-data/    # Market data visualization
│   │   │   ├── portfolio/      # Portfolio management
│   │   │   ├── strategies/     # Strategy management
│   │   │   └── risk-management/ # Risk controls
│   │   ├── services/           # Angular services
│   │   │   ├── api.service.ts  # HTTP API communication
│   │   │   ├── websocket.service.ts # WebSocket management
│   │   │   └── strategy.service.ts  # Strategy operations
│   │   └── shared/             # Shared utilities
│   └── styles.css              # Global styles
```

### **2. Core Services**

#### **Market Data Gateway** (`apps/core-services/market-data-gateway`)
- **Framework**: Hono + Bun
- **Port**: 3001
- **Purpose**: Market data aggregation and real-time distribution
- **Database**: QuestDB (time-series), Redis (caching)

**Responsibilities:**
- Aggregate data from multiple market data providers
- Real-time WebSocket streaming to clients
- Historical data storage and retrieval
- Rate limiting and API management
- Data normalization and validation

**API Endpoints:**
```
GET  /health                    # Health check
GET  /api/market-data/:symbol   # Get latest market data
GET  /api/historical/:symbol    # Get historical data
WS   /ws                        # WebSocket for real-time data
```

#### **Risk Guardian** (`apps/core-services/risk-guardian`)
- **Framework**: Hono + Bun
- **Port**: 3002
- **Purpose**: Real-time risk monitoring and controls
- **Database**: PostgreSQL (persistent), Redis (real-time)

**Responsibilities:**
- Position size monitoring
- Daily loss tracking
- Portfolio risk assessment
- Volatility monitoring
- Real-time risk alerts
- Risk threshold management

**API Endpoints:**
```
GET  /health                    # Health check
GET  /api/risk/thresholds       # Get risk thresholds
PUT  /api/risk/thresholds       # Update risk thresholds
POST /api/risk/evaluate         # Evaluate position risk
GET  /api/risk/history          # Risk evaluation history
WS   /ws                        # WebSocket for risk alerts
```

#### **Strategy Orchestrator** (`apps/intelligence-services/strategy-orchestrator`)
- **Framework**: Hono + Bun
- **Port**: 4001
- **Purpose**: Strategy lifecycle management and execution
- **Database**: MongoDB (strategies), PostgreSQL (trades), Redis (signals)

**Responsibilities:**
- Strategy creation and management
- Backtesting engine (vectorized & event-based)
- Real-time strategy execution
- Signal generation and broadcasting
- Performance analytics
- Strategy optimization

**Current Structure:**
```
strategy-orchestrator/
├── src/
│   ├── core/
│   │   ├── backtesting/
│   │   │   ├── BacktestEngine.ts      # Main backtesting engine
│   │   │   ├── BacktestService.ts     # Backtesting service layer
│   │   │   ├── MarketDataFeed.ts      # Historical data provider
│   │   │   └── PerformanceAnalytics.ts # Performance metrics
│   │   ├── execution/
│   │   │   └── StrategyExecutionService.ts # Real-time execution
│   │   ├── strategies/
│   │   │   ├── Strategy.ts            # Base strategy interface
│   │   │   ├── StrategyRegistry.ts    # Strategy management
│   │   │   ├── BaseStrategy.ts        # Abstract base class
│   │   │   ├── VectorizedStrategy.ts  # Vectorized base class
│   │   │   ├── MovingAverageCrossover.ts # MA strategy
│   │   │   └── MeanReversionStrategy.ts  # Mean reversion
│   │   └── indicators/
│   │       └── TechnicalIndicators.ts # Technical analysis
│   ├── controllers/
│   │   └── StrategyController.ts      # API endpoints
│   └── index.ts                       # Main entry point
```

**API Endpoints:**
```
GET  /health                           # Health check
GET  /api/strategies                   # List strategies
POST /api/strategies                   # Create strategy
PUT  /api/strategies/:id               # Update strategy
POST /api/strategies/:id/:action       # Start/stop/pause strategy
GET  /api/strategies/:id/signals       # Get strategy signals
POST /api/strategies/:id/backtest      # Run backtest
GET  /api/strategies/:id/performance   # Get performance metrics
WS   /ws                              # WebSocket for strategy updates
```

### **3. Shared Packages**

#### **Shared Types** (`packages/shared-types`)
```typescript
export interface MarketData {
  symbol: string;
  price: number;
  volume: number;
  timestamp: Date;
  bid: number;
  ask: number;
}

export interface Strategy {
  id: string;
  name: string;
  symbols: string[];
  parameters: Record<string, any>;
  status: 'ACTIVE' | 'PAUSED' | 'STOPPED';
}

export interface BacktestResult {
  totalReturn: number;
  sharpeRatio: number;
  maxDrawdown: number;
  winRate: number;
  totalTrades: number;
}
```

#### **Configuration** (`packages/config`)
```typescript
export const config = {
  redis: {
    host: process.env.REDIS_HOST || 'localhost',
    port: parseInt(process.env.REDIS_PORT || '6379')
  },
  database: {
    postgres: process.env.POSTGRES_URL,
    questdb: process.env.QUESTDB_URL,
    mongodb: process.env.MONGODB_URL
  },
  marketData: {
    alphaVantageKey: process.env.ALPHA_VANTAGE_KEY,
    iexKey: process.env.IEX_KEY
  }
};
```

---

## 🔄 Data Flow

### **Real-Time Market Data Flow**
```mermaid
sequenceDiagram
    participant EXT as External APIs
    participant MDG as Market Data Gateway
    participant Redis as Redis Cache
    participant QDB as QuestDB
    participant UI as Trading Dashboard
    
    EXT->>MDG: Market data feed
    MDG->>Redis: Cache latest prices
    MDG->>QDB: Store historical data
    MDG->>UI: WebSocket broadcast
    UI->>UI: Update charts/tables
```

### **Strategy Execution Flow**
```mermaid
sequenceDiagram
    participant UI as Trading Dashboard
    participant SO as Strategy Orchestrator
    participant MDG as Market Data Gateway
    participant RG as Risk Guardian
    participant Redis as Redis
    
    UI->>SO: Start strategy
    SO->>MDG: Subscribe to market data
    MDG->>SO: Real-time price updates
    SO->>SO: Generate trading signals
    SO->>RG: Risk evaluation
    RG->>SO: Risk approval/rejection
    SO->>Redis: Store signals
    SO->>UI: WebSocket signal broadcast
```

---

## 💻 Technology Stack

### **Backend**
- **Runtime**: Bun (ultra-fast JavaScript runtime)
- **Web Framework**: Hono (lightweight, fast web framework)
- **Language**: TypeScript (type safety)
- **Build Tool**: Turbo (monorepo management)

### **Frontend**
- **Framework**: Angular 20 (latest stable)
- **UI Library**: Angular Material + Tailwind CSS
- **State Management**: Angular Signals (reactive programming)
- **WebSocket**: Native WebSocket API

### **Databases**
- **Time-Series**: QuestDB (market data storage)
- **Relational**: PostgreSQL (structured data)
- **Document**: MongoDB (strategy configurations)
- **Cache/Pub-Sub**: Redis (real-time data)

### **Infrastructure**
- **Containerization**: Docker + Docker Compose
- **Process Management**: PM2 (production)
- **Monitoring**: Built-in health checks
- **Development**: Hot reload, TypeScript compilation

---

## 🚀 Future Architecture

### **Phase 1: Enhanced Microservices (Q2 2025)**
```mermaid
graph TB
    subgraph "API Gateway Layer"
        GW[Kong/Envoy API Gateway]
        LB[Load Balancer]
    end
    
    subgraph "Authentication"
        AUTH[Auth Service<br/>JWT + OAuth]
    end
    
    subgraph "Core Services"
        MDG[Market Data Gateway]
        RG[Risk Guardian]
        SO[Strategy Orchestrator]
        OE[Order Execution Engine]
        NS[Notification Service]
    end
    
    subgraph "Analytics Services"
        BA[Backtest Analytics]
        PA[Performance Analytics]
        ML[ML Prediction Service]
    end
    
    subgraph "Message Queue"
        NATS[NATS/Apache Kafka]
    end
```

### **Phase 2: Machine Learning Integration (Q3 2025)**
- **ML Pipeline**: Python-based ML services
- **Feature Engineering**: Real-time feature computation
- **Model Training**: Automated model retraining
- **Prediction API**: Real-time predictions

### **Phase 3: Multi-Asset Support (Q4 2025)**
- **Crypto Trading**: Binance, Coinbase integration
- **Forex Trading**: OANDA, FXCM integration
- **Options Trading**: Interactive Brokers integration
- **Futures Trading**: CME, ICE integration

---

## 📈 Improvement Recommendations

### **1. High Priority Improvements**

#### **API Gateway Implementation**
```typescript
// Implement Kong or Envoy for:
- Rate limiting per service
- Authentication/authorization
- Request/response transformation
- Circuit breaker patterns
- Load balancing
```

#### **Enhanced Error Handling**
```typescript
// Implement structured error handling:
interface ServiceError {
  code: string;
  message: string;
  service: string;
  timestamp: Date;
  correlationId: string;
}
```

#### **Comprehensive Logging**
```typescript
// Implement structured logging:
interface LogEntry {
  level: 'debug' | 'info' | 'warn' | 'error';
  service: string;
  message: string;
  metadata: Record<string, any>;
  timestamp: Date;
  correlationId: string;
}
```

### **2. Medium Priority Improvements**

#### **Database Optimization**
```sql
-- QuestDB optimizations for market data:
CREATE TABLE market_data (
  symbol SYMBOL,
  timestamp TIMESTAMP,
  price DOUBLE,
  volume DOUBLE,
  bid DOUBLE,
  ask DOUBLE
) timestamp(timestamp) PARTITION BY DAY;

-- Add indexes for fast queries:
CREATE INDEX idx_symbol_timestamp ON market_data (symbol, timestamp);
```

#### **Caching Strategy**
```typescript
// Implement multi-layer caching:
interface CacheStrategy {
  L1: Map<string, any>;        // In-memory cache
  L2: Redis;                   // Distributed cache
  L3: Database;                // Persistent storage
}
```

#### **WebSocket Optimization**
```typescript
// Implement WebSocket connection pooling:
interface WSConnectionPool {
  connections: Map<string, WebSocket[]>;
  balancer: RoundRobinBalancer;
  heartbeat: HeartbeatManager;
}
```

### **3. Low Priority Improvements**

#### **Code Quality**
- Implement comprehensive unit tests (>90% coverage)
- Add integration tests for all services
- Implement E2E tests for critical user flows
- Add performance benchmarks

#### **Documentation**
- API documentation with OpenAPI/Swagger
- Developer onboarding guide
- Deployment runbooks
- Architecture decision records (ADRs)

---

## 🏗️ Deployment Architecture

### **Development Environment**
```yaml
# docker-compose.dev.yml
version: '3.8'
services:
  # Databases
  postgres:
    image: postgres:15
    ports: ["5432:5432"]
  
  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]
  
  questdb:
    image: questdb/questdb:latest
    ports: ["9000:9000", "8812:8812"]
  
  mongodb:
    image: mongo:6
    ports: ["27017:27017"]
  
  # Services
  market-data-gateway:
    build: ./apps/core-services/market-data-gateway
    ports: ["3001:3001"]
    depends_on: [redis, questdb]
  
  risk-guardian:
    build: ./apps/core-services/risk-guardian
    ports: ["3002:3002"]
    depends_on: [postgres, redis]
  
  strategy-orchestrator:
    build: ./apps/intelligence-services/strategy-orchestrator
    ports: ["4001:4001"]
    depends_on: [mongodb, postgres, redis]
  
  trading-dashboard:
    build: ./apps/interface-services/trading-dashboard
    ports: ["4200:4200"]
```

### **Production Environment**
```yaml
# kubernetes deployment example
apiVersion: apps/v1
kind: Deployment
metadata:
  name: market-data-gateway
spec:
  replicas: 3
  selector:
    matchLabels:
      app: market-data-gateway
  template:
    metadata:
      labels:
        app: market-data-gateway
    spec:
      containers:
      - name: market-data-gateway
        image: stockbot/market-data-gateway:latest
        ports:
        - containerPort: 3001
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
```

---

## 🔒 Security Architecture

### **Authentication & Authorization**
```typescript
// JWT-based authentication
interface AuthToken {
  userId: string;
  roles: string[];
  permissions: string[];
  expiresAt: Date;
}

// Role-based access control
enum UserRole {
  TRADER = 'TRADER',
  ADMIN = 'ADMIN',
  VIEWER = 'VIEWER'
}

enum Permission {
  READ_MARKET_DATA = 'READ_MARKET_DATA',
  EXECUTE_TRADES = 'EXECUTE_TRADES',
  MANAGE_STRATEGIES = 'MANAGE_STRATEGIES',
  CONFIGURE_RISK = 'CONFIGURE_RISK'
}
```

### **API Security**
```typescript
// Rate limiting configuration
interface RateLimit {
  windowMs: number;      // 15 minutes
  maxRequests: number;   // 100 requests per window
  skipIf: (req: Request) => boolean;
}

// Input validation
interface ApiValidation {
  schema: JSONSchema;
  sanitization: SanitizationRules;
  authentication: AuthenticationMiddleware;
}
```

### **Data Security**
- **Encryption at Rest**: AES-256 for sensitive data
- **Encryption in Transit**: TLS 1.3 for all communications
- **Secrets Management**: Kubernetes secrets or HashiCorp Vault
- **Network Security**: VPC, security groups, firewalls

---

## 📊 Monitoring & Observability

### **Metrics Collection**
```typescript
interface ServiceMetrics {
  // Performance metrics
  requestLatency: Histogram;
  requestRate: Counter;
  errorRate: Counter;
  
  // Business metrics
  tradesExecuted: Counter;
  strategiesActive: Gauge;
  portfolioValue: Gauge;
  
  // System metrics
  memoryUsage: Gauge;
  cpuUsage: Gauge;
  dbConnections: Gauge;
}
```

### **Health Checks**
```typescript
interface HealthCheck {
  service: string;
  status: 'healthy' | 'degraded' | 'unhealthy';
  checks: {
    database: boolean;
    redis: boolean;
    externalApis: boolean;
    webSocket: boolean;
  };
  uptime: number;
  version: string;
}
```

### **Alerting Rules**
```yaml
# Prometheus alerting rules
groups:
- name: stockbot
  rules:
  - alert: HighErrorRate
    expr: rate(http_requests_total{status=~"5.."}[5m]) > 0.1
    for: 2m
    
  - alert: HighLatency
    expr: http_request_duration_seconds{quantile="0.95"} > 1
    for: 2m
    
  - alert: ServiceDown
    expr: up{job="stockbot"} == 0
    for: 30s
```

---

## 📋 Migration Plan

### **Phase 1: Current → Enhanced (1-2 months)**
1. **Week 1-2**: Implement API Gateway and authentication
2. **Week 3-4**: Add comprehensive logging and monitoring
3. **Week 5-6**: Enhance error handling and resilience
4. **Week 7-8**: Performance optimization and testing

### **Phase 2: Enhanced → ML-Ready (2-3 months)**
1. **Month 1**: Implement ML pipeline infrastructure
2. **Month 2**: Develop feature engineering services
3. **Month 3**: Integrate ML predictions into strategies

### **Phase 3: ML-Ready → Multi-Asset (3-4 months)**
1. **Month 1**: Abstract market data interfaces
2. **Month 2**: Implement crypto trading support
3. **Month 3**: Add forex and options trading
4. **Month 4**: Performance optimization and testing

---

## 🎯 Success Metrics

### **Technical KPIs**
- **Latency**: < 100ms for market data updates
- **Throughput**: > 10,000 requests/second
- **Availability**: 99.9% uptime
- **Error Rate**: < 0.1% of requests

### **Business KPIs**
- **Strategy Performance**: Sharpe ratio > 1.5
- **Risk Management**: Max drawdown < 5%
- **Execution Quality**: Slippage < 0.01%
- **System Adoption**: > 90% user satisfaction

---

This architecture document serves as a living blueprint for the Stock Bot Trading System, providing clear guidance for current development and future scaling decisions.