# Stock Bot - System Architecture > **Updated**: June 2025 ## Overview TypeScript microservices architecture for automated stock trading with real-time data processing and multi-database storage. ## Core Services ``` ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ Data Service │ │Processing Service│ │Strategy Service │ │ • Market Data │────▶│ • Indicators │────▶│ • Strategies │ │ • Providers │ │ • Analytics │ │ • Backtesting │ │ • QuestDB │ │ • Validation │ │ • Signal Gen │ └─────────────────┘ └─────────────────┘ └─────────────────┘ │ │ │ │ ┌─────────────────┐ │ └──────────────▶│ Event Bus │◀─────────────┘ │ (Dragonfly) │ └─────────────────┘ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │Execution Service│ │Portfolio Service│ │ Dashboard │ │ • Order Mgmt │ │ • Positions │ │ • Angular UI │ │ • Risk Control │ │ • Risk Mgmt │ │ • Real-time │ │ • Execution │ │ • Performance │ │ • Analytics │ └─────────────────┘ └─────────────────┘ └─────────────────┘ ``` ## Services Structure ``` stock-bot/ ├── apps/ │ ├── data-service/ # Market data ingestion & storage │ ├── execution-service/ # Order execution & broker integration │ ├── portfolio-service/ # Position & risk management │ ├── processing-service/ # Data processing & indicators │ ├── strategy-service/ # Trading strategies & backtesting │ └── dashboard/ # Angular UI (port 4200) │ ├── libs/ # Shared libraries │ ├── logger/ # Centralized logging w/ Loki │ ├── config/ # Configuration management │ ├── event-bus/ # Event system │ ├── mongodb-client/ # MongoDB operations │ ├── postgres-client/ # PostgreSQL operations │ ├── questdb-client/ # Time-series data │ ├── http/ # HTTP client w/ proxy support │ ├── cache/ # Caching layer │ └── utils/ # Common utilities │ └── database/ # Database configurations ├── mongodb/init/ └── postgres/init/ ``` ## Technology Stack | Component | Technology | Purpose | |-----------|------------|---------| | **Runtime** | Bun | Fast JavaScript runtime | | **Language** | TypeScript | Type-safe development | | **Databases** | PostgreSQL, MongoDB, QuestDB | Multi-database architecture | | **Caching** | Dragonfly (Redis) | Event bus & caching | | **Frontend** | Angular 18 | Modern reactive UI | | **Monitoring** | Prometheus, Grafana, Loki | Observability stack | ## Quick Start ```bash # Install dependencies bun install # Start infrastructure bun run infra:up # Start services bun run dev # Access dashboard # http://localhost:4200 ``` ## Key Features - **Real-time Trading**: Live market data & order execution - **Multi-Database**: PostgreSQL, MongoDB, QuestDB for different data types - **Event-Driven**: Asynchronous communication via Dragonfly - **Monitoring**: Full observability with metrics, logs, and tracing - **Modular**: Shared libraries for common functionality - **Type-Safe**: Full TypeScript coverage │ ├── processing-service/ # Combined processing & indicators │ │ ├── src/ │ │ │ ├── indicators/ # Technical indicators (uses @stock-bot/utils) │ │ │ ├── processors/ # Data processing pipeline │ │ │ ├── vectorized/ # Vectorized calculations │ │ │ ├── services/ │ │ │ └── index.ts │ │ └── package.json │ │ │ ├── strategy-service/ # Combined strategy & backtesting │ │ ├── src/ │ │ │ ├── strategies/ # Strategy implementations │ │ │ ├── backtesting/ # Multi-mode backtesting engine │ │ │ │ ├── modes/ # Backtesting modes │ │ │ │ │ ├── live-mode.ts # Live trading mode │ │ │ │ │ ├── event-mode.ts # Event-driven backtest │ │ │ │ │ └── vector-mode.ts # Vectorized backtest │ │ │ │ ├── engines/ # Execution engines │ │ │ │ │ ├── event-engine.ts # Event-based simulation │ │ │ │ │ ├── vector-engine.ts # Vectorized calculations │ │ │ │ │ └── hybrid-engine.ts # Combined validation │ │ │ │ ├── simulator.ts # Market simulator │ │ │ │ ├── runner.ts # Backtest orchestrator │ │ │ │ └── metrics.ts # Performance analysis │ │ │ ├── live/ # Live strategy execution │ │ │ ├── framework/ # Strategy framework │ │ │ │ ├── base-strategy.ts │ │ │ │ ├── execution-mode.ts │ │ │ │ └── mode-factory.ts │ │ │ └── index.ts │ │ └── package.json │ │ │ ├── execution-service/ # Combined order execution & simulation │ │ ├── src/ │ │ │ ├── brokers/ # Live broker adapters │ │ │ ├── simulation/ # Simulated execution │ │ │ ├── unified/ # Unified execution interface │ │ │ │ ├── executor.ts # Abstract executor │ │ │ │ ├── live-executor.ts │ │ │ │ ├── sim-executor.ts │ │ │ │ └── vector-executor.ts │ │ │ └── index.ts │ │ └── package.json │ │ │ ├── portfolio-service/ # Combined portfolio & risk management │ │ ├── src/ │ │ │ ├── portfolio/ # Portfolio tracking │ │ │ ├── risk/ # Risk management (uses @stock-bot/utils) │ │ │ ├── positions/ # Position management │ │ │ ├── performance/ # Performance tracking │ │ │ └── index.ts │ │ └── package.json │ │ │ └── dashboard/ # Combined API & reporting │ ├── src/ │ │ ├── api/ # REST API │ │ ├── web/ # Web interface (Angular) │ │ ├── reports/ # Report generation │ │ ├── websockets/ # Real-time updates │ │ └── index.ts │ └── package.json │ ├── libs/ # ✅ Your existing shared libraries │ ├── config/ # ✅ Environment configuration │ ├── http/ # ✅ HTTP utilities │ ├── logger/ # ✅ Loki-integrated logging │ ├── mongodb-client/ # ✅ MongoDB operations │ ├── postgres-client/ # ✅ PostgreSQL operations │ ├── questdb-client/ # ✅ Time-series data │ ├── types/ # ✅ Shared TypeScript types │ ├── utils/ # ✅ Calculations & utilities │ ├── event-bus/ # 🆕 Dragonfly event system │ ├── strategy-engine/ # 🆕 Strategy framework │ ├── vector-engine/ # 🆕 Vectorized calculations │ └── data-frame/ # 🆕 DataFrame operations ``` ## Multi-Mode Backtesting Architecture ### 1. Execution Mode Framework ```typescript export abstract class ExecutionMode { protected logger = createLogger(this.constructor.name); protected config = new ServiceConfig(); abstract name: string; abstract executeOrder(order: Order): Promise; abstract getCurrentTime(): Date; abstract getMarketData(symbol: string): Promise; abstract publishEvent(event: string, data: any): Promise; } export enum BacktestMode { LIVE = 'live', EVENT_DRIVEN = 'event-driven', VECTORIZED = 'vectorized', HYBRID = 'hybrid' } ``` ### 2. Live Trading Mode ```typescript export class LiveMode extends ExecutionMode { name = 'live'; private broker = new BrokerClient(this.config.getBrokerConfig()); private eventBus = new EventBus(); async executeOrder(order: Order): Promise { this.logger.info('Executing live order', { orderId: order.id }); // Execute via real broker const result = await this.broker.placeOrder(order); // Publish to event bus await this.eventBus.publish('order.executed', result); return result; } getCurrentTime(): Date { return new Date(); // Real time } async getMarketData(symbol: string): Promise { // Get live market data return await this.marketDataService.getLiveData(symbol); } async publishEvent(event: string, data: any): Promise { await this.eventBus.publish(event, data); } } ``` ### 3. Event-Driven Backtesting Mode ```typescript export class EventBacktestMode extends ExecutionMode { name = 'event-driven'; private simulator = new MarketSimulator(); private eventBus = new InMemoryEventBus(); // In-memory for simulation private simulationTime: Date; private historicalData: Map; constructor(private config: BacktestConfig) { super(); this.simulationTime = config.startDate; } async executeOrder(order: Order): Promise { this.logger.debug('Simulating order execution', { orderId: order.id, simulationTime: this.simulationTime }); // Realistic order simulation with slippage, fees const result = await this.simulator.executeOrder(order, { currentTime: this.simulationTime, marketData: await this.getMarketData(order.symbol), slippageModel: this.config.slippageModel, commissionModel: this.config.commissionModel }); // Publish to simulation event bus await this.eventBus.publish('order.executed', result); return result; } getCurrentTime(): Date { return this.simulationTime; } async getMarketData(symbol: string): Promise { const data = this.historicalData.get(symbol) || []; return data.find(d => d.timestamp <= this.simulationTime) || null; } async publishEvent(event: string, data: any): Promise { await this.eventBus.publish(event, data); } // Progress simulation time advanceTime(newTime: Date): void { this.simulationTime = newTime; } } ``` ### 4. Vectorized Backtesting Mode ```typescript export class VectorBacktestMode extends ExecutionMode { name = 'vectorized'; private dataFrame: DataFrame; private currentIndex: number = 0; constructor(private config: VectorBacktestConfig) { super(); this.dataFrame = new DataFrame(config.historicalData); } // Vectorized execution - processes entire dataset at once async executeVectorizedBacktest(strategy: VectorizedStrategy): Promise { const startTime = Date.now(); this.logger.info('Starting vectorized backtest', { strategy: strategy.name, dataPoints: this.dataFrame.length }); // Generate all signals at once using your utils library const signals = this.generateVectorizedSignals(strategy); // Calculate performance metrics vectorized const performance = this.calculateVectorizedPerformance(signals); // Apply trading costs if specified if (this.config.tradingCosts) { this.applyTradingCosts(performance, signals); } const executionTime = Date.now() - startTime; this.logger.info('Vectorized backtest completed', { executionTime, totalReturn: performance.totalReturn, sharpeRatio: performance.sharpeRatio }); return { mode: 'vectorized', strategy: strategy.name, performance, executionTime, signals }; } private generateVectorizedSignals(strategy: VectorizedStrategy): DataFrame { const prices = this.dataFrame.get('close'); // Use your existing technical indicators from @stock-bot/utils const indicators = { sma20: sma(prices, 20), sma50: sma(prices, 50), rsi: rsi(prices, 14), macd: macd(prices) }; // Generate position signals vectorized const positions = strategy.generatePositions(this.dataFrame, indicators); return new DataFrame({ ...this.dataFrame.toObject(), ...indicators, positions }); } private calculateVectorizedPerformance(signals: DataFrame): PerformanceMetrics { const prices = signals.get('close'); const positions = signals.get('positions'); // Calculate returns vectorized const returns = prices.slice(1).map((price, i) => (price - prices[i]) / prices[i] ); // Strategy returns = position[t-1] * market_return[t] const strategyReturns = returns.map((ret, i) => (positions[i] || 0) * ret ); // Use your existing performance calculation utilities return { totalReturn: calculateTotalReturn(strategyReturns), sharpeRatio: calculateSharpeRatio(strategyReturns), maxDrawdown: calculateMaxDrawdown(strategyReturns), volatility: calculateVolatility(strategyReturns), winRate: calculateWinRate(strategyReturns) }; } // Standard interface methods (not used in vectorized mode) async executeOrder(order: Order): Promise { throw new Error('Use executeVectorizedBacktest for vectorized mode'); } getCurrentTime(): Date { return this.dataFrame.getTimestamp(this.currentIndex); } async getMarketData(symbol: string): Promise { return this.dataFrame.getRow(this.currentIndex); } async publishEvent(event: string, data: any): Promise { // No-op for vectorized mode } } ``` ### 5. Hybrid Validation Mode ```typescript export class HybridBacktestMode extends ExecutionMode { name = 'hybrid'; private eventMode: EventBacktestMode; private vectorMode: VectorBacktestMode; constructor(config: BacktestConfig) { super(); this.eventMode = new EventBacktestMode(config); this.vectorMode = new VectorBacktestMode(config); } async validateStrategy( strategy: BaseStrategy, tolerance: number = 0.001 ): Promise { this.logger.info('Starting hybrid validation', { strategy: strategy.name, tolerance }); // Run vectorized backtest (fast) const vectorResult = await this.vectorMode.executeVectorizedBacktest( strategy as VectorizedStrategy ); // Run event-driven backtest (realistic) const eventResult = await this.runEventBacktest(strategy); // Compare results const performanceDiff = Math.abs( vectorResult.performance.totalReturn - eventResult.performance.totalReturn ); const isValid = performanceDiff < tolerance; this.logger.info('Hybrid validation completed', { isValid, performanceDifference: performanceDiff, recommendation: isValid ? 'vectorized' : 'event-driven' }); return { isValid, performanceDifference: performanceDiff, vectorizedResult: vectorResult, eventResult, recommendation: isValid ? 'Vectorized results are reliable for this strategy' : 'Use event-driven backtesting for accurate results' }; } // Standard interface methods delegate to event mode async executeOrder(order: Order): Promise { return await this.eventMode.executeOrder(order); } getCurrentTime(): Date { return this.eventMode.getCurrentTime(); } async getMarketData(symbol: string): Promise { return await this.eventMode.getMarketData(symbol); } async publishEvent(event: string, data: any): Promise { await this.eventMode.publishEvent(event, data); } } ``` ## Unified Strategy Implementation ### Base Strategy Framework ```typescript export abstract class BaseStrategy { protected mode: ExecutionMode; protected logger = createLogger(this.constructor.name); abstract name: string; abstract parameters: Record; constructor(mode: ExecutionMode) { this.mode = mode; } // Works identically across all modes abstract onPriceUpdate(data: PriceData): Promise; abstract onIndicatorUpdate(data: IndicatorData): Promise; protected async emitSignal(signal: TradeSignal): Promise { this.logger.debug('Emitting trade signal', { signal }); // Mode handles whether this is live, simulated, or vectorized const order = this.createOrder(signal); const result = await this.mode.executeOrder(order); await this.mode.publishEvent('trade.executed', { signal, order, result, timestamp: this.mode.getCurrentTime() }); } private createOrder(signal: TradeSignal): Order { return { id: generateId(), symbol: signal.symbol, side: signal.action, quantity: signal.quantity, type: 'market', timestamp: this.mode.getCurrentTime() }; } } // Vectorized strategy interface export interface VectorizedStrategy { name: string; parameters: Record; generatePositions(data: DataFrame, indicators: any): number[]; } ``` ### Example Strategy Implementation ```typescript export class SMAStrategy extends BaseStrategy implements VectorizedStrategy { name = 'SMA-Crossover'; parameters = { fastPeriod: 10, slowPeriod: 20 }; private fastSMA: number[] = []; private slowSMA: number[] = []; async onPriceUpdate(data: PriceData): Promise { // Same logic for live, event-driven, and hybrid modes this.fastSMA.push(data.close); this.slowSMA.push(data.close); if (this.fastSMA.length > this.parameters.fastPeriod) { this.fastSMA.shift(); } if (this.slowSMA.length > this.parameters.slowPeriod) { this.slowSMA.shift(); } if (this.fastSMA.length === this.parameters.fastPeriod && this.slowSMA.length === this.parameters.slowPeriod) { const fastAvg = sma(this.fastSMA, this.parameters.fastPeriod)[0]; const slowAvg = sma(this.slowSMA, this.parameters.slowPeriod)[0]; if (fastAvg > slowAvg) { await this.emitSignal({ symbol: data.symbol, action: 'BUY', quantity: 100, confidence: 0.8 }); } else if (fastAvg < slowAvg) { await this.emitSignal({ symbol: data.symbol, action: 'SELL', quantity: 100, confidence: 0.8 }); } } } async onIndicatorUpdate(data: IndicatorData): Promise { // Handle pre-calculated indicators } // Vectorized implementation for fast backtesting generatePositions(data: DataFrame, indicators: any): number[] { const { sma20: fastSMA, sma50: slowSMA } = indicators; return fastSMA.map((fast, i) => { const slow = slowSMA[i]; if (isNaN(fast) || isNaN(slow)) return 0; // Long when fast > slow, short when fast < slow return fast > slow ? 1 : (fast < slow ? -1 : 0); }); } } ``` ## Mode Factory and Service Integration ### Mode Factory ```typescript export class ModeFactory { static create(mode: BacktestMode, config: any): ExecutionMode { switch (mode) { case BacktestMode.LIVE: return new LiveMode(); case BacktestMode.EVENT_DRIVEN: return new EventBacktestMode(config); case BacktestMode.VECTORIZED: return new VectorBacktestMode(config); case BacktestMode.HYBRID: return new HybridBacktestMode(config); default: throw new Error(`Unknown mode: ${mode}`); } } } ``` ### Strategy Service Integration ```typescript export class StrategyService { private logger = createLogger('strategy-service'); async runStrategy( strategyName: string, mode: BacktestMode, config: any ): Promise { const executionMode = ModeFactory.create(mode, config); const strategy = await this.loadStrategy(strategyName, executionMode); this.logger.info('Starting strategy execution', { strategy: strategyName, mode, config }); switch (mode) { case BacktestMode.LIVE: return await this.runLiveStrategy(strategy); case BacktestMode.EVENT_DRIVEN: return await this.runEventBacktest(strategy, config); case BacktestMode.VECTORIZED: return await (executionMode as VectorBacktestMode) .executeVectorizedBacktest(strategy as VectorizedStrategy); case BacktestMode.HYBRID: return await (executionMode as HybridBacktestMode) .validateStrategy(strategy, config.tolerance); default: throw new Error(`Unsupported mode: ${mode}`); } } async optimizeStrategy( strategyName: string, parameterGrid: Record, config: BacktestConfig ): Promise { const results: OptimizationResult[] = []; const combinations = this.generateParameterCombinations(parameterGrid); this.logger.info('Starting parameter optimization', { strategy: strategyName, combinations: combinations.length }); // Use vectorized mode for fast parameter optimization const vectorMode = new VectorBacktestMode(config); // Can be parallelized await Promise.all( combinations.map(async (params) => { const strategy = await this.loadStrategy(strategyName, vectorMode, params); const result = await vectorMode.executeVectorizedBacktest( strategy as VectorizedStrategy ); results.push({ parameters: params, performance: result.performance, executionTime: result.executionTime }); }) ); // Sort by Sharpe ratio return results.sort((a, b) => b.performance.sharpeRatio - a.performance.sharpeRatio ); } } ``` ## Service Configuration ### Environment-Based Mode Selection ```typescript export class ServiceConfig { getTradingConfig(): TradingConfig { return { mode: (process.env.TRADING_MODE as BacktestMode) || BacktestMode.LIVE, brokerConfig: { apiKey: process.env.BROKER_API_KEY, sandbox: process.env.BROKER_SANDBOX === 'true' }, backtestConfig: { startDate: new Date(process.env.BACKTEST_START_DATE || '2023-01-01'), endDate: new Date(process.env.BACKTEST_END_DATE || '2024-01-01'), initialCapital: parseFloat(process.env.INITIAL_CAPITAL || '100000'), slippageModel: process.env.SLIPPAGE_MODEL || 'linear', commissionModel: process.env.COMMISSION_MODEL || 'fixed' } }; } } ``` ### CLI Interface ```typescript // CLI for running different modes import { Command } from 'commander'; const program = new Command(); program .name('stock-bot') .description('Stock Trading Bot with Multi-Mode Backtesting'); program .command('live') .description('Run live trading') .option('-s, --strategy ', 'Strategy to run') .action(async (options) => { const strategyService = new StrategyService(); await strategyService.runStrategy( options.strategy, BacktestMode.LIVE, {} ); }); program .command('backtest') .description('Run backtesting') .option('-s, --strategy ', 'Strategy to test') .option('-m, --mode ', 'Backtest mode (event|vector|hybrid)', 'event') .option('-f, --from ', 'Start date') .option('-t, --to ', 'End date') .action(async (options) => { const strategyService = new StrategyService(); await strategyService.runStrategy( options.strategy, options.mode as BacktestMode, { startDate: new Date(options.from), endDate: new Date(options.to) } ); }); program .command('optimize') .description('Optimize strategy parameters') .option('-s, --strategy ', 'Strategy to optimize') .option('-p, --params ', 'Parameter grid JSON') .action(async (options) => { const strategyService = new StrategyService(); const paramGrid = JSON.parse(options.params); await strategyService.optimizeStrategy( options.strategy, paramGrid, {} ); }); program.parse(); ``` ## Performance Comparison ### Execution Speed by Mode | Mode | Data Points/Second | Memory Usage | Use Case | |------|-------------------|--------------|----------| | **Live** | Real-time | Low | Production trading | | **Event-Driven** | ~1,000 | Medium | Realistic validation | | **Vectorized** | ~100,000+ | High | Parameter optimization | | **Hybrid** | Combined | Medium | Strategy validation | ### When to Use Each Mode - **Live Mode**: Production trading with real money - **Event-Driven**: Final strategy validation, complex order logic - **Vectorized**: Initial development, parameter optimization, quick testing - **Hybrid**: Validating vectorized results against realistic simulation ## Integration with Your Existing Libraries This architecture leverages all your existing infrastructure: - **@stock-bot/config**: Environment management - **@stock-bot/logger**: Comprehensive logging with Loki - **@stock-bot/utils**: All technical indicators and calculations - **@stock-bot/questdb-client**: Time-series data storage - **@stock-bot/postgres-client**: Transactional data - **@stock-bot/mongodb-client**: Configuration storage ## Key Benefits 1. **Unified Codebase**: Same strategy logic across all modes 2. **Performance Flexibility**: Choose speed vs accuracy based on needs 3. **Validation Pipeline**: Hybrid mode ensures vectorized results are accurate 4. **Production Ready**: Live mode for actual trading 5. **Development Friendly**: Fast iteration with vectorized backtesting This simplified architecture reduces complexity while providing comprehensive backtesting capabilities that scale from rapid prototyping to production trading.