28 KiB
28 KiB
Stock Bot - System Architecture
Updated: June 2025
Overview
TypeScript microservices architecture for automated stock trading with real-time data processing and multi-database storage.
Core Services
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Data Service │ │Processing Service│ │Strategy Service │
│ • Market Data │────▶│ • Indicators │────▶│ • Strategies │
│ • Providers │ │ • Analytics │ │ • Backtesting │
│ • QuestDB │ │ • Validation │ │ • Signal Gen │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│ │ │
│ ┌─────────────────┐ │
└──────────────▶│ Event Bus │◀─────────────┘
│ (Dragonfly) │
└─────────────────┘
│
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│Execution Service│ │Portfolio Service│ │ Dashboard │
│ • Order Mgmt │ │ • Positions │ │ • Angular UI │
│ • Risk Control │ │ • Risk Mgmt │ │ • Real-time │
│ • Execution │ │ • Performance │ │ • Analytics │
└─────────────────┘ └─────────────────┘ └─────────────────┘
Services Structure
stock-bot/
├── apps/
│ ├── data-service/ # Market data ingestion & storage
│ ├── execution-service/ # Order execution & broker integration
│ ├── portfolio-service/ # Position & risk management
│ ├── processing-service/ # Data processing & indicators
│ ├── strategy-service/ # Trading strategies & backtesting
│ └── dashboard/ # Angular UI (port 4200)
│
├── libs/ # Shared libraries
│ ├── logger/ # Centralized logging w/ Loki
│ ├── config/ # Configuration management
│ ├── event-bus/ # Event system
│ ├── mongodb-client/ # MongoDB operations
│ ├── postgres-client/ # PostgreSQL operations
│ ├── questdb-client/ # Time-series data
│ ├── http/ # HTTP client w/ proxy support
│ ├── cache/ # Caching layer
│ └── utils/ # Common utilities
│
└── database/ # Database configurations
├── mongodb/init/
└── postgres/init/
Technology Stack
| Component | Technology | Purpose |
|---|---|---|
| Runtime | Bun | Fast JavaScript runtime |
| Language | TypeScript | Type-safe development |
| Databases | PostgreSQL, MongoDB, QuestDB | Multi-database architecture |
| Caching | Dragonfly (Redis) | Event bus & caching |
| Frontend | Angular 18 | Modern reactive UI |
| Monitoring | Prometheus, Grafana, Loki | Observability stack |
Quick Start
# Install dependencies
bun install
# Start infrastructure
bun run infra:up
# Start services
bun run dev
# Access dashboard
# http://localhost:4200
Key Features
- Real-time Trading: Live market data & order execution
- Multi-Database: PostgreSQL, MongoDB, QuestDB for different data types
- Event-Driven: Asynchronous communication via Dragonfly
- Monitoring: Full observability with metrics, logs, and tracing
- Modular: Shared libraries for common functionality
- Type-Safe: Full TypeScript coverage │ ├── processing-service/ # Combined processing & indicators │ │ ├── src/ │ │ │ ├── indicators/ # Technical indicators (uses @stock-bot/utils) │ │ │ ├── processors/ # Data processing pipeline │ │ │ ├── vectorized/ # Vectorized calculations │ │ │ ├── services/ │ │ │ └── index.ts │ │ └── package.json │ │ │ ├── strategy-service/ # Combined strategy & backtesting │ │ ├── src/ │ │ │ ├── strategies/ # Strategy implementations │ │ │ ├── backtesting/ # Multi-mode backtesting engine │ │ │ │ ├── modes/ # Backtesting modes │ │ │ │ │ ├── live-mode.ts # Live trading mode │ │ │ │ │ ├── event-mode.ts # Event-driven backtest │ │ │ │ │ └── vector-mode.ts # Vectorized backtest │ │ │ │ ├── engines/ # Execution engines │ │ │ │ │ ├── event-engine.ts # Event-based simulation │ │ │ │ │ ├── vector-engine.ts # Vectorized calculations │ │ │ │ │ └── hybrid-engine.ts # Combined validation │ │ │ │ ├── simulator.ts # Market simulator │ │ │ │ ├── runner.ts # Backtest orchestrator │ │ │ │ └── metrics.ts # Performance analysis │ │ │ ├── live/ # Live strategy execution │ │ │ ├── framework/ # Strategy framework │ │ │ │ ├── base-strategy.ts │ │ │ │ ├── execution-mode.ts │ │ │ │ └── mode-factory.ts │ │ │ └── index.ts │ │ └── package.json │ │ │ ├── execution-service/ # Combined order execution & simulation │ │ ├── src/ │ │ │ ├── brokers/ # Live broker adapters │ │ │ ├── simulation/ # Simulated execution │ │ │ ├── unified/ # Unified execution interface │ │ │ │ ├── executor.ts # Abstract executor │ │ │ │ ├── live-executor.ts │ │ │ │ ├── sim-executor.ts │ │ │ │ └── vector-executor.ts │ │ │ └── index.ts │ │ └── package.json │ │ │ ├── portfolio-service/ # Combined portfolio & risk management │ │ ├── src/ │ │ │ ├── portfolio/ # Portfolio tracking │ │ │ ├── risk/ # Risk management (uses @stock-bot/utils) │ │ │ ├── positions/ # Position management │ │ │ ├── performance/ # Performance tracking │ │ │ └── index.ts │ │ └── package.json │ │ │ └── dashboard/ # Combined API & reporting │ ├── src/ │ │ ├── api/ # REST API │ │ ├── web/ # Web interface (Angular) │ │ ├── reports/ # Report generation │ │ ├── websockets/ # Real-time updates │ │ └── index.ts │ └── package.json │ ├── libs/ # ✅ Your existing shared libraries │ ├── config/ # ✅ Environment configuration │ ├── http/ # ✅ HTTP utilities │ ├── logger/ # ✅ Loki-integrated logging │ ├── mongodb-client/ # ✅ MongoDB operations │ ├── postgres-client/ # ✅ PostgreSQL operations │ ├── questdb-client/ # ✅ Time-series data │ ├── types/ # ✅ Shared TypeScript types │ ├── utils/ # ✅ Calculations & utilities │ ├── event-bus/ # 🆕 Dragonfly event system │ ├── strategy-engine/ # 🆕 Strategy framework │ ├── vector-engine/ # 🆕 Vectorized calculations │ └── data-frame/ # 🆕 DataFrame operations
## Multi-Mode Backtesting Architecture
### 1. Execution Mode Framework
```typescript
export abstract class ExecutionMode {
protected logger = createLogger(this.constructor.name);
protected config = new ServiceConfig();
abstract name: string;
abstract executeOrder(order: Order): Promise<OrderResult>;
abstract getCurrentTime(): Date;
abstract getMarketData(symbol: string): Promise<MarketData>;
abstract publishEvent(event: string, data: any): Promise<void>;
}
export enum BacktestMode {
LIVE = 'live',
EVENT_DRIVEN = 'event-driven',
VECTORIZED = 'vectorized',
HYBRID = 'hybrid'
}
2. Live Trading Mode
export class LiveMode extends ExecutionMode {
name = 'live';
private broker = new BrokerClient(this.config.getBrokerConfig());
private eventBus = new EventBus();
async executeOrder(order: Order): Promise<OrderResult> {
this.logger.info('Executing live order', { orderId: order.id });
// Execute via real broker
const result = await this.broker.placeOrder(order);
// Publish to event bus
await this.eventBus.publish('order.executed', result);
return result;
}
getCurrentTime(): Date {
return new Date(); // Real time
}
async getMarketData(symbol: string): Promise<MarketData> {
// Get live market data
return await this.marketDataService.getLiveData(symbol);
}
async publishEvent(event: string, data: any): Promise<void> {
await this.eventBus.publish(event, data);
}
}
3. Event-Driven Backtesting Mode
export class EventBacktestMode extends ExecutionMode {
name = 'event-driven';
private simulator = new MarketSimulator();
private eventBus = new InMemoryEventBus(); // In-memory for simulation
private simulationTime: Date;
private historicalData: Map<string, MarketData[]>;
constructor(private config: BacktestConfig) {
super();
this.simulationTime = config.startDate;
}
async executeOrder(order: Order): Promise<OrderResult> {
this.logger.debug('Simulating order execution', {
orderId: order.id,
simulationTime: this.simulationTime
});
// Realistic order simulation with slippage, fees
const result = await this.simulator.executeOrder(order, {
currentTime: this.simulationTime,
marketData: await this.getMarketData(order.symbol),
slippageModel: this.config.slippageModel,
commissionModel: this.config.commissionModel
});
// Publish to simulation event bus
await this.eventBus.publish('order.executed', result);
return result;
}
getCurrentTime(): Date {
return this.simulationTime;
}
async getMarketData(symbol: string): Promise<MarketData> {
const data = this.historicalData.get(symbol) || [];
return data.find(d => d.timestamp <= this.simulationTime) || null;
}
async publishEvent(event: string, data: any): Promise<void> {
await this.eventBus.publish(event, data);
}
// Progress simulation time
advanceTime(newTime: Date): void {
this.simulationTime = newTime;
}
}
4. Vectorized Backtesting Mode
export class VectorBacktestMode extends ExecutionMode {
name = 'vectorized';
private dataFrame: DataFrame;
private currentIndex: number = 0;
constructor(private config: VectorBacktestConfig) {
super();
this.dataFrame = new DataFrame(config.historicalData);
}
// Vectorized execution - processes entire dataset at once
async executeVectorizedBacktest(strategy: VectorizedStrategy): Promise<BacktestResult> {
const startTime = Date.now();
this.logger.info('Starting vectorized backtest', {
strategy: strategy.name,
dataPoints: this.dataFrame.length
});
// Generate all signals at once using your utils library
const signals = this.generateVectorizedSignals(strategy);
// Calculate performance metrics vectorized
const performance = this.calculateVectorizedPerformance(signals);
// Apply trading costs if specified
if (this.config.tradingCosts) {
this.applyTradingCosts(performance, signals);
}
const executionTime = Date.now() - startTime;
this.logger.info('Vectorized backtest completed', {
executionTime,
totalReturn: performance.totalReturn,
sharpeRatio: performance.sharpeRatio
});
return {
mode: 'vectorized',
strategy: strategy.name,
performance,
executionTime,
signals
};
}
private generateVectorizedSignals(strategy: VectorizedStrategy): DataFrame {
const prices = this.dataFrame.get('close');
// Use your existing technical indicators from @stock-bot/utils
const indicators = {
sma20: sma(prices, 20),
sma50: sma(prices, 50),
rsi: rsi(prices, 14),
macd: macd(prices)
};
// Generate position signals vectorized
const positions = strategy.generatePositions(this.dataFrame, indicators);
return new DataFrame({
...this.dataFrame.toObject(),
...indicators,
positions
});
}
private calculateVectorizedPerformance(signals: DataFrame): PerformanceMetrics {
const prices = signals.get('close');
const positions = signals.get('positions');
// Calculate returns vectorized
const returns = prices.slice(1).map((price, i) =>
(price - prices[i]) / prices[i]
);
// Strategy returns = position[t-1] * market_return[t]
const strategyReturns = returns.map((ret, i) =>
(positions[i] || 0) * ret
);
// Use your existing performance calculation utilities
return {
totalReturn: calculateTotalReturn(strategyReturns),
sharpeRatio: calculateSharpeRatio(strategyReturns),
maxDrawdown: calculateMaxDrawdown(strategyReturns),
volatility: calculateVolatility(strategyReturns),
winRate: calculateWinRate(strategyReturns)
};
}
// Standard interface methods (not used in vectorized mode)
async executeOrder(order: Order): Promise<OrderResult> {
throw new Error('Use executeVectorizedBacktest for vectorized mode');
}
getCurrentTime(): Date {
return this.dataFrame.getTimestamp(this.currentIndex);
}
async getMarketData(symbol: string): Promise<MarketData> {
return this.dataFrame.getRow(this.currentIndex);
}
async publishEvent(event: string, data: any): Promise<void> {
// No-op for vectorized mode
}
}
5. Hybrid Validation Mode
export class HybridBacktestMode extends ExecutionMode {
name = 'hybrid';
private eventMode: EventBacktestMode;
private vectorMode: VectorBacktestMode;
constructor(config: BacktestConfig) {
super();
this.eventMode = new EventBacktestMode(config);
this.vectorMode = new VectorBacktestMode(config);
}
async validateStrategy(
strategy: BaseStrategy,
tolerance: number = 0.001
): Promise<ValidationResult> {
this.logger.info('Starting hybrid validation', {
strategy: strategy.name,
tolerance
});
// Run vectorized backtest (fast)
const vectorResult = await this.vectorMode.executeVectorizedBacktest(
strategy as VectorizedStrategy
);
// Run event-driven backtest (realistic)
const eventResult = await this.runEventBacktest(strategy);
// Compare results
const performanceDiff = Math.abs(
vectorResult.performance.totalReturn -
eventResult.performance.totalReturn
);
const isValid = performanceDiff < tolerance;
this.logger.info('Hybrid validation completed', {
isValid,
performanceDifference: performanceDiff,
recommendation: isValid ? 'vectorized' : 'event-driven'
});
return {
isValid,
performanceDifference: performanceDiff,
vectorizedResult: vectorResult,
eventResult,
recommendation: isValid ?
'Vectorized results are reliable for this strategy' :
'Use event-driven backtesting for accurate results'
};
}
// Standard interface methods delegate to event mode
async executeOrder(order: Order): Promise<OrderResult> {
return await this.eventMode.executeOrder(order);
}
getCurrentTime(): Date {
return this.eventMode.getCurrentTime();
}
async getMarketData(symbol: string): Promise<MarketData> {
return await this.eventMode.getMarketData(symbol);
}
async publishEvent(event: string, data: any): Promise<void> {
await this.eventMode.publishEvent(event, data);
}
}
Unified Strategy Implementation
Base Strategy Framework
export abstract class BaseStrategy {
protected mode: ExecutionMode;
protected logger = createLogger(this.constructor.name);
abstract name: string;
abstract parameters: Record<string, any>;
constructor(mode: ExecutionMode) {
this.mode = mode;
}
// Works identically across all modes
abstract onPriceUpdate(data: PriceData): Promise<void>;
abstract onIndicatorUpdate(data: IndicatorData): Promise<void>;
protected async emitSignal(signal: TradeSignal): Promise<void> {
this.logger.debug('Emitting trade signal', { signal });
// Mode handles whether this is live, simulated, or vectorized
const order = this.createOrder(signal);
const result = await this.mode.executeOrder(order);
await this.mode.publishEvent('trade.executed', {
signal,
order,
result,
timestamp: this.mode.getCurrentTime()
});
}
private createOrder(signal: TradeSignal): Order {
return {
id: generateId(),
symbol: signal.symbol,
side: signal.action,
quantity: signal.quantity,
type: 'market',
timestamp: this.mode.getCurrentTime()
};
}
}
// Vectorized strategy interface
export interface VectorizedStrategy {
name: string;
parameters: Record<string, any>;
generatePositions(data: DataFrame, indicators: any): number[];
}
Example Strategy Implementation
export class SMAStrategy extends BaseStrategy implements VectorizedStrategy {
name = 'SMA-Crossover';
parameters = { fastPeriod: 10, slowPeriod: 20 };
private fastSMA: number[] = [];
private slowSMA: number[] = [];
async onPriceUpdate(data: PriceData): Promise<void> {
// Same logic for live, event-driven, and hybrid modes
this.fastSMA.push(data.close);
this.slowSMA.push(data.close);
if (this.fastSMA.length > this.parameters.fastPeriod) {
this.fastSMA.shift();
}
if (this.slowSMA.length > this.parameters.slowPeriod) {
this.slowSMA.shift();
}
if (this.fastSMA.length === this.parameters.fastPeriod &&
this.slowSMA.length === this.parameters.slowPeriod) {
const fastAvg = sma(this.fastSMA, this.parameters.fastPeriod)[0];
const slowAvg = sma(this.slowSMA, this.parameters.slowPeriod)[0];
if (fastAvg > slowAvg) {
await this.emitSignal({
symbol: data.symbol,
action: 'BUY',
quantity: 100,
confidence: 0.8
});
} else if (fastAvg < slowAvg) {
await this.emitSignal({
symbol: data.symbol,
action: 'SELL',
quantity: 100,
confidence: 0.8
});
}
}
}
async onIndicatorUpdate(data: IndicatorData): Promise<void> {
// Handle pre-calculated indicators
}
// Vectorized implementation for fast backtesting
generatePositions(data: DataFrame, indicators: any): number[] {
const { sma20: fastSMA, sma50: slowSMA } = indicators;
return fastSMA.map((fast, i) => {
const slow = slowSMA[i];
if (isNaN(fast) || isNaN(slow)) return 0;
// Long when fast > slow, short when fast < slow
return fast > slow ? 1 : (fast < slow ? -1 : 0);
});
}
}
Mode Factory and Service Integration
Mode Factory
export class ModeFactory {
static create(mode: BacktestMode, config: any): ExecutionMode {
switch (mode) {
case BacktestMode.LIVE:
return new LiveMode();
case BacktestMode.EVENT_DRIVEN:
return new EventBacktestMode(config);
case BacktestMode.VECTORIZED:
return new VectorBacktestMode(config);
case BacktestMode.HYBRID:
return new HybridBacktestMode(config);
default:
throw new Error(`Unknown mode: ${mode}`);
}
}
}
Strategy Service Integration
export class StrategyService {
private logger = createLogger('strategy-service');
async runStrategy(
strategyName: string,
mode: BacktestMode,
config: any
): Promise<any> {
const executionMode = ModeFactory.create(mode, config);
const strategy = await this.loadStrategy(strategyName, executionMode);
this.logger.info('Starting strategy execution', {
strategy: strategyName,
mode,
config
});
switch (mode) {
case BacktestMode.LIVE:
return await this.runLiveStrategy(strategy);
case BacktestMode.EVENT_DRIVEN:
return await this.runEventBacktest(strategy, config);
case BacktestMode.VECTORIZED:
return await (executionMode as VectorBacktestMode)
.executeVectorizedBacktest(strategy as VectorizedStrategy);
case BacktestMode.HYBRID:
return await (executionMode as HybridBacktestMode)
.validateStrategy(strategy, config.tolerance);
default:
throw new Error(`Unsupported mode: ${mode}`);
}
}
async optimizeStrategy(
strategyName: string,
parameterGrid: Record<string, any[]>,
config: BacktestConfig
): Promise<OptimizationResult[]> {
const results: OptimizationResult[] = [];
const combinations = this.generateParameterCombinations(parameterGrid);
this.logger.info('Starting parameter optimization', {
strategy: strategyName,
combinations: combinations.length
});
// Use vectorized mode for fast parameter optimization
const vectorMode = new VectorBacktestMode(config);
// Can be parallelized
await Promise.all(
combinations.map(async (params) => {
const strategy = await this.loadStrategy(strategyName, vectorMode, params);
const result = await vectorMode.executeVectorizedBacktest(
strategy as VectorizedStrategy
);
results.push({
parameters: params,
performance: result.performance,
executionTime: result.executionTime
});
})
);
// Sort by Sharpe ratio
return results.sort((a, b) =>
b.performance.sharpeRatio - a.performance.sharpeRatio
);
}
}
Service Configuration
Environment-Based Mode Selection
export class ServiceConfig {
getTradingConfig(): TradingConfig {
return {
mode: (process.env.TRADING_MODE as BacktestMode) || BacktestMode.LIVE,
brokerConfig: {
apiKey: process.env.BROKER_API_KEY,
sandbox: process.env.BROKER_SANDBOX === 'true'
},
backtestConfig: {
startDate: new Date(process.env.BACKTEST_START_DATE || '2023-01-01'),
endDate: new Date(process.env.BACKTEST_END_DATE || '2024-01-01'),
initialCapital: parseFloat(process.env.INITIAL_CAPITAL || '100000'),
slippageModel: process.env.SLIPPAGE_MODEL || 'linear',
commissionModel: process.env.COMMISSION_MODEL || 'fixed'
}
};
}
}
CLI Interface
// CLI for running different modes
import { Command } from 'commander';
const program = new Command();
program
.name('stock-bot')
.description('Stock Trading Bot with Multi-Mode Backtesting');
program
.command('live')
.description('Run live trading')
.option('-s, --strategy <strategy>', 'Strategy to run')
.action(async (options) => {
const strategyService = new StrategyService();
await strategyService.runStrategy(
options.strategy,
BacktestMode.LIVE,
{}
);
});
program
.command('backtest')
.description('Run backtesting')
.option('-s, --strategy <strategy>', 'Strategy to test')
.option('-m, --mode <mode>', 'Backtest mode (event|vector|hybrid)', 'event')
.option('-f, --from <date>', 'Start date')
.option('-t, --to <date>', 'End date')
.action(async (options) => {
const strategyService = new StrategyService();
await strategyService.runStrategy(
options.strategy,
options.mode as BacktestMode,
{
startDate: new Date(options.from),
endDate: new Date(options.to)
}
);
});
program
.command('optimize')
.description('Optimize strategy parameters')
.option('-s, --strategy <strategy>', 'Strategy to optimize')
.option('-p, --params <params>', 'Parameter grid JSON')
.action(async (options) => {
const strategyService = new StrategyService();
const paramGrid = JSON.parse(options.params);
await strategyService.optimizeStrategy(
options.strategy,
paramGrid,
{}
);
});
program.parse();
Performance Comparison
Execution Speed by Mode
| Mode | Data Points/Second | Memory Usage | Use Case |
|---|---|---|---|
| Live | Real-time | Low | Production trading |
| Event-Driven | ~1,000 | Medium | Realistic validation |
| Vectorized | ~100,000+ | High | Parameter optimization |
| Hybrid | Combined | Medium | Strategy validation |
When to Use Each Mode
- Live Mode: Production trading with real money
- Event-Driven: Final strategy validation, complex order logic
- Vectorized: Initial development, parameter optimization, quick testing
- Hybrid: Validating vectorized results against realistic simulation
Integration with Your Existing Libraries
This architecture leverages all your existing infrastructure:
- @stock-bot/config: Environment management
- @stock-bot/logger: Comprehensive logging with Loki
- @stock-bot/utils: All technical indicators and calculations
- @stock-bot/questdb-client: Time-series data storage
- @stock-bot/postgres-client: Transactional data
- @stock-bot/mongodb-client: Configuration storage
Key Benefits
- Unified Codebase: Same strategy logic across all modes
- Performance Flexibility: Choose speed vs accuracy based on needs
- Validation Pipeline: Hybrid mode ensures vectorized results are accurate
- Production Ready: Live mode for actual trading
- Development Friendly: Fast iteration with vectorized backtesting
This simplified architecture reduces complexity while providing comprehensive backtesting capabilities that scale from rapid prototyping to production trading.