stock-bot/docs/mongodb-multi-database-migration.md

MongoDB Client Multi-Database Migration Guide

Overview

Your MongoDB client has been enhanced to support multiple databases dynamically while maintaining full backward compatibility.

Key Features Added

1. Dynamic Database Switching

// Set default database (all operations will use this unless overridden)
client.setDefaultDatabase('analytics');

// Get current default database
const currentDb = client.getDefaultDatabase(); // Returns: 'analytics'

2. Database Parameter in Methods

All methods now accept an optional database parameter:

// Old way (still works - uses default database)
await client.batchUpsert('symbols', data, 'symbol');

// New way (specify database explicitly)
await client.batchUpsert('symbols', data, 'symbol', { database: 'stock' });

3. Convenience Methods

Pre-configured methods for common databases:

// Stock database operations
await client.batchUpsertStock('symbols', data, 'symbol');

// Analytics database operations  
await client.batchUpsertAnalytics('metrics', data, 'metric_name');

// Trading documents database operations
await client.batchUpsertTrading('orders', data, 'order_id');

4. Direct Database Access

// Get specific database instances
const stockDb = client.getDatabase('stock');
const analyticsDb = client.getDatabase('analytics');

// Get collections with database override
const collection = client.getCollection('symbols', 'stock');

Migration Steps

Step 1: No Changes Required (Backward Compatible)

Your existing code continues to work unchanged:

// This still works exactly as before
const client = MongoDBClient.getInstance();
await client.connect();
await client.batchUpsert('exchanges', exchangeData, 'exchange_id');

Step 2: Organize Data by Database (Recommended)

Update your data service to use appropriate databases:

// In your data service initialization
export class DataService {
  private mongoClient = MongoDBClient.getInstance();

  async initialize() {
    await this.mongoClient.connect();
    
    // Set stock as default for most operations
    this.mongoClient.setDefaultDatabase('stock');
  }

  async saveInteractiveBrokersData(exchanges: any[], symbols: any[]) {
    // Stock market data goes to 'stock' database (default)
    await this.mongoClient.batchUpsert('exchanges', exchanges, 'exchange_id');
    await this.mongoClient.batchUpsert('symbols', symbols, 'symbol');
  }

  async saveAnalyticsData(performance: any[]) {
    // Analytics data goes to 'analytics' database
    await this.mongoClient.batchUpsert(
      'performance', 
      performance, 
      'date',
      { database: 'analytics' }
    );
  }
}

Step 3: Use Convenience Methods (Optional)

Replace explicit database parameters with convenience methods:

// Instead of:
await client.batchUpsert('symbols', data, 'symbol', { database: 'stock' });

// Use:
await client.batchUpsertStock('symbols', data, 'symbol');

Factory Functions

New factory functions are available for easier database management:

import { 
  connectMongoDB, 
  setDefaultDatabase, 
  getCurrentDatabase,
  getDatabase 
} from '@stock-bot/mongodb-client';

// Set default database globally
setDefaultDatabase('analytics');

// Get current default
const current = getCurrentDatabase();

// Get specific database
const stockDb = getDatabase('stock');

Database Recommendations

Stock Database (stock)

• Market data (symbols, exchanges, prices)
• Financial instruments
• Market events
• Real-time data

Analytics Database (analytics)

• Performance metrics
• Calculated indicators
• Reports and dashboards
• Aggregated data

Trading Documents Database (trading_documents)

• Trade orders and executions
• User portfolios
• Transaction logs
• Audit trails

Example: Updating Your Data Service

// Before (still works)
export class DataService {
  async saveExchanges(exchanges: any[]) {
    const client = MongoDBClient.getInstance();
    await client.batchUpsert('exchanges', exchanges, 'exchange_id');
  }
}

// After (recommended)
export class DataService {
  private mongoClient = MongoDBClient.getInstance();

  async initialize() {
    await this.mongoClient.connect();
    this.mongoClient.setDefaultDatabase('stock'); // Set appropriate default
  }

  async saveExchanges(exchanges: any[]) {
    // Uses default 'stock' database
    await this.mongoClient.batchUpsert('exchanges', exchanges, 'exchange_id');
    
    // Or use convenience method
    await this.mongoClient.batchUpsertStock('exchanges', exchanges, 'exchange_id');
  }

  async savePerformanceMetrics(metrics: any[]) {
    // Save to analytics database
    await this.mongoClient.batchUpsertAnalytics('metrics', metrics, 'metric_name');
  }
}

Testing

Your existing tests continue to work. For new multi-database features:

import { MongoDBClient } from '@stock-bot/mongodb-client';

const client = MongoDBClient.getInstance();
await client.connect();

// Test database switching
client.setDefaultDatabase('test_db');
expect(client.getDefaultDatabase()).toBe('test_db');

// Test explicit database parameter
await client.batchUpsert('test_collection', data, 'id', { database: 'other_db' });

Benefits

1. Organized Data: Separate databases for different data types
2. Better Performance: Smaller, focused databases
3. Easier Maintenance: Clear data boundaries
4. Scalability: Can scale databases independently
5. Backward Compatibility: No breaking changes

Next Steps

1. Update your data service to use appropriate default database
2. Gradually migrate to using specific databases for different data types
3. Consider using convenience methods for cleaner code
4. Update tests to cover multi-database scenarios