stock-bot/MIGRATION-TO-CONNECTION-POOLS.md

# Migration Guide: From Singleton to Connection Pool Pattern

## Overview

This guide explains how to migrate from the singleton anti-pattern to a proper connection pool pattern using the new `@stock-bot/connection-factory` library.

## Current State (Singleton Anti-Pattern)

```typescript
// ❌ Old pattern - global singleton
import { connectMongoDB, getMongoDBClient } from '@stock-bot/mongodb-client';
import { connectPostgreSQL, getPostgreSQLClient } from '@stock-bot/postgres-client';

// Initialize once at startup
await connectMongoDB(config);
await connectPostgreSQL(config);

// Use everywhere
const mongo = getMongoDBClient();
const postgres = getPostgreSQLClient();
```

### Problems with this approach:
- Global state makes testing difficult
- All operations share the same connection pool
- Can't optimize pool sizes for different use cases
- Memory leaks from persistent connections
- Hard to implement graceful shutdown

## New Pattern (Connection Factory + Service Container)

### Step 1: Set up Connection Factory

```typescript
// ✅ New pattern - connection factory
import { setupServiceContainer } from './setup/database-setup';

// Initialize service container at startup
const container = await setupServiceContainer();

// Register cleanup
shutdown.register(async () => {
  await container.dispose();
});
```

### Step 2: Update Handlers to Use Container

```typescript
// ✅ Use OperationContext with container
export class MyHandler {
  constructor(private readonly container: ServiceContainer) {}
  
  async handleOperation(data: any) {
    const context = OperationContext.create('my-handler', 'operation', {
      container: this.container
    });
    
    try {
      // Connections are managed by the container
      await context.mongodb.insertOne(data);
      await context.postgres.query('...');
      await context.cache.set('key', 'value');
    } finally {
      // Clean up resources
      await context.dispose();
    }
  }
}
```

### Step 3: Update Route Handlers

```typescript
// Pass container to route handlers
export function createRoutes(container: ServiceContainer) {
  const router = new Hono();
  const handler = new MyHandler(container);
  
  router.get('/data', async (c) => {
    const result = await handler.handleOperation(c.req.query());
    return c.json(result);
  });
  
  return router;
}
```

## Migration Checklist

### For Each Service:

1. **Create database setup module**
   ```typescript
   // apps/[service-name]/src/setup/database-setup.ts
   export async function setupServiceContainer(): Promise<ServiceContainer> {
     // Configure connection pools based on service needs
   }
   ```

2. **Update main index.ts**
   - Remove direct `connectMongoDB()` and `connectPostgreSQL()` calls
   - Replace with `setupServiceContainer()`
   - Pass container to route handlers and job processors

3. **Update handlers**
   - Accept `ServiceContainer` in constructor
   - Create `OperationContext` with container
   - Remove direct database client imports
   - Add `context.dispose()` in finally blocks

4. **Update job handlers**
   ```typescript
   // Before
   export async function myJobHandler(job: Job) {
     const mongo = getMongoDBClient();
     // ...
   }
   
   // After
   export function createMyJobHandler(container: ServiceContainer) {
     return async (job: Job) => {
       const context = OperationContext.create('job', job.name, {
         container
       });
       try {
         // Use context.mongodb, context.postgres, etc.
       } finally {
         await context.dispose();
       }
     };
   }
   ```

## Pool Size Recommendations

The `PoolSizeCalculator` provides optimal pool sizes based on service type:

| Service | Min | Max | Use Case |
|---------|-----|-----|----------|
| data-ingestion | 5 | 50 | High-volume batch imports |
| data-pipeline | 3 | 30 | Data processing pipelines |
| web-api | 2 | 10 | Low-latency API requests |
| processing-service | 2 | 20 | CPU-intensive operations |
| portfolio-service | 2 | 15 | Portfolio calculations |
| strategy-service | 3 | 25 | Strategy backtesting |

## Benefits After Migration

1. **Better Resource Management**
   - Each service gets appropriately sized connection pools
   - Automatic cleanup with dispose pattern
   - No more connection leaks

2. **Improved Testing**
   - Easy to mock containers for tests
   - No global state to reset between tests
   - Can test with different configurations

3. **Enhanced Performance**
   - Optimized pool sizes per service
   - Isolated pools for heavy operations
   - Better connection reuse

4. **Operational Benefits**
   - Connection pool metrics per service
   - Graceful shutdown handling
   - Better error isolation

## Backward Compatibility

The `OperationContext` maintains backward compatibility:
- If no container is provided, it falls back to singleton pattern
- This allows gradual migration service by service
- Warning logs indicate when fallback is used

## Example: Complete Service Migration

See `/apps/data-ingestion/src/handlers/example-handler.ts` for a complete example of:
- Using the service container
- Creating operation contexts
- Handling batch operations with scoped containers
- Proper resource cleanup