Migration Guide: From Singleton to Connection Pool Pattern

Overview

This guide explains how to migrate from the singleton anti-pattern to a proper connection pool pattern using the new @stock-bot/connection-factory library.

Current State (Singleton Anti-Pattern)

// ❌ Old pattern - global singleton
import { connectMongoDB, getMongoDBClient } from '@stock-bot/mongodb-client';
import { connectPostgreSQL, getPostgreSQLClient } from '@stock-bot/postgres-client';

// Initialize once at startup
await connectMongoDB(config);
await connectPostgreSQL(config);

// Use everywhere
const mongo = getMongoDBClient();
const postgres = getPostgreSQLClient();

Problems with this approach:

Global state makes testing difficult
All operations share the same connection pool
Can't optimize pool sizes for different use cases
Memory leaks from persistent connections
Hard to implement graceful shutdown

New Pattern (Connection Factory + Service Container)

Step 1: Set up Connection Factory

// ✅ New pattern - connection factory
import { setupServiceContainer } from './setup/database-setup';

// Initialize service container at startup
const container = await setupServiceContainer();

// Register cleanup
shutdown.register(async () => {
  await container.dispose();
});

Step 2: Update Handlers to Use Container

// ✅ Use OperationContext with container
export class MyHandler {
  constructor(private readonly container: ServiceContainer) {}
  
  async handleOperation(data: any) {
    const context = OperationContext.create('my-handler', 'operation', {
      container: this.container
    });
    
    try {
      // Connections are managed by the container
      await context.mongodb.insertOne(data);
      await context.postgres.query('...');
      await context.cache.set('key', 'value');
    } finally {
      // Clean up resources
      await context.dispose();
    }
  }
}

Step 3: Update Route Handlers

// Pass container to route handlers
export function createRoutes(container: ServiceContainer) {
  const router = new Hono();
  const handler = new MyHandler(container);
  
  router.get('/data', async (c) => {
    const result = await handler.handleOperation(c.req.query());
    return c.json(result);
  });
  
  return router;
}

Migration Checklist

For Each Service:

Create database setup module

// apps/[service-name]/src/setup/database-setup.ts
export async function setupServiceContainer(): Promise<ServiceContainer> {
  // Configure connection pools based on service needs
}

Update main index.ts
- Remove direct connectMongoDB() and connectPostgreSQL() calls
- Replace with setupServiceContainer()
- Pass container to route handlers and job processors
Update handlers
- Accept ServiceContainer in constructor
- Create OperationContext with container
- Remove direct database client imports
- Add context.dispose() in finally blocks

Update job handlers

// Before
export async function myJobHandler(job: Job) {
  const mongo = getMongoDBClient();
  // ...
}

// After
export function createMyJobHandler(container: ServiceContainer) {
  return async (job: Job) => {
    const context = OperationContext.create('job', job.name, {
      container
    });
    try {
      // Use context.mongodb, context.postgres, etc.
    } finally {
      await context.dispose();
    }
  };
}

Pool Size Recommendations

The PoolSizeCalculator provides optimal pool sizes based on service type:

Service	Min	Max	Use Case
data-ingestion	5	50	High-volume batch imports
data-pipeline	3	30	Data processing pipelines
web-api	2	10	Low-latency API requests
processing-service	2	20	CPU-intensive operations
portfolio-service	2	15	Portfolio calculations
strategy-service	3	25	Strategy backtesting

Benefits After Migration

Better Resource Management
- Each service gets appropriately sized connection pools
- Automatic cleanup with dispose pattern
- No more connection leaks
Improved Testing
- Easy to mock containers for tests
- No global state to reset between tests
- Can test with different configurations
Enhanced Performance
- Optimized pool sizes per service
- Isolated pools for heavy operations
- Better connection reuse
Operational Benefits
- Connection pool metrics per service
- Graceful shutdown handling
- Better error isolation

Backward Compatibility

The OperationContext maintains backward compatibility:

If no container is provided, it falls back to singleton pattern
This allows gradual migration service by service
Warning logs indicate when fallback is used

Example: Complete Service Migration

See /apps/data-ingestion/src/handlers/example-handler.ts for a complete example of:

Using the service container
Creating operation contexts
Handling batch operations with scoped containers
Proper resource cleanup

5.3 KiB Raw Blame History