added new di with connection pool and rebuild of the database/cache services

CONNECTION-POOL-IMPLEMENTATION.md

@@ -0,0 +1,145 @@
+# Connection Pool Implementation Summary
+
+## What We've Implemented
+
+### 1. Removed Singleton Pattern ✅
+- Deleted `libs/mongodb-client/src/singleton.ts`
+- Deleted `libs/postgres-client/src/singleton.ts`
+- Removed singleton exports from both index files
+- Forces all new code to use the connection factory pattern
+
+### 2. Added Connection Pool Monitoring ✅
+Both MongoDB and PostgreSQL clients now support:
+- `getPoolMetrics()` method returning:
+  - totalConnections
+  - activeConnections
+  - idleConnections
+  - waitingRequests
+  - errors count
+  - lastError message
+  - avgResponseTime
+  - created timestamp
+  - lastUsed timestamp
+
+### 3. Implemented Connection Lifecycle Events ✅
+Both clients now support event callbacks:
+```typescript
+interface ConnectionEvents {
+  onConnect?: () => void | Promise<void>;
+  onDisconnect?: () => void | Promise<void>;
+  onError?: (error: Error) => void | Promise<void>;
+  onPoolCreated?: () => void | Promise<void>;
+}
+```
+
+### 4. Dynamic Pool Sizing ✅
+New feature for automatically adjusting pool size based on load:
+```typescript
+interface DynamicPoolConfig {
+  enabled: boolean;
+  minSize: number;
+  maxSize: number;
+  scaleUpThreshold: number;    // % utilization to trigger scale up
+  scaleDownThreshold: number;  // % utilization to trigger scale down
+  scaleUpIncrement: number;     // connections to add
+  scaleDownIncrement: number;   // connections to remove
+  evaluationInterval: number;   // ms between evaluations
+}
+```
+
+### 5. Pool Warmup Strategy ✅
+- `warmupPool()` method pre-creates minimum connections
+- Runs parallel queries/pings to establish connections
+- Reduces cold start latency
+- Validates connections before marking pool as ready
+
+## Enhanced Components
+
+### Connection Factory (`@stock-bot/connection-factory`)
+- Manages connection pools across services
+- Provides consistent configuration
+- Handles lifecycle events
+- Supports pool metrics collection
+
+### Enhanced OperationContext
+- Backward compatible with singleton pattern
+- Supports dependency injection via ServiceContainer
+- Lazy loads database connections
+- Proper resource disposal with `dispose()` method
+
+### Service Container
+- Lightweight dependency injection
+- Scoped containers for request isolation
+- Automatic resource cleanup
+- Service lifecycle management
+
+## Usage Examples
+
+### Basic Setup
+```typescript
+// Setup service container with connection pools
+const container = await setupServiceContainer();
+
+// Create operation context with container
+const context = OperationContext.create('handler', 'operation', {
+  container
+});
+
+// Use databases normally
+await context.mongodb.insertOne(data);
+await context.postgres.query('SELECT...');
+
+// Clean up when done
+await context.dispose();
+```
+
+### Dynamic Pool Configuration
+```typescript
+const dynamicConfig: DynamicPoolConfig = {
+  enabled: true,
+  minSize: 5,
+  maxSize: 100,
+  scaleUpThreshold: 70,
+  scaleDownThreshold: 30,
+  scaleUpIncrement: 10,
+  scaleDownIncrement: 5,
+  evaluationInterval: 10000
+};
+
+client.setDynamicPoolConfig(dynamicConfig);
+```
+
+### Monitoring Pool Health
+```typescript
+// Get current metrics
+const metrics = client.getPoolMetrics();
+console.log({
+  active: metrics.activeConnections,
+  total: metrics.totalConnections,
+  waiting: metrics.waitingRequests
+});
+
+// Listen to events
+const events: ConnectionEvents = {
+  onError: (error) => logger.error('Pool error', { error }),
+  onConnect: () => logger.info('Connected')
+};
+```
+
+## Next Steps
+
+1. **Migrate Services**: Update each service to use the new pattern
+2. **Add Monitoring**: Set up dashboards for pool metrics
+3. **Configure Alerts**: Alert on pool exhaustion or high wait times
+4. **Performance Testing**: Validate dynamic sizing under load
+5. **Documentation**: Update service documentation with pool sizing guidelines
+
+## Benefits Achieved
+
+- ✅ No more global state
+- ✅ Per-service connection pool optimization
+- ✅ Automatic scaling based on load
+- ✅ Better resource utilization
+- ✅ Improved observability
+- ✅ Graceful degradation under load
+- ✅ Proper cleanup and lifecycle management