added new exchanges system

apps/data-sync-service/README.md

@@ -0,0 +1,184 @@
+# Data Sync Service
+
+The Data Sync Service handles synchronization of raw MongoDB data to PostgreSQL master records, providing a unified data layer for the stock-bot application.
+
+## Features
+
+### Original Sync Manager
+- Basic QM (QuoteMedia) symbol and exchange synchronization
+- Simple static exchange mapping
+- Manual sync triggers via REST API
+
+### Enhanced Sync Manager ✨ NEW
+- **Multi-provider support**: Syncs from EOD, Interactive Brokers, and QuoteMedia
+- **Comprehensive exchange handling**: Leverages all 4 MongoDB exchange collections
+- **Intelligent exchange mapping**: Dynamic mapping with fallback logic
+- **Transaction safety**: Full ACID compliance with rollback on errors
+- **Performance optimization**: Exchange caching for faster lookups
+- **Enhanced error handling**: Detailed error tracking and reporting
+
+## API Endpoints
+
+### Health Check
+- `GET /health` - Service health status
+
+### Original Sync Operations
+- `POST /sync/symbols` - Sync QM symbols to PostgreSQL
+- `POST /sync/exchanges` - Sync QM exchanges to PostgreSQL
+- `GET /sync/status` - Get basic sync status
+
+### Enhanced Sync Operations ✨ NEW
+- `POST /sync/exchanges/all?clear=true` - Comprehensive exchange sync from all providers (clear=true removes dummy data first)
+- `POST /sync/symbols/:provider?clear=true` - Sync symbols from specific provider (qm, eod, ib)
+- `POST /sync/clear` - Clear all PostgreSQL data (exchanges, symbols, mappings)
+- `GET /sync/status/enhanced` - Get detailed sync status
+- `GET /sync/stats/exchanges` - Get exchange statistics
+
+## Data Sources
+
+### MongoDB Collections
+1. **exchanges** (34 records) - Unified exchange reference
+2. **eodExchanges** (78 records) - EOD provider with currency/MIC data
+3. **ibExchanges** (214 records) - Interactive Brokers with asset types
+4. **qmExchanges** (25 records) - QuoteMedia exchanges
+
+### PostgreSQL Tables
+1. **master_exchanges** - Unified exchange master data
+2. **master_symbols** - Symbol master records
+3. **provider_symbol_mappings** - Multi-provider symbol mappings
+4. **sync_status** - Synchronization tracking
+
+## Key Improvements
+
+### 1. Multi-Provider Exchange Sync
+Instead of only syncing QM exchanges, the enhanced sync manager:
+- Syncs from unified `exchanges` collection first (primary source)
+- Enriches with EOD exchanges (comprehensive global data with currencies)
+- Adds IB exchanges for additional coverage (214 exchanges vs 25 in QM)
+
+### 2. Intelligent Exchange Mapping
+Replaces hard-coded mapping with dynamic resolution:
+```typescript
+// Before: Static mapping
+const exchangeMap = { 'NASDAQ': 'NASDAQ', 'NYSE': 'NYSE' };
+
+// After: Dynamic mapping with variations
+const codeMap = {
+  'NASDAQ': 'NASDAQ', 'NAS': 'NASDAQ',
+  'NYSE': 'NYSE', 'NYQ': 'NYSE',
+  'LSE': 'LSE', 'LON': 'LSE', 'LN': 'LSE',
+  'US': 'NYSE' // EOD uses 'US' for US markets
+};
+```
+
+### 3. Transaction Safety
+All sync operations use database transactions:
+- `BEGIN` transaction at start
+- `COMMIT` on success
+- `ROLLBACK` on any error
+- Ensures data consistency
+
+### 4. Performance Optimization
+- Exchange cache preloaded at startup
+- Reduced database queries during symbol processing
+- Batch operations where possible
+
+### 5. Enhanced Error Handling
+- Detailed error logging with context
+- Separate error counting in sync results
+- Graceful handling of missing/invalid data
+
+## Usage Examples
+
+### Clear All Data and Start Fresh Exchange Sync
+```bash
+curl -X POST "http://localhost:3005/sync/exchanges/all?clear=true"
+```
+
+### Sync Symbols from Specific Provider
+```bash
+# Sync QuoteMedia symbols (clear existing symbols first)
+curl -X POST "http://localhost:3005/sync/symbols/qm?clear=true"
+
+# Sync EOD symbols
+curl -X POST http://localhost:3005/sync/symbols/eod
+
+# Sync Interactive Brokers symbols
+curl -X POST http://localhost:3005/sync/symbols/ib
+```
+
+### Clear All PostgreSQL Data
+```bash
+curl -X POST http://localhost:3005/sync/clear
+```
+
+### Get Enhanced Status
+```bash
+curl http://localhost:3005/sync/status/enhanced
+curl http://localhost:3005/sync/stats/exchanges
+```
+
+## Configuration
+
+### Environment Variables
+- `DATA_SYNC_SERVICE_PORT` - Service port (default: 3005)
+- `NODE_ENV` - Environment mode
+
+### Database Connections
+- **MongoDB**: `mongodb://trading_admin:trading_mongo_dev@localhost:27017/stock?authSource=admin`
+- **PostgreSQL**: `postgresql://trading_user:trading_pass_dev@localhost:5432/trading_bot`
+
+## Development
+
+### Build and Run
+```bash
+# Development mode
+bun run dev
+
+# Build
+bun run build
+
+# Production
+bun run start
+```
+
+### Testing
+```bash
+# Run tests
+bun test
+
+# Start infrastructure
+bun run infra:up
+
+# Test sync operations
+curl -X POST http://localhost:3005/sync/exchanges/all
+curl -X POST http://localhost:3005/sync/symbols/qm
+```
+
+## Architecture
+
+```
+MongoDB Collections          PostgreSQL Tables
+┌─ exchanges (34)        ┐   ┌─ master_exchanges
+├─ eodExchanges (78)     ├──▶├─ master_symbols
+├─ ibExchanges (214)     │   ├─ provider_symbol_mappings
+└─ qmExchanges (25)      ┘   └─ sync_status
+        │
+        ▼
+Enhanced Sync Manager
+- Exchange caching
+- Dynamic mapping
+- Transaction safety
+- Multi-provider support
+```
+
+## Migration Path
+
+The enhanced sync manager is designed to work alongside the original sync manager:
+
+1. **Immediate**: Use enhanced exchange sync for better coverage
+2. **Phase 1**: Test enhanced symbol sync with each provider
+3. **Phase 2**: Replace original sync manager when confident
+4. **Phase 3**: Remove original sync manager and endpoints
+
+Both managers can be used simultaneously during the transition period.

apps/data-sync-service/package.json

@@ -0,0 +1,25 @@
+{
+  "name": "@stock-bot/data-sync-service",
+  "version": "1.0.0",
+  "description": "Sync service from MongoDB raw data to PostgreSQL master records",
+  "main": "dist/index.js",
+  "type": "module",
+  "scripts": {
+    "dev": "bun --watch src/index.ts",
+    "build": "bun build src/index.ts --outdir dist --target node",
+    "start": "bun dist/index.js",
+    "test": "bun test",
+    "clean": "rm -rf dist"
+  },
+  "dependencies": {
+    "@stock-bot/config": "*",
+    "@stock-bot/logger": "*",
+    "@stock-bot/mongodb-client": "*",
+    "@stock-bot/postgres-client": "*",
+    "@stock-bot/shutdown": "*",
+    "hono": "^4.0.0"
+  },
+  "devDependencies": {
+    "typescript": "^5.0.0"
+  }
+}

apps/data-sync-service/src/index.ts

@@ -0,0 +1,263 @@
+/**
+ * Data Sync Service - Sync raw MongoDB data to PostgreSQL master records
+ */
+import { Hono } from 'hono';
+import { cors } from 'hono/cors';
+import { loadEnvVariables } from '@stock-bot/config';
+import { getLogger, shutdownLoggers } from '@stock-bot/logger';
+import { connectMongoDB, disconnectMongoDB } from '@stock-bot/mongodb-client';
+import { connectPostgreSQL, disconnectPostgreSQL } from '@stock-bot/postgres-client';
+import { Shutdown } from '@stock-bot/shutdown';
+import { enhancedSyncManager } from './services/enhanced-sync-manager';
+import { syncManager } from './services/sync-manager';
+
+// Load environment variables
+loadEnvVariables();
+
+const app = new Hono();
+
+// Add CORS middleware
+app.use(
+  '*',
+  cors({
+    origin: '*',
+    allowMethods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
+    allowHeaders: ['Content-Type', 'Authorization'],
+    credentials: false,
+  })
+);
+
+const logger = getLogger('data-sync-service');
+const PORT = parseInt(process.env.DATA_SYNC_SERVICE_PORT || '3005');
+let server: ReturnType<typeof Bun.serve> | null = null;
+
+// Initialize shutdown manager
+const shutdown = Shutdown.getInstance({ timeout: 15000 });
+
+// Basic health check endpoint
+app.get('/health', c => {
+  return c.json({
+    status: 'healthy',
+    service: 'data-sync-service',
+    timestamp: new Date().toISOString(),
+  });
+});
+
+// Manual sync trigger endpoints
+app.post('/sync/symbols', async c => {
+  try {
+    const result = await syncManager.syncQMSymbols();
+    return c.json({ success: true, result });
+  } catch (error) {
+    logger.error('Manual symbol sync failed', { error });
+    return c.json(
+      { success: false, error: error instanceof Error ? error.message : 'Unknown error' },
+      500
+    );
+  }
+});
+
+app.post('/sync/exchanges', async c => {
+  try {
+    const result = await syncManager.syncQMExchanges();
+    return c.json({ success: true, result });
+  } catch (error) {
+    logger.error('Manual exchange sync failed', { error });
+    return c.json(
+      { success: false, error: error instanceof Error ? error.message : 'Unknown error' },
+      500
+    );
+  }
+});
+
+// Get sync status
+app.get('/sync/status', async c => {
+  try {
+    const status = await syncManager.getSyncStatus();
+    return c.json(status);
+  } catch (error) {
+    logger.error('Failed to get sync status', { error });
+    return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
+  }
+});
+
+// Enhanced sync endpoints
+app.post('/sync/exchanges/all', async c => {
+  try {
+    const clearFirst = c.req.query('clear') === 'true';
+    const result = await enhancedSyncManager.syncAllExchanges(clearFirst);
+    return c.json({ success: true, result });
+  } catch (error) {
+    logger.error('Enhanced exchange sync failed', { error });
+    return c.json(
+      { success: false, error: error instanceof Error ? error.message : 'Unknown error' },
+      500
+    );
+  }
+});
+
+app.post('/sync/provider-mappings/qm', async c => {
+  try {
+    const result = await enhancedSyncManager.syncQMProviderMappings();
+    return c.json({ success: true, result });
+  } catch (error) {
+    logger.error('QM provider mappings sync failed', { error });
+    return c.json(
+      { success: false, error: error instanceof Error ? error.message : 'Unknown error' },
+      500
+    );
+  }
+});
+
+app.post('/sync/symbols/:provider', async c => {
+  try {
+    const provider = c.req.param('provider');
+    const clearFirst = c.req.query('clear') === 'true';
+    const result = await enhancedSyncManager.syncSymbolsFromProvider(provider, clearFirst);
+    return c.json({ success: true, result });
+  } catch (error) {
+    logger.error('Enhanced symbol sync failed', { error });
+    return c.json(
+      { success: false, error: error instanceof Error ? error.message : 'Unknown error' },
+      500
+    );
+  }
+});
+
+// Clear data endpoint
+app.post('/sync/clear', async c => {
+  try {
+    const result = await enhancedSyncManager.clearPostgreSQLData();
+    return c.json({ success: true, result });
+  } catch (error) {
+    logger.error('Clear PostgreSQL data failed', { error });
+    return c.json(
+      { success: false, error: error instanceof Error ? error.message : 'Unknown error' },
+      500
+    );
+  }
+});
+
+// Enhanced status endpoints
+app.get('/sync/status/enhanced', async c => {
+  try {
+    const status = await enhancedSyncManager.getSyncStatus();
+    return c.json(status);
+  } catch (error) {
+    logger.error('Failed to get enhanced sync status', { error });
+    return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
+  }
+});
+
+app.get('/sync/stats/exchanges', async c => {
+  try {
+    const stats = await enhancedSyncManager.getExchangeStats();
+    return c.json(stats);
+  } catch (error) {
+    logger.error('Failed to get exchange stats', { error });
+    return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
+  }
+});
+
+app.get('/sync/stats/provider-mappings', async c => {
+  try {
+    const stats = await enhancedSyncManager.getProviderMappingStats();
+    return c.json(stats);
+  } catch (error) {
+    logger.error('Failed to get provider mapping stats', { error });
+    return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
+  }
+});
+
+// Initialize services
+async function initializeServices() {
+  logger.info('Initializing data sync service...');
+
+  try {
+    // Initialize MongoDB client
+    logger.info('Connecting to MongoDB...');
+    await connectMongoDB();
+    logger.info('MongoDB connected');
+
+    // Initialize PostgreSQL client
+    logger.info('Connecting to PostgreSQL...');
+    await connectPostgreSQL();
+    logger.info('PostgreSQL connected');
+
+    // Initialize sync managers
+    logger.info('Initializing sync managers...');
+    await syncManager.initialize();
+    await enhancedSyncManager.initialize();
+    logger.info('Sync managers initialized');
+
+    logger.info('All services initialized successfully');
+  } catch (error) {
+    logger.error('Failed to initialize services', { error });
+    throw error;
+  }
+}
+
+// Start server
+async function startServer() {
+  await initializeServices();
+
+  server = Bun.serve({
+    port: PORT,
+    fetch: app.fetch,
+    development: process.env.NODE_ENV === 'development',
+  });
+
+  logger.info(`Data Sync Service started on port ${PORT}`);
+}
+
+// Register shutdown handlers
+shutdown.onShutdown(async () => {
+  if (server) {
+    logger.info('Stopping HTTP server...');
+    try {
+      server.stop();
+      logger.info('HTTP server stopped');
+    } catch (error) {
+      logger.error('Error stopping HTTP server', { error });
+    }
+  }
+});
+
+shutdown.onShutdown(async () => {
+  logger.info('Shutting down sync managers...');
+  try {
+    await syncManager.shutdown();
+    await enhancedSyncManager.shutdown();
+    logger.info('Sync managers shut down');
+  } catch (error) {
+    logger.error('Error shutting down sync managers', { error });
+  }
+});
+
+shutdown.onShutdown(async () => {
+  logger.info('Disconnecting from databases...');
+  try {
+    await disconnectMongoDB();
+    await disconnectPostgreSQL();
+    logger.info('Database connections closed');
+  } catch (error) {
+    logger.error('Error closing database connections', { error });
+  }
+});
+
+shutdown.onShutdown(async () => {
+  try {
+    await shutdownLoggers();
+    process.stdout.write('Data sync service loggers shut down\n');
+  } catch (error) {
+    process.stderr.write(`Error shutting down loggers: ${error}\n`);
+  }
+});
+
+// Start the service
+startServer().catch(error => {
+  logger.error('Failed to start data sync service', { error });
+  process.exit(1);
+});
+
+logger.info('Data sync service startup initiated');

apps/data-sync-service/src/services/enhanced-sync-manager.ts

@@ -0,0 +1,798 @@
+/**
+ * Enhanced Sync Manager - Improved syncing with comprehensive exchange data
+ */
+import { getLogger } from '@stock-bot/logger';
+import { getMongoDBClient } from '@stock-bot/mongodb-client';
+import { getPostgreSQLClient } from '@stock-bot/postgres-client';
+
+const logger = getLogger('enhanced-sync-manager');
+
+interface ExchangeMapping {
+  id: string;
+  code: string;
+  name: string;
+  country: string;
+  currency: string;
+}
+
+interface SyncResult {
+  processed: number;
+  created: number;
+  updated: number;
+  skipped: number;
+  errors: number;
+}
+
+interface SyncStatus {
+  provider: string;
+  dataType: string;
+  lastSyncAt?: Date;
+  lastSyncCount: number;
+  syncErrors?: string;
+}
+
+export class EnhancedSyncManager {
+  private isInitialized = false;
+  private mongoClient: any;
+  private postgresClient: any;
+  private exchangeCache: Map<string, ExchangeMapping> = new Map();
+
+  async initialize(): Promise<void> {
+    if (this.isInitialized) {
+      logger.warn('Enhanced sync manager already initialized');
+      return;
+    }
+
+    try {
+      this.mongoClient = getMongoDBClient();
+      this.postgresClient = getPostgreSQLClient();
+
+      // Pre-load exchange mappings for performance
+      await this.loadExchangeCache();
+
+      this.isInitialized = true;
+      logger.info('Enhanced sync manager initialized successfully');
+    } catch (error) {
+      logger.error('Failed to initialize enhanced sync manager', { error });
+      throw error;
+    }
+  }
+
+  /**
+   * Helper method to get MongoDB database for direct queries
+   */
+  private getMongoDatabase() {
+    return this.mongoClient.getDatabase();
+  }
+
+  async shutdown(): Promise<void> {
+    if (!this.isInitialized) {
+      return;
+    }
+
+    logger.info('Shutting down enhanced sync manager...');
+    this.exchangeCache.clear();
+    this.isInitialized = false;
+    logger.info('Enhanced sync manager shut down successfully');
+  }
+
+  /**
+   * Clear all exchange and symbol data from PostgreSQL
+   */
+  async clearPostgreSQLData(): Promise<{
+    exchangesCleared: number;
+    symbolsCleared: number;
+    mappingsCleared: number;
+  }> {
+    if (!this.isInitialized) {
+      throw new Error('Enhanced sync manager not initialized');
+    }
+
+    logger.info('Clearing existing PostgreSQL data...');
+
+    try {
+      // Start transaction for atomic operations
+      await this.postgresClient.query('BEGIN');
+
+      // Get counts before clearing
+      const exchangeCountResult = await this.postgresClient.query(
+        'SELECT COUNT(*) as count FROM exchanges'
+      );
+      const symbolCountResult = await this.postgresClient.query(
+        'SELECT COUNT(*) as count FROM symbols'
+      );
+      const mappingCountResult = await this.postgresClient.query(
+        'SELECT COUNT(*) as count FROM provider_mappings'
+      );
+
+      const exchangesCleared = parseInt(exchangeCountResult.rows[0].count);
+      const symbolsCleared = parseInt(symbolCountResult.rows[0].count);
+      const mappingsCleared = parseInt(mappingCountResult.rows[0].count);
+
+      // Clear data in correct order (respect foreign keys)
+      await this.postgresClient.query('DELETE FROM provider_mappings');
+      await this.postgresClient.query('DELETE FROM symbols');
+      await this.postgresClient.query('DELETE FROM exchanges');
+
+      // Reset sync status
+      await this.postgresClient.query(
+        'UPDATE sync_status SET last_sync_at = NULL, last_sync_count = 0, sync_errors = NULL'
+      );
+
+      await this.postgresClient.query('COMMIT');
+
+      logger.info('PostgreSQL data cleared successfully', {
+        exchangesCleared,
+        symbolsCleared,
+        mappingsCleared,
+      });
+
+      return { exchangesCleared, symbolsCleared, mappingsCleared };
+    } catch (error) {
+      await this.postgresClient.query('ROLLBACK');
+      logger.error('Failed to clear PostgreSQL data', { error });
+      throw error;
+    }
+  }
+
+  /**
+   * Comprehensive exchange sync from all MongoDB providers
+   */
+  async syncAllExchanges(clearFirst: boolean = true): Promise<SyncResult> {
+    if (!this.isInitialized) {
+      throw new Error('Enhanced sync manager not initialized');
+    }
+
+    logger.info('Starting comprehensive exchange sync...', { clearFirst });
+
+    const result: SyncResult = {
+      processed: 0,
+      created: 0,
+      updated: 0,
+      skipped: 0,
+      errors: 0,
+    };
+
+    try {
+      // Clear existing data if requested
+      if (clearFirst) {
+        await this.clearPostgreSQLData();
+      }
+
+      // Start transaction for atomic operations
+      await this.postgresClient.query('BEGIN');
+
+      // 1. Sync from unified exchanges collection (primary source)
+      const unifiedResult = await this.syncUnifiedExchanges();
+      this.mergeResults(result, unifiedResult);
+
+      // 2. Sync from EOD exchanges (comprehensive global data)
+      const eodResult = await this.syncEODExchanges();
+      this.mergeResults(result, eodResult);
+
+      // 3. Sync from IB exchanges (detailed asset information)
+      const ibResult = await this.syncIBExchanges();
+      this.mergeResults(result, ibResult);
+
+      // 4. Update sync status
+      await this.updateSyncStatus('all', 'exchanges', result.processed);
+
+      await this.postgresClient.query('COMMIT');
+
+      // Refresh exchange cache with new data
+      await this.loadExchangeCache();
+
+      logger.info('Comprehensive exchange sync completed', result);
+      return result;
+    } catch (error) {
+      await this.postgresClient.query('ROLLBACK');
+      logger.error('Comprehensive exchange sync failed', { error });
+      throw error;
+    }
+  }
+
+  /**
+   * Sync QM provider exchange mappings
+   */
+  async syncQMProviderMappings(): Promise<SyncResult> {
+    if (!this.isInitialized) {
+      throw new Error('Enhanced sync manager not initialized');
+    }
+
+    logger.info('Starting QM provider exchange mappings sync...');
+
+    const result: SyncResult = {
+      processed: 0,
+      created: 0,
+      updated: 0,
+      skipped: 0,
+      errors: 0,
+    };
+
+    try {
+      // Start transaction
+      await this.postgresClient.query('BEGIN');
+
+      // Get unique exchange combinations from QM symbols
+      const db = this.getMongoDatabase();
+      const pipeline = [
+        {
+          $group: {
+            _id: {
+              exchangeCode: '$exchangeCode',
+              exchange: '$exchange',
+              countryCode: '$countryCode',
+            },
+            count: { $sum: 1 },
+            sampleExchange: { $first: '$exchange' },
+          },
+        },
+        {
+          $project: {
+            exchangeCode: '$_id.exchangeCode',
+            exchange: '$_id.exchange',
+            countryCode: '$_id.countryCode',
+            count: 1,
+            sampleExchange: 1,
+          },
+        },
+      ];
+
+      const qmExchanges = await db.collection('qmSymbols').aggregate(pipeline).toArray();
+      logger.info(`Found ${qmExchanges.length} unique QM exchange combinations`);
+
+      for (const exchange of qmExchanges) {
+        try {
+          // Create provider exchange mapping for QM
+          await this.createProviderExchangeMapping(
+            'qm', // provider
+            exchange.exchangeCode,
+            exchange.sampleExchange || exchange.exchangeCode,
+            exchange.countryCode,
+            exchange.countryCode === 'CA' ? 'CAD' : 'USD', // Simple currency mapping
+            0.8 // good confidence for QM data
+          );
+
+          result.processed++;
+          result.created++;
+        } catch (error) {
+          logger.error('Failed to process QM exchange mapping', { error, exchange });
+          result.errors++;
+        }
+      }
+
+      await this.postgresClient.query('COMMIT');
+
+      logger.info('QM provider exchange mappings sync completed', result);
+      return result;
+    } catch (error) {
+      await this.postgresClient.query('ROLLBACK');
+      logger.error('QM provider exchange mappings sync failed', { error });
+      throw error;
+    }
+  }
+
+  /**
+   * Enhanced symbol sync with multi-provider mapping
+   */
+  async syncSymbolsFromProvider(
+    provider: string,
+    clearFirst: boolean = false
+  ): Promise<SyncResult> {
+    if (!this.isInitialized) {
+      throw new Error('Enhanced sync manager not initialized');
+    }
+
+    logger.info(`Starting ${provider} symbols sync...`, { clearFirst });
+
+    const result: SyncResult = {
+      processed: 0,
+      created: 0,
+      updated: 0,
+      skipped: 0,
+      errors: 0,
+    };
+
+    try {
+      // Clear existing data if requested (only symbols and mappings, keep exchanges)
+      if (clearFirst) {
+        await this.postgresClient.query('BEGIN');
+        await this.postgresClient.query('DELETE FROM provider_mappings');
+        await this.postgresClient.query('DELETE FROM symbols');
+        await this.postgresClient.query('COMMIT');
+        logger.info('Cleared existing symbols and mappings before sync');
+      }
+
+      // Start transaction
+      await this.postgresClient.query('BEGIN');
+
+      let symbols: any[] = [];
+
+      // Get symbols based on provider
+      const db = this.getMongoDatabase();
+      switch (provider.toLowerCase()) {
+        case 'qm':
+          symbols = await db.collection('qmSymbols').find({}).toArray();
+          break;
+        case 'eod':
+          symbols = await db.collection('eodSymbols').find({}).toArray();
+          break;
+        case 'ib':
+          symbols = await db.collection('ibSymbols').find({}).toArray();
+          break;
+        default:
+          throw new Error(`Unsupported provider: ${provider}`);
+      }
+
+      logger.info(`Found ${symbols.length} ${provider} symbols to process`);
+      result.processed = symbols.length;
+
+      for (const symbol of symbols) {
+        try {
+          await this.processSingleSymbol(symbol, provider, result);
+        } catch (error) {
+          logger.error('Failed to process symbol', {
+            error,
+            symbol: symbol.symbol || symbol.code,
+            provider,
+          });
+          result.errors++;
+        }
+      }
+
+      // Update sync status
+      await this.updateSyncStatus(provider, 'symbols', result.processed);
+
+      await this.postgresClient.query('COMMIT');
+
+      logger.info(`${provider} symbols sync completed`, result);
+      return result;
+    } catch (error) {
+      await this.postgresClient.query('ROLLBACK');
+      logger.error(`${provider} symbols sync failed`, { error });
+      throw error;
+    }
+  }
+
+  /**
+   * Get comprehensive sync status
+   */
+  async getSyncStatus(): Promise<SyncStatus[]> {
+    const query = `
+      SELECT provider, data_type as "dataType", last_sync_at as "lastSyncAt",
+             last_sync_count as "lastSyncCount", sync_errors as "syncErrors"
+      FROM sync_status 
+      ORDER BY provider, data_type
+    `;
+    const result = await this.postgresClient.query(query);
+    return result.rows;
+  }
+
+  /**
+   * Get exchange statistics
+   */
+  async getExchangeStats(): Promise<any> {
+    const query = `
+      SELECT 
+        COUNT(*) as total_exchanges,
+        COUNT(CASE WHEN active = true THEN 1 END) as active_exchanges,
+        COUNT(DISTINCT country) as countries,
+        COUNT(DISTINCT currency) as currencies
+      FROM exchanges
+    `;
+    const result = await this.postgresClient.query(query);
+    return result.rows[0];
+  }
+
+  /**
+   * Get provider exchange mapping statistics
+   */
+  async getProviderMappingStats(): Promise<any> {
+    const query = `
+      SELECT 
+        provider,
+        COUNT(*) as total_mappings,
+        COUNT(CASE WHEN active = true THEN 1 END) as active_mappings,
+        COUNT(CASE WHEN verified = true THEN 1 END) as verified_mappings,
+        COUNT(CASE WHEN auto_mapped = true THEN 1 END) as auto_mapped,
+        AVG(confidence) as avg_confidence
+      FROM provider_exchange_mappings
+      GROUP BY provider
+      ORDER BY provider
+    `;
+    const result = await this.postgresClient.query(query);
+    return result.rows;
+  }
+
+  // Private helper methods
+
+  private async loadExchangeCache(): Promise<void> {
+    const query = 'SELECT id, code, name, country, currency FROM exchanges';
+    const result = await this.postgresClient.query(query);
+
+    this.exchangeCache.clear();
+    for (const exchange of result.rows) {
+      this.exchangeCache.set(exchange.code.toUpperCase(), exchange);
+    }
+
+    logger.info(`Loaded ${this.exchangeCache.size} exchanges into cache`);
+  }
+
+  private async syncUnifiedExchanges(): Promise<SyncResult> {
+    const db = this.getMongoDatabase();
+    const exchanges = await db.collection('exchanges').find({}).toArray();
+    const result: SyncResult = { processed: 0, created: 0, updated: 0, skipped: 0, errors: 0 };
+
+    for (const exchange of exchanges) {
+      try {
+        // Create provider exchange mapping for unified collection
+        await this.createProviderExchangeMapping(
+          'unified', // provider
+          exchange.sourceCode || exchange.code,
+          exchange.sourceName || exchange.name,
+          exchange.sourceRegion,
+          null, // currency not in unified
+          0.9 // high confidence for unified mappings
+        );
+
+        result.processed++;
+        result.created++; // Count as created mapping
+      } catch (error) {
+        logger.error('Failed to process unified exchange', { error, exchange });
+        result.errors++;
+      }
+    }
+
+    return result;
+  }
+
+  private async syncEODExchanges(): Promise<SyncResult> {
+    const db = this.getMongoDatabase();
+    const exchanges = await db.collection('eodExchanges').find({ active: true }).toArray();
+    const result: SyncResult = { processed: 0, created: 0, updated: 0, skipped: 0, errors: 0 };
+
+    for (const exchange of exchanges) {
+      try {
+        // Create provider exchange mapping for EOD
+        await this.createProviderExchangeMapping(
+          'eod', // provider
+          exchange.Code,
+          exchange.Name,
+          exchange.CountryISO2,
+          exchange.Currency,
+          0.95 // very high confidence for EOD data
+        );
+
+        result.processed++;
+        result.created++; // Count as created mapping
+      } catch (error) {
+        logger.error('Failed to process EOD exchange', { error, exchange });
+        result.errors++;
+      }
+    }
+
+    return result;
+  }
+
+  private async syncIBExchanges(): Promise<SyncResult> {
+    const db = this.getMongoDatabase();
+    const exchanges = await db.collection('ibExchanges').find({}).toArray();
+    const result: SyncResult = { processed: 0, created: 0, updated: 0, skipped: 0, errors: 0 };
+
+    for (const exchange of exchanges) {
+      try {
+        // Create provider exchange mapping for IB
+        await this.createProviderExchangeMapping(
+          'ib', // provider
+          exchange.exchange_id,
+          exchange.name,
+          exchange.country_code,
+          'USD', // IB doesn't specify currency, default to USD
+          0.85 // good confidence for IB data
+        );
+
+        result.processed++;
+        result.created++; // Count as created mapping
+      } catch (error) {
+        logger.error('Failed to process IB exchange', { error, exchange });
+        result.errors++;
+      }
+    }
+
+    return result;
+  }
+
+  /**
+   * Create or update a provider exchange mapping
+   * This method intelligently maps provider exchanges to master exchanges
+   */
+  private async createProviderExchangeMapping(
+    provider: string,
+    providerExchangeCode: string,
+    providerExchangeName: string,
+    countryCode: string | null,
+    currency: string | null,
+    confidence: number
+  ): Promise<void> {
+    if (!providerExchangeCode) return;
+
+    // Check if mapping already exists
+    const existingMapping = await this.findProviderExchangeMapping(provider, providerExchangeCode);
+    if (existingMapping) {
+      // Don't override existing mappings to preserve manual work
+      return;
+    }
+
+    // Find or create master exchange
+    const masterExchange = await this.findOrCreateMasterExchange(
+      providerExchangeCode,
+      providerExchangeName,
+      countryCode,
+      currency
+    );
+
+    // Create the provider exchange mapping
+    const query = `
+      INSERT INTO provider_exchange_mappings 
+      (provider, provider_exchange_code, provider_exchange_name, master_exchange_id, 
+       country_code, currency, confidence, active, auto_mapped)
+      VALUES ($1, $2, $3, $4, $5, $6, $7, false, true)
+      ON CONFLICT (provider, provider_exchange_code) DO NOTHING
+    `;
+
+    await this.postgresClient.query(query, [
+      provider,
+      providerExchangeCode,
+      providerExchangeName,
+      masterExchange.id,
+      countryCode,
+      currency,
+      confidence,
+    ]);
+  }
+
+  /**
+   * Find or create a master exchange based on provider data
+   */
+  private async findOrCreateMasterExchange(
+    providerCode: string,
+    providerName: string,
+    countryCode: string | null,
+    currency: string | null
+  ): Promise<any> {
+    // First, try to find exact match
+    let masterExchange = await this.findExchangeByCode(providerCode);
+
+    if (masterExchange) {
+      return masterExchange;
+    }
+
+    // Try to find by similar codes (basic mapping)
+    const basicMapping = this.getBasicExchangeMapping(providerCode);
+    if (basicMapping) {
+      masterExchange = await this.findExchangeByCode(basicMapping);
+      if (masterExchange) {
+        return masterExchange;
+      }
+    }
+
+    // Create new master exchange (inactive by default)
+    const query = `
+      INSERT INTO exchanges (code, name, country, currency, active)
+      VALUES ($1, $2, $3, $4, false)
+      ON CONFLICT (code) DO UPDATE SET
+        name = COALESCE(EXCLUDED.name, exchanges.name),
+        country = COALESCE(EXCLUDED.country, exchanges.country),
+        currency = COALESCE(EXCLUDED.currency, exchanges.currency)
+      RETURNING id, code, name, country, currency
+    `;
+
+    const result = await this.postgresClient.query(query, [
+      providerCode,
+      providerName || providerCode,
+      countryCode || 'US',
+      currency || 'USD',
+    ]);
+
+    const newExchange = result.rows[0];
+
+    // Update cache
+    this.exchangeCache.set(newExchange.code.toUpperCase(), newExchange);
+
+    return newExchange;
+  }
+
+  /**
+   * Basic exchange code mapping for common cases
+   */
+  private getBasicExchangeMapping(providerCode: string): string | null {
+    const mappings: Record<string, string> = {
+      NYE: 'NYSE',
+      NAS: 'NASDAQ',
+      TO: 'TSX',
+      LN: 'LSE',
+      LON: 'LSE',
+    };
+
+    return mappings[providerCode.toUpperCase()] || null;
+  }
+
+  private async findProviderExchangeMapping(
+    provider: string,
+    providerExchangeCode: string
+  ): Promise<any> {
+    const query =
+      'SELECT * FROM provider_exchange_mappings WHERE provider = $1 AND provider_exchange_code = $2';
+    const result = await this.postgresClient.query(query, [provider, providerExchangeCode]);
+    return result.rows[0] || null;
+  }
+
+  private async processSingleSymbol(
+    symbol: any,
+    provider: string,
+    result: SyncResult
+  ): Promise<void> {
+    const symbolCode = symbol.symbol || symbol.code;
+    const exchangeCode = symbol.exchangeCode || symbol.exchange || symbol.exchange_id;
+
+    if (!symbolCode || !exchangeCode) {
+      result.skipped++;
+      return;
+    }
+
+    // Find active provider exchange mapping
+    const providerMapping = await this.findActiveProviderExchangeMapping(provider, exchangeCode);
+
+    if (!providerMapping) {
+      result.skipped++;
+      return;
+    }
+
+    // Check if symbol exists
+    const existingSymbol = await this.findSymbolByCodeAndExchange(
+      symbolCode,
+      providerMapping.master_exchange_id
+    );
+
+    if (existingSymbol) {
+      await this.updateSymbol(existingSymbol.id, symbol);
+      await this.upsertProviderMapping(existingSymbol.id, provider, symbol);
+      result.updated++;
+    } else {
+      const newSymbolId = await this.createSymbol(symbol, providerMapping.master_exchange_id);
+      await this.upsertProviderMapping(newSymbolId, provider, symbol);
+      result.created++;
+    }
+  }
+
+  private async findActiveProviderExchangeMapping(
+    provider: string,
+    providerExchangeCode: string
+  ): Promise<any> {
+    const query = `
+      SELECT pem.*, e.code as master_exchange_code 
+      FROM provider_exchange_mappings pem
+      JOIN exchanges e ON pem.master_exchange_id = e.id
+      WHERE pem.provider = $1 AND pem.provider_exchange_code = $2 AND pem.active = true
+    `;
+    const result = await this.postgresClient.query(query, [provider, providerExchangeCode]);
+    return result.rows[0] || null;
+  }
+
+  private async findExchangeByCode(code: string): Promise<any> {
+    const query = 'SELECT * FROM exchanges WHERE code = $1';
+    const result = await this.postgresClient.query(query, [code]);
+    return result.rows[0] || null;
+  }
+
+  private async findSymbolByCodeAndExchange(symbol: string, exchangeId: string): Promise<any> {
+    const query = 'SELECT * FROM symbols WHERE symbol = $1 AND exchange_id = $2';
+    const result = await this.postgresClient.query(query, [symbol, exchangeId]);
+    return result.rows[0] || null;
+  }
+
+  private async createSymbol(symbol: any, exchangeId: string): Promise<string> {
+    const query = `
+      INSERT INTO symbols (symbol, exchange_id, company_name, country, currency)
+      VALUES ($1, $2, $3, $4, $5)
+      RETURNING id
+    `;
+
+    const result = await this.postgresClient.query(query, [
+      symbol.symbol || symbol.code,
+      exchangeId,
+      symbol.companyName || symbol.name || symbol.company_name,
+      symbol.countryCode || symbol.country_code || 'US',
+      symbol.currency || 'USD',
+    ]);
+
+    return result.rows[0].id;
+  }
+
+  private async updateSymbol(symbolId: string, symbol: any): Promise<void> {
+    const query = `
+      UPDATE symbols 
+      SET company_name = COALESCE($2, company_name),
+          country = COALESCE($3, country),
+          currency = COALESCE($4, currency),
+          updated_at = NOW()
+      WHERE id = $1
+    `;
+
+    await this.postgresClient.query(query, [
+      symbolId,
+      symbol.companyName || symbol.name || symbol.company_name,
+      symbol.countryCode || symbol.country_code,
+      symbol.currency,
+    ]);
+  }
+
+  private async upsertProviderMapping(
+    symbolId: string,
+    provider: string,
+    symbol: any
+  ): Promise<void> {
+    const query = `
+      INSERT INTO provider_mappings 
+      (symbol_id, provider, provider_symbol, provider_exchange, last_seen)
+      VALUES ($1, $2, $3, $4, NOW())
+      ON CONFLICT (provider, provider_symbol)
+      DO UPDATE SET 
+        symbol_id = EXCLUDED.symbol_id,
+        provider_exchange = EXCLUDED.provider_exchange,
+        last_seen = NOW()
+    `;
+
+    await this.postgresClient.query(query, [
+      symbolId,
+      provider,
+      symbol.qmSearchCode || symbol.symbol || symbol.code,
+      symbol.exchangeCode || symbol.exchange || symbol.exchange_id,
+    ]);
+  }
+
+  private async updateSyncStatus(provider: string, dataType: string, count: number): Promise<void> {
+    const query = `
+      INSERT INTO sync_status (provider, data_type, last_sync_at, last_sync_count, sync_errors)
+      VALUES ($1, $2, NOW(), $3, NULL)
+      ON CONFLICT (provider, data_type)
+      DO UPDATE SET 
+        last_sync_at = NOW(),
+        last_sync_count = EXCLUDED.last_sync_count,
+        sync_errors = NULL,
+        updated_at = NOW()
+    `;
+
+    await this.postgresClient.query(query, [provider, dataType, count]);
+  }
+
+  private normalizeCountryCode(countryCode: string): string | null {
+    if (!countryCode) return null;
+
+    // Map common country variations to ISO 2-letter codes
+    const countryMap: Record<string, string> = {
+      'United States': 'US',
+      USA: 'US',
+      Canada: 'CA',
+      'United Kingdom': 'GB',
+      UK: 'GB',
+      Germany: 'DE',
+      Japan: 'JP',
+      Australia: 'AU',
+    };
+
+    const normalized = countryMap[countryCode];
+    return normalized || (countryCode.length === 2 ? countryCode.toUpperCase() : null);
+  }
+
+  private mergeResults(target: SyncResult, source: SyncResult): void {
+    target.processed += source.processed;
+    target.created += source.created;
+    target.updated += source.updated;
+    target.skipped += source.skipped;
+    target.errors += source.errors;
+  }
+}
+
+// Export singleton instance
+export const enhancedSyncManager = new EnhancedSyncManager();

apps/data-sync-service/src/services/index.ts

@@ -0,0 +1,2 @@
+export { syncManager } from './sync-manager';
+export { enhancedSyncManager } from './enhanced-sync-manager';

apps/data-sync-service/src/services/sync-manager.ts

@@ -0,0 +1,306 @@
+/**
+ * Sync Manager - Handles syncing raw MongoDB data to PostgreSQL master records
+ */
+import { getLogger } from '@stock-bot/logger';
+import { getMongoDBClient } from '@stock-bot/mongodb-client';
+import { getPostgreSQLClient } from '@stock-bot/postgres-client';
+
+const logger = getLogger('sync-manager');
+
+export class SyncManager {
+  private isInitialized = false;
+  private mongoClient: any;
+  private postgresClient: any;
+
+  async initialize(): Promise<void> {
+    if (this.isInitialized) {
+      logger.warn('Sync manager already initialized');
+      return;
+    }
+
+    try {
+      this.mongoClient = getMongoDBClient();
+      this.postgresClient = getPostgreSQLClient();
+
+      this.isInitialized = true;
+      logger.info('Sync manager initialized successfully');
+    } catch (error) {
+      logger.error('Failed to initialize sync manager', { error });
+      throw error;
+    }
+  }
+
+  async shutdown(): Promise<void> {
+    if (!this.isInitialized) {
+      return;
+    }
+
+    logger.info('Shutting down sync manager...');
+    this.isInitialized = false;
+    logger.info('Sync manager shut down successfully');
+  }
+
+  /**
+   * Sync QM symbols from MongoDB to PostgreSQL
+   */
+  async syncQMSymbols(): Promise<{ processed: number; created: number; updated: number }> {
+    if (!this.isInitialized) {
+      throw new Error('Sync manager not initialized');
+    }
+
+    logger.info('Starting QM symbols sync...');
+
+    try {
+      // 1. Get all QM symbols from MongoDB
+      const qmSymbols = await this.mongoClient.find('qmSymbols', {});
+      logger.info(`Found ${qmSymbols.length} QM symbols to process`);
+
+      let created = 0;
+      let updated = 0;
+
+      for (const symbol of qmSymbols) {
+        try {
+          // 2. Resolve exchange
+          const exchangeId = await this.resolveExchange(symbol.exchangeCode || symbol.exchange);
+
+          if (!exchangeId) {
+            logger.warn('Unknown exchange, skipping symbol', {
+              symbol: symbol.symbol,
+              exchange: symbol.exchangeCode || symbol.exchange,
+            });
+            continue;
+          }
+
+          // 3. Check if symbol exists
+          const existingSymbol = await this.findSymbol(symbol.symbol, exchangeId);
+
+          if (existingSymbol) {
+            // Update existing
+            await this.updateSymbol(existingSymbol.id, symbol);
+            await this.upsertProviderMapping(existingSymbol.id, 'qm', symbol);
+            updated++;
+          } else {
+            // Create new
+            const newSymbolId = await this.createSymbol(symbol, exchangeId);
+            await this.upsertProviderMapping(newSymbolId, 'qm', symbol);
+            created++;
+          }
+        } catch (error) {
+          logger.error('Failed to process symbol', { error, symbol: symbol.symbol });
+        }
+      }
+
+      // 4. Update sync status
+      await this.updateSyncStatus('qm', 'symbols', qmSymbols.length);
+
+      const result = { processed: qmSymbols.length, created, updated };
+      logger.info('QM symbols sync completed', result);
+      return result;
+    } catch (error) {
+      logger.error('QM symbols sync failed', { error });
+      throw error;
+    }
+  }
+
+  /**
+   * Sync QM exchanges from MongoDB to PostgreSQL
+   */
+  async syncQMExchanges(): Promise<{ processed: number; created: number; updated: number }> {
+    if (!this.isInitialized) {
+      throw new Error('Sync manager not initialized');
+    }
+
+    logger.info('Starting QM exchanges sync...');
+
+    try {
+      // 1. Get all QM exchanges from MongoDB
+      const qmExchanges = await this.mongoClient.find('qmExchanges', {});
+      logger.info(`Found ${qmExchanges.length} QM exchanges to process`);
+
+      let created = 0;
+      let updated = 0;
+
+      for (const exchange of qmExchanges) {
+        try {
+          // 2. Check if exchange exists
+          const existingExchange = await this.findExchange(exchange.exchangeCode);
+
+          if (existingExchange) {
+            // Update existing
+            await this.updateExchange(existingExchange.id, exchange);
+            updated++;
+          } else {
+            // Create new
+            await this.createExchange(exchange);
+            created++;
+          }
+        } catch (error) {
+          logger.error('Failed to process exchange', { error, exchange: exchange.exchangeCode });
+        }
+      }
+
+      // 3. Update sync status
+      await this.updateSyncStatus('qm', 'exchanges', qmExchanges.length);
+
+      const result = { processed: qmExchanges.length, created, updated };
+      logger.info('QM exchanges sync completed', result);
+      return result;
+    } catch (error) {
+      logger.error('QM exchanges sync failed', { error });
+      throw error;
+    }
+  }
+
+  /**
+   * Get sync status for all providers
+   */
+  async getSyncStatus(): Promise<any[]> {
+    const query = 'SELECT * FROM sync_status ORDER BY provider, data_type';
+    const result = await this.postgresClient.query(query);
+    return result.rows;
+  }
+
+  // Helper methods
+
+  private async resolveExchange(exchangeCode: string): Promise<string | null> {
+    if (!exchangeCode) return null;
+
+    // Simple mapping - expand this as needed
+    const exchangeMap: Record<string, string> = {
+      NASDAQ: 'NASDAQ',
+      NYSE: 'NYSE',
+      TSX: 'TSX',
+      TSE: 'TSX', // TSE maps to TSX
+      LSE: 'LSE',
+      CME: 'CME',
+    };
+
+    const normalizedCode = exchangeMap[exchangeCode.toUpperCase()];
+    if (!normalizedCode) {
+      return null;
+    }
+
+    const query = 'SELECT id FROM exchanges WHERE code = $1';
+    const result = await this.postgresClient.query(query, [normalizedCode]);
+    return result.rows[0]?.id || null;
+  }
+
+  private async findSymbol(symbol: string, exchangeId: string): Promise<any> {
+    const query = 'SELECT * FROM symbols WHERE symbol = $1 AND exchange_id = $2';
+    const result = await this.postgresClient.query(query, [symbol, exchangeId]);
+    return result.rows[0] || null;
+  }
+
+  private async createSymbol(qmSymbol: any, exchangeId: string): Promise<string> {
+    const query = `
+      INSERT INTO symbols (symbol, exchange_id, company_name, country, currency)
+      VALUES ($1, $2, $3, $4, $5)
+      RETURNING id
+    `;
+
+    const result = await this.postgresClient.query(query, [
+      qmSymbol.symbol,
+      exchangeId,
+      qmSymbol.companyName || qmSymbol.name,
+      qmSymbol.countryCode || 'US',
+      qmSymbol.currency || 'USD',
+    ]);
+
+    return result.rows[0].id;
+  }
+
+  private async updateSymbol(symbolId: string, qmSymbol: any): Promise<void> {
+    const query = `
+      UPDATE symbols 
+      SET company_name = COALESCE($2, company_name),
+          country = COALESCE($3, country),
+          currency = COALESCE($4, currency),
+          updated_at = NOW()
+      WHERE id = $1
+    `;
+
+    await this.postgresClient.query(query, [
+      symbolId,
+      qmSymbol.companyName || qmSymbol.name,
+      qmSymbol.countryCode,
+      qmSymbol.currency,
+    ]);
+  }
+
+  private async upsertProviderMapping(
+    symbolId: string,
+    provider: string,
+    qmSymbol: any
+  ): Promise<void> {
+    const query = `
+      INSERT INTO provider_mappings 
+      (symbol_id, provider, provider_symbol, provider_exchange, last_seen)
+      VALUES ($1, $2, $3, $4, NOW())
+      ON CONFLICT (provider, provider_symbol)
+      DO UPDATE SET 
+        symbol_id = EXCLUDED.symbol_id,
+        provider_exchange = EXCLUDED.provider_exchange,
+        last_seen = NOW()
+    `;
+
+    await this.postgresClient.query(query, [
+      symbolId,
+      provider,
+      qmSymbol.qmSearchCode || qmSymbol.symbol,
+      qmSymbol.exchangeCode || qmSymbol.exchange,
+    ]);
+  }
+
+  private async findExchange(exchangeCode: string): Promise<any> {
+    const query = 'SELECT * FROM exchanges WHERE code = $1';
+    const result = await this.postgresClient.query(query, [exchangeCode]);
+    return result.rows[0] || null;
+  }
+
+  private async createExchange(qmExchange: any): Promise<void> {
+    const query = `
+      INSERT INTO exchanges (code, name, country, currency)
+      VALUES ($1, $2, $3, $4)
+      ON CONFLICT (code) DO NOTHING
+    `;
+
+    await this.postgresClient.query(query, [
+      qmExchange.exchangeCode || qmExchange.exchange,
+      qmExchange.exchangeShortName || qmExchange.name,
+      qmExchange.countryCode || 'US',
+      'USD', // Default currency, can be improved
+    ]);
+  }
+
+  private async updateExchange(exchangeId: string, qmExchange: any): Promise<void> {
+    const query = `
+      UPDATE exchanges 
+      SET name = COALESCE($2, name),
+          country = COALESCE($3, country),
+          updated_at = NOW()
+      WHERE id = $1
+    `;
+
+    await this.postgresClient.query(query, [
+      exchangeId,
+      qmExchange.exchangeShortName || qmExchange.name,
+      qmExchange.countryCode,
+    ]);
+  }
+
+  private async updateSyncStatus(provider: string, dataType: string, count: number): Promise<void> {
+    const query = `
+      UPDATE sync_status 
+      SET last_sync_at = NOW(),
+          last_sync_count = $3,
+          sync_errors = NULL,
+          updated_at = NOW()
+      WHERE provider = $1 AND data_type = $2
+    `;
+
+    await this.postgresClient.query(query, [provider, dataType, count]);
+  }
+}
+
+// Export singleton instance
+export const syncManager = new SyncManager();

apps/data-sync-service/tsconfig.json

@@ -0,0 +1,22 @@
+{
+  "compilerOptions": {
+    "target": "ES2022",
+    "module": "ESNext",
+    "moduleResolution": "bundler",
+    "strict": true,
+    "esModuleInterop": true,
+    "skipLibCheck": true,
+    "forceConsistentCasingInFileNames": true,
+    "outDir": "./dist",
+    "rootDir": "./src",
+    "declaration": true,
+    "declarationMap": true,
+    "sourceMap": true,
+    "resolveJsonModule": true,
+    "allowImportingTsExtensions": false,
+    "noEmit": false,
+    "allowSyntheticDefaultImports": true
+  },
+  "include": ["src/**/*"],
+  "exclude": ["node_modules", "dist", "**/*.test.ts"]
+}

apps/data-sync-service/turbo.json

@@ -0,0 +1,3 @@
+{
+  "extends": ["//"]
+}