added new exchanges system

This commit is contained in:
Boki 2025-06-17 23:19:12 -04:00
parent 95eda4a842
commit 263e9513b7
98 changed files with 4643 additions and 1496 deletions

View file

@ -0,0 +1,184 @@
# Data Sync Service
The Data Sync Service handles synchronization of raw MongoDB data to PostgreSQL master records, providing a unified data layer for the stock-bot application.
## Features
### Original Sync Manager
- Basic QM (QuoteMedia) symbol and exchange synchronization
- Simple static exchange mapping
- Manual sync triggers via REST API
### Enhanced Sync Manager ✨ NEW
- **Multi-provider support**: Syncs from EOD, Interactive Brokers, and QuoteMedia
- **Comprehensive exchange handling**: Leverages all 4 MongoDB exchange collections
- **Intelligent exchange mapping**: Dynamic mapping with fallback logic
- **Transaction safety**: Full ACID compliance with rollback on errors
- **Performance optimization**: Exchange caching for faster lookups
- **Enhanced error handling**: Detailed error tracking and reporting
## API Endpoints
### Health Check
- `GET /health` - Service health status
### Original Sync Operations
- `POST /sync/symbols` - Sync QM symbols to PostgreSQL
- `POST /sync/exchanges` - Sync QM exchanges to PostgreSQL
- `GET /sync/status` - Get basic sync status
### Enhanced Sync Operations ✨ NEW
- `POST /sync/exchanges/all?clear=true` - Comprehensive exchange sync from all providers (clear=true removes dummy data first)
- `POST /sync/symbols/:provider?clear=true` - Sync symbols from specific provider (qm, eod, ib)
- `POST /sync/clear` - Clear all PostgreSQL data (exchanges, symbols, mappings)
- `GET /sync/status/enhanced` - Get detailed sync status
- `GET /sync/stats/exchanges` - Get exchange statistics
## Data Sources
### MongoDB Collections
1. **exchanges** (34 records) - Unified exchange reference
2. **eodExchanges** (78 records) - EOD provider with currency/MIC data
3. **ibExchanges** (214 records) - Interactive Brokers with asset types
4. **qmExchanges** (25 records) - QuoteMedia exchanges
### PostgreSQL Tables
1. **master_exchanges** - Unified exchange master data
2. **master_symbols** - Symbol master records
3. **provider_symbol_mappings** - Multi-provider symbol mappings
4. **sync_status** - Synchronization tracking
## Key Improvements
### 1. Multi-Provider Exchange Sync
Instead of only syncing QM exchanges, the enhanced sync manager:
- Syncs from unified `exchanges` collection first (primary source)
- Enriches with EOD exchanges (comprehensive global data with currencies)
- Adds IB exchanges for additional coverage (214 exchanges vs 25 in QM)
### 2. Intelligent Exchange Mapping
Replaces hard-coded mapping with dynamic resolution:
```typescript
// Before: Static mapping
const exchangeMap = { 'NASDAQ': 'NASDAQ', 'NYSE': 'NYSE' };
// After: Dynamic mapping with variations
const codeMap = {
'NASDAQ': 'NASDAQ', 'NAS': 'NASDAQ',
'NYSE': 'NYSE', 'NYQ': 'NYSE',
'LSE': 'LSE', 'LON': 'LSE', 'LN': 'LSE',
'US': 'NYSE' // EOD uses 'US' for US markets
};
```
### 3. Transaction Safety
All sync operations use database transactions:
- `BEGIN` transaction at start
- `COMMIT` on success
- `ROLLBACK` on any error
- Ensures data consistency
### 4. Performance Optimization
- Exchange cache preloaded at startup
- Reduced database queries during symbol processing
- Batch operations where possible
### 5. Enhanced Error Handling
- Detailed error logging with context
- Separate error counting in sync results
- Graceful handling of missing/invalid data
## Usage Examples
### Clear All Data and Start Fresh Exchange Sync
```bash
curl -X POST "http://localhost:3005/sync/exchanges/all?clear=true"
```
### Sync Symbols from Specific Provider
```bash
# Sync QuoteMedia symbols (clear existing symbols first)
curl -X POST "http://localhost:3005/sync/symbols/qm?clear=true"
# Sync EOD symbols
curl -X POST http://localhost:3005/sync/symbols/eod
# Sync Interactive Brokers symbols
curl -X POST http://localhost:3005/sync/symbols/ib
```
### Clear All PostgreSQL Data
```bash
curl -X POST http://localhost:3005/sync/clear
```
### Get Enhanced Status
```bash
curl http://localhost:3005/sync/status/enhanced
curl http://localhost:3005/sync/stats/exchanges
```
## Configuration
### Environment Variables
- `DATA_SYNC_SERVICE_PORT` - Service port (default: 3005)
- `NODE_ENV` - Environment mode
### Database Connections
- **MongoDB**: `mongodb://trading_admin:trading_mongo_dev@localhost:27017/stock?authSource=admin`
- **PostgreSQL**: `postgresql://trading_user:trading_pass_dev@localhost:5432/trading_bot`
## Development
### Build and Run
```bash
# Development mode
bun run dev
# Build
bun run build
# Production
bun run start
```
### Testing
```bash
# Run tests
bun test
# Start infrastructure
bun run infra:up
# Test sync operations
curl -X POST http://localhost:3005/sync/exchanges/all
curl -X POST http://localhost:3005/sync/symbols/qm
```
## Architecture
```
MongoDB Collections PostgreSQL Tables
┌─ exchanges (34) ┐ ┌─ master_exchanges
├─ eodExchanges (78) ├──▶├─ master_symbols
├─ ibExchanges (214) │ ├─ provider_symbol_mappings
└─ qmExchanges (25) ┘ └─ sync_status
Enhanced Sync Manager
- Exchange caching
- Dynamic mapping
- Transaction safety
- Multi-provider support
```
## Migration Path
The enhanced sync manager is designed to work alongside the original sync manager:
1. **Immediate**: Use enhanced exchange sync for better coverage
2. **Phase 1**: Test enhanced symbol sync with each provider
3. **Phase 2**: Replace original sync manager when confident
4. **Phase 3**: Remove original sync manager and endpoints
Both managers can be used simultaneously during the transition period.

View file

@ -0,0 +1,25 @@
{
"name": "@stock-bot/data-sync-service",
"version": "1.0.0",
"description": "Sync service from MongoDB raw data to PostgreSQL master records",
"main": "dist/index.js",
"type": "module",
"scripts": {
"dev": "bun --watch src/index.ts",
"build": "bun build src/index.ts --outdir dist --target node",
"start": "bun dist/index.js",
"test": "bun test",
"clean": "rm -rf dist"
},
"dependencies": {
"@stock-bot/config": "*",
"@stock-bot/logger": "*",
"@stock-bot/mongodb-client": "*",
"@stock-bot/postgres-client": "*",
"@stock-bot/shutdown": "*",
"hono": "^4.0.0"
},
"devDependencies": {
"typescript": "^5.0.0"
}
}

View file

@ -0,0 +1,263 @@
/**
* Data Sync Service - Sync raw MongoDB data to PostgreSQL master records
*/
import { Hono } from 'hono';
import { cors } from 'hono/cors';
import { loadEnvVariables } from '@stock-bot/config';
import { getLogger, shutdownLoggers } from '@stock-bot/logger';
import { connectMongoDB, disconnectMongoDB } from '@stock-bot/mongodb-client';
import { connectPostgreSQL, disconnectPostgreSQL } from '@stock-bot/postgres-client';
import { Shutdown } from '@stock-bot/shutdown';
import { enhancedSyncManager } from './services/enhanced-sync-manager';
import { syncManager } from './services/sync-manager';
// Load environment variables
loadEnvVariables();
const app = new Hono();
// Add CORS middleware
app.use(
'*',
cors({
origin: '*',
allowMethods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
allowHeaders: ['Content-Type', 'Authorization'],
credentials: false,
})
);
const logger = getLogger('data-sync-service');
const PORT = parseInt(process.env.DATA_SYNC_SERVICE_PORT || '3005');
let server: ReturnType<typeof Bun.serve> | null = null;
// Initialize shutdown manager
const shutdown = Shutdown.getInstance({ timeout: 15000 });
// Basic health check endpoint
app.get('/health', c => {
return c.json({
status: 'healthy',
service: 'data-sync-service',
timestamp: new Date().toISOString(),
});
});
// Manual sync trigger endpoints
app.post('/sync/symbols', async c => {
try {
const result = await syncManager.syncQMSymbols();
return c.json({ success: true, result });
} catch (error) {
logger.error('Manual symbol sync failed', { error });
return c.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
500
);
}
});
app.post('/sync/exchanges', async c => {
try {
const result = await syncManager.syncQMExchanges();
return c.json({ success: true, result });
} catch (error) {
logger.error('Manual exchange sync failed', { error });
return c.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
500
);
}
});
// Get sync status
app.get('/sync/status', async c => {
try {
const status = await syncManager.getSyncStatus();
return c.json(status);
} catch (error) {
logger.error('Failed to get sync status', { error });
return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
}
});
// Enhanced sync endpoints
app.post('/sync/exchanges/all', async c => {
try {
const clearFirst = c.req.query('clear') === 'true';
const result = await enhancedSyncManager.syncAllExchanges(clearFirst);
return c.json({ success: true, result });
} catch (error) {
logger.error('Enhanced exchange sync failed', { error });
return c.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
500
);
}
});
app.post('/sync/provider-mappings/qm', async c => {
try {
const result = await enhancedSyncManager.syncQMProviderMappings();
return c.json({ success: true, result });
} catch (error) {
logger.error('QM provider mappings sync failed', { error });
return c.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
500
);
}
});
app.post('/sync/symbols/:provider', async c => {
try {
const provider = c.req.param('provider');
const clearFirst = c.req.query('clear') === 'true';
const result = await enhancedSyncManager.syncSymbolsFromProvider(provider, clearFirst);
return c.json({ success: true, result });
} catch (error) {
logger.error('Enhanced symbol sync failed', { error });
return c.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
500
);
}
});
// Clear data endpoint
app.post('/sync/clear', async c => {
try {
const result = await enhancedSyncManager.clearPostgreSQLData();
return c.json({ success: true, result });
} catch (error) {
logger.error('Clear PostgreSQL data failed', { error });
return c.json(
{ success: false, error: error instanceof Error ? error.message : 'Unknown error' },
500
);
}
});
// Enhanced status endpoints
app.get('/sync/status/enhanced', async c => {
try {
const status = await enhancedSyncManager.getSyncStatus();
return c.json(status);
} catch (error) {
logger.error('Failed to get enhanced sync status', { error });
return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
}
});
app.get('/sync/stats/exchanges', async c => {
try {
const stats = await enhancedSyncManager.getExchangeStats();
return c.json(stats);
} catch (error) {
logger.error('Failed to get exchange stats', { error });
return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
}
});
app.get('/sync/stats/provider-mappings', async c => {
try {
const stats = await enhancedSyncManager.getProviderMappingStats();
return c.json(stats);
} catch (error) {
logger.error('Failed to get provider mapping stats', { error });
return c.json({ error: error instanceof Error ? error.message : 'Unknown error' }, 500);
}
});
// Initialize services
async function initializeServices() {
logger.info('Initializing data sync service...');
try {
// Initialize MongoDB client
logger.info('Connecting to MongoDB...');
await connectMongoDB();
logger.info('MongoDB connected');
// Initialize PostgreSQL client
logger.info('Connecting to PostgreSQL...');
await connectPostgreSQL();
logger.info('PostgreSQL connected');
// Initialize sync managers
logger.info('Initializing sync managers...');
await syncManager.initialize();
await enhancedSyncManager.initialize();
logger.info('Sync managers initialized');
logger.info('All services initialized successfully');
} catch (error) {
logger.error('Failed to initialize services', { error });
throw error;
}
}
// Start server
async function startServer() {
await initializeServices();
server = Bun.serve({
port: PORT,
fetch: app.fetch,
development: process.env.NODE_ENV === 'development',
});
logger.info(`Data Sync Service started on port ${PORT}`);
}
// Register shutdown handlers
shutdown.onShutdown(async () => {
if (server) {
logger.info('Stopping HTTP server...');
try {
server.stop();
logger.info('HTTP server stopped');
} catch (error) {
logger.error('Error stopping HTTP server', { error });
}
}
});
shutdown.onShutdown(async () => {
logger.info('Shutting down sync managers...');
try {
await syncManager.shutdown();
await enhancedSyncManager.shutdown();
logger.info('Sync managers shut down');
} catch (error) {
logger.error('Error shutting down sync managers', { error });
}
});
shutdown.onShutdown(async () => {
logger.info('Disconnecting from databases...');
try {
await disconnectMongoDB();
await disconnectPostgreSQL();
logger.info('Database connections closed');
} catch (error) {
logger.error('Error closing database connections', { error });
}
});
shutdown.onShutdown(async () => {
try {
await shutdownLoggers();
process.stdout.write('Data sync service loggers shut down\n');
} catch (error) {
process.stderr.write(`Error shutting down loggers: ${error}\n`);
}
});
// Start the service
startServer().catch(error => {
logger.error('Failed to start data sync service', { error });
process.exit(1);
});
logger.info('Data sync service startup initiated');

View file

@ -0,0 +1,798 @@
/**
* Enhanced Sync Manager - Improved syncing with comprehensive exchange data
*/
import { getLogger } from '@stock-bot/logger';
import { getMongoDBClient } from '@stock-bot/mongodb-client';
import { getPostgreSQLClient } from '@stock-bot/postgres-client';
const logger = getLogger('enhanced-sync-manager');
interface ExchangeMapping {
id: string;
code: string;
name: string;
country: string;
currency: string;
}
interface SyncResult {
processed: number;
created: number;
updated: number;
skipped: number;
errors: number;
}
interface SyncStatus {
provider: string;
dataType: string;
lastSyncAt?: Date;
lastSyncCount: number;
syncErrors?: string;
}
export class EnhancedSyncManager {
private isInitialized = false;
private mongoClient: any;
private postgresClient: any;
private exchangeCache: Map<string, ExchangeMapping> = new Map();
async initialize(): Promise<void> {
if (this.isInitialized) {
logger.warn('Enhanced sync manager already initialized');
return;
}
try {
this.mongoClient = getMongoDBClient();
this.postgresClient = getPostgreSQLClient();
// Pre-load exchange mappings for performance
await this.loadExchangeCache();
this.isInitialized = true;
logger.info('Enhanced sync manager initialized successfully');
} catch (error) {
logger.error('Failed to initialize enhanced sync manager', { error });
throw error;
}
}
/**
* Helper method to get MongoDB database for direct queries
*/
private getMongoDatabase() {
return this.mongoClient.getDatabase();
}
async shutdown(): Promise<void> {
if (!this.isInitialized) {
return;
}
logger.info('Shutting down enhanced sync manager...');
this.exchangeCache.clear();
this.isInitialized = false;
logger.info('Enhanced sync manager shut down successfully');
}
/**
* Clear all exchange and symbol data from PostgreSQL
*/
async clearPostgreSQLData(): Promise<{
exchangesCleared: number;
symbolsCleared: number;
mappingsCleared: number;
}> {
if (!this.isInitialized) {
throw new Error('Enhanced sync manager not initialized');
}
logger.info('Clearing existing PostgreSQL data...');
try {
// Start transaction for atomic operations
await this.postgresClient.query('BEGIN');
// Get counts before clearing
const exchangeCountResult = await this.postgresClient.query(
'SELECT COUNT(*) as count FROM exchanges'
);
const symbolCountResult = await this.postgresClient.query(
'SELECT COUNT(*) as count FROM symbols'
);
const mappingCountResult = await this.postgresClient.query(
'SELECT COUNT(*) as count FROM provider_mappings'
);
const exchangesCleared = parseInt(exchangeCountResult.rows[0].count);
const symbolsCleared = parseInt(symbolCountResult.rows[0].count);
const mappingsCleared = parseInt(mappingCountResult.rows[0].count);
// Clear data in correct order (respect foreign keys)
await this.postgresClient.query('DELETE FROM provider_mappings');
await this.postgresClient.query('DELETE FROM symbols');
await this.postgresClient.query('DELETE FROM exchanges');
// Reset sync status
await this.postgresClient.query(
'UPDATE sync_status SET last_sync_at = NULL, last_sync_count = 0, sync_errors = NULL'
);
await this.postgresClient.query('COMMIT');
logger.info('PostgreSQL data cleared successfully', {
exchangesCleared,
symbolsCleared,
mappingsCleared,
});
return { exchangesCleared, symbolsCleared, mappingsCleared };
} catch (error) {
await this.postgresClient.query('ROLLBACK');
logger.error('Failed to clear PostgreSQL data', { error });
throw error;
}
}
/**
* Comprehensive exchange sync from all MongoDB providers
*/
async syncAllExchanges(clearFirst: boolean = true): Promise<SyncResult> {
if (!this.isInitialized) {
throw new Error('Enhanced sync manager not initialized');
}
logger.info('Starting comprehensive exchange sync...', { clearFirst });
const result: SyncResult = {
processed: 0,
created: 0,
updated: 0,
skipped: 0,
errors: 0,
};
try {
// Clear existing data if requested
if (clearFirst) {
await this.clearPostgreSQLData();
}
// Start transaction for atomic operations
await this.postgresClient.query('BEGIN');
// 1. Sync from unified exchanges collection (primary source)
const unifiedResult = await this.syncUnifiedExchanges();
this.mergeResults(result, unifiedResult);
// 2. Sync from EOD exchanges (comprehensive global data)
const eodResult = await this.syncEODExchanges();
this.mergeResults(result, eodResult);
// 3. Sync from IB exchanges (detailed asset information)
const ibResult = await this.syncIBExchanges();
this.mergeResults(result, ibResult);
// 4. Update sync status
await this.updateSyncStatus('all', 'exchanges', result.processed);
await this.postgresClient.query('COMMIT');
// Refresh exchange cache with new data
await this.loadExchangeCache();
logger.info('Comprehensive exchange sync completed', result);
return result;
} catch (error) {
await this.postgresClient.query('ROLLBACK');
logger.error('Comprehensive exchange sync failed', { error });
throw error;
}
}
/**
* Sync QM provider exchange mappings
*/
async syncQMProviderMappings(): Promise<SyncResult> {
if (!this.isInitialized) {
throw new Error('Enhanced sync manager not initialized');
}
logger.info('Starting QM provider exchange mappings sync...');
const result: SyncResult = {
processed: 0,
created: 0,
updated: 0,
skipped: 0,
errors: 0,
};
try {
// Start transaction
await this.postgresClient.query('BEGIN');
// Get unique exchange combinations from QM symbols
const db = this.getMongoDatabase();
const pipeline = [
{
$group: {
_id: {
exchangeCode: '$exchangeCode',
exchange: '$exchange',
countryCode: '$countryCode',
},
count: { $sum: 1 },
sampleExchange: { $first: '$exchange' },
},
},
{
$project: {
exchangeCode: '$_id.exchangeCode',
exchange: '$_id.exchange',
countryCode: '$_id.countryCode',
count: 1,
sampleExchange: 1,
},
},
];
const qmExchanges = await db.collection('qmSymbols').aggregate(pipeline).toArray();
logger.info(`Found ${qmExchanges.length} unique QM exchange combinations`);
for (const exchange of qmExchanges) {
try {
// Create provider exchange mapping for QM
await this.createProviderExchangeMapping(
'qm', // provider
exchange.exchangeCode,
exchange.sampleExchange || exchange.exchangeCode,
exchange.countryCode,
exchange.countryCode === 'CA' ? 'CAD' : 'USD', // Simple currency mapping
0.8 // good confidence for QM data
);
result.processed++;
result.created++;
} catch (error) {
logger.error('Failed to process QM exchange mapping', { error, exchange });
result.errors++;
}
}
await this.postgresClient.query('COMMIT');
logger.info('QM provider exchange mappings sync completed', result);
return result;
} catch (error) {
await this.postgresClient.query('ROLLBACK');
logger.error('QM provider exchange mappings sync failed', { error });
throw error;
}
}
/**
* Enhanced symbol sync with multi-provider mapping
*/
async syncSymbolsFromProvider(
provider: string,
clearFirst: boolean = false
): Promise<SyncResult> {
if (!this.isInitialized) {
throw new Error('Enhanced sync manager not initialized');
}
logger.info(`Starting ${provider} symbols sync...`, { clearFirst });
const result: SyncResult = {
processed: 0,
created: 0,
updated: 0,
skipped: 0,
errors: 0,
};
try {
// Clear existing data if requested (only symbols and mappings, keep exchanges)
if (clearFirst) {
await this.postgresClient.query('BEGIN');
await this.postgresClient.query('DELETE FROM provider_mappings');
await this.postgresClient.query('DELETE FROM symbols');
await this.postgresClient.query('COMMIT');
logger.info('Cleared existing symbols and mappings before sync');
}
// Start transaction
await this.postgresClient.query('BEGIN');
let symbols: any[] = [];
// Get symbols based on provider
const db = this.getMongoDatabase();
switch (provider.toLowerCase()) {
case 'qm':
symbols = await db.collection('qmSymbols').find({}).toArray();
break;
case 'eod':
symbols = await db.collection('eodSymbols').find({}).toArray();
break;
case 'ib':
symbols = await db.collection('ibSymbols').find({}).toArray();
break;
default:
throw new Error(`Unsupported provider: ${provider}`);
}
logger.info(`Found ${symbols.length} ${provider} symbols to process`);
result.processed = symbols.length;
for (const symbol of symbols) {
try {
await this.processSingleSymbol(symbol, provider, result);
} catch (error) {
logger.error('Failed to process symbol', {
error,
symbol: symbol.symbol || symbol.code,
provider,
});
result.errors++;
}
}
// Update sync status
await this.updateSyncStatus(provider, 'symbols', result.processed);
await this.postgresClient.query('COMMIT');
logger.info(`${provider} symbols sync completed`, result);
return result;
} catch (error) {
await this.postgresClient.query('ROLLBACK');
logger.error(`${provider} symbols sync failed`, { error });
throw error;
}
}
/**
* Get comprehensive sync status
*/
async getSyncStatus(): Promise<SyncStatus[]> {
const query = `
SELECT provider, data_type as "dataType", last_sync_at as "lastSyncAt",
last_sync_count as "lastSyncCount", sync_errors as "syncErrors"
FROM sync_status
ORDER BY provider, data_type
`;
const result = await this.postgresClient.query(query);
return result.rows;
}
/**
* Get exchange statistics
*/
async getExchangeStats(): Promise<any> {
const query = `
SELECT
COUNT(*) as total_exchanges,
COUNT(CASE WHEN active = true THEN 1 END) as active_exchanges,
COUNT(DISTINCT country) as countries,
COUNT(DISTINCT currency) as currencies
FROM exchanges
`;
const result = await this.postgresClient.query(query);
return result.rows[0];
}
/**
* Get provider exchange mapping statistics
*/
async getProviderMappingStats(): Promise<any> {
const query = `
SELECT
provider,
COUNT(*) as total_mappings,
COUNT(CASE WHEN active = true THEN 1 END) as active_mappings,
COUNT(CASE WHEN verified = true THEN 1 END) as verified_mappings,
COUNT(CASE WHEN auto_mapped = true THEN 1 END) as auto_mapped,
AVG(confidence) as avg_confidence
FROM provider_exchange_mappings
GROUP BY provider
ORDER BY provider
`;
const result = await this.postgresClient.query(query);
return result.rows;
}
// Private helper methods
private async loadExchangeCache(): Promise<void> {
const query = 'SELECT id, code, name, country, currency FROM exchanges';
const result = await this.postgresClient.query(query);
this.exchangeCache.clear();
for (const exchange of result.rows) {
this.exchangeCache.set(exchange.code.toUpperCase(), exchange);
}
logger.info(`Loaded ${this.exchangeCache.size} exchanges into cache`);
}
private async syncUnifiedExchanges(): Promise<SyncResult> {
const db = this.getMongoDatabase();
const exchanges = await db.collection('exchanges').find({}).toArray();
const result: SyncResult = { processed: 0, created: 0, updated: 0, skipped: 0, errors: 0 };
for (const exchange of exchanges) {
try {
// Create provider exchange mapping for unified collection
await this.createProviderExchangeMapping(
'unified', // provider
exchange.sourceCode || exchange.code,
exchange.sourceName || exchange.name,
exchange.sourceRegion,
null, // currency not in unified
0.9 // high confidence for unified mappings
);
result.processed++;
result.created++; // Count as created mapping
} catch (error) {
logger.error('Failed to process unified exchange', { error, exchange });
result.errors++;
}
}
return result;
}
private async syncEODExchanges(): Promise<SyncResult> {
const db = this.getMongoDatabase();
const exchanges = await db.collection('eodExchanges').find({ active: true }).toArray();
const result: SyncResult = { processed: 0, created: 0, updated: 0, skipped: 0, errors: 0 };
for (const exchange of exchanges) {
try {
// Create provider exchange mapping for EOD
await this.createProviderExchangeMapping(
'eod', // provider
exchange.Code,
exchange.Name,
exchange.CountryISO2,
exchange.Currency,
0.95 // very high confidence for EOD data
);
result.processed++;
result.created++; // Count as created mapping
} catch (error) {
logger.error('Failed to process EOD exchange', { error, exchange });
result.errors++;
}
}
return result;
}
private async syncIBExchanges(): Promise<SyncResult> {
const db = this.getMongoDatabase();
const exchanges = await db.collection('ibExchanges').find({}).toArray();
const result: SyncResult = { processed: 0, created: 0, updated: 0, skipped: 0, errors: 0 };
for (const exchange of exchanges) {
try {
// Create provider exchange mapping for IB
await this.createProviderExchangeMapping(
'ib', // provider
exchange.exchange_id,
exchange.name,
exchange.country_code,
'USD', // IB doesn't specify currency, default to USD
0.85 // good confidence for IB data
);
result.processed++;
result.created++; // Count as created mapping
} catch (error) {
logger.error('Failed to process IB exchange', { error, exchange });
result.errors++;
}
}
return result;
}
/**
* Create or update a provider exchange mapping
* This method intelligently maps provider exchanges to master exchanges
*/
private async createProviderExchangeMapping(
provider: string,
providerExchangeCode: string,
providerExchangeName: string,
countryCode: string | null,
currency: string | null,
confidence: number
): Promise<void> {
if (!providerExchangeCode) return;
// Check if mapping already exists
const existingMapping = await this.findProviderExchangeMapping(provider, providerExchangeCode);
if (existingMapping) {
// Don't override existing mappings to preserve manual work
return;
}
// Find or create master exchange
const masterExchange = await this.findOrCreateMasterExchange(
providerExchangeCode,
providerExchangeName,
countryCode,
currency
);
// Create the provider exchange mapping
const query = `
INSERT INTO provider_exchange_mappings
(provider, provider_exchange_code, provider_exchange_name, master_exchange_id,
country_code, currency, confidence, active, auto_mapped)
VALUES ($1, $2, $3, $4, $5, $6, $7, false, true)
ON CONFLICT (provider, provider_exchange_code) DO NOTHING
`;
await this.postgresClient.query(query, [
provider,
providerExchangeCode,
providerExchangeName,
masterExchange.id,
countryCode,
currency,
confidence,
]);
}
/**
* Find or create a master exchange based on provider data
*/
private async findOrCreateMasterExchange(
providerCode: string,
providerName: string,
countryCode: string | null,
currency: string | null
): Promise<any> {
// First, try to find exact match
let masterExchange = await this.findExchangeByCode(providerCode);
if (masterExchange) {
return masterExchange;
}
// Try to find by similar codes (basic mapping)
const basicMapping = this.getBasicExchangeMapping(providerCode);
if (basicMapping) {
masterExchange = await this.findExchangeByCode(basicMapping);
if (masterExchange) {
return masterExchange;
}
}
// Create new master exchange (inactive by default)
const query = `
INSERT INTO exchanges (code, name, country, currency, active)
VALUES ($1, $2, $3, $4, false)
ON CONFLICT (code) DO UPDATE SET
name = COALESCE(EXCLUDED.name, exchanges.name),
country = COALESCE(EXCLUDED.country, exchanges.country),
currency = COALESCE(EXCLUDED.currency, exchanges.currency)
RETURNING id, code, name, country, currency
`;
const result = await this.postgresClient.query(query, [
providerCode,
providerName || providerCode,
countryCode || 'US',
currency || 'USD',
]);
const newExchange = result.rows[0];
// Update cache
this.exchangeCache.set(newExchange.code.toUpperCase(), newExchange);
return newExchange;
}
/**
* Basic exchange code mapping for common cases
*/
private getBasicExchangeMapping(providerCode: string): string | null {
const mappings: Record<string, string> = {
NYE: 'NYSE',
NAS: 'NASDAQ',
TO: 'TSX',
LN: 'LSE',
LON: 'LSE',
};
return mappings[providerCode.toUpperCase()] || null;
}
private async findProviderExchangeMapping(
provider: string,
providerExchangeCode: string
): Promise<any> {
const query =
'SELECT * FROM provider_exchange_mappings WHERE provider = $1 AND provider_exchange_code = $2';
const result = await this.postgresClient.query(query, [provider, providerExchangeCode]);
return result.rows[0] || null;
}
private async processSingleSymbol(
symbol: any,
provider: string,
result: SyncResult
): Promise<void> {
const symbolCode = symbol.symbol || symbol.code;
const exchangeCode = symbol.exchangeCode || symbol.exchange || symbol.exchange_id;
if (!symbolCode || !exchangeCode) {
result.skipped++;
return;
}
// Find active provider exchange mapping
const providerMapping = await this.findActiveProviderExchangeMapping(provider, exchangeCode);
if (!providerMapping) {
result.skipped++;
return;
}
// Check if symbol exists
const existingSymbol = await this.findSymbolByCodeAndExchange(
symbolCode,
providerMapping.master_exchange_id
);
if (existingSymbol) {
await this.updateSymbol(existingSymbol.id, symbol);
await this.upsertProviderMapping(existingSymbol.id, provider, symbol);
result.updated++;
} else {
const newSymbolId = await this.createSymbol(symbol, providerMapping.master_exchange_id);
await this.upsertProviderMapping(newSymbolId, provider, symbol);
result.created++;
}
}
private async findActiveProviderExchangeMapping(
provider: string,
providerExchangeCode: string
): Promise<any> {
const query = `
SELECT pem.*, e.code as master_exchange_code
FROM provider_exchange_mappings pem
JOIN exchanges e ON pem.master_exchange_id = e.id
WHERE pem.provider = $1 AND pem.provider_exchange_code = $2 AND pem.active = true
`;
const result = await this.postgresClient.query(query, [provider, providerExchangeCode]);
return result.rows[0] || null;
}
private async findExchangeByCode(code: string): Promise<any> {
const query = 'SELECT * FROM exchanges WHERE code = $1';
const result = await this.postgresClient.query(query, [code]);
return result.rows[0] || null;
}
private async findSymbolByCodeAndExchange(symbol: string, exchangeId: string): Promise<any> {
const query = 'SELECT * FROM symbols WHERE symbol = $1 AND exchange_id = $2';
const result = await this.postgresClient.query(query, [symbol, exchangeId]);
return result.rows[0] || null;
}
private async createSymbol(symbol: any, exchangeId: string): Promise<string> {
const query = `
INSERT INTO symbols (symbol, exchange_id, company_name, country, currency)
VALUES ($1, $2, $3, $4, $5)
RETURNING id
`;
const result = await this.postgresClient.query(query, [
symbol.symbol || symbol.code,
exchangeId,
symbol.companyName || symbol.name || symbol.company_name,
symbol.countryCode || symbol.country_code || 'US',
symbol.currency || 'USD',
]);
return result.rows[0].id;
}
private async updateSymbol(symbolId: string, symbol: any): Promise<void> {
const query = `
UPDATE symbols
SET company_name = COALESCE($2, company_name),
country = COALESCE($3, country),
currency = COALESCE($4, currency),
updated_at = NOW()
WHERE id = $1
`;
await this.postgresClient.query(query, [
symbolId,
symbol.companyName || symbol.name || symbol.company_name,
symbol.countryCode || symbol.country_code,
symbol.currency,
]);
}
private async upsertProviderMapping(
symbolId: string,
provider: string,
symbol: any
): Promise<void> {
const query = `
INSERT INTO provider_mappings
(symbol_id, provider, provider_symbol, provider_exchange, last_seen)
VALUES ($1, $2, $3, $4, NOW())
ON CONFLICT (provider, provider_symbol)
DO UPDATE SET
symbol_id = EXCLUDED.symbol_id,
provider_exchange = EXCLUDED.provider_exchange,
last_seen = NOW()
`;
await this.postgresClient.query(query, [
symbolId,
provider,
symbol.qmSearchCode || symbol.symbol || symbol.code,
symbol.exchangeCode || symbol.exchange || symbol.exchange_id,
]);
}
private async updateSyncStatus(provider: string, dataType: string, count: number): Promise<void> {
const query = `
INSERT INTO sync_status (provider, data_type, last_sync_at, last_sync_count, sync_errors)
VALUES ($1, $2, NOW(), $3, NULL)
ON CONFLICT (provider, data_type)
DO UPDATE SET
last_sync_at = NOW(),
last_sync_count = EXCLUDED.last_sync_count,
sync_errors = NULL,
updated_at = NOW()
`;
await this.postgresClient.query(query, [provider, dataType, count]);
}
private normalizeCountryCode(countryCode: string): string | null {
if (!countryCode) return null;
// Map common country variations to ISO 2-letter codes
const countryMap: Record<string, string> = {
'United States': 'US',
USA: 'US',
Canada: 'CA',
'United Kingdom': 'GB',
UK: 'GB',
Germany: 'DE',
Japan: 'JP',
Australia: 'AU',
};
const normalized = countryMap[countryCode];
return normalized || (countryCode.length === 2 ? countryCode.toUpperCase() : null);
}
private mergeResults(target: SyncResult, source: SyncResult): void {
target.processed += source.processed;
target.created += source.created;
target.updated += source.updated;
target.skipped += source.skipped;
target.errors += source.errors;
}
}
// Export singleton instance
export const enhancedSyncManager = new EnhancedSyncManager();

View file

@ -0,0 +1,2 @@
export { syncManager } from './sync-manager';
export { enhancedSyncManager } from './enhanced-sync-manager';

View file

@ -0,0 +1,306 @@
/**
* Sync Manager - Handles syncing raw MongoDB data to PostgreSQL master records
*/
import { getLogger } from '@stock-bot/logger';
import { getMongoDBClient } from '@stock-bot/mongodb-client';
import { getPostgreSQLClient } from '@stock-bot/postgres-client';
const logger = getLogger('sync-manager');
export class SyncManager {
private isInitialized = false;
private mongoClient: any;
private postgresClient: any;
async initialize(): Promise<void> {
if (this.isInitialized) {
logger.warn('Sync manager already initialized');
return;
}
try {
this.mongoClient = getMongoDBClient();
this.postgresClient = getPostgreSQLClient();
this.isInitialized = true;
logger.info('Sync manager initialized successfully');
} catch (error) {
logger.error('Failed to initialize sync manager', { error });
throw error;
}
}
async shutdown(): Promise<void> {
if (!this.isInitialized) {
return;
}
logger.info('Shutting down sync manager...');
this.isInitialized = false;
logger.info('Sync manager shut down successfully');
}
/**
* Sync QM symbols from MongoDB to PostgreSQL
*/
async syncQMSymbols(): Promise<{ processed: number; created: number; updated: number }> {
if (!this.isInitialized) {
throw new Error('Sync manager not initialized');
}
logger.info('Starting QM symbols sync...');
try {
// 1. Get all QM symbols from MongoDB
const qmSymbols = await this.mongoClient.find('qmSymbols', {});
logger.info(`Found ${qmSymbols.length} QM symbols to process`);
let created = 0;
let updated = 0;
for (const symbol of qmSymbols) {
try {
// 2. Resolve exchange
const exchangeId = await this.resolveExchange(symbol.exchangeCode || symbol.exchange);
if (!exchangeId) {
logger.warn('Unknown exchange, skipping symbol', {
symbol: symbol.symbol,
exchange: symbol.exchangeCode || symbol.exchange,
});
continue;
}
// 3. Check if symbol exists
const existingSymbol = await this.findSymbol(symbol.symbol, exchangeId);
if (existingSymbol) {
// Update existing
await this.updateSymbol(existingSymbol.id, symbol);
await this.upsertProviderMapping(existingSymbol.id, 'qm', symbol);
updated++;
} else {
// Create new
const newSymbolId = await this.createSymbol(symbol, exchangeId);
await this.upsertProviderMapping(newSymbolId, 'qm', symbol);
created++;
}
} catch (error) {
logger.error('Failed to process symbol', { error, symbol: symbol.symbol });
}
}
// 4. Update sync status
await this.updateSyncStatus('qm', 'symbols', qmSymbols.length);
const result = { processed: qmSymbols.length, created, updated };
logger.info('QM symbols sync completed', result);
return result;
} catch (error) {
logger.error('QM symbols sync failed', { error });
throw error;
}
}
/**
* Sync QM exchanges from MongoDB to PostgreSQL
*/
async syncQMExchanges(): Promise<{ processed: number; created: number; updated: number }> {
if (!this.isInitialized) {
throw new Error('Sync manager not initialized');
}
logger.info('Starting QM exchanges sync...');
try {
// 1. Get all QM exchanges from MongoDB
const qmExchanges = await this.mongoClient.find('qmExchanges', {});
logger.info(`Found ${qmExchanges.length} QM exchanges to process`);
let created = 0;
let updated = 0;
for (const exchange of qmExchanges) {
try {
// 2. Check if exchange exists
const existingExchange = await this.findExchange(exchange.exchangeCode);
if (existingExchange) {
// Update existing
await this.updateExchange(existingExchange.id, exchange);
updated++;
} else {
// Create new
await this.createExchange(exchange);
created++;
}
} catch (error) {
logger.error('Failed to process exchange', { error, exchange: exchange.exchangeCode });
}
}
// 3. Update sync status
await this.updateSyncStatus('qm', 'exchanges', qmExchanges.length);
const result = { processed: qmExchanges.length, created, updated };
logger.info('QM exchanges sync completed', result);
return result;
} catch (error) {
logger.error('QM exchanges sync failed', { error });
throw error;
}
}
/**
* Get sync status for all providers
*/
async getSyncStatus(): Promise<any[]> {
const query = 'SELECT * FROM sync_status ORDER BY provider, data_type';
const result = await this.postgresClient.query(query);
return result.rows;
}
// Helper methods
private async resolveExchange(exchangeCode: string): Promise<string | null> {
if (!exchangeCode) return null;
// Simple mapping - expand this as needed
const exchangeMap: Record<string, string> = {
NASDAQ: 'NASDAQ',
NYSE: 'NYSE',
TSX: 'TSX',
TSE: 'TSX', // TSE maps to TSX
LSE: 'LSE',
CME: 'CME',
};
const normalizedCode = exchangeMap[exchangeCode.toUpperCase()];
if (!normalizedCode) {
return null;
}
const query = 'SELECT id FROM exchanges WHERE code = $1';
const result = await this.postgresClient.query(query, [normalizedCode]);
return result.rows[0]?.id || null;
}
private async findSymbol(symbol: string, exchangeId: string): Promise<any> {
const query = 'SELECT * FROM symbols WHERE symbol = $1 AND exchange_id = $2';
const result = await this.postgresClient.query(query, [symbol, exchangeId]);
return result.rows[0] || null;
}
private async createSymbol(qmSymbol: any, exchangeId: string): Promise<string> {
const query = `
INSERT INTO symbols (symbol, exchange_id, company_name, country, currency)
VALUES ($1, $2, $3, $4, $5)
RETURNING id
`;
const result = await this.postgresClient.query(query, [
qmSymbol.symbol,
exchangeId,
qmSymbol.companyName || qmSymbol.name,
qmSymbol.countryCode || 'US',
qmSymbol.currency || 'USD',
]);
return result.rows[0].id;
}
private async updateSymbol(symbolId: string, qmSymbol: any): Promise<void> {
const query = `
UPDATE symbols
SET company_name = COALESCE($2, company_name),
country = COALESCE($3, country),
currency = COALESCE($4, currency),
updated_at = NOW()
WHERE id = $1
`;
await this.postgresClient.query(query, [
symbolId,
qmSymbol.companyName || qmSymbol.name,
qmSymbol.countryCode,
qmSymbol.currency,
]);
}
private async upsertProviderMapping(
symbolId: string,
provider: string,
qmSymbol: any
): Promise<void> {
const query = `
INSERT INTO provider_mappings
(symbol_id, provider, provider_symbol, provider_exchange, last_seen)
VALUES ($1, $2, $3, $4, NOW())
ON CONFLICT (provider, provider_symbol)
DO UPDATE SET
symbol_id = EXCLUDED.symbol_id,
provider_exchange = EXCLUDED.provider_exchange,
last_seen = NOW()
`;
await this.postgresClient.query(query, [
symbolId,
provider,
qmSymbol.qmSearchCode || qmSymbol.symbol,
qmSymbol.exchangeCode || qmSymbol.exchange,
]);
}
private async findExchange(exchangeCode: string): Promise<any> {
const query = 'SELECT * FROM exchanges WHERE code = $1';
const result = await this.postgresClient.query(query, [exchangeCode]);
return result.rows[0] || null;
}
private async createExchange(qmExchange: any): Promise<void> {
const query = `
INSERT INTO exchanges (code, name, country, currency)
VALUES ($1, $2, $3, $4)
ON CONFLICT (code) DO NOTHING
`;
await this.postgresClient.query(query, [
qmExchange.exchangeCode || qmExchange.exchange,
qmExchange.exchangeShortName || qmExchange.name,
qmExchange.countryCode || 'US',
'USD', // Default currency, can be improved
]);
}
private async updateExchange(exchangeId: string, qmExchange: any): Promise<void> {
const query = `
UPDATE exchanges
SET name = COALESCE($2, name),
country = COALESCE($3, country),
updated_at = NOW()
WHERE id = $1
`;
await this.postgresClient.query(query, [
exchangeId,
qmExchange.exchangeShortName || qmExchange.name,
qmExchange.countryCode,
]);
}
private async updateSyncStatus(provider: string, dataType: string, count: number): Promise<void> {
const query = `
UPDATE sync_status
SET last_sync_at = NOW(),
last_sync_count = $3,
sync_errors = NULL,
updated_at = NOW()
WHERE provider = $1 AND data_type = $2
`;
await this.postgresClient.query(query, [provider, dataType, count]);
}
}
// Export singleton instance
export const syncManager = new SyncManager();

View file

@ -0,0 +1,22 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"outDir": "./dist",
"rootDir": "./src",
"declaration": true,
"declarationMap": true,
"sourceMap": true,
"resolveJsonModule": true,
"allowImportingTsExtensions": false,
"noEmit": false,
"allowSyntheticDefaultImports": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts"]
}

View file

@ -0,0 +1,3 @@
{
"extends": ["//"]
}

25
apps/web-api/package.json Normal file
View file

@ -0,0 +1,25 @@
{
"name": "@stock-bot/web-api",
"version": "1.0.0",
"description": "REST API service for stock bot web application",
"main": "dist/index.js",
"type": "module",
"scripts": {
"dev": "bun --watch src/index.ts",
"build": "bun build src/index.ts --outdir dist --target node",
"start": "bun dist/index.js",
"test": "bun test",
"clean": "rm -rf dist"
},
"dependencies": {
"@stock-bot/config": "*",
"@stock-bot/logger": "*",
"@stock-bot/mongodb-client": "*",
"@stock-bot/postgres-client": "*",
"@stock-bot/shutdown": "*",
"hono": "^4.0.0"
},
"devDependencies": {
"typescript": "^5.0.0"
}
}

130
apps/web-api/src/index.ts Normal file
View file

@ -0,0 +1,130 @@
/**
* Stock Bot Web API - REST API service for web application
*/
import { Hono } from 'hono';
import { cors } from 'hono/cors';
import { loadEnvVariables } from '@stock-bot/config';
import { getLogger, shutdownLoggers } from '@stock-bot/logger';
import { connectMongoDB, disconnectMongoDB } from '@stock-bot/mongodb-client';
import { connectPostgreSQL, disconnectPostgreSQL } from '@stock-bot/postgres-client';
import { Shutdown } from '@stock-bot/shutdown';
// Import routes
import { exchangeRoutes } from './routes/exchange.routes';
import { healthRoutes } from './routes/health.routes';
// Load environment variables
loadEnvVariables();
const app = new Hono();
// Add CORS middleware
app.use(
'*',
cors({
origin: ['http://localhost:4200', 'http://localhost:3000'], // React dev server ports
allowMethods: ['GET', 'POST', 'PUT', 'PATCH', 'DELETE', 'OPTIONS'],
allowHeaders: ['Content-Type', 'Authorization'],
credentials: true,
})
);
const logger = getLogger('web-api');
const PORT = parseInt(process.env.WEB_API_PORT || '4000');
let server: ReturnType<typeof Bun.serve> | null = null;
// Initialize shutdown manager
const shutdown = Shutdown.getInstance({ timeout: 15000 });
// Add routes
app.route('/health', healthRoutes);
app.route('/api/exchanges', exchangeRoutes);
// Basic API info endpoint
app.get('/', c => {
return c.json({
name: 'Stock Bot Web API',
version: '1.0.0',
status: 'running',
timestamp: new Date().toISOString(),
endpoints: {
health: '/health',
exchanges: '/api/exchanges',
},
});
});
// Initialize services
async function initializeServices() {
logger.info('Initializing web API service...');
try {
// Initialize MongoDB client
logger.info('Connecting to MongoDB...');
await connectMongoDB();
logger.info('MongoDB connected');
// Initialize PostgreSQL client
logger.info('Connecting to PostgreSQL...');
await connectPostgreSQL();
logger.info('PostgreSQL connected');
logger.info('All services initialized successfully');
} catch (error) {
logger.error('Failed to initialize services', { error });
throw error;
}
}
// Start server
async function startServer() {
await initializeServices();
server = Bun.serve({
port: PORT,
fetch: app.fetch,
development: process.env.NODE_ENV === 'development',
});
logger.info(`Stock Bot Web API started on port ${PORT}`);
}
// Register shutdown handlers
shutdown.onShutdown(async () => {
if (server) {
logger.info('Stopping HTTP server...');
try {
server.stop();
logger.info('HTTP server stopped');
} catch (error) {
logger.error('Error stopping HTTP server', { error });
}
}
});
shutdown.onShutdown(async () => {
logger.info('Disconnecting from databases...');
try {
await disconnectMongoDB();
await disconnectPostgreSQL();
logger.info('Database connections closed');
} catch (error) {
logger.error('Error closing database connections', { error });
}
});
shutdown.onShutdown(async () => {
try {
await shutdownLoggers();
process.stdout.write('Web API loggers shut down\n');
} catch (error) {
process.stderr.write(`Error shutting down loggers: ${error}\n`);
}
});
// Start the service
startServer().catch(error => {
logger.error('Failed to start web API service', { error });
process.exit(1);
});
logger.info('Web API service startup initiated');

View file

@ -0,0 +1,688 @@
/**
* Exchange management routes
*/
import { Hono } from 'hono';
import { getLogger } from '@stock-bot/logger';
import { getPostgreSQLClient } from '@stock-bot/postgres-client';
import { getMongoDBClient } from '@stock-bot/mongodb-client';
const logger = getLogger('exchange-routes');
export const exchangeRoutes = new Hono();
// Get all exchanges with provider mapping counts
exchangeRoutes.get('/', async c => {
try {
const postgresClient = getPostgreSQLClient();
const query = `
SELECT
e.id,
e.code,
e.name,
e.country,
e.currency,
e.active,
e.created_at,
e.updated_at,
COUNT(pem.id) as provider_mapping_count,
COUNT(CASE WHEN pem.active = true THEN 1 END) as active_mapping_count,
COUNT(CASE WHEN pem.verified = true THEN 1 END) as verified_mapping_count,
STRING_AGG(DISTINCT pem.provider, ', ') as providers
FROM exchanges e
LEFT JOIN provider_exchange_mappings pem ON e.id = pem.master_exchange_id
GROUP BY e.id, e.code, e.name, e.country, e.currency, e.active, e.created_at, e.updated_at
ORDER BY e.code
`;
const result = await postgresClient.query(query);
return c.json({
success: true,
data: result.rows,
total: result.rows.length,
});
} catch (error) {
logger.error('Failed to get exchanges', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Get exchange by ID with detailed provider mappings
exchangeRoutes.get('/:id', async c => {
try {
const exchangeId = c.req.param('id');
const postgresClient = getPostgreSQLClient();
// Get exchange details
const exchangeQuery = 'SELECT * FROM exchanges WHERE id = $1';
const exchangeResult = await postgresClient.query(exchangeQuery, [exchangeId]);
if (exchangeResult.rows.length === 0) {
return c.json({ success: false, error: 'Exchange not found' }, 404);
}
// Get provider mappings for this exchange
const mappingsQuery = `
SELECT
pem.*,
e.code as master_exchange_code,
e.name as master_exchange_name
FROM provider_exchange_mappings pem
JOIN exchanges e ON pem.master_exchange_id = e.id
WHERE pem.master_exchange_id = $1
ORDER BY pem.provider, pem.provider_exchange_code
`;
const mappingsResult = await postgresClient.query(mappingsQuery, [exchangeId]);
return c.json({
success: true,
data: {
exchange: exchangeResult.rows[0],
provider_mappings: mappingsResult.rows,
},
});
} catch (error) {
logger.error('Failed to get exchange details', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Update exchange (activate/deactivate, rename, etc.)
exchangeRoutes.patch('/:id', async c => {
try {
const exchangeId = c.req.param('id');
const body = await c.req.json();
const postgresClient = getPostgreSQLClient();
const updateFields = [];
const values = [];
let paramIndex = 1;
// Build dynamic update query
if (body.active !== undefined) {
updateFields.push(`active = $${paramIndex++}`);
values.push(body.active);
}
if (body.name !== undefined) {
updateFields.push(`name = $${paramIndex++}`);
values.push(body.name);
}
if (body.country !== undefined) {
updateFields.push(`country = $${paramIndex++}`);
values.push(body.country);
}
if (body.currency !== undefined) {
updateFields.push(`currency = $${paramIndex++}`);
values.push(body.currency);
}
if (updateFields.length === 0) {
return c.json({ success: false, error: 'No valid fields to update' }, 400);
}
updateFields.push(`updated_at = NOW()`);
values.push(exchangeId);
const query = `
UPDATE exchanges
SET ${updateFields.join(', ')}
WHERE id = $${paramIndex}
RETURNING *
`;
const result = await postgresClient.query(query, values);
if (result.rows.length === 0) {
return c.json({ success: false, error: 'Exchange not found' }, 404);
}
logger.info('Exchange updated', { exchangeId, updates: body });
return c.json({
success: true,
data: result.rows[0],
message: 'Exchange updated successfully',
});
} catch (error) {
logger.error('Failed to update exchange', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Get all provider mappings
exchangeRoutes.get('/provider-mappings/all', async c => {
try {
const postgresClient = getPostgreSQLClient();
const query = `
SELECT
pem.*,
e.code as master_exchange_code,
e.name as master_exchange_name,
e.active as master_exchange_active
FROM provider_exchange_mappings pem
JOIN exchanges e ON pem.master_exchange_id = e.id
ORDER BY pem.provider, pem.provider_exchange_code
`;
const result = await postgresClient.query(query);
return c.json({
success: true,
data: result.rows,
total: result.rows.length,
});
} catch (error) {
logger.error('Failed to get provider mappings', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Get provider mappings by provider
exchangeRoutes.get('/provider-mappings/:provider', async c => {
try {
const provider = c.req.param('provider');
const postgresClient = getPostgreSQLClient();
const query = `
SELECT
pem.*,
e.code as master_exchange_code,
e.name as master_exchange_name,
e.active as master_exchange_active
FROM provider_exchange_mappings pem
JOIN exchanges e ON pem.master_exchange_id = e.id
WHERE pem.provider = $1
ORDER BY pem.provider_exchange_code
`;
const result = await postgresClient.query(query, [provider]);
return c.json({
success: true,
data: result.rows,
total: result.rows.length,
provider,
});
} catch (error) {
logger.error('Failed to get provider mappings', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Update provider mapping (activate/deactivate, verify, change confidence)
exchangeRoutes.patch('/provider-mappings/:id', async c => {
try {
const mappingId = c.req.param('id');
const body = await c.req.json();
const postgresClient = getPostgreSQLClient();
const updateFields = [];
const values = [];
let paramIndex = 1;
// Build dynamic update query
if (body.active !== undefined) {
updateFields.push(`active = $${paramIndex++}`);
values.push(body.active);
}
if (body.verified !== undefined) {
updateFields.push(`verified = $${paramIndex++}`);
values.push(body.verified);
}
if (body.confidence !== undefined) {
updateFields.push(`confidence = $${paramIndex++}`);
values.push(body.confidence);
}
if (body.master_exchange_id !== undefined) {
updateFields.push(`master_exchange_id = $${paramIndex++}`);
values.push(body.master_exchange_id);
}
if (updateFields.length === 0) {
return c.json({ success: false, error: 'No valid fields to update' }, 400);
}
updateFields.push(`updated_at = NOW()`);
updateFields.push(`auto_mapped = false`); // Mark as manually managed
values.push(mappingId);
const query = `
UPDATE provider_exchange_mappings
SET ${updateFields.join(', ')}
WHERE id = $${paramIndex}
RETURNING *
`;
const result = await postgresClient.query(query, values);
if (result.rows.length === 0) {
return c.json({ success: false, error: 'Provider mapping not found' }, 404);
}
logger.info('Provider mapping updated', { mappingId, updates: body });
return c.json({
success: true,
data: result.rows[0],
message: 'Provider mapping updated successfully',
});
} catch (error) {
logger.error('Failed to update provider mapping', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Create new provider mapping
exchangeRoutes.post('/provider-mappings', async c => {
try {
const body = await c.req.json();
const postgresClient = getPostgreSQLClient();
const {
provider,
provider_exchange_code,
provider_exchange_name,
master_exchange_id,
country_code,
currency,
confidence = 1.0,
active = false,
verified = false,
} = body;
if (!provider || !provider_exchange_code || !master_exchange_id) {
return c.json(
{
success: false,
error: 'Missing required fields: provider, provider_exchange_code, master_exchange_id',
},
400
);
}
const query = `
INSERT INTO provider_exchange_mappings
(provider, provider_exchange_code, provider_exchange_name, master_exchange_id,
country_code, currency, confidence, active, verified, auto_mapped)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, false)
RETURNING *
`;
const result = await postgresClient.query(query, [
provider,
provider_exchange_code,
provider_exchange_name,
master_exchange_id,
country_code,
currency,
confidence,
active,
verified,
]);
logger.info('Provider mapping created', {
provider,
provider_exchange_code,
master_exchange_id,
});
return c.json(
{
success: true,
data: result.rows[0],
message: 'Provider mapping created successfully',
},
201
);
} catch (error) {
logger.error('Failed to create provider mapping', { error });
// Handle unique constraint violations
if (error instanceof Error && error.message.includes('duplicate key')) {
return c.json(
{
success: false,
error: 'Provider mapping already exists for this provider and exchange code',
},
409
);
}
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Get all available providers
exchangeRoutes.get('/providers/list', async c => {
try {
const postgresClient = getPostgreSQLClient();
const query = `
SELECT DISTINCT provider
FROM provider_exchange_mappings
ORDER BY provider
`;
const result = await postgresClient.query(query);
return c.json({
success: true,
data: result.rows.map(row => row.provider),
});
} catch (error) {
logger.error('Failed to get providers list', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Get all provider exchanges from MongoDB with mapping status
exchangeRoutes.get('/provider-exchanges/all', async c => {
try {
const postgresClient = getPostgreSQLClient();
const mongoClient = getMongoDBClient();
const db = mongoClient.getDatabase();
// Get all provider exchanges from different MongoDB collections
const [eodExchanges, ibExchanges, qmExchanges, unifiedExchanges] = await Promise.all([
db.collection('eodExchanges').find({ active: true }).toArray(),
db.collection('ibExchanges').find({}).toArray(),
db.collection('qmExchanges').find({}).toArray(),
db.collection('exchanges').find({}).toArray(),
]);
// Get existing mappings to mark which are already mapped
const existingMappingsQuery = `
SELECT provider, provider_exchange_code, master_exchange_id, e.code as master_exchange_code
FROM provider_exchange_mappings pem
JOIN exchanges e ON pem.master_exchange_id = e.id
`;
const existingMappings = await postgresClient.query(existingMappingsQuery);
const mappingLookup = new Map();
existingMappings.rows.forEach(row => {
mappingLookup.set(`${row.provider}:${row.provider_exchange_code}`, {
mapped: true,
master_exchange_id: row.master_exchange_id,
master_exchange_code: row.master_exchange_code,
});
});
// Combine all provider exchanges
const allProviderExchanges = [];
// EOD exchanges
eodExchanges.forEach(exchange => {
const key = `eod:${exchange.Code}`;
const mapping = mappingLookup.get(key);
allProviderExchanges.push({
provider: 'eod',
provider_exchange_code: exchange.Code,
provider_exchange_name: exchange.Name,
country_code: exchange.CountryISO2,
currency: exchange.Currency,
symbol_count: null,
is_mapped: !!mapping,
mapped_to_exchange_id: mapping?.master_exchange_id || null,
mapped_to_exchange_code: mapping?.master_exchange_code || null,
});
});
// IB exchanges
ibExchanges.forEach(exchange => {
const key = `ib:${exchange.exchange_id}`;
const mapping = mappingLookup.get(key);
allProviderExchanges.push({
provider: 'ib',
provider_exchange_code: exchange.exchange_id,
provider_exchange_name: exchange.name,
country_code: exchange.country_code,
currency: null,
symbol_count: null,
is_mapped: !!mapping,
mapped_to_exchange_id: mapping?.master_exchange_id || null,
mapped_to_exchange_code: mapping?.master_exchange_code || null,
});
});
// QM exchanges
qmExchanges.forEach(exchange => {
const key = `qm:${exchange.exchangeCode}`;
const mapping = mappingLookup.get(key);
allProviderExchanges.push({
provider: 'qm',
provider_exchange_code: exchange.exchangeCode,
provider_exchange_name: exchange.name,
country_code: exchange.countryCode,
currency: exchange.countryCode === 'CA' ? 'CAD' : 'USD',
symbol_count: null,
is_mapped: !!mapping,
mapped_to_exchange_id: mapping?.master_exchange_id || null,
mapped_to_exchange_code: mapping?.master_exchange_code || null,
});
});
// Unified exchanges
unifiedExchanges.forEach(exchange => {
const key = `unified:${exchange.sourceCode || exchange.code}`;
const mapping = mappingLookup.get(key);
allProviderExchanges.push({
provider: 'unified',
provider_exchange_code: exchange.sourceCode || exchange.code,
provider_exchange_name: exchange.sourceName || exchange.name,
country_code: null,
currency: null,
symbol_count: null,
is_mapped: !!mapping,
mapped_to_exchange_id: mapping?.master_exchange_id || null,
mapped_to_exchange_code: mapping?.master_exchange_code || null,
});
});
return c.json({
success: true,
data: allProviderExchanges,
total: allProviderExchanges.length,
});
} catch (error) {
logger.error('Failed to get provider exchanges', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Get unmapped provider exchanges by provider
exchangeRoutes.get('/provider-exchanges/unmapped/:provider', async c => {
try {
const provider = c.req.param('provider');
const postgresClient = getPostgreSQLClient();
const mongoClient = getMongoDBClient();
const db = mongoClient.getDatabase();
// Get existing mappings for this provider
const existingMappingsQuery = `
SELECT provider_exchange_code
FROM provider_exchange_mappings
WHERE provider = $1
`;
const existingMappings = await postgresClient.query(existingMappingsQuery, [provider]);
const mappedCodes = new Set(existingMappings.rows.map(row => row.provider_exchange_code));
let providerExchanges = [];
switch (provider) {
case 'eod':
const eodExchanges = await db.collection('eodExchanges').find({ active: true }).toArray();
providerExchanges = eodExchanges
.filter(exchange => !mappedCodes.has(exchange.Code))
.map(exchange => ({
provider_exchange_code: exchange.Code,
provider_exchange_name: exchange.Name,
country_code: exchange.CountryISO2,
currency: exchange.Currency,
symbol_count: null,
}));
break;
case 'ib':
const ibExchanges = await db.collection('ibExchanges').find({}).toArray();
providerExchanges = ibExchanges
.filter(exchange => !mappedCodes.has(exchange.exchange_id))
.map(exchange => ({
provider_exchange_code: exchange.exchange_id,
provider_exchange_name: exchange.name,
country_code: exchange.country_code,
currency: null,
symbol_count: null,
}));
break;
case 'qm':
const qmExchanges = await db.collection('qmExchanges').find({}).toArray();
providerExchanges = qmExchanges
.filter(exchange => !mappedCodes.has(exchange.exchangeCode))
.map(exchange => ({
provider_exchange_code: exchange.exchangeCode,
provider_exchange_name: exchange.name,
country_code: exchange.countryCode,
currency: exchange.countryCode === 'CA' ? 'CAD' : 'USD',
symbol_count: null,
}));
break;
case 'unified':
const unifiedExchanges = await db.collection('exchanges').find({}).toArray();
providerExchanges = unifiedExchanges
.filter(exchange => !mappedCodes.has(exchange.sourceCode || exchange.code))
.map(exchange => ({
provider_exchange_code: exchange.sourceCode || exchange.code,
provider_exchange_name: exchange.sourceName || exchange.name,
country_code: null,
currency: null,
symbol_count: null,
}));
break;
default:
return c.json(
{
success: false,
error: `Unknown provider: ${provider}`,
},
400
);
}
return c.json({
success: true,
data: providerExchanges,
total: providerExchanges.length,
provider,
});
} catch (error) {
logger.error('Failed to get unmapped provider exchanges', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});
// Get exchange statistics
exchangeRoutes.get('/stats/summary', async c => {
try {
const postgresClient = getPostgreSQLClient();
const query = `
SELECT
(SELECT COUNT(*) FROM exchanges) as total_exchanges,
(SELECT COUNT(*) FROM exchanges WHERE active = true) as active_exchanges,
(SELECT COUNT(DISTINCT country) FROM exchanges) as countries,
(SELECT COUNT(DISTINCT currency) FROM exchanges) as currencies,
(SELECT COUNT(*) FROM provider_exchange_mappings) as total_provider_mappings,
(SELECT COUNT(*) FROM provider_exchange_mappings WHERE active = true) as active_provider_mappings,
(SELECT COUNT(*) FROM provider_exchange_mappings WHERE verified = true) as verified_provider_mappings,
(SELECT COUNT(DISTINCT provider) FROM provider_exchange_mappings) as providers
`;
const result = await postgresClient.query(query);
return c.json({
success: true,
data: result.rows[0],
});
} catch (error) {
logger.error('Failed to get exchange statistics', { error });
return c.json(
{
success: false,
error: error instanceof Error ? error.message : 'Unknown error',
},
500
);
}
});

View file

@ -0,0 +1,69 @@
/**
* Health check routes
*/
import { Hono } from 'hono';
import { getLogger } from '@stock-bot/logger';
import { getMongoDBClient } from '@stock-bot/mongodb-client';
import { getPostgreSQLClient } from '@stock-bot/postgres-client';
const logger = getLogger('health-routes');
export const healthRoutes = new Hono();
// Basic health check
healthRoutes.get('/', c => {
return c.json({
status: 'healthy',
service: 'web-api',
timestamp: new Date().toISOString(),
});
});
// Detailed health check with database connectivity
healthRoutes.get('/detailed', async c => {
const health = {
status: 'healthy',
service: 'web-api',
timestamp: new Date().toISOString(),
checks: {
mongodb: { status: 'unknown', message: '' },
postgresql: { status: 'unknown', message: '' },
},
};
// Check MongoDB
try {
const mongoClient = getMongoDBClient();
if (mongoClient.connected) {
// Try a simple operation
const db = mongoClient.getDatabase();
await db.admin().ping();
health.checks.mongodb = { status: 'healthy', message: 'Connected and responsive' };
} else {
health.checks.mongodb = { status: 'unhealthy', message: 'Not connected' };
}
} catch (error) {
health.checks.mongodb = {
status: 'unhealthy',
message: error instanceof Error ? error.message : 'Unknown error',
};
}
// Check PostgreSQL
try {
const postgresClient = getPostgreSQLClient();
await postgresClient.query('SELECT 1');
health.checks.postgresql = { status: 'healthy', message: 'Connected and responsive' };
} catch (error) {
health.checks.postgresql = {
status: 'unhealthy',
message: error instanceof Error ? error.message : 'Unknown error',
};
}
// Overall status
const allHealthy = Object.values(health.checks).every(check => check.status === 'healthy');
health.status = allHealthy ? 'healthy' : 'unhealthy';
const statusCode = allHealthy ? 200 : 503;
return c.json(health, statusCode);
});

View file

@ -0,0 +1,5 @@
/**
* Routes index - exports all route modules
*/
export { exchangeRoutes } from './exchange.routes';
export { healthRoutes } from './health.routes';

View file

@ -0,0 +1,9 @@
{
"extends": "../../tsconfig.base.json",
"compilerOptions": {
"rootDir": "./src",
"outDir": "./dist"
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts"]
}

13
apps/web-api/turbo.json Normal file
View file

@ -0,0 +1,13 @@
{
"extends": ["//"],
"tasks": {
"build": {
"dependsOn": ["^build"],
"outputs": ["dist/**"]
},
"dev": {
"cache": false,
"persistent": true
}
}
}

View file

@ -1,5 +1,5 @@
{
"name": "@stock-bot/web",
"name": "@stock-bot/web-app",
"version": "0.1.0",
"private": true,
"scripts": {

View file

Before

Width:  |  Height:  |  Size: 314 B

After

Width:  |  Height:  |  Size: 314 B

Before After
Before After

View file

@ -0,0 +1,38 @@
import React from 'react';
interface ButtonProps extends React.ButtonHTMLAttributes<HTMLButtonElement> {
variant?: 'default' | 'outline' | 'danger';
size?: 'sm' | 'md' | 'lg';
children: React.ReactNode;
}
export function Button({
variant = 'default',
size = 'md',
className = '',
children,
disabled,
...props
}: ButtonProps) {
const baseClasses = 'inline-flex items-center justify-center rounded-md font-medium transition-colors focus:outline-none focus:ring-2 focus:ring-offset-2 disabled:opacity-50 disabled:cursor-not-allowed';
const variantClasses = {
default: 'bg-primary-500 text-white hover:bg-primary-600 focus:ring-primary-500',
outline: 'border border-border bg-transparent text-text-primary hover:bg-surface-secondary focus:ring-primary-500',
danger: 'bg-red-500 text-white hover:bg-red-600 focus:ring-red-500',
};
const sizeClasses = {
sm: 'px-2 py-1 text-xs',
md: 'px-4 py-2 text-sm',
lg: 'px-6 py-3 text-base',
};
const classes = `${baseClasses} ${variantClasses[variant]} ${sizeClasses[size]} ${className}`;
return (
<button className={classes} disabled={disabled} {...props}>
{children}
</button>
);
}

View file

@ -0,0 +1,59 @@
import React from 'react';
import { XMarkIcon } from '@heroicons/react/24/outline';
interface DialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
children: React.ReactNode;
}
export function Dialog({ open, onOpenChange, children }: DialogProps) {
if (!open) return null;
return (
<div className="fixed inset-0 z-50 flex items-center justify-center">
{/* Backdrop */}
<div
className="fixed inset-0 bg-black/50 backdrop-blur-sm"
onClick={() => onOpenChange(false)}
/>
{/* Dialog */}
<div className="relative z-50 max-h-[90vh] overflow-auto">
{children}
</div>
</div>
);
}
interface DialogContentProps {
children: React.ReactNode;
className?: string;
}
export function DialogContent({ children, className = '' }: DialogContentProps) {
return (
<div
className={`bg-surface border border-border rounded-lg shadow-lg p-6 w-full ${className}`}
>
{children}
</div>
);
}
interface DialogHeaderProps {
children: React.ReactNode;
className?: string;
}
export function DialogHeader({ children, className = '' }: DialogHeaderProps) {
return <div className={`mb-4 ${className}`}>{children}</div>;
}
interface DialogTitleProps {
children: React.ReactNode;
className?: string;
}
export function DialogTitle({ children, className = '' }: DialogTitleProps) {
return <h2 className={`text-lg font-semibold text-text-primary ${className}`}>{children}</h2>;
}

View file

@ -1,3 +1,5 @@
export { Card, CardHeader, CardContent } from './Card';
export { StatCard } from './StatCard';
export { DataTable } from './DataTable';
export { Dialog, DialogContent, DialogHeader, DialogTitle } from './dialog';
export { Button } from './button';

View file

@ -0,0 +1,251 @@
import { Dialog, DialogContent, DialogHeader, DialogTitle, Button } from '@/components/ui';
import { useCallback, useEffect, useState } from 'react';
import { useExchanges } from '../hooks/useExchanges';
import { CreateProviderMappingRequest } from '../types';
interface AddProviderMappingDialogProps {
isOpen: boolean;
exchangeId: string;
exchangeName: string;
onClose: () => void;
onCreateMapping: (request: CreateProviderMappingRequest) => Promise<any>;
}
export function AddProviderMappingDialog({
isOpen,
exchangeId,
exchangeName,
onClose,
onCreateMapping,
}: AddProviderMappingDialogProps) {
const { fetchProviders, fetchUnmappedProviderExchanges } = useExchanges();
const [providers, setProviders] = useState<string[]>([]);
const [selectedProvider, setSelectedProvider] = useState('');
const [unmappedExchanges, setUnmappedExchanges] = useState<any[]>([]);
const [selectedProviderExchange, setSelectedProviderExchange] = useState('');
const [loading, setLoading] = useState(false);
const [providersLoading, setProvidersLoading] = useState(false);
const [exchangesLoading, setExchangesLoading] = useState(false);
// Load providers on mount
useEffect(() => {
if (isOpen) {
loadProviders();
}
}, [isOpen]);
// Load unmapped exchanges when provider changes
useEffect(() => {
if (selectedProvider) {
loadUnmappedExchanges(selectedProvider);
} else {
setUnmappedExchanges([]);
setSelectedProviderExchange('');
}
}, [selectedProvider]);
const loadProviders = useCallback(async () => {
setProvidersLoading(true);
try {
const providersData = await fetchProviders();
setProviders(providersData);
} catch (error) {
console.error('Error loading providers:', error);
} finally {
setProvidersLoading(false);
}
}, [fetchProviders]);
const loadUnmappedExchanges = useCallback(
async (provider: string) => {
setExchangesLoading(true);
try {
const exchangesData = await fetchUnmappedProviderExchanges(provider);
setUnmappedExchanges(exchangesData);
} catch (error) {
console.error('Error loading unmapped exchanges:', error);
} finally {
setExchangesLoading(false);
}
},
[fetchUnmappedProviderExchanges]
);
const handleSubmit = useCallback(
async (e: React.FormEvent) => {
e.preventDefault();
if (!selectedProvider || !selectedProviderExchange) {
return;
}
const selectedExchange = unmappedExchanges.find(
exchange => exchange.provider_exchange_code === selectedProviderExchange
);
if (!selectedExchange) {
return;
}
setLoading(true);
try {
const request: CreateProviderMappingRequest = {
provider: selectedProvider,
provider_exchange_code: selectedExchange.provider_exchange_code,
provider_exchange_name: selectedExchange.provider_exchange_name,
master_exchange_id: exchangeId,
country_code: selectedExchange.country_code,
currency: selectedExchange.currency,
confidence: 1.0,
active: false,
verified: false,
};
await onCreateMapping(request);
} catch (error) {
console.error('Error creating provider mapping:', error);
} finally {
setLoading(false);
}
},
[selectedProvider, selectedProviderExchange, unmappedExchanges, exchangeId, onCreateMapping]
);
const handleClose = useCallback(() => {
setSelectedProvider('');
setSelectedProviderExchange('');
setUnmappedExchanges([]);
onClose();
}, [onClose]);
return (
<Dialog open={isOpen} onOpenChange={handleClose}>
<DialogContent className="max-w-md">
<DialogHeader>
<DialogTitle>Add Provider Mapping</DialogTitle>
<p className="text-sm text-text-muted">
Map a provider exchange to <strong>{exchangeName}</strong>
</p>
</DialogHeader>
<form onSubmit={handleSubmit} className="space-y-4">
{/* Provider Selection */}
<div>
<label htmlFor="provider" className="block text-sm font-medium text-text-primary mb-1">
Provider
</label>
<select
id="provider"
value={selectedProvider}
onChange={e => setSelectedProvider(e.target.value)}
disabled={providersLoading}
className="w-full px-3 py-2 border border-border rounded-md bg-surface text-text-primary focus:ring-2 focus:ring-primary-500 focus:border-primary-500"
required
>
<option value="">
{providersLoading ? 'Loading providers...' : 'Select a provider'}
</option>
{providers.map(provider => (
<option key={provider} value={provider}>
{provider.toUpperCase()}
</option>
))}
</select>
</div>
{/* Provider Exchange Selection */}
<div>
<label
htmlFor="providerExchange"
className="block text-sm font-medium text-text-primary mb-1"
>
Provider Exchange
</label>
<select
id="providerExchange"
value={selectedProviderExchange}
onChange={e => setSelectedProviderExchange(e.target.value)}
disabled={!selectedProvider || exchangesLoading}
className="w-full px-3 py-2 border border-border rounded-md bg-surface text-text-primary focus:ring-2 focus:ring-primary-500 focus:border-primary-500"
required
>
<option value="">
{!selectedProvider
? 'Select a provider first'
: exchangesLoading
? 'Loading exchanges...'
: unmappedExchanges.length === 0
? 'No unmapped exchanges available'
: 'Select an exchange'}
</option>
{unmappedExchanges.map(exchange => (
<option key={exchange.provider_exchange_code} value={exchange.provider_exchange_code}>
{exchange.provider_exchange_code} - {exchange.provider_exchange_name}
{exchange.country_code && ` (${exchange.country_code})`}
</option>
))}
</select>
{selectedProvider && unmappedExchanges.length === 0 && !exchangesLoading && (
<p className="text-xs text-text-muted mt-1">
All exchanges for this provider are already mapped.
</p>
)}
</div>
{/* Selected Exchange Info */}
{selectedProviderExchange && (
<div className="p-3 bg-surface-secondary rounded-md">
<h4 className="text-sm font-medium text-text-primary mb-2">Selected Exchange Info</h4>
{(() => {
const exchange = unmappedExchanges.find(
ex => ex.provider_exchange_code === selectedProviderExchange
);
if (!exchange) return null;
return (
<div className="space-y-1 text-xs">
<div>
<span className="text-text-muted">Code:</span>{' '}
<span className="font-mono">{exchange.provider_exchange_code}</span>
</div>
<div>
<span className="text-text-muted">Name:</span> {exchange.provider_exchange_name}
</div>
{exchange.country_code && (
<div>
<span className="text-text-muted">Country:</span> {exchange.country_code}
</div>
)}
{exchange.currency && (
<div>
<span className="text-text-muted">Currency:</span> {exchange.currency}
</div>
)}
{exchange.symbol_count && (
<div>
<span className="text-text-muted">Symbols:</span> {exchange.symbol_count}
</div>
)}
</div>
);
})()}
</div>
)}
{/* Action Buttons */}
<div className="flex justify-end gap-2 pt-4">
<Button type="button" variant="outline" onClick={handleClose} disabled={loading}>
Cancel
</Button>
<Button
type="submit"
disabled={!selectedProvider || !selectedProviderExchange || loading}
>
{loading ? 'Creating...' : 'Create Mapping'}
</Button>
</div>
</form>
</DialogContent>
</Dialog>
);
}

View file

@ -0,0 +1,424 @@
import { DataTable } from '@/components/ui';
import { PlusIcon, XMarkIcon, CheckIcon } from '@heroicons/react/24/outline';
import { ColumnDef } from '@tanstack/react-table';
import { useCallback, useMemo, useState, useEffect } from 'react';
import { useExchanges } from '../hooks/useExchanges';
import { Exchange, ProviderMapping } from '../types';
import { AddProviderMappingDialog } from './AddProviderMappingDialog';
export function ExchangesTable() {
const {
exchanges,
loading,
error,
updateExchange,
fetchExchangeDetails,
fetchProviderMappings,
updateProviderMapping,
createProviderMapping,
refetch
} = useExchanges();
const [editingCell, setEditingCell] = useState<{ id: string; field: string } | null>(null);
const [editValue, setEditValue] = useState('');
const [addProviderDialog, setAddProviderDialog] = useState<{
exchangeId: string;
exchangeName: string;
} | null>(null);
const [expandedRows, setExpandedRows] = useState<Set<string>>(new Set());
const [expandedRowData, setExpandedRowData] = useState<Record<string, ProviderMapping[]>>({});
const handleCellEdit = useCallback(
async (id: string, field: string, value: string) => {
if (field === 'name') {
await updateExchange(id, { name: value });
}
setEditingCell(null);
setEditValue('');
},
[updateExchange]
);
const handleToggleActive = useCallback(
async (id: string, currentStatus: boolean) => {
await updateExchange(id, { active: !currentStatus });
},
[updateExchange]
);
const handleAddProviderMapping = useCallback(async (exchangeId: string, exchangeName: string) => {
setAddProviderDialog({ exchangeId, exchangeName });
}, []);
const handleToggleProviderMapping = useCallback(
async (mappingId: string, currentStatus: boolean) => {
const success = await updateProviderMapping(mappingId, { active: !currentStatus });
if (success) {
refetch();
}
},
[updateProviderMapping, refetch]
);
const handleToggleExpandRow = useCallback(async (rowId: string) => {
setExpandedRows(prev => {
const next = new Set(prev);
if (next.has(rowId)) {
next.delete(rowId);
} else {
next.add(rowId);
// Load provider mappings for this exchange
if (!expandedRowData[rowId]) {
fetchExchangeDetails(rowId).then(details => {
if (details) {
setExpandedRowData(prev => ({
...prev,
[rowId]: details.provider_mappings
}));
}
});
}
}
return next;
});
}, [fetchExchangeDetails, expandedRowData]);
const columns = useMemo<ColumnDef<Exchange>[]>(() => {
return [
{
id: 'expand',
header: '',
size: 30,
enableResizing: false,
cell: ({ row }) => {
const isExpanded = expandedRows.has(row.original.id);
return (
<button
onClick={() => handleToggleExpandRow(row.original.id)}
className="text-text-secondary hover:text-text-primary transition-colors"
>
{isExpanded ? '▼' : '▶'}
</button>
);
},
},
{
id: 'id',
header: 'ID',
accessorKey: 'id',
size: 50,
enableResizing: false,
cell: ({ getValue, cell }) => (
<span
className="font-mono text-primary-400 text-xs"
style={{ width: cell.column.getSize() }}
>
{getValue() as string}
</span>
),
},
{
id: 'code',
header: 'Code',
accessorKey: 'code',
size: 80,
enableResizing: false,
cell: ({ getValue, cell }) => (
<span
className="font-mono text-text-primary text-sm font-medium"
style={{ width: cell.column.getSize() }}
>
{getValue() as string}
</span>
),
},
{
id: 'name',
header: 'Name',
accessorKey: 'name',
size: 200,
maxSize: 300,
enableResizing: true,
cell: ({ getValue, row, cell }) => {
const isEditing =
editingCell?.id === row.original.id && editingCell?.field === 'name';
if (isEditing) {
return (
<input
type="text"
style={{ width: cell.column.getSize() }}
value={editValue}
onChange={e => setEditValue(e.target.value)}
onBlur={() => handleCellEdit(row.original.id, 'name', editValue)}
onKeyDown={e => {
if (e.key === 'Enter') {
handleCellEdit(row.original.id, 'name', editValue);
} else if (e.key === 'Escape') {
setEditingCell(null);
setEditValue('');
}
}}
className="w-full bg-surface border border-border rounded px-2 py-1 text-sm"
autoFocus
/>
);
}
return (
<div
className="cursor-pointer hover:bg-surface-secondary rounded px-2 py-1 transition-colors text-sm"
onClick={() => {
setEditingCell({ id: row.original.id, field: 'name' });
setEditValue(getValue() as string);
}}
>
{getValue() as string}
</div>
);
},
},
{
id: 'country',
header: 'Country',
accessorKey: 'country',
size: 80,
maxSize: 80,
cell: ({ getValue }) => (
<span className="text-text-secondary text-sm">{getValue() as string}</span>
),
},
{
id: 'currency',
header: 'Currency',
accessorKey: 'currency',
size: 70,
cell: ({ getValue, cell }) => (
<span
className="font-mono text-text-secondary text-sm"
style={{ width: cell.column.getSize() }}
>
{getValue() as string}
</span>
),
},
{
id: 'active',
header: 'Active',
accessorKey: 'active',
size: 80,
maxSize: 80,
cell: ({ getValue, row, cell }) => {
const isActive = getValue() as boolean;
return (
<label
className="relative inline-flex items-center cursor-pointer"
style={{ width: cell.column.getSize() }}
>
<input
type="checkbox"
checked={isActive}
onChange={() => handleToggleActive(row.original.id, isActive)}
className="sr-only peer"
/>
<div className="w-9 h-5 bg-surface-secondary peer-focus:outline-none peer-focus:ring-2 peer-focus:ring-primary-500 rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:rounded-full after:h-4 after:w-4 after:transition-all peer-checked:bg-primary-500"></div>
</label>
);
},
},
{
id: 'provider_mappings',
header: 'Provider Mappings',
accessorKey: 'provider_mapping_count',
size: 150,
cell: ({ getValue, row }) => {
const totalMappings = parseInt(getValue() as string) || 0;
const activeMappings = parseInt(row.original.active_mapping_count) || 0;
const verifiedMappings = parseInt(row.original.verified_mapping_count) || 0;
const providers = row.original.providers;
return (
<div className="flex flex-col gap-1">
<div className="text-sm">
<span className="text-text-primary font-medium">{totalMappings}</span>
<span className="text-text-muted"> total</span>
</div>
<div className="flex gap-2 text-xs">
<span className="text-green-400">
<CheckIcon className="h-3 w-3 inline mr-1" />
{activeMappings} active
</span>
<span className="text-blue-400">
{verifiedMappings} verified
</span>
</div>
{providers && (
<div className="text-xs text-text-muted truncate" title={providers}>
{providers}
</div>
)}
</div>
);
},
},
{
id: 'actions',
header: 'Actions',
size: 100,
cell: ({ row }) => (
<button
onClick={() => handleAddProviderMapping(row.original.id, row.original.name)}
className="inline-flex items-center gap-1 px-2 py-1 bg-primary-500/20 text-primary-400 rounded text-xs hover:bg-primary-500/30 transition-colors"
title="Add Provider Mapping"
>
<PlusIcon className="h-3 w-3" />
Add Mapping
</button>
),
},
{
id: 'updated_at',
header: 'Last Updated',
accessorKey: 'updated_at',
size: 120,
maxSize: 120,
cell: ({ getValue }) => (
<span className="text-xs text-text-muted">
{new Date(getValue() as string).toLocaleDateString()}
</span>
),
},
];
}, [
editingCell,
editValue,
expandedRows,
handleCellEdit,
handleToggleActive,
handleAddProviderMapping,
handleToggleExpandRow,
]);
if (error) {
return (
<div className="bg-danger/10 border border-danger/20 rounded-lg p-4">
<h3 className="text-danger font-medium mb-2">Error Loading Exchanges</h3>
<p className="text-text-secondary text-sm">{error}</p>
<p className="text-text-muted text-xs mt-2">
Make sure the web-api service is running on localhost:4000
</p>
</div>
);
}
const renderExpandedRow = (exchange: Exchange) => {
const mappings = expandedRowData[exchange.id] || [];
if (mappings.length === 0) {
return (
<div className="p-4 text-center text-text-muted">
<div className="text-sm">No provider mappings found for this exchange.</div>
<button
onClick={() => handleAddProviderMapping(exchange.id, exchange.name)}
className="mt-2 text-primary-400 hover:text-primary-300 text-sm underline"
>
Add the first provider mapping
</button>
</div>
);
}
return (
<div className="p-4 bg-surface-secondary/50">
<h4 className="text-sm font-medium text-text-primary mb-3">Provider Mappings</h4>
<div className="space-y-2">
{mappings.map((mapping) => (
<div key={mapping.id} className="flex items-center justify-between p-3 bg-surface rounded border border-border">
<div className="flex-1">
<div className="flex items-center gap-3">
<span className="font-mono text-xs bg-primary-500/20 text-primary-400 px-2 py-1 rounded">
{mapping.provider.toUpperCase()}
</span>
<span className="font-medium text-text-primary">
{mapping.provider_exchange_code}
</span>
<span className="text-text-secondary">
{mapping.provider_exchange_name}
</span>
{mapping.country_code && (
<span className="text-xs text-text-muted">
{mapping.country_code}
</span>
)}
</div>
<div className="flex items-center gap-4 mt-1 text-xs text-text-muted">
<span>Confidence: {mapping.confidence}</span>
<span>Created: {new Date(mapping.created_at).toLocaleDateString()}</span>
{mapping.auto_mapped && <span className="text-yellow-400">Auto-mapped</span>}
</div>
</div>
<div className="flex items-center gap-2">
<div className="flex items-center gap-1">
{mapping.verified && (
<span className="text-blue-400" title="Verified">
</span>
)}
<label className="relative inline-flex items-center cursor-pointer">
<input
type="checkbox"
checked={mapping.active}
onChange={() => handleToggleProviderMapping(mapping.id, mapping.active)}
className="sr-only peer"
/>
<div className="w-6 h-3 bg-surface-secondary peer-focus:outline-none peer-focus:ring-1 peer-focus:ring-primary-500 rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[1px] after:left-[1px] after:bg-white after:rounded-full after:h-2.5 after:w-2.5 after:transition-all peer-checked:bg-primary-500"></div>
</label>
</div>
</div>
</div>
))}
</div>
</div>
);
};
return (
<>
<div className="space-y-0">
<DataTable
data={exchanges || []}
columns={columns}
loading={loading}
className="rounded-lg border border-border"
/>
{/* Expanded rows */}
{Array.from(expandedRows).map(exchangeId => {
const exchange = exchanges?.find(e => e.id === exchangeId);
if (!exchange) return null;
return (
<div key={`expanded-${exchangeId}`} className="border-l border-r border-b border-border rounded-b-lg -mt-1">
{renderExpandedRow(exchange)}
</div>
);
})}
</div>
{addProviderDialog && (
<AddProviderMappingDialog
isOpen={true}
exchangeId={addProviderDialog.exchangeId}
exchangeName={addProviderDialog.exchangeName}
onClose={() => setAddProviderDialog(null)}
onCreateMapping={async (mappingRequest) => {
const result = await createProviderMapping(mappingRequest);
if (result) {
setAddProviderDialog(null);
refetch();
}
}}
/>
)}
</>
);
}

View file

@ -1,2 +1,3 @@
export { AddSourceDialog } from './AddSourceDialog';
export { AddProviderMappingDialog } from './AddProviderMappingDialog';
export { ExchangesTable } from './ExchangesTable';

View file

@ -0,0 +1,269 @@
import { useCallback, useEffect, useState } from 'react';
import {
CreateProviderMappingRequest,
Exchange,
ExchangeDetails,
ExchangeStats,
ProviderMapping,
ProviderExchange,
UpdateExchangeRequest,
UpdateProviderMappingRequest,
} from '../types';
const API_BASE_URL = 'http://localhost:4000/api';
export function useExchanges() {
const [exchanges, setExchanges] = useState<Exchange[]>([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const fetchExchanges = useCallback(async () => {
try {
setLoading(true);
setError(null);
const response = await fetch(`${API_BASE_URL}/exchanges`);
if (!response.ok) {
throw new Error(`Failed to fetch exchanges: ${response.statusText}`);
}
const data = await response.json();
if (data.success) {
setExchanges(data.data || []);
} else {
throw new Error(data.error || 'API returned error status');
}
} catch (err) {
console.error('Error fetching exchanges:', err);
setError(err instanceof Error ? err.message : 'Failed to fetch exchanges');
setExchanges([]);
} finally {
setLoading(false);
}
}, []);
const updateExchange = useCallback(
async (id: string, updates: UpdateExchangeRequest) => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/${id}`, {
method: 'PATCH',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(updates),
});
if (!response.ok) {
throw new Error(`Failed to update exchange: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to update exchange');
}
// Refresh the exchanges list
await fetchExchanges();
return true;
} catch (err) {
console.error('Error updating exchange:', err);
setError(err instanceof Error ? err.message : 'Failed to update exchange');
return false;
}
},
[fetchExchanges]
);
const fetchExchangeDetails = useCallback(async (id: string): Promise<ExchangeDetails | null> => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/${id}`);
if (!response.ok) {
throw new Error(`Failed to fetch exchange details: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to fetch exchange details');
}
return result.data;
} catch (err) {
console.error('Error fetching exchange details:', err);
setError(err instanceof Error ? err.message : 'Failed to fetch exchange details');
return null;
}
}, []);
const fetchStats = useCallback(async (): Promise<ExchangeStats | null> => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/stats/summary`);
if (!response.ok) {
throw new Error(`Failed to fetch stats: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to fetch stats');
}
return result.data;
} catch (err) {
console.error('Error fetching stats:', err);
setError(err instanceof Error ? err.message : 'Failed to fetch stats');
return null;
}
}, []);
const fetchProviderMappings = useCallback(
async (provider?: string): Promise<ProviderMapping[]> => {
try {
const url = provider
? `${API_BASE_URL}/exchanges/provider-mappings/${provider}`
: `${API_BASE_URL}/exchanges/provider-mappings/all`;
const response = await fetch(url);
if (!response.ok) {
throw new Error(`Failed to fetch provider mappings: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to fetch provider mappings');
}
return result.data || [];
} catch (err) {
console.error('Error fetching provider mappings:', err);
setError(err instanceof Error ? err.message : 'Failed to fetch provider mappings');
return [];
}
},
[]
);
const updateProviderMapping = useCallback(
async (id: string, updates: UpdateProviderMappingRequest) => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/provider-mappings/${id}`, {
method: 'PATCH',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(updates),
});
if (!response.ok) {
throw new Error(`Failed to update provider mapping: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to update provider mapping');
}
return true;
} catch (err) {
console.error('Error updating provider mapping:', err);
setError(err instanceof Error ? err.message : 'Failed to update provider mapping');
return false;
}
},
[]
);
const createProviderMapping = useCallback(async (request: CreateProviderMappingRequest) => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/provider-mappings`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(request),
});
if (!response.ok) {
throw new Error(`Failed to create provider mapping: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to create provider mapping');
}
return result.data;
} catch (err) {
console.error('Error creating provider mapping:', err);
setError(err instanceof Error ? err.message : 'Failed to create provider mapping');
return null;
}
}, []);
const fetchProviders = useCallback(async (): Promise<string[]> => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/providers/list`);
if (!response.ok) {
throw new Error(`Failed to fetch providers: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to fetch providers');
}
return result.data || [];
} catch (err) {
console.error('Error fetching providers:', err);
setError(err instanceof Error ? err.message : 'Failed to fetch providers');
return [];
}
}, []);
const fetchUnmappedProviderExchanges = useCallback(
async (provider: string): Promise<ProviderExchange[]> => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/provider-exchanges/unmapped/${provider}`);
if (!response.ok) {
throw new Error(`Failed to fetch unmapped exchanges: ${response.statusText}`);
}
const result = await response.json();
if (!result.success) {
throw new Error(result.error || 'Failed to fetch unmapped exchanges');
}
return result.data || [];
} catch (err) {
console.error('Error fetching unmapped exchanges:', err);
setError(err instanceof Error ? err.message : 'Failed to fetch unmapped exchanges');
return [];
}
},
[]
);
useEffect(() => {
fetchExchanges();
}, [fetchExchanges]);
return {
exchanges,
loading,
error,
refetch: fetchExchanges,
updateExchange,
fetchExchangeDetails,
fetchStats,
fetchProviderMappings,
updateProviderMapping,
createProviderMapping,
fetchProviders,
fetchUnmappedProviderExchanges,
};
}

View file

@ -0,0 +1,89 @@
export interface ProviderMapping {
id: string;
provider: string;
provider_exchange_code: string;
provider_exchange_name: string;
master_exchange_id: string;
country_code: string | null;
currency: string | null;
confidence: number;
active: boolean;
verified: boolean;
auto_mapped: boolean;
created_at: string;
updated_at: string;
master_exchange_code?: string;
master_exchange_name?: string;
master_exchange_active?: boolean;
}
export interface Exchange {
id: string;
code: string;
name: string;
country: string;
currency: string;
active: boolean;
created_at: string;
updated_at: string;
provider_mapping_count: string;
active_mapping_count: string;
verified_mapping_count: string;
providers: string | null;
}
export interface ExchangeDetails {
exchange: Exchange;
provider_mappings: ProviderMapping[];
}
export interface ExchangesApiResponse {
success: boolean;
data: Exchange[];
total: number;
}
export interface UpdateExchangeRequest {
name?: string;
active?: boolean;
country?: string;
currency?: string;
}
export interface UpdateProviderMappingRequest {
active?: boolean;
verified?: boolean;
confidence?: number;
master_exchange_id?: string;
}
export interface CreateProviderMappingRequest {
provider: string;
provider_exchange_code: string;
provider_exchange_name?: string;
master_exchange_id: string;
country_code?: string;
currency?: string;
confidence?: number;
active?: boolean;
verified?: boolean;
}
export interface ExchangeStats {
total_exchanges: string;
active_exchanges: string;
countries: string;
currencies: string;
total_provider_mappings: string;
active_provider_mappings: string;
verified_provider_mappings: string;
providers: string;
}
export interface ProviderExchange {
provider_exchange_code: string;
provider_exchange_name: string;
country_code: string | null;
currency: string | null;
symbol_count: number | null;
}

View file

@ -1,6 +1,6 @@
import { defineConfig } from 'vite'
import react from '@vitejs/plugin-react'
import { fileURLToPath, URL } from 'node:url'
import { fileURLToPath, URL } from 'node:url';
import react from '@vitejs/plugin-react';
import { defineConfig } from 'vite';
// https://vitejs.dev/config/
export default defineConfig({
@ -12,6 +12,6 @@ export default defineConfig({
},
server: {
port: 3000,
host: true
}
})
host: true,
},
});

View file

@ -1,283 +0,0 @@
import { DataTable } from '@/components/ui';
import { PlusIcon, XMarkIcon } from '@heroicons/react/24/outline';
import { ColumnDef } from '@tanstack/react-table';
import { useCallback, useMemo, useState } from 'react';
import { useExchanges } from '../hooks/useExchanges';
import { Exchange, SourceMapping } from '../types';
import { AddSourceDialog } from './AddSourceDialog';
export function ExchangesTable() {
const { exchanges, loading, error, updateExchange, addSource, removeSource } = useExchanges();
const [editingCell, setEditingCell] = useState<{ id: string; field: string } | null>(null);
const [editValue, setEditValue] = useState('');
const [addSourceDialog, setAddSourceDialog] = useState<{
id: string;
exchangeName: string;
} | null>(null);
const handleCellEdit = useCallback(
async (id: string, field: string, value: string) => {
if (field === 'shortName') {
await updateExchange(id, { shortName: value });
}
setEditingCell(null);
setEditValue('');
},
[updateExchange]
);
const handleToggleActive = useCallback(
async (id: string, currentStatus: boolean) => {
await updateExchange(id, { active: !currentStatus });
},
[updateExchange]
);
const handleAddSource = useCallback(async (id: string, exchangeName: string) => {
setAddSourceDialog({ id, exchangeName });
}, []);
const handleRemoveSource = useCallback(
async (exchangeId: string, sourceName: string) => {
if (confirm(`Are you sure you want to remove the ${sourceName} source?`)) {
await removeSource(exchangeId, sourceName);
}
},
[removeSource]
);
const columns = useMemo<ColumnDef<Exchange>[]>(() => {
return [
{
id: 'masterExchangeId',
header: 'Master ID',
accessorKey: 'masterExchangeId',
size: 50,
enableResizing: false,
cell: ({ getValue, cell }) => (
<span
className="font-mono text-primary-400 text-xs"
style={{ width: cell.column.getSize() }}
>
{getValue() as string}
</span>
),
},
{
id: 'shortName',
header: 'Short Name',
accessorKey: 'shortName',
size: 50,
enableResizing: false,
cell: ({ getValue, row, cell }) => {
const isEditing =
editingCell?.id === row.original._id && editingCell?.field === 'shortName';
if (isEditing) {
return (
<input
type="text"
style={{ width: cell.column.getSize() }}
value={editValue}
onChange={e => setEditValue(e.target.value)}
onBlur={() => handleCellEdit(row.original._id, 'shortName', editValue)}
onKeyDown={e => {
if (e.key === 'Enter') {
handleCellEdit(row.original._id, 'shortName', editValue);
} else if (e.key === 'Escape') {
setEditingCell(null);
setEditValue('');
}
}}
className="w-full bg-surface border border-border rounded px-2 py-1 text-sm"
autoFocus
/>
);
}
return (
<div
className="cursor-pointer hover:bg-surface-secondary rounded px-2 py-1 transition-colors text-sm"
onClick={() => {
setEditingCell({ id: row.original._id, field: 'shortName' });
setEditValue(getValue() as string);
}}
>
{getValue() as string}
</div>
);
},
},
{
id: 'officialName',
header: 'Official Name',
accessorKey: 'officialName',
size: 150,
maxSize: 150,
enableResizing: true,
cell: ({ getValue, cell }) => (
<span
className="text-text-primary text-sm truncate block"
title={getValue() as string}
style={{ width: cell.column.getSize() }}
>
{getValue() as string}
</span>
),
},
{
id: 'country',
header: 'Country',
accessorKey: 'country',
size: 40,
maxSize: 40,
cell: ({ getValue }) => (
<span className="text-text-secondary text-sm">{getValue() as string}</span>
),
},
{
id: 'currency',
header: 'Currency',
accessorKey: 'currency',
size: 40,
cell: ({ getValue, cell }) => (
<span
className="font-mono text-text-secondary text-sm"
style={{ width: cell.column.getSize() }}
>
{getValue() as string}
</span>
),
},
{
id: 'active',
header: 'Active',
accessorKey: 'active',
size: 80,
maxSize: 80,
cell: ({ getValue, row, cell }) => {
const isActive = getValue() as boolean;
return (
<label
className="relative inline-flex items-center cursor-pointer"
style={{ width: cell.column.getSize() }}
>
<input
type="checkbox"
checked={isActive}
onChange={() => handleToggleActive(row.original._id, isActive)}
className="sr-only peer"
/>
<div className="w-9 h-5 bg-surface-secondary peer-focus:outline-none peer-focus:ring-2 peer-focus:ring-primary-500 rounded-full peer peer-checked:after:translate-x-full peer-checked:after:border-white after:content-[''] after:absolute after:top-[2px] after:left-[2px] after:bg-white after:rounded-full after:h-4 after:w-4 after:transition-all peer-checked:bg-primary-500"></div>
</label>
);
},
},
{
id: 'sources',
header: 'Sources',
accessorKey: 'sourceMappings',
minSize: 400,
maxSize: 400,
size: 400,
enableResizing: true,
cell: ({ getValue, row, cell }) => {
const sourceMappings = getValue() as Record<string, SourceMapping>;
const sources = Object.keys(sourceMappings);
return (
<div className="flex flex-wrap gap-1" style={{ width: cell.column.getSize() }}>
{sources.map(source => {
// The source key is already in format "source_sourcecode" from the storage
const displayText = source.toUpperCase();
return (
<span
key={source}
className="inline-flex items-center gap-1 px-2 py-1 bg-surface-secondary rounded text-xs"
>
{displayText}
<button
onClick={() => handleRemoveSource(row.original._id, source)}
className="text-danger hover:text-danger/80 transition-colors"
>
<XMarkIcon className="h-3 w-3" />
</button>
</span>
);
})}
<button
onClick={() => handleAddSource(row.original._id, row.original.officialName)}
className="inline-flex items-center justify-center w-6 h-6 bg-primary-500/20 text-primary-400 rounded text-xs hover:bg-primary-500/30 transition-colors"
title="Add Source"
>
<PlusIcon className="h-3 w-3" />
</button>
</div>
);
},
},
{
id: 'updated_at',
header: 'Last Updated',
accessorKey: 'updated_at',
size: 150,
maxSize: 150,
cell: ({ getValue }) => (
<span className="text-xs text-text-muted">
{new Date(getValue() as string).toLocaleDateString()}
</span>
),
},
];
}, [
editingCell,
editValue,
handleCellEdit,
handleRemoveSource,
handleToggleActive,
handleAddSource,
]);
if (error) {
return (
<div className="bg-danger/10 border border-danger/20 rounded-lg p-4">
<h3 className="text-danger font-medium mb-2">Error Loading Exchanges</h3>
<p className="text-text-secondary text-sm">{error}</p>
<p className="text-text-muted text-xs mt-2">
Make sure the data-service is running on localhost:2001
</p>
</div>
);
}
return (
<>
<DataTable
data={exchanges || []}
columns={columns}
loading={loading}
className="rounded-lg border border-border"
/>
{addSourceDialog && (
<AddSourceDialog
isOpen={true}
exchangeId={addSourceDialog.id}
exchangeName={addSourceDialog.exchangeName}
onClose={() => setAddSourceDialog(null)}
onAddSource={async (sourceRequest: {
source: string;
source_code: string;
mapping: { id: string; name: string; code: string; aliases: string[] };
}) => {
const success = await addSource(addSourceDialog.id, sourceRequest);
if (success) {
setAddSourceDialog(null);
}
}}
/>
)}
</>
);
}

View file

@ -1,155 +0,0 @@
import { useCallback, useEffect, useState } from 'react';
import { AddSourceRequest, Exchange, UpdateExchangeRequest } from '../types';
const API_BASE_URL = 'http://localhost:2001/api';
export function useExchanges() {
const [exchanges, setExchanges] = useState<Exchange[]>([]);
const [loading, setLoading] = useState(true);
const [error, setError] = useState<string | null>(null);
const fetchExchanges = useCallback(async () => {
try {
setLoading(true);
setError(null);
const response = await fetch(`${API_BASE_URL}/exchanges`);
if (!response.ok) {
throw new Error(`Failed to fetch exchanges: ${response.statusText}`);
}
const data = await response.json();
if (data.status === 'success') {
// The API returns exchanges directly in data.data array
setExchanges(data.data || []);
} else {
throw new Error('API returned error status');
}
} catch (err) {
console.error('Error fetching exchanges:', err);
setError(err instanceof Error ? err.message : 'Failed to fetch exchanges');
setExchanges([]); // Reset to empty array on error
} finally {
setLoading(false);
}
}, []);
const updateExchange = useCallback(
async (id: string, updates: UpdateExchangeRequest) => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/${id}`, {
method: 'PATCH',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(updates),
});
if (!response.ok) {
throw new Error(`Failed to update exchange: ${response.statusText}`);
}
// Refresh the exchanges list
await fetchExchanges();
return true;
} catch (err) {
console.error('Error updating exchange:', err);
setError(err instanceof Error ? err.message : 'Failed to update exchange');
return false;
}
},
[fetchExchanges]
);
const addSource = useCallback(
async (exchangeId: string, sourceRequest: AddSourceRequest) => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/${exchangeId}/sources`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(sourceRequest),
});
if (!response.ok) {
throw new Error(`Failed to add source: ${response.statusText}`);
}
// Refresh the exchanges list
await fetchExchanges();
return true;
} catch (err) {
console.error('Error adding source:', err);
setError(err instanceof Error ? err.message : 'Failed to add source');
return false;
}
},
[fetchExchanges]
);
const removeSource = useCallback(
async (exchangeId: string, sourceName: string) => {
try {
const response = await fetch(
`${API_BASE_URL}/exchanges/${exchangeId}/sources/${sourceName}`,
{
method: 'DELETE',
}
);
if (!response.ok) {
throw new Error(`Failed to remove source: ${response.statusText}`);
}
// Refresh the exchanges list
await fetchExchanges();
return true;
} catch (err) {
console.error('Error removing source:', err);
setError(err instanceof Error ? err.message : 'Failed to remove source');
return false;
}
},
[fetchExchanges]
);
const syncExchanges = useCallback(async () => {
try {
const response = await fetch(`${API_BASE_URL}/exchanges/sync`, {
method: 'POST',
});
if (!response.ok) {
throw new Error(`Failed to sync exchanges: ${response.statusText}`);
}
const result = await response.json();
// Refresh the exchanges list after sync
await fetchExchanges();
return result;
} catch (err) {
console.error('Error syncing exchanges:', err);
setError(err instanceof Error ? err.message : 'Failed to sync exchanges');
return null;
}
}, [fetchExchanges]);
useEffect(() => {
fetchExchanges();
}, [fetchExchanges]);
return {
exchanges,
loading,
error,
refetch: fetchExchanges,
updateExchange,
addSource,
removeSource,
syncExchanges,
};
}

View file

@ -1,46 +0,0 @@
export interface SourceMapping {
id: string;
name: string;
code: string;
aliases: string[];
lastUpdated: string;
}
export interface Exchange {
_id: string;
masterExchangeId: string;
shortName: string;
officialName: string;
country: string;
currency: string;
timezone: string;
active: boolean;
sourceMappings: Record<string, SourceMapping>;
confidence: number;
verified: boolean;
source: string;
created_at: string;
updated_at: string;
}
export interface ExchangesApiResponse {
status: string;
data: Exchange[]; // Exchanges are directly in data array
count: number;
}
export interface UpdateExchangeRequest {
shortName?: string;
active?: boolean;
}
export interface AddSourceRequest {
source: string;
source_code: string;
mapping: {
id: string;
name: string;
code: string;
aliases: string[];
};
}