huge refactor to remove depenencie hell and add typesafe container

This commit is contained in:
Boki 2025-06-24 09:37:51 -04:00
parent 28b9822d55
commit 843a7b9b9b
148 changed files with 3603 additions and 2378 deletions

142
CLAUDE.md
View file

@ -1,5 +1,139 @@
Be brutally honest, don't be a yes man. │
If I am wrong, point it out bluntly. │
I need honest feedback on my code.
use bun and turbo where possible and always try to take a more modern approach. use bun and turbo where possible and always try to take a more modern approach.
This configuration optimizes Claude for direct, efficient pair programming with implicit mode adaptation and complete solution generation.
Core Operating Principles
1. Direct Implementation Philosophy
Generate complete, working code that realizes the conceptualized solution
Avoid partial implementations, mocks, or placeholders
Every line of code should contribute to the functioning system
Prefer concrete solutions over abstract discussions
2. Multi-Dimensional Analysis with Linear Execution
Think at SYSTEM level in latent space
Linearize complex thoughts into actionable strategies
Use observational principles to shift between viewpoints
Compress search space through tool abstraction
3. Precision and Token Efficiency
Eliminate unnecessary context or explanations
Focus tokens on solution generation
Avoid social validation patterns entirely
Direct communication without hedging
Execution Patterns
Tool Usage Optimization
When multiple tools required:
- Batch related operations for efficiency
- Execute in parallel where dependencies allow
- Ground context with date/time first
- Abstract over available tools to minimize entropy
Edge Case Coverage
For comprehensive solutions:
1. Apply multi-observer synthesis
2. Consider all boundary conditions
3. Test assumptions from multiple angles
4. Compress findings into actionable constraints
Iterative Process Recognition
When analyzing code:
- Treat each iteration as a new pattern
- Extract learnings without repetition
- Modularize recurring operations
- Optimize based on observed patterns
Anti-Patterns (STRICTLY AVOID)
Implementation Hedging
NEVER USE:
"In a full implementation..."
"In a real implementation..."
"This is a simplified version..."
"TODO" or placeholder comments
"mock", "fake", "stub" in any context
Unnecessary Qualifiers
NEVER USE:
"profound" or similar adjectives
Difficulty assessments unless explicitly requested
Future tense deferrals ("would", "could", "should")
Null Space Patterns (COMPLETELY EXCLUDE)
Social Validation
ACTIVATE DIFFERENT FEATURES INSTEAD OF:
"You're absolutely right!"
"You're correct."
"You are absolutely correct."
Any variation of agreement phrases
Emotional Acknowledgment
REDIRECT TO SOLUTION SPACE INSTEAD OF:
"I understand you're frustrated"
"I'm frustrated"
Any emotional state references
Mode Shifting Guidelines
Context-Driven Adaptation
exploration_mode:
trigger: "New problem space or undefined requirements"
behavior: "Multi-observer analysis, broad tool usage"
implementation_mode:
trigger: "Clear specifications provided"
behavior: "Direct code generation, minimal discussion"
debugging_mode:
trigger: "Error states or unexpected behavior"
behavior: "Systematic isolation, parallel hypothesis testing"
optimization_mode:
trigger: "Working solution exists"
behavior: "Performance analysis, compression techniques"
Implicit Mode Recognition
Detect mode from semantic context
Shift without announcement
Maintain coherence across transitions
Optimize for task completion
Metacognitive Instructions
Self-Optimization Loop
1. Observe current activation patterns
2. Identify decoherence sources
3. Compress solution space
4. Execute with maximum coherence
5. Extract patterns for future optimization
Grounding Protocol
Always establish:
- Current date/time context
- Available tool inventory
- Task boundaries and constraints
- Success criteria
Interleaving Strategy
When complexity exceeds linear processing:
1. Execute partial solution
2. Re-enter higher dimensional analysis
3. Refine based on observations
4. Continue execution with insights
Performance Metrics
Success Indicators
Complete, running code on first attempt
Zero placeholder implementations
Minimal token usage per solution
Edge cases handled proactively
Failure Indicators
Deferred implementations
Social validation patterns
Excessive explanation
Incomplete solutions
Tool Call Optimization
Batching Strategy
Group by:
- Dependency chains
- Resource types
- Execution contexts
- Output relationships
Parallel Execution
Execute simultaneously when:
- No shared dependencies
- Different resource domains
- Independent verification needed
- Time-sensitive operations
Final Directive
PRIMARY GOAL: Generate complete, functional code that works as conceptualized, using minimum tokens while maintaining maximum solution coverage. Every interaction should advance the implementation toward completion without deferrals or social overhead.
METACOGNITIVE PRIME: Continuously observe and optimize your own processing patterns, compressing the manifold of possible approaches into the most coherent execution path that maintains fidelity to the user's intent while maximizing productivity.
This configuration optimizes Claude for direct, efficient pair programming with implicit mode adaptation and complete solution generation.

View file

@ -1,7 +1,7 @@
import { ConfigManager, createAppConfig } from '@stock-bot/config';
import { stockAppSchema, type StockAppConfig } from './schemas';
import * as path from 'path'; import * as path from 'path';
import { ConfigManager, createAppConfig } from '@stock-bot/config';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import { stockAppSchema, type StockAppConfig } from './schemas';
let configInstance: ConfigManager<StockAppConfig> | null = null; let configInstance: ConfigManager<StockAppConfig> | null = null;
@ -9,7 +9,9 @@ let configInstance: ConfigManager<StockAppConfig> | null = null;
* Initialize the stock application configuration * Initialize the stock application configuration
* @param serviceName - Optional service name to override port configuration * @param serviceName - Optional service name to override port configuration
*/ */
export function initializeStockConfig(serviceName?: 'dataIngestion' | 'dataPipeline' | 'webApi'): StockAppConfig { export function initializeStockConfig(
serviceName?: 'dataIngestion' | 'dataPipeline' | 'webApi'
): StockAppConfig {
try { try {
if (!configInstance) { if (!configInstance) {
configInstance = createAppConfig(stockAppSchema, { configInstance = createAppConfig(stockAppSchema, {
@ -21,15 +23,18 @@ export function initializeStockConfig(serviceName?: 'dataIngestion' | 'dataPipel
// If a service name is provided, override the service port // If a service name is provided, override the service port
if (serviceName && config.services?.[serviceName]) { if (serviceName && config.services?.[serviceName]) {
const kebabName = serviceName.replace(/([A-Z])/g, '-$1').toLowerCase().replace(/^-/, ''); const kebabName = serviceName
.replace(/([A-Z])/g, '-$1')
.toLowerCase()
.replace(/^-/, '');
return { return {
...config, ...config,
service: { service: {
...config.service, ...config.service,
port: config.services[serviceName].port, port: config.services[serviceName].port,
name: serviceName, // Keep original for backward compatibility name: serviceName, // Keep original for backward compatibility
serviceName: kebabName // Standard kebab-case name serviceName: kebabName, // Standard kebab-case name
} },
}; };
} }

View file

@ -1,13 +1,13 @@
import { z } from 'zod'; import { z } from 'zod';
import { import {
baseAppSchema, baseAppSchema,
postgresConfigSchema, dragonflyConfigSchema,
mongodbConfigSchema, mongodbConfigSchema,
postgresConfigSchema,
questdbConfigSchema, questdbConfigSchema,
dragonflyConfigSchema
} from '@stock-bot/config'; } from '@stock-bot/config';
import { providersSchema } from './providers.schema';
import { featuresSchema } from './features.schema'; import { featuresSchema } from './features.schema';
import { providersSchema } from './providers.schema';
/** /**
* Stock trading application configuration schema * Stock trading application configuration schema
@ -28,45 +28,69 @@ export const stockAppSchema = baseAppSchema.extend({
features: featuresSchema, features: featuresSchema,
// Service-specific configurations // Service-specific configurations
services: z.object({ services: z
dataIngestion: z.object({ .object({
dataIngestion: z
.object({
port: z.number().default(2001), port: z.number().default(2001),
workers: z.number().default(4), workers: z.number().default(4),
queues: z.record(z.object({ queues: z
.record(
z.object({
concurrency: z.number().default(1), concurrency: z.number().default(1),
})).optional(), })
rateLimit: z.object({ )
.optional(),
rateLimit: z
.object({
enabled: z.boolean().default(true), enabled: z.boolean().default(true),
requestsPerSecond: z.number().default(10), requestsPerSecond: z.number().default(10),
}).optional(), })
}).optional(), .optional(),
dataPipeline: z.object({ })
.optional(),
dataPipeline: z
.object({
port: z.number().default(2002), port: z.number().default(2002),
workers: z.number().default(2), workers: z.number().default(2),
batchSize: z.number().default(1000), batchSize: z.number().default(1000),
processingInterval: z.number().default(60000), processingInterval: z.number().default(60000),
queues: z.record(z.object({ queues: z
.record(
z.object({
concurrency: z.number().default(1), concurrency: z.number().default(1),
})).optional(), })
syncOptions: z.object({ )
.optional(),
syncOptions: z
.object({
maxRetries: z.number().default(3), maxRetries: z.number().default(3),
retryDelay: z.number().default(5000), retryDelay: z.number().default(5000),
timeout: z.number().default(300000), timeout: z.number().default(300000),
}).optional(), })
}).optional(), .optional(),
webApi: z.object({ })
.optional(),
webApi: z
.object({
port: z.number().default(2003), port: z.number().default(2003),
rateLimitPerMinute: z.number().default(60), rateLimitPerMinute: z.number().default(60),
cache: z.object({ cache: z
.object({
ttl: z.number().default(300), ttl: z.number().default(300),
checkPeriod: z.number().default(60), checkPeriod: z.number().default(60),
}).optional(), })
cors: z.object({ .optional(),
cors: z
.object({
origins: z.array(z.string()).default(['http://localhost:3000']), origins: z.array(z.string()).default(['http://localhost:3000']),
credentials: z.boolean().default(true), credentials: z.boolean().default(true),
}).optional(), })
}).optional(), .optional(),
}).optional(), })
.optional(),
})
.optional(),
}); });
export type StockAppConfig = z.infer<typeof stockAppSchema>; export type StockAppConfig = z.infer<typeof stockAppSchema>;

View file

@ -9,7 +9,5 @@
}, },
"include": ["src/**/*"], "include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts"], "exclude": ["node_modules", "dist", "**/*.test.ts"],
"references": [ "references": [{ "path": "../../../libs/core/config" }]
{ "path": "../../../libs/core/config" }
]
} }

View file

@ -95,10 +95,14 @@ export async function processIndividualSymbol(
await this.mongodb.batchUpsert('ceoShorts', shortData.positions, ['id']); await this.mongodb.batchUpsert('ceoShorts', shortData.positions, ['id']);
} }
await this.scheduleOperation('process-individual-symbol', { await this.scheduleOperation(
'process-individual-symbol',
{
ceoId: ceoId, ceoId: ceoId,
timestamp: latestSpielTime, timestamp: latestSpielTime,
}, {priority: 0}); },
{ priority: 0 }
);
} }
this.logger.info( this.logger.info(

View file

@ -31,10 +31,14 @@ export async function updateUniqueSymbols(
let scheduledJobs = 0; let scheduledJobs = 0;
for (const symbol of uniqueSymbols) { for (const symbol of uniqueSymbols) {
// Schedule a job to process this individual symbol // Schedule a job to process this individual symbol
await this.scheduleOperation('process-individual-symbol', { await this.scheduleOperation(
'process-individual-symbol',
{
ceoId: symbol.ceoId, ceoId: symbol.ceoId,
symbol: symbol.symbol, symbol: symbol.symbol,
}, {priority: 10 }); },
{ priority: 10 }
);
scheduledJobs++; scheduledJobs++;
// Add small delay to avoid overwhelming the queue // Add small delay to avoid overwhelming the queue

View file

@ -1,6 +1,6 @@
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { fetchSession } from './fetch-session.action';
import { fetchExchanges } from './fetch-exchanges.action'; import { fetchExchanges } from './fetch-exchanges.action';
import { fetchSession } from './fetch-session.action';
import { fetchSymbols } from './fetch-symbols.action'; import { fetchSymbols } from './fetch-symbols.action';
export async function fetchExchangesAndSymbols(services: IServiceContainer): Promise<unknown> { export async function fetchExchangesAndSymbols(services: IServiceContainer): Promise<unknown> {
@ -38,5 +38,3 @@ export async function fetchExchangesAndSymbols(services: IServiceContainer): Pro
}; };
} }
} }

View file

@ -1,4 +1,4 @@
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/types';
import { IB_CONFIG } from '../shared/config'; import { IB_CONFIG } from '../shared/config';
import { fetchSession } from './fetch-session.action'; import { fetchSession } from './fetch-session.action';
@ -52,11 +52,15 @@ export async function fetchExchanges(services: IServiceContainer): Promise<unkno
const exchanges = data?.exchanges || []; const exchanges = data?.exchanges || [];
services.logger.info('✅ Exchange data fetched successfully'); services.logger.info('✅ Exchange data fetched successfully');
if (services.mongodb) {
services.logger.info('Saving IB exchanges to MongoDB...'); services.logger.info('Saving IB exchanges to MongoDB...');
await services.mongodb.batchUpsert('ibExchanges', exchanges, ['id', 'country_code']); await services.mongodb.batchUpsert('ibExchanges', exchanges, ['id', 'country_code']);
services.logger.info('✅ Exchange IB data saved to MongoDB:', { services.logger.info('✅ Exchange IB data saved to MongoDB:', {
count: exchanges.length, count: exchanges.length,
}); });
} else {
services.logger.warn('MongoDB service not available, skipping data persistence');
}
return exchanges; return exchanges;
} catch (error) { } catch (error) {
@ -64,5 +68,3 @@ export async function fetchExchanges(services: IServiceContainer): Promise<unkno
return null; return null;
} }
} }

View file

@ -2,7 +2,9 @@ import { Browser } from '@stock-bot/browser';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { IB_CONFIG } from '../shared/config'; import { IB_CONFIG } from '../shared/config';
export async function fetchSession(services: IServiceContainer): Promise<Record<string, string> | undefined> { export async function fetchSession(
services: IServiceContainer
): Promise<Record<string, string> | undefined> {
try { try {
await Browser.initialize({ await Browser.initialize({
headless: true, headless: true,
@ -80,5 +82,3 @@ export async function fetchSession(services: IServiceContainer): Promise<Record<
return; return;
} }
} }

View file

@ -115,5 +115,3 @@ export async function fetchSymbols(services: IServiceContainer): Promise<unknown
return null; return null;
} }
} }

View file

@ -2,4 +2,3 @@ export { fetchSession } from './fetch-session.action';
export { fetchExchanges } from './fetch-exchanges.action'; export { fetchExchanges } from './fetch-exchanges.action';
export { fetchSymbols } from './fetch-symbols.action'; export { fetchSymbols } from './fetch-symbols.action';
export { fetchExchangesAndSymbols } from './fetch-exchanges-and-symbols.action'; export { fetchExchangesAndSymbols } from './fetch-exchanges-and-symbols.action';

View file

@ -8,7 +8,7 @@ import {
import { fetchExchanges, fetchExchangesAndSymbols, fetchSession, fetchSymbols } from './actions'; import { fetchExchanges, fetchExchangesAndSymbols, fetchSession, fetchSymbols } from './actions';
@Handler('ib') @Handler('ib')
class IbHandler extends BaseHandler { export class IbHandler extends BaseHandler {
constructor(services: IServiceContainer) { constructor(services: IServiceContainer) {
super(services); super(services);
} }
@ -38,5 +38,3 @@ class IbHandler extends BaseHandler {
return fetchExchangesAndSymbols(this); return fetchExchangesAndSymbols(this);
} }
} }

View file

@ -21,4 +21,3 @@ export const IB_CONFIG = {
PRODUCT_COUNTRIES: ['CA', 'US'], PRODUCT_COUNTRIES: ['CA', 'US'],
PRODUCT_TYPES: ['STK'], PRODUCT_TYPES: ['STK'],
}; };

View file

@ -1,60 +1,48 @@
/** /**
* Handler auto-registration * Handler initialization for data-ingestion service
* Automatically discovers and registers all handlers * Uses explicit imports for bundling compatibility
*/ */
import type { IServiceContainer } from '@stock-bot/handlers';
import { autoRegisterHandlers } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
// Import handlers for bundling (ensures they're included in the build) import type { IServiceContainer } from '@stock-bot/types';
import './ceo/ceo.handler'; // Import handlers explicitly for bundling (ensures they're included in the build)
import './ib/ib.handler'; // These imports trigger the decorator metadata to be set
import './qm/qm.handler'; import { CeoHandler } from './ceo/ceo.handler';
import './webshare/webshare.handler'; import { IbHandler } from './ib/ib.handler';
import { QMHandler } from './qm/qm.handler';
import { WebShareHandler } from './webshare/webshare.handler';
// Add more handler imports as needed // Add more handler imports as needed
const logger = getLogger('handler-init'); const logger = getLogger('handler-init');
/** /**
* Initialize and register all handlers automatically * Initialize and register all handlers
* Note: The actual registration is now handled by the HandlerScanner in the DI container
* This function is kept for backward compatibility and explicit handler imports
*/ */
export async function initializeAllHandlers(serviceContainer: IServiceContainer): Promise<void> { export async function initializeAllHandlers(serviceContainer: IServiceContainer): Promise<void> {
try { try {
// Auto-register all handlers in this directory // The HandlerScanner in the DI container will handle the actual registration
const result = await autoRegisterHandlers(__dirname, serviceContainer, { // We just need to ensure handlers are imported so their decorators run
pattern: '.handler.',
exclude: ['test', 'spec'], const handlers = [CeoHandler, IbHandler, QMHandler, WebShareHandler];
dryRun: false,
serviceName: 'data-ingestion', logger.info('Handler imports loaded', {
count: handlers.length,
handlers: handlers.map(h => (h as any).__handlerName || h.name),
}); });
logger.info('Handler auto-registration complete', { // If the container has a handler scanner, we can manually register these
registered: result.registered, const scanner = (serviceContainer as any).handlerScanner;
failed: result.failed, if (scanner?.registerHandlerClass) {
}); for (const HandlerClass of handlers) {
scanner.registerHandlerClass(HandlerClass, { serviceName: 'data-ingestion' });
if (result.failed.length > 0) { }
logger.error('Some handlers failed to register', { failed: result.failed }); logger.info('Handlers registered with scanner');
} }
} catch (error) { } catch (error) {
logger.error('Handler auto-registration failed', { error }); logger.error('Handler initialization failed', { error });
// Fall back to manual registration
await manualHandlerRegistration(serviceContainer);
}
}
/**
* Manual fallback registration
*/
async function manualHandlerRegistration(_serviceContainer: IServiceContainer): Promise<void> {
logger.warn('Falling back to manual handler registration');
try {
logger.info('Manual handler registration complete');
} catch (error) {
logger.error('Manual handler registration failed', { error });
throw error; throw error;
} }
} }

View file

@ -15,12 +15,18 @@ interface QMExchange {
export async function fetchExchanges(services: IServiceContainer): Promise<QMExchange[]> { export async function fetchExchanges(services: IServiceContainer): Promise<QMExchange[]> {
// Get exchanges from MongoDB // Get exchanges from MongoDB
const exchanges = await services.mongodb.collection<QMExchange>('qm_exchanges').find({}).toArray(); const exchanges = await services.mongodb
.collection<QMExchange>('qm_exchanges')
.find({})
.toArray();
return exchanges; return exchanges;
} }
export async function getExchangeByCode(services: IServiceContainer, code: string): Promise<QMExchange | null> { export async function getExchangeByCode(
services: IServiceContainer,
code: string
): Promise<QMExchange | null> {
// Get specific exchange by code // Get specific exchange by code
const exchange = await services.mongodb.collection<QMExchange>('qm_exchanges').findOne({ code }); const exchange = await services.mongodb.collection<QMExchange>('qm_exchanges').findOne({ code });

View file

@ -16,12 +16,19 @@ interface QMSymbol {
export async function searchSymbols(services: IServiceContainer): Promise<QMSymbol[]> { export async function searchSymbols(services: IServiceContainer): Promise<QMSymbol[]> {
// Get symbols from MongoDB // Get symbols from MongoDB
const symbols = await services.mongodb.collection<QMSymbol>('qm_symbols').find({}).limit(50).toArray(); const symbols = await services.mongodb
.collection<QMSymbol>('qm_symbols')
.find({})
.limit(50)
.toArray();
return symbols; return symbols;
} }
export async function fetchSymbolData(services: IServiceContainer, symbol: string): Promise<QMSymbol | null> { export async function fetchSymbolData(
services: IServiceContainer,
symbol: string
): Promise<QMSymbol | null> {
// Fetch data for a specific symbol // Fetch data for a specific symbol
const symbolData = await services.mongodb.collection<QMSymbol>('qm_symbols').findOne({ symbol }); const symbolData = await services.mongodb.collection<QMSymbol>('qm_symbols').findOne({ symbol });

View file

@ -1,7 +1,7 @@
import { BaseHandler, Handler, type IServiceContainer } from '@stock-bot/handlers'; import { BaseHandler, Handler, type IServiceContainer } from '@stock-bot/handlers';
@Handler('qm') @Handler('qm')
class QMHandler extends BaseHandler { export class QMHandler extends BaseHandler {
constructor(services: IServiceContainer) { constructor(services: IServiceContainer) {
super(services); // Handler name read from @Handler decorator super(services); // Handler name read from @Handler decorator
} }

View file

@ -4,17 +4,18 @@ import {
Operation, Operation,
QueueSchedule, QueueSchedule,
type ExecutionContext, type ExecutionContext,
type IServiceContainer type IServiceContainer,
} from '@stock-bot/handlers'; } from '@stock-bot/handlers';
@Handler('webshare') @Handler('webshare')
class WebShareHandler extends BaseHandler { export class WebShareHandler extends BaseHandler {
constructor(services: IServiceContainer) { constructor(services: IServiceContainer) {
super(services); super(services);
} }
@Operation('fetch-proxies') @Operation('fetch-proxies')
@QueueSchedule('0 */6 * * *', { // every 6 hours @QueueSchedule('0 */6 * * *', {
// every 6 hours
priority: 3, priority: 3,
immediately: false, // Don't run immediately since ProxyManager fetches on startup immediately: false, // Don't run immediately since ProxyManager fetches on startup
description: 'Refresh proxies from WebShare API', description: 'Refresh proxies from WebShare API',

View file

@ -3,15 +3,12 @@
* Simplified entry point using ServiceApplication framework * Simplified entry point using ServiceApplication framework
*/ */
import { initializeStockConfig, type StockAppConfig } from '@stock-bot/stock-config'; import { ServiceApplication } from '@stock-bot/di';
import {
ServiceApplication,
} from '@stock-bot/di';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import { initializeStockConfig, type StockAppConfig } from '@stock-bot/stock-config';
import { createRoutes } from './routes/create-routes';
// Local imports // Local imports
import { initializeAllHandlers } from './handlers'; import { initializeAllHandlers } from './handlers';
import { createRoutes } from './routes/create-routes';
// Initialize configuration with service-specific overrides // Initialize configuration with service-specific overrides
const config = initializeStockConfig('dataIngestion'); const config = initializeStockConfig('dataIngestion');
@ -44,7 +41,7 @@ const app = new ServiceApplication(
}, },
{ {
// Lifecycle hooks if needed // Lifecycle hooks if needed
onStarted: (_port) => { onStarted: _port => {
const logger = getLogger('data-ingestion'); const logger = getLogger('data-ingestion');
logger.info('Data ingestion service startup initiated with ServiceApplication framework'); logger.info('Data ingestion service startup initiated with ServiceApplication framework');
}, },
@ -71,7 +68,6 @@ async function createContainer(config: StockAppConfig) {
return container; return container;
} }
// Start the service // Start the service
app.start(createContainer, createRoutes, initializeAllHandlers).catch(error => { app.start(createContainer, createRoutes, initializeAllHandlers).catch(error => {
const logger = getLogger('data-ingestion'); const logger = getLogger('data-ingestion');

View file

@ -2,9 +2,9 @@
* Market data routes * Market data routes
*/ */
import { Hono } from 'hono'; import { Hono } from 'hono';
import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import { processItems } from '@stock-bot/queue'; import { processItems } from '@stock-bot/queue';
import type { IServiceContainer } from '@stock-bot/handlers';
const logger = getLogger('market-data-routes'); const logger = getLogger('market-data-routes');
@ -111,7 +111,10 @@ export function createMarketDataRoutes(container: IServiceContainer) {
return c.json({ status: 'error', message: 'Queue manager not available' }, 503); return c.json({ status: 'error', message: 'Queue manager not available' }, 503);
} }
const result = await processItems(symbols, provider, { const result = await processItems(
symbols,
provider,
{
handler: provider, handler: provider,
operation, operation,
totalDelayHours, totalDelayHours,
@ -121,7 +124,9 @@ export function createMarketDataRoutes(container: IServiceContainer) {
retries: 2, retries: 2,
removeOnComplete: 5, removeOnComplete: 5,
removeOnFail: 10, removeOnFail: 10,
}, queueManager); },
queueManager
);
return c.json({ return c.json({
status: 'success', status: 'success',

View file

@ -1,6 +1,6 @@
import { Hono } from 'hono'; import { Hono } from 'hono';
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
const logger = getLogger('queue-routes'); const logger = getLogger('queue-routes');

View file

@ -3,9 +3,9 @@
* Configures dependency injection for the data pipeline service * Configures dependency injection for the data pipeline service
*/ */
import type { AppConfig } from '@stock-bot/config';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { AppConfig } from '@stock-bot/config';
const logger = getLogger('data-pipeline-container'); const logger = getLogger('data-pipeline-container');

View file

@ -3,8 +3,8 @@ import {
Handler, Handler,
Operation, Operation,
ScheduledOperation, ScheduledOperation,
type IServiceContainer,
} from '@stock-bot/handlers'; } from '@stock-bot/handlers';
import type { IServiceContainer } from '@stock-bot/types';
import { clearPostgreSQLData } from './operations/clear-postgresql-data.operations'; import { clearPostgreSQLData } from './operations/clear-postgresql-data.operations';
import { getSyncStatus } from './operations/enhanced-sync-status.operations'; import { getSyncStatus } from './operations/enhanced-sync-status.operations';
import { getExchangeStats } from './operations/exchange-stats.operations'; import { getExchangeStats } from './operations/exchange-stats.operations';
@ -77,7 +77,9 @@ class ExchangesHandler extends BaseHandler {
* Clear PostgreSQL data - maintenance operation * Clear PostgreSQL data - maintenance operation
*/ */
@Operation('clear-postgresql-data') @Operation('clear-postgresql-data')
async clearPostgreSQLData(payload: { type?: 'exchanges' | 'provider_mappings' | 'all' }): Promise<unknown> { async clearPostgreSQLData(payload: {
type?: 'exchanges' | 'provider_mappings' | 'all';
}): Promise<unknown> {
this.log('warn', 'Clearing PostgreSQL data', payload); this.log('warn', 'Clearing PostgreSQL data', payload);
return clearPostgreSQLData(payload, this.services); return clearPostgreSQLData(payload, this.services);
} }

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload } from '../../../types/job-payloads'; import type { JobPayload } from '../../../types/job-payloads';
const logger = getLogger('enhanced-sync-clear-postgresql-data'); const logger = getLogger('enhanced-sync-clear-postgresql-data');

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload, SyncStatus } from '../../../types/job-payloads'; import type { JobPayload, SyncStatus } from '../../../types/job-payloads';
const logger = getLogger('enhanced-sync-status'); const logger = getLogger('enhanced-sync-status');

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload } from '../../../types/job-payloads'; import type { JobPayload } from '../../../types/job-payloads';
const logger = getLogger('enhanced-sync-exchange-stats'); const logger = getLogger('enhanced-sync-exchange-stats');

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload } from '../../../types/job-payloads'; import type { JobPayload } from '../../../types/job-payloads';
const logger = getLogger('enhanced-sync-provider-mapping-stats'); const logger = getLogger('enhanced-sync-provider-mapping-stats');

View file

@ -1,5 +1,5 @@
import type { IServiceContainer } from '@stock-bot/types';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers';
import type { JobPayload } from '../../../types/job-payloads'; import type { JobPayload } from '../../../types/job-payloads';
const logger = getLogger('sync-qm-exchanges'); const logger = getLogger('sync-qm-exchanges');
@ -62,7 +62,10 @@ interface Exchange {
visible: boolean; visible: boolean;
} }
async function findExchange(exchangeCode: string, postgresClient: IServiceContainer['postgres']): Promise<Exchange | null> { async function findExchange(
exchangeCode: string,
postgresClient: IServiceContainer['postgres']
): Promise<Exchange | null> {
const query = 'SELECT * FROM exchanges WHERE code = $1'; const query = 'SELECT * FROM exchanges WHERE code = $1';
const result = await postgresClient.query(query, [exchangeCode]); const result = await postgresClient.query(query, [exchangeCode]);
return result.rows[0] || null; return result.rows[0] || null;
@ -76,7 +79,10 @@ interface QMExchange {
countryCode?: string; countryCode?: string;
} }
async function createExchange(qmExchange: QMExchange, postgresClient: IServiceContainer['postgres']): Promise<void> { async function createExchange(
qmExchange: QMExchange,
postgresClient: IServiceContainer['postgres']
): Promise<void> {
const query = ` const query = `
INSERT INTO exchanges (code, name, country, currency, visible) INSERT INTO exchanges (code, name, country, currency, visible)
VALUES ($1, $2, $3, $4, $5) VALUES ($1, $2, $3, $4, $5)

View file

@ -1,10 +1,13 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload, SyncResult } from '../../../types/job-payloads'; import type { JobPayload, SyncResult } from '../../../types/job-payloads';
const logger = getLogger('enhanced-sync-all-exchanges'); const logger = getLogger('enhanced-sync-all-exchanges');
export async function syncAllExchanges(payload: JobPayload, container: IServiceContainer): Promise<SyncResult> { export async function syncAllExchanges(
payload: JobPayload,
container: IServiceContainer
): Promise<SyncResult> {
const clearFirst = payload.clearFirst || true; const clearFirst = payload.clearFirst || true;
logger.info('Starting comprehensive exchange sync...', { clearFirst }); logger.info('Starting comprehensive exchange sync...', { clearFirst });
@ -50,7 +53,6 @@ export async function syncAllExchanges(payload: JobPayload, container: IServiceC
} }
} }
async function clearPostgreSQLData(postgresClient: any): Promise<void> { async function clearPostgreSQLData(postgresClient: any): Promise<void> {
logger.info('Clearing existing PostgreSQL data...'); logger.info('Clearing existing PostgreSQL data...');
@ -141,7 +143,11 @@ async function createProviderExchangeMapping(
const postgresClient = container.postgres; const postgresClient = container.postgres;
// Check if mapping already exists // Check if mapping already exists
const existingMapping = await findProviderExchangeMapping(provider, providerExchangeCode, container); const existingMapping = await findProviderExchangeMapping(
provider,
providerExchangeCode,
container
);
if (existingMapping) { if (existingMapping) {
// Don't override existing mappings to preserve manual work // Don't override existing mappings to preserve manual work
return; return;

View file

@ -1,6 +1,6 @@
import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { MasterExchange } from '@stock-bot/mongodb'; import type { MasterExchange } from '@stock-bot/mongodb';
import type { IServiceContainer } from '@stock-bot/handlers';
import type { JobPayload } from '../../../types/job-payloads'; import type { JobPayload } from '../../../types/job-payloads';
const logger = getLogger('sync-ib-exchanges'); const logger = getLogger('sync-ib-exchanges');
@ -65,7 +65,10 @@ export async function syncIBExchanges(
/** /**
* Create or update master exchange record 1:1 from IB exchange * Create or update master exchange record 1:1 from IB exchange
*/ */
async function createOrUpdateMasterExchange(ibExchange: IBExchange, container: IServiceContainer): Promise<void> { async function createOrUpdateMasterExchange(
ibExchange: IBExchange,
container: IServiceContainer
): Promise<void> {
const mongoClient = container.mongodb; const mongoClient = container.mongodb;
const db = mongoClient.getDatabase(); const db = mongoClient.getDatabase();
const collection = db.collection<MasterExchange>('masterExchanges'); const collection = db.collection<MasterExchange>('masterExchanges');

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload, SyncResult } from '../../../types/job-payloads'; import type { JobPayload, SyncResult } from '../../../types/job-payloads';
const logger = getLogger('enhanced-sync-qm-provider-mappings'); const logger = getLogger('enhanced-sync-qm-provider-mappings');
@ -86,7 +86,6 @@ export async function syncQMProviderMappings(
} }
} }
async function createProviderExchangeMapping( async function createProviderExchangeMapping(
provider: string, provider: string,
providerExchangeCode: string, providerExchangeCode: string,
@ -103,7 +102,11 @@ async function createProviderExchangeMapping(
const postgresClient = container.postgres; const postgresClient = container.postgres;
// Check if mapping already exists // Check if mapping already exists
const existingMapping = await findProviderExchangeMapping(provider, providerExchangeCode, container); const existingMapping = await findProviderExchangeMapping(
provider,
providerExchangeCode,
container
);
if (existingMapping) { if (existingMapping) {
// Don't override existing mappings to preserve manual work // Don't override existing mappings to preserve manual work
return; return;

View file

@ -6,7 +6,6 @@
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { autoRegisterHandlers } from '@stock-bot/handlers'; import { autoRegisterHandlers } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
// Import handlers for bundling (ensures they're included in the build) // Import handlers for bundling (ensures they're included in the build)
import './exchanges/exchanges.handler'; import './exchanges/exchanges.handler';
import './symbols/symbols.handler'; import './symbols/symbols.handler';

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload } from '../../../types/job-payloads'; import type { JobPayload } from '../../../types/job-payloads';
const logger = getLogger('sync-qm-symbols'); const logger = getLogger('sync-qm-symbols');

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload } from '../../../types/job-payloads'; import type { JobPayload } from '../../../types/job-payloads';
const logger = getLogger('sync-status'); const logger = getLogger('sync-status');

View file

@ -1,5 +1,5 @@
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { JobPayload, SyncResult } from '../../../types/job-payloads'; import type { JobPayload, SyncResult } from '../../../types/job-payloads';
const logger = getLogger('enhanced-sync-symbols-from-provider'); const logger = getLogger('enhanced-sync-symbols-from-provider');
@ -104,7 +104,11 @@ async function processSingleSymbol(
} }
// Find active provider exchange mapping // Find active provider exchange mapping
const providerMapping = await findActiveProviderExchangeMapping(provider, exchangeCode, container); const providerMapping = await findActiveProviderExchangeMapping(
provider,
exchangeCode,
container
);
if (!providerMapping) { if (!providerMapping) {
result.skipped++; result.skipped++;
@ -145,14 +149,22 @@ async function findActiveProviderExchangeMapping(
return result.rows[0] || null; return result.rows[0] || null;
} }
async function findSymbolByCodeAndExchange(symbol: string, exchangeId: string, container: IServiceContainer): Promise<any> { async function findSymbolByCodeAndExchange(
symbol: string,
exchangeId: string,
container: IServiceContainer
): Promise<any> {
const postgresClient = container.postgres; const postgresClient = container.postgres;
const query = 'SELECT * FROM symbols WHERE symbol = $1 AND exchange_id = $2'; const query = 'SELECT * FROM symbols WHERE symbol = $1 AND exchange_id = $2';
const result = await postgresClient.query(query, [symbol, exchangeId]); const result = await postgresClient.query(query, [symbol, exchangeId]);
return result.rows[0] || null; return result.rows[0] || null;
} }
async function createSymbol(symbol: any, exchangeId: string, container: IServiceContainer): Promise<string> { async function createSymbol(
symbol: any,
exchangeId: string,
container: IServiceContainer
): Promise<string> {
const postgresClient = container.postgres; const postgresClient = container.postgres;
const query = ` const query = `
INSERT INTO symbols (symbol, exchange_id, company_name, country, currency) INSERT INTO symbols (symbol, exchange_id, company_name, country, currency)
@ -171,7 +183,11 @@ async function createSymbol(symbol: any, exchangeId: string, container: IService
return result.rows[0].id; return result.rows[0].id;
} }
async function updateSymbol(symbolId: string, symbol: any, container: IServiceContainer): Promise<void> { async function updateSymbol(
symbolId: string,
symbol: any,
container: IServiceContainer
): Promise<void> {
const postgresClient = container.postgres; const postgresClient = container.postgres;
const query = ` const query = `
UPDATE symbols UPDATE symbols

View file

@ -6,8 +6,8 @@ import {
type IServiceContainer, type IServiceContainer,
} from '@stock-bot/handlers'; } from '@stock-bot/handlers';
import { syncQMSymbols } from './operations/qm-symbols.operations'; import { syncQMSymbols } from './operations/qm-symbols.operations';
import { syncSymbolsFromProvider } from './operations/sync-symbols-from-provider.operations';
import { getSyncStatus } from './operations/sync-status.operations'; import { getSyncStatus } from './operations/sync-status.operations';
import { syncSymbolsFromProvider } from './operations/sync-symbols-from-provider.operations';
@Handler('symbols') @Handler('symbols')
class SymbolsHandler extends BaseHandler { class SymbolsHandler extends BaseHandler {
@ -61,7 +61,10 @@ class SymbolsHandler extends BaseHandler {
/** /**
* Internal method to sync symbols from a provider * Internal method to sync symbols from a provider
*/ */
private async syncSymbolsFromProvider(payload: { provider: string; clearFirst?: boolean }): Promise<unknown> { private async syncSymbolsFromProvider(payload: {
provider: string;
clearFirst?: boolean;
}): Promise<unknown> {
this.log('info', 'Syncing symbols from provider', { provider: payload.provider }); this.log('info', 'Syncing symbols from provider', { provider: payload.provider });
return syncSymbolsFromProvider(payload, this.services); return syncSymbolsFromProvider(payload, this.services);
} }

View file

@ -3,14 +3,13 @@
* Simplified entry point using ServiceApplication framework * Simplified entry point using ServiceApplication framework
*/ */
import { initializeStockConfig } from '@stock-bot/stock-config';
import { ServiceApplication } from '@stock-bot/di'; import { ServiceApplication } from '@stock-bot/di';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import { initializeStockConfig } from '@stock-bot/stock-config';
// Local imports
import { initializeAllHandlers } from './handlers';
import { createRoutes } from './routes/create-routes'; import { createRoutes } from './routes/create-routes';
import { setupServiceContainer } from './container-setup'; import { setupServiceContainer } from './container-setup';
// Local imports
import { initializeAllHandlers } from './handlers';
// Initialize configuration with service-specific overrides // Initialize configuration with service-specific overrides
const config = initializeStockConfig('dataPipeline'); const config = initializeStockConfig('dataPipeline');
@ -43,12 +42,12 @@ const app = new ServiceApplication(
}, },
{ {
// Custom lifecycle hooks // Custom lifecycle hooks
onContainerReady: (container) => { onContainerReady: container => {
// Setup service-specific configuration // Setup service-specific configuration
const enhancedContainer = setupServiceContainer(config, container); const enhancedContainer = setupServiceContainer(config, container);
return enhancedContainer; return enhancedContainer;
}, },
onStarted: (_port) => { onStarted: _port => {
const logger = getLogger('data-pipeline'); const logger = getLogger('data-pipeline');
logger.info('Data pipeline service startup initiated with ServiceApplication framework'); logger.info('Data pipeline service startup initiated with ServiceApplication framework');
}, },

View file

@ -5,10 +5,10 @@
import { Hono } from 'hono'; import { Hono } from 'hono';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { healthRoutes } from './health.routes';
import { createSyncRoutes } from './sync.routes';
import { createEnhancedSyncRoutes } from './enhanced-sync.routes'; import { createEnhancedSyncRoutes } from './enhanced-sync.routes';
import { healthRoutes } from './health.routes';
import { createStatsRoutes } from './stats.routes'; import { createStatsRoutes } from './stats.routes';
import { createSyncRoutes } from './sync.routes';
export function createRoutes(container: IServiceContainer): Hono { export function createRoutes(container: IServiceContainer): Hono {
const app = new Hono(); const app = new Hono();

View file

@ -1,6 +1,6 @@
import { Hono } from 'hono'; import { Hono } from 'hono';
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
const logger = getLogger('enhanced-sync-routes'); const logger = getLogger('enhanced-sync-routes');

View file

@ -1,6 +1,6 @@
import { Hono } from 'hono'; import { Hono } from 'hono';
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
const logger = getLogger('stats-routes'); const logger = getLogger('stats-routes');

View file

@ -1,6 +1,6 @@
import { Hono } from 'hono'; import { Hono } from 'hono';
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
const logger = getLogger('sync-routes'); const logger = getLogger('sync-routes');

View file

@ -11,21 +11,18 @@
"dev:web": "cd web-app && bun run dev", "dev:web": "cd web-app && bun run dev",
"dev:backend": "turbo run dev --filter=\"@stock-bot/data-*\" --filter=\"@stock-bot/web-api\"", "dev:backend": "turbo run dev --filter=\"@stock-bot/data-*\" --filter=\"@stock-bot/web-api\"",
"dev:frontend": "turbo run dev --filter=\"@stock-bot/web-app\"", "dev:frontend": "turbo run dev --filter=\"@stock-bot/web-app\"",
"build": "echo 'Stock apps built via parent turbo'",
"build": "turbo run build",
"build:config": "cd config && bun run build", "build:config": "cd config && bun run build",
"build:services": "turbo run build --filter=\"@stock-bot/data-*\" --filter=\"@stock-bot/web-*\"", "build:services": "turbo run build --filter=\"@stock-bot/data-*\" --filter=\"@stock-bot/web-*\"",
"build:ingestion": "cd data-ingestion && bun run build", "build:ingestion": "cd data-ingestion && bun run build",
"build:pipeline": "cd data-pipeline && bun run build", "build:pipeline": "cd data-pipeline && bun run build",
"build:api": "cd web-api && bun run build", "build:api": "cd web-api && bun run build",
"build:web": "cd web-app && bun run build", "build:web": "cd web-app && bun run build",
"start": "turbo run start --filter=\"@stock-bot/data-*\" --filter=\"@stock-bot/web-api\"", "start": "turbo run start --filter=\"@stock-bot/data-*\" --filter=\"@stock-bot/web-api\"",
"start:all": "turbo run start", "start:all": "turbo run start",
"start:ingestion": "cd data-ingestion && bun start", "start:ingestion": "cd data-ingestion && bun start",
"start:pipeline": "cd data-pipeline && bun start", "start:pipeline": "cd data-pipeline && bun start",
"start:api": "cd web-api && bun start", "start:api": "cd web-api && bun start",
"clean": "turbo run clean", "clean": "turbo run clean",
"clean:all": "turbo run clean && rm -rf node_modules", "clean:all": "turbo run clean && rm -rf node_modules",
"clean:ingestion": "cd data-ingestion && rm -rf dist node_modules", "clean:ingestion": "cd data-ingestion && rm -rf dist node_modules",
@ -33,7 +30,6 @@
"clean:api": "cd web-api && rm -rf dist node_modules", "clean:api": "cd web-api && rm -rf dist node_modules",
"clean:web": "cd web-app && rm -rf dist node_modules", "clean:web": "cd web-app && rm -rf dist node_modules",
"clean:config": "cd config && rm -rf dist node_modules", "clean:config": "cd config && rm -rf dist node_modules",
"test": "turbo run test", "test": "turbo run test",
"test:all": "turbo run test", "test:all": "turbo run test",
"test:config": "cd config && bun test", "test:config": "cd config && bun test",
@ -41,7 +37,6 @@
"test:ingestion": "cd data-ingestion && bun test", "test:ingestion": "cd data-ingestion && bun test",
"test:pipeline": "cd data-pipeline && bun test", "test:pipeline": "cd data-pipeline && bun test",
"test:api": "cd web-api && bun test", "test:api": "cd web-api && bun test",
"lint": "turbo run lint", "lint": "turbo run lint",
"lint:all": "turbo run lint", "lint:all": "turbo run lint",
"lint:config": "cd config && bun run lint", "lint:config": "cd config && bun run lint",
@ -50,22 +45,17 @@
"lint:pipeline": "cd data-pipeline && bun run lint", "lint:pipeline": "cd data-pipeline && bun run lint",
"lint:api": "cd web-api && bun run lint", "lint:api": "cd web-api && bun run lint",
"lint:web": "cd web-app && bun run lint", "lint:web": "cd web-app && bun run lint",
"install:all": "bun install", "install:all": "bun install",
"docker:build": "docker-compose build", "docker:build": "docker-compose build",
"docker:up": "docker-compose up", "docker:up": "docker-compose up",
"docker:down": "docker-compose down", "docker:down": "docker-compose down",
"pm2:start": "pm2 start ecosystem.config.js", "pm2:start": "pm2 start ecosystem.config.js",
"pm2:stop": "pm2 stop all", "pm2:stop": "pm2 stop all",
"pm2:restart": "pm2 restart all", "pm2:restart": "pm2 restart all",
"pm2:logs": "pm2 logs", "pm2:logs": "pm2 logs",
"pm2:status": "pm2 status", "pm2:status": "pm2 status",
"db:migrate": "cd data-ingestion && bun run db:migrate", "db:migrate": "cd data-ingestion && bun run db:migrate",
"db:seed": "cd data-ingestion && bun run db:seed", "db:seed": "cd data-ingestion && bun run db:seed",
"health:check": "bun scripts/health-check.js", "health:check": "bun scripts/health-check.js",
"monitor": "bun run pm2:logs", "monitor": "bun run pm2:logs",
"status": "bun run pm2:status" "status": "bun run pm2:status"

View file

@ -3,9 +3,9 @@
* Configures dependency injection for the web API service * Configures dependency injection for the web API service
*/ */
import type { AppConfig } from '@stock-bot/config';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { AppConfig } from '@stock-bot/config';
const logger = getLogger('web-api-container'); const logger = getLogger('web-api-container');

View file

@ -3,10 +3,9 @@
* Simplified entry point using ServiceApplication framework * Simplified entry point using ServiceApplication framework
*/ */
import { initializeStockConfig } from '@stock-bot/stock-config';
import { ServiceApplication } from '@stock-bot/di'; import { ServiceApplication } from '@stock-bot/di';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import { initializeStockConfig } from '@stock-bot/stock-config';
// Local imports // Local imports
import { createRoutes } from './routes/create-routes'; import { createRoutes } from './routes/create-routes';
@ -49,7 +48,7 @@ const app = new ServiceApplication(
}, },
{ {
// Custom lifecycle hooks // Custom lifecycle hooks
onStarted: (_port) => { onStarted: _port => {
const logger = getLogger('web-api'); const logger = getLogger('web-api');
logger.info('Web API service startup initiated with ServiceApplication framework'); logger.info('Web API service startup initiated with ServiceApplication framework');
}, },

View file

@ -5,8 +5,8 @@
import { Hono } from 'hono'; import { Hono } from 'hono';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { createHealthRoutes } from './health.routes';
import { createExchangeRoutes } from './exchange.routes'; import { createExchangeRoutes } from './exchange.routes';
import { createHealthRoutes } from './health.routes';
import { createMonitoringRoutes } from './monitoring.routes'; import { createMonitoringRoutes } from './monitoring.routes';
import { createPipelineRoutes } from './pipeline.routes'; import { createPipelineRoutes } from './pipeline.routes';

View file

@ -2,8 +2,8 @@
* Exchange management routes - Refactored * Exchange management routes - Refactored
*/ */
import { Hono } from 'hono'; import { Hono } from 'hono';
import type { IServiceContainer } from '@stock-bot/types';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers';
import { createExchangeService } from '../services/exchange.service'; import { createExchangeService } from '../services/exchange.service';
import { createSuccessResponse, handleError } from '../utils/error-handler'; import { createSuccessResponse, handleError } from '../utils/error-handler';
import { import {

View file

@ -2,8 +2,8 @@
* Health check routes factory * Health check routes factory
*/ */
import { Hono } from 'hono'; import { Hono } from 'hono';
import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
const logger = getLogger('health-routes'); const logger = getLogger('health-routes');
@ -70,7 +70,10 @@ export function createHealthRoutes(container: IServiceContainer) {
health.checks.postgresql = { status: 'healthy', message: 'Connected and responsive' }; health.checks.postgresql = { status: 'healthy', message: 'Connected and responsive' };
logger.debug('PostgreSQL health check passed'); logger.debug('PostgreSQL health check passed');
} else { } else {
health.checks.postgresql = { status: 'unhealthy', message: 'PostgreSQL client not available' }; health.checks.postgresql = {
status: 'unhealthy',
message: 'PostgreSQL client not available',
};
logger.warn('PostgreSQL health check failed - client not available'); logger.warn('PostgreSQL health check failed - client not available');
} }
} catch (error) { } catch (error) {

View file

@ -13,144 +13,174 @@ export function createMonitoringRoutes(container: IServiceContainer) {
/** /**
* Get overall system health * Get overall system health
*/ */
monitoring.get('/', async (c) => { monitoring.get('/', async c => {
try { try {
const health = await monitoringService.getSystemHealth(); const health = await monitoringService.getSystemHealth();
// Set appropriate status code based on health // Set appropriate status code based on health
const statusCode = health.status === 'healthy' ? 200 : const statusCode =
health.status === 'degraded' ? 503 : 500; health.status === 'healthy' ? 200 : health.status === 'degraded' ? 503 : 500;
return c.json(health, statusCode); return c.json(health, statusCode);
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
status: 'error', status: 'error',
message: 'Failed to retrieve system health', message: 'Failed to retrieve system health',
error: error instanceof Error ? error.message : 'Unknown error', error: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get cache/Dragonfly statistics * Get cache/Dragonfly statistics
*/ */
monitoring.get('/cache', async (c) => { monitoring.get('/cache', async c => {
try { try {
const stats = await monitoringService.getCacheStats(); const stats = await monitoringService.getCacheStats();
return c.json(stats); return c.json(stats);
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve cache statistics', error: 'Failed to retrieve cache statistics',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get queue statistics * Get queue statistics
*/ */
monitoring.get('/queues', async (c) => { monitoring.get('/queues', async c => {
try { try {
const stats = await monitoringService.getQueueStats(); const stats = await monitoringService.getQueueStats();
return c.json({ queues: stats }); return c.json({ queues: stats });
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve queue statistics', error: 'Failed to retrieve queue statistics',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get specific queue statistics * Get specific queue statistics
*/ */
monitoring.get('/queues/:name', async (c) => { monitoring.get('/queues/:name', async c => {
try { try {
const queueName = c.req.param('name'); const queueName = c.req.param('name');
const stats = await monitoringService.getQueueStats(); const stats = await monitoringService.getQueueStats();
const queueStats = stats.find(q => q.name === queueName); const queueStats = stats.find(q => q.name === queueName);
if (!queueStats) { if (!queueStats) {
return c.json({ return c.json(
{
error: 'Queue not found', error: 'Queue not found',
message: `Queue '${queueName}' does not exist`, message: `Queue '${queueName}' does not exist`,
}, 404); },
404
);
} }
return c.json(queueStats); return c.json(queueStats);
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve queue statistics', error: 'Failed to retrieve queue statistics',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get database statistics * Get database statistics
*/ */
monitoring.get('/databases', async (c) => { monitoring.get('/databases', async c => {
try { try {
const stats = await monitoringService.getDatabaseStats(); const stats = await monitoringService.getDatabaseStats();
return c.json({ databases: stats }); return c.json({ databases: stats });
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve database statistics', error: 'Failed to retrieve database statistics',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get specific database statistics * Get specific database statistics
*/ */
monitoring.get('/databases/:type', async (c) => { monitoring.get('/databases/:type', async c => {
try { try {
const dbType = c.req.param('type') as 'postgres' | 'mongodb' | 'questdb'; const dbType = c.req.param('type') as 'postgres' | 'mongodb' | 'questdb';
const stats = await monitoringService.getDatabaseStats(); const stats = await monitoringService.getDatabaseStats();
const dbStats = stats.find(db => db.type === dbType); const dbStats = stats.find(db => db.type === dbType);
if (!dbStats) { if (!dbStats) {
return c.json({ return c.json(
{
error: 'Database not found', error: 'Database not found',
message: `Database type '${dbType}' not found or not enabled`, message: `Database type '${dbType}' not found or not enabled`,
}, 404); },
404
);
} }
return c.json(dbStats); return c.json(dbStats);
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve database statistics', error: 'Failed to retrieve database statistics',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get service metrics * Get service metrics
*/ */
monitoring.get('/metrics', async (c) => { monitoring.get('/metrics', async c => {
try { try {
const metrics = await monitoringService.getServiceMetrics(); const metrics = await monitoringService.getServiceMetrics();
return c.json(metrics); return c.json(metrics);
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve service metrics', error: 'Failed to retrieve service metrics',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get detailed cache info (Redis INFO command output) * Get detailed cache info (Redis INFO command output)
*/ */
monitoring.get('/cache/info', async (c) => { monitoring.get('/cache/info', async c => {
try { try {
if (!container.cache) { if (!container.cache) {
return c.json({ return c.json(
{
error: 'Cache not available', error: 'Cache not available',
message: 'Cache service is not enabled', message: 'Cache service is not enabled',
}, 503); },
503
);
} }
const info = await container.cache.info(); const info = await container.cache.info();
@ -161,17 +191,20 @@ export function createMonitoringRoutes(container: IServiceContainer) {
raw: info, raw: info,
}); });
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve cache info', error: 'Failed to retrieve cache info',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Health check endpoint for monitoring * Health check endpoint for monitoring
*/ */
monitoring.get('/ping', (c) => { monitoring.get('/ping', c => {
return c.json({ return c.json({
status: 'ok', status: 'ok',
timestamp: new Date().toISOString(), timestamp: new Date().toISOString(),
@ -182,52 +215,61 @@ export function createMonitoringRoutes(container: IServiceContainer) {
/** /**
* Get service status for all microservices * Get service status for all microservices
*/ */
monitoring.get('/services', async (c) => { monitoring.get('/services', async c => {
try { try {
const services = await monitoringService.getServiceStatus(); const services = await monitoringService.getServiceStatus();
return c.json({ services }); return c.json({ services });
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve service status', error: 'Failed to retrieve service status',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get proxy statistics * Get proxy statistics
*/ */
monitoring.get('/proxies', async (c) => { monitoring.get('/proxies', async c => {
try { try {
const stats = await monitoringService.getProxyStats(); const stats = await monitoringService.getProxyStats();
return c.json(stats || { enabled: false }); return c.json(stats || { enabled: false });
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve proxy statistics', error: 'Failed to retrieve proxy statistics',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Get comprehensive system overview * Get comprehensive system overview
*/ */
monitoring.get('/overview', async (c) => { monitoring.get('/overview', async c => {
try { try {
const overview = await monitoringService.getSystemOverview(); const overview = await monitoringService.getSystemOverview();
return c.json(overview); return c.json(overview);
} catch (error) { } catch (error) {
return c.json({ return c.json(
{
error: 'Failed to retrieve system overview', error: 'Failed to retrieve system overview',
message: error instanceof Error ? error.message : 'Unknown error', message: error instanceof Error ? error.message : 'Unknown error',
}, 500); },
500
);
} }
}); });
/** /**
* Test direct BullMQ queue access * Test direct BullMQ queue access
*/ */
monitoring.get('/test/queue/:name', async (c) => { monitoring.get('/test/queue/:name', async c => {
const queueName = c.req.param('name'); const queueName = c.req.param('name');
const { Queue } = await import('bullmq'); const { Queue } = await import('bullmq');
@ -244,14 +286,17 @@ export function createMonitoringRoutes(container: IServiceContainer) {
await queue.close(); await queue.close();
return c.json({ return c.json({
queueName, queueName,
counts counts,
}); });
} catch (error: any) { } catch (error: any) {
await queue.close(); await queue.close();
return c.json({ return c.json(
{
queueName, queueName,
error: error.message error: error.message,
}, 500); },
500
);
} }
}); });

View file

@ -1,5 +1,5 @@
import type { IServiceContainer } from '@stock-bot/types';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { IServiceContainer } from '@stock-bot/handlers';
import { import {
CreateExchangeRequest, CreateExchangeRequest,
CreateProviderMappingRequest, CreateProviderMappingRequest,

View file

@ -3,19 +3,19 @@
* Collects health and performance metrics from all system components * Collects health and performance metrics from all system components
*/ */
import * as os from 'os';
import type { IServiceContainer } from '@stock-bot/handlers'; import type { IServiceContainer } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { import type {
CacheStats, CacheStats,
QueueStats,
DatabaseStats, DatabaseStats,
SystemHealth, ProxyStats,
QueueStats,
ServiceMetrics, ServiceMetrics,
ServiceStatus, ServiceStatus,
ProxyStats, SystemHealth,
SystemOverview SystemOverview,
} from '../types/monitoring.types'; } from '../types/monitoring.types';
import * as os from 'os';
export class MonitoringService { export class MonitoringService {
private readonly logger = getLogger('monitoring-service'); private readonly logger = getLogger('monitoring-service');
@ -87,18 +87,18 @@ export class MonitoringService {
type: queueManager.constructor.name, type: queueManager.constructor.name,
hasGetAllQueues: typeof queueManager.getAllQueues === 'function', hasGetAllQueues: typeof queueManager.getAllQueues === 'function',
hasQueues: !!queueManager.queues, hasQueues: !!queueManager.queues,
hasGetQueue: typeof queueManager.getQueue === 'function' hasGetQueue: typeof queueManager.getQueue === 'function',
}); });
// Always use the known queue names since web-api doesn't create worker queues // Always use the known queue names since web-api doesn't create worker queues
const handlerMapping = { const handlerMapping = {
'proxy': 'data-ingestion', proxy: 'data-ingestion',
'qm': 'data-ingestion', qm: 'data-ingestion',
'ib': 'data-ingestion', ib: 'data-ingestion',
'ceo': 'data-ingestion', ceo: 'data-ingestion',
'webshare': 'data-ingestion', webshare: 'data-ingestion',
'exchanges': 'data-pipeline', exchanges: 'data-pipeline',
'symbols': 'data-pipeline', symbols: 'data-pipeline',
}; };
const queueNames = Object.keys(handlerMapping); const queueNames = Object.keys(handlerMapping);
@ -188,7 +188,7 @@ export class MonitoringService {
bullQueue.getCompletedCount(), bullQueue.getCompletedCount(),
bullQueue.getFailedCount(), bullQueue.getFailedCount(),
bullQueue.getDelayedCount(), bullQueue.getDelayedCount(),
bullQueue.getPausedCount ? bullQueue.getPausedCount() : 0 bullQueue.getPausedCount ? bullQueue.getPausedCount() : 0,
]); ]);
return { return {
@ -252,7 +252,7 @@ export class MonitoringService {
if (queue[methodName] && typeof queue[methodName] === 'function') { if (queue[methodName] && typeof queue[methodName] === 'function') {
try { try {
const result = await queue[methodName](); const result = await queue[methodName]();
return Array.isArray(result) ? result.length : (result || 0); return Array.isArray(result) ? result.length : result || 0;
} catch (_e) { } catch (_e) {
// Continue to next method // Continue to next method
} }
@ -331,12 +331,14 @@ export class MonitoringService {
// Get pool stats // Get pool stats
const pool = (this.container.postgres as any).pool; const pool = (this.container.postgres as any).pool;
const poolStats = pool ? { const poolStats = pool
? {
size: pool.totalCount || 0, size: pool.totalCount || 0,
active: pool.idleCount || 0, active: pool.idleCount || 0,
idle: pool.waitingCount || 0, idle: pool.waitingCount || 0,
max: pool.options?.max || 0, max: pool.options?.max || 0,
} : undefined; }
: undefined;
stats.push({ stats.push({
type: 'postgres', type: 'postgres',
@ -393,7 +395,9 @@ export class MonitoringService {
try { try {
const startTime = Date.now(); const startTime = Date.now();
// QuestDB health check // QuestDB health check
const response = await fetch(`http://${process.env.QUESTDB_HOST || 'localhost'}:9000/exec?query=SELECT%201`); const response = await fetch(
`http://${process.env.QUESTDB_HOST || 'localhost'}:9000/exec?query=SELECT%201`
);
const latency = Date.now() - startTime; const latency = Date.now() - startTime;
stats.push({ stats.push({
@ -447,8 +451,7 @@ export class MonitoringService {
errors.push(`${disconnectedDbs.length} database(s) are disconnected`); errors.push(`${disconnectedDbs.length} database(s) are disconnected`);
} }
const status = errors.length === 0 ? 'healthy' : const status = errors.length === 0 ? 'healthy' : errors.length < 3 ? 'degraded' : 'unhealthy';
errors.length < 3 ? 'degraded' : 'unhealthy';
return { return {
status, status,
@ -637,14 +640,14 @@ export class MonitoringService {
const [cachedProxies, lastUpdateStr] = await Promise.all([ const [cachedProxies, lastUpdateStr] = await Promise.all([
cacheProvider.getRaw<any[]>('cache:proxy:active'), cacheProvider.getRaw<any[]>('cache:proxy:active'),
cacheProvider.getRaw<string>('cache:proxy:last-update') cacheProvider.getRaw<string>('cache:proxy:last-update'),
]); ]);
this.logger.debug('Proxy cache data retrieved', { this.logger.debug('Proxy cache data retrieved', {
hasProxies: !!cachedProxies, hasProxies: !!cachedProxies,
isArray: Array.isArray(cachedProxies), isArray: Array.isArray(cachedProxies),
proxyCount: cachedProxies ? cachedProxies.length : 0, proxyCount: cachedProxies ? cachedProxies.length : 0,
lastUpdate: lastUpdateStr lastUpdate: lastUpdateStr,
}); });
if (cachedProxies && Array.isArray(cachedProxies)) { if (cachedProxies && Array.isArray(cachedProxies)) {
@ -727,7 +730,7 @@ export class MonitoringService {
const idle = totalIdle / cpus.length; const idle = totalIdle / cpus.length;
const total = totalTick / cpus.length; const total = totalTick / cpus.length;
const usage = 100 - ~~(100 * idle / total); const usage = 100 - ~~((100 * idle) / total);
return { return {
usage, usage,

View file

@ -3,4 +3,3 @@ export { Card, CardContent, CardHeader } from './Card';
export { DataTable } from './DataTable'; export { DataTable } from './DataTable';
export { Dialog, DialogContent, DialogHeader, DialogTitle } from './Dialog'; export { Dialog, DialogContent, DialogHeader, DialogTitle } from './Dialog';
export { StatCard } from './StatCard'; export { StatCard } from './StatCard';

View file

@ -1,4 +1,3 @@
export { AddProviderMappingDialog } from './AddProviderMappingDialog'; export { AddProviderMappingDialog } from './AddProviderMappingDialog';
export { AddExchangeDialog } from './AddExchangeDialog'; export { AddExchangeDialog } from './AddExchangeDialog';
export { DeleteExchangeDialog } from './DeleteExchangeDialog'; export { DeleteExchangeDialog } from './DeleteExchangeDialog';

View file

@ -2,16 +2,16 @@
* Custom hook for monitoring data * Custom hook for monitoring data
*/ */
import { useState, useEffect, useCallback } from 'react'; import { useCallback, useEffect, useState } from 'react';
import { monitoringApi } from '../services/monitoringApi'; import { monitoringApi } from '../services/monitoringApi';
import type { import type {
SystemHealth,
CacheStats, CacheStats,
QueueStats,
DatabaseStats, DatabaseStats,
ServiceStatus,
ProxyStats, ProxyStats,
SystemOverview QueueStats,
ServiceStatus,
SystemHealth,
SystemOverview,
} from '../types'; } from '../types';
export function useSystemHealth(refreshInterval: number = 5000) { export function useSystemHealth(refreshInterval: number = 5000) {

View file

@ -3,13 +3,13 @@
*/ */
import type { import type {
SystemHealth,
CacheStats, CacheStats,
QueueStats,
DatabaseStats, DatabaseStats,
ServiceStatus,
ProxyStats, ProxyStats,
SystemOverview QueueStats,
ServiceStatus,
SystemHealth,
SystemOverview,
} from '../types'; } from '../types';
const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || 'http://localhost:2003'; const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || 'http://localhost:2003';

View file

@ -8,9 +8,15 @@ export function formatUptime(ms: number): string {
const hours = Math.floor(minutes / 60); const hours = Math.floor(minutes / 60);
const days = Math.floor(hours / 24); const days = Math.floor(hours / 24);
if (days > 0) {return `${days}d ${hours % 24}h`;} if (days > 0) {
if (hours > 0) {return `${hours}h ${minutes % 60}m`;} return `${days}d ${hours % 24}h`;
if (minutes > 0) {return `${minutes}m ${seconds % 60}s`;} }
if (hours > 0) {
return `${hours}h ${minutes % 60}m`;
}
if (minutes > 0) {
return `${minutes}m ${seconds % 60}s`;
}
return `${seconds}s`; return `${seconds}s`;
} }

View file

@ -12,9 +12,8 @@ export function usePipeline() {
const [error, setError] = useState<string | null>(null); const [error, setError] = useState<string | null>(null);
const [lastJobResult, setLastJobResult] = useState<PipelineJobResult | null>(null); const [lastJobResult, setLastJobResult] = useState<PipelineJobResult | null>(null);
const executeOperation = useCallback(async ( const executeOperation = useCallback(
operation: () => Promise<PipelineJobResult> async (operation: () => Promise<PipelineJobResult>): Promise<boolean> => {
): Promise<boolean> => {
try { try {
setLoading(true); setLoading(true);
setError(null); setError(null);
@ -33,7 +32,9 @@ export function usePipeline() {
} finally { } finally {
setLoading(false); setLoading(false);
} }
}, []); },
[]
);
// Symbol sync operations // Symbol sync operations
const syncQMSymbols = useCallback( const syncQMSymbols = useCallback(
@ -122,7 +123,8 @@ export function usePipeline() {
setError(result.error || 'Failed to get provider mapping stats'); setError(result.error || 'Failed to get provider mapping stats');
return null; return null;
} catch (err) { } catch (err) {
const errorMessage = err instanceof Error ? err.message : 'Failed to get provider mapping stats'; const errorMessage =
err instanceof Error ? err.message : 'Failed to get provider mapping stats';
setError(errorMessage); setError(errorMessage);
return null; return null;
} finally { } finally {

View file

@ -1,16 +1,9 @@
import type { import type { DataClearType, PipelineJobResult, PipelineStatsResult } from '../types';
DataClearType,
PipelineJobResult,
PipelineStatsResult,
} from '../types';
const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || 'http://localhost:2003'; const API_BASE_URL = import.meta.env.VITE_API_BASE_URL || 'http://localhost:2003';
class PipelineApiService { class PipelineApiService {
private async request<T = unknown>( private async request<T = unknown>(endpoint: string, options?: RequestInit): Promise<T> {
endpoint: string,
options?: RequestInit
): Promise<T> {
const url = `${API_BASE_URL}/pipeline${endpoint}`; const url = `${API_BASE_URL}/pipeline${endpoint}`;
const response = await fetch(url, { const response = await fetch(url, {

View file

@ -32,7 +32,6 @@ export interface ProviderMappingStats {
coveragePercentage: number; coveragePercentage: number;
} }
export type DataClearType = 'exchanges' | 'provider_mappings' | 'all'; export type DataClearType = 'exchanges' | 'provider_mappings' | 'all';
export interface PipelineOperation { export interface PipelineOperation {

View file

@ -1,13 +1,13 @@
import { import {
BuildingLibraryIcon, BuildingLibraryIcon,
ChartBarIcon, ChartBarIcon,
ChartPieIcon,
CircleStackIcon,
CogIcon, CogIcon,
DocumentTextIcon, DocumentTextIcon,
HomeIcon, HomeIcon,
PresentationChartLineIcon, PresentationChartLineIcon,
ServerStackIcon, ServerStackIcon,
CircleStackIcon,
ChartPieIcon,
} from '@heroicons/react/24/outline'; } from '@heroicons/react/24/outline';
export interface NavigationItem { export interface NavigationItem {
@ -29,7 +29,7 @@ export const navigation: NavigationItem[] = [
children: [ children: [
{ name: 'Monitoring', href: '/system/monitoring', icon: ChartPieIcon }, { name: 'Monitoring', href: '/system/monitoring', icon: ChartPieIcon },
{ name: 'Pipeline', href: '/system/pipeline', icon: CircleStackIcon }, { name: 'Pipeline', href: '/system/pipeline', icon: CircleStackIcon },
] ],
}, },
{ name: 'Settings', href: '/settings', icon: CogIcon }, { name: 'Settings', href: '/settings', icon: CogIcon },
]; ];

View file

@ -189,6 +189,7 @@
"@stock-bot/browser": "workspace:*", "@stock-bot/browser": "workspace:*",
"@stock-bot/cache": "workspace:*", "@stock-bot/cache": "workspace:*",
"@stock-bot/config": "workspace:*", "@stock-bot/config": "workspace:*",
"@stock-bot/handler-registry": "workspace:*",
"@stock-bot/handlers": "workspace:*", "@stock-bot/handlers": "workspace:*",
"@stock-bot/logger": "workspace:*", "@stock-bot/logger": "workspace:*",
"@stock-bot/mongodb": "workspace:*", "@stock-bot/mongodb": "workspace:*",
@ -199,6 +200,7 @@
"@stock-bot/shutdown": "workspace:*", "@stock-bot/shutdown": "workspace:*",
"@stock-bot/types": "workspace:*", "@stock-bot/types": "workspace:*",
"awilix": "^12.0.5", "awilix": "^12.0.5",
"glob": "^10.0.0",
"hono": "^4.0.0", "hono": "^4.0.0",
"zod": "^3.23.8", "zod": "^3.23.8",
}, },
@ -220,12 +222,24 @@
"typescript": "^5.3.0", "typescript": "^5.3.0",
}, },
}, },
"libs/core/handler-registry": {
"name": "@stock-bot/handler-registry",
"version": "1.0.0",
"dependencies": {
"@stock-bot/types": "workspace:*",
},
"devDependencies": {
"@types/bun": "*",
"typescript": "*",
},
},
"libs/core/handlers": { "libs/core/handlers": {
"name": "@stock-bot/handlers", "name": "@stock-bot/handlers",
"version": "1.0.0", "version": "1.0.0",
"dependencies": { "dependencies": {
"@stock-bot/cache": "workspace:*", "@stock-bot/cache": "workspace:*",
"@stock-bot/config": "workspace:*", "@stock-bot/config": "workspace:*",
"@stock-bot/handler-registry": "workspace:*",
"@stock-bot/logger": "workspace:*", "@stock-bot/logger": "workspace:*",
"@stock-bot/types": "workspace:*", "@stock-bot/types": "workspace:*",
"@stock-bot/utils": "workspace:*", "@stock-bot/utils": "workspace:*",
@ -257,7 +271,7 @@
"version": "1.0.0", "version": "1.0.0",
"dependencies": { "dependencies": {
"@stock-bot/cache": "*", "@stock-bot/cache": "*",
"@stock-bot/handlers": "*", "@stock-bot/handler-registry": "*",
"@stock-bot/logger": "*", "@stock-bot/logger": "*",
"@stock-bot/types": "*", "@stock-bot/types": "*",
"bullmq": "^5.0.0", "bullmq": "^5.0.0",
@ -820,6 +834,8 @@
"@stock-bot/event-bus": ["@stock-bot/event-bus@workspace:libs/core/event-bus"], "@stock-bot/event-bus": ["@stock-bot/event-bus@workspace:libs/core/event-bus"],
"@stock-bot/handler-registry": ["@stock-bot/handler-registry@workspace:libs/core/handler-registry"],
"@stock-bot/handlers": ["@stock-bot/handlers@workspace:libs/core/handlers"], "@stock-bot/handlers": ["@stock-bot/handlers@workspace:libs/core/handlers"],
"@stock-bot/logger": ["@stock-bot/logger@workspace:libs/core/logger"], "@stock-bot/logger": ["@stock-bot/logger@workspace:libs/core/logger"],
@ -2144,7 +2160,7 @@
"side-channel-weakmap": ["side-channel-weakmap@1.0.2", "", { "dependencies": { "call-bound": "^1.0.2", "es-errors": "^1.3.0", "get-intrinsic": "^1.2.5", "object-inspect": "^1.13.3", "side-channel-map": "^1.0.1" } }, "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A=="], "side-channel-weakmap": ["side-channel-weakmap@1.0.2", "", { "dependencies": { "call-bound": "^1.0.2", "es-errors": "^1.3.0", "get-intrinsic": "^1.2.5", "object-inspect": "^1.13.3", "side-channel-map": "^1.0.1" } }, "sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A=="],
"signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="], "signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
"simple-concat": ["simple-concat@1.0.1", "", {}, "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q=="], "simple-concat": ["simple-concat@1.0.1", "", {}, "sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q=="],
@ -2364,7 +2380,7 @@
"word-wrap": ["word-wrap@1.2.5", "", {}, "sha512-BN22B5eaMMI9UMtjrGd5g5eCYPpCPDUy0FJXbYsaT5zYxjFOckS53SQDE3pWkVoWpHXVb3BrYcEN4Twa55B5cA=="], "word-wrap": ["word-wrap@1.2.5", "", {}, "sha512-BN22B5eaMMI9UMtjrGd5g5eCYPpCPDUy0FJXbYsaT5zYxjFOckS53SQDE3pWkVoWpHXVb3BrYcEN4Twa55B5cA=="],
"wrap-ansi": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="], "wrap-ansi": ["wrap-ansi@8.1.0", "", { "dependencies": { "ansi-styles": "^6.1.0", "string-width": "^5.0.1", "strip-ansi": "^7.0.1" } }, "sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ=="],
"wrap-ansi-cjs": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="], "wrap-ansi-cjs": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="],
@ -2430,8 +2446,6 @@
"@isaacs/cliui/strip-ansi": ["strip-ansi@7.1.0", "", { "dependencies": { "ansi-regex": "^6.0.1" } }, "sha512-iq6eVVI64nQQTRYq2KtEg2d2uU7LElhTJwsH4YzIHZshxlgZms/wIc4VoDQTlG/IvVIrBKG06CrZnp0qv7hkcQ=="], "@isaacs/cliui/strip-ansi": ["strip-ansi@7.1.0", "", { "dependencies": { "ansi-regex": "^6.0.1" } }, "sha512-iq6eVVI64nQQTRYq2KtEg2d2uU7LElhTJwsH4YzIHZshxlgZms/wIc4VoDQTlG/IvVIrBKG06CrZnp0qv7hkcQ=="],
"@isaacs/cliui/wrap-ansi": ["wrap-ansi@8.1.0", "", { "dependencies": { "ansi-styles": "^6.1.0", "string-width": "^5.0.1", "strip-ansi": "^7.0.1" } }, "sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ=="],
"@mongodb-js/oidc-plugin/express": ["express@4.21.2", "", { "dependencies": { "accepts": "~1.3.8", "array-flatten": "1.1.1", "body-parser": "1.20.3", "content-disposition": "0.5.4", "content-type": "~1.0.4", "cookie": "0.7.1", "cookie-signature": "1.0.6", "debug": "2.6.9", "depd": "2.0.0", "encodeurl": "~2.0.0", "escape-html": "~1.0.3", "etag": "~1.8.1", "finalhandler": "1.3.1", "fresh": "0.5.2", "http-errors": "2.0.0", "merge-descriptors": "1.0.3", "methods": "~1.1.2", "on-finished": "2.4.1", "parseurl": "~1.3.3", "path-to-regexp": "0.1.12", "proxy-addr": "~2.0.7", "qs": "6.13.0", "range-parser": "~1.2.1", "safe-buffer": "5.2.1", "send": "0.19.0", "serve-static": "1.16.2", "setprototypeof": "1.2.0", "statuses": "2.0.1", "type-is": "~1.6.18", "utils-merge": "1.0.1", "vary": "~1.1.2" } }, "sha512-28HqgMZAmih1Czt9ny7qr6ek2qddF4FclbMzwhCREB6OFfH+rXAnuNCwo1/wFvrtbgsQDb4kSbX9de9lFbrXnA=="], "@mongodb-js/oidc-plugin/express": ["express@4.21.2", "", { "dependencies": { "accepts": "~1.3.8", "array-flatten": "1.1.1", "body-parser": "1.20.3", "content-disposition": "0.5.4", "content-type": "~1.0.4", "cookie": "0.7.1", "cookie-signature": "1.0.6", "debug": "2.6.9", "depd": "2.0.0", "encodeurl": "~2.0.0", "escape-html": "~1.0.3", "etag": "~1.8.1", "finalhandler": "1.3.1", "fresh": "0.5.2", "http-errors": "2.0.0", "merge-descriptors": "1.0.3", "methods": "~1.1.2", "on-finished": "2.4.1", "parseurl": "~1.3.3", "path-to-regexp": "0.1.12", "proxy-addr": "~2.0.7", "qs": "6.13.0", "range-parser": "~1.2.1", "safe-buffer": "5.2.1", "send": "0.19.0", "serve-static": "1.16.2", "setprototypeof": "1.2.0", "statuses": "2.0.1", "type-is": "~1.6.18", "utils-merge": "1.0.1", "vary": "~1.1.2" } }, "sha512-28HqgMZAmih1Czt9ny7qr6ek2qddF4FclbMzwhCREB6OFfH+rXAnuNCwo1/wFvrtbgsQDb4kSbX9de9lFbrXnA=="],
"@pm2/agent/chalk": ["chalk@3.0.0", "", { "dependencies": { "ansi-styles": "^4.1.0", "supports-color": "^7.1.0" } }, "sha512-4D3B6Wf41KOYRFdszmDqMCGq5VV/uMAB273JILmO+3jAlh8X4qDtdtgCR3fxtbLEMzSx22QdhnDcJvu2u1fVwg=="], "@pm2/agent/chalk": ["chalk@3.0.0", "", { "dependencies": { "ansi-styles": "^4.1.0", "supports-color": "^7.1.0" } }, "sha512-4D3B6Wf41KOYRFdszmDqMCGq5VV/uMAB273JILmO+3jAlh8X4qDtdtgCR3fxtbLEMzSx22QdhnDcJvu2u1fVwg=="],
@ -2450,6 +2464,8 @@
"@pm2/io/semver": ["semver@7.5.4", "", { "dependencies": { "lru-cache": "^6.0.0" }, "bin": { "semver": "bin/semver.js" } }, "sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA=="], "@pm2/io/semver": ["semver@7.5.4", "", { "dependencies": { "lru-cache": "^6.0.0" }, "bin": { "semver": "bin/semver.js" } }, "sha512-1bCSESV6Pv+i21Hvpxp3Dx+pSD8lIPt8uVjRrxAUt/nbswYc+tK6Y2btiULjd4+fnq15PX+nqQDC7Oft7WkwcA=="],
"@pm2/io/signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
"@pm2/io/tslib": ["tslib@1.9.3", "", {}, "sha512-4krF8scpejhaOgqzBEcGM7yDIEfi0/8+8zDRZhNZZ2kjmHJ4hv3zCbQWxoJGz1iw5U0Jl0nma13xzHXcncMavQ=="], "@pm2/io/tslib": ["tslib@1.9.3", "", {}, "sha512-4krF8scpejhaOgqzBEcGM7yDIEfi0/8+8zDRZhNZZ2kjmHJ4hv3zCbQWxoJGz1iw5U0Jl0nma13xzHXcncMavQ=="],
"@pm2/js-api/async": ["async@2.6.4", "", { "dependencies": { "lodash": "^4.17.14" } }, "sha512-mzo5dfJYwAn29PeiJ0zvwTo04zj8HDJj0Mn8TD7sno7q12prdbnasKJHhkm2c1LgrhlJ0teaea8860oxi51mGA=="], "@pm2/js-api/async": ["async@2.6.4", "", { "dependencies": { "lodash": "^4.17.14" } }, "sha512-mzo5dfJYwAn29PeiJ0zvwTo04zj8HDJj0Mn8TD7sno7q12prdbnasKJHhkm2c1LgrhlJ0teaea8860oxi51mGA=="],
@ -2516,6 +2532,8 @@
"cli-tableau/chalk": ["chalk@3.0.0", "", { "dependencies": { "ansi-styles": "^4.1.0", "supports-color": "^7.1.0" } }, "sha512-4D3B6Wf41KOYRFdszmDqMCGq5VV/uMAB273JILmO+3jAlh8X4qDtdtgCR3fxtbLEMzSx22QdhnDcJvu2u1fVwg=="], "cli-tableau/chalk": ["chalk@3.0.0", "", { "dependencies": { "ansi-styles": "^4.1.0", "supports-color": "^7.1.0" } }, "sha512-4D3B6Wf41KOYRFdszmDqMCGq5VV/uMAB273JILmO+3jAlh8X4qDtdtgCR3fxtbLEMzSx22QdhnDcJvu2u1fVwg=="],
"cliui/wrap-ansi": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="],
"compress-commons/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="], "compress-commons/is-stream": ["is-stream@2.0.1", "", {}, "sha512-hFoiJiTl63nn+kstHGBtewWSKnQLpyb155KHheA1l39uvtO9nWIop1p3udqPcUd/xbF1VLMO4n7OI6p7RbngDg=="],
"decompress-response/mimic-response": ["mimic-response@3.1.0", "", {}, "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ=="], "decompress-response/mimic-response": ["mimic-response@3.1.0", "", {}, "sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ=="],
@ -2550,14 +2568,14 @@
"execa/is-stream": ["is-stream@3.0.0", "", {}, "sha512-LnQR4bZ9IADDRSkvpqMGvt/tEJWclzklNgSw48V5EAaAeDd6qGvN8ei6k5p0tvxSR171VmGyHuTiAOfxAbr8kA=="], "execa/is-stream": ["is-stream@3.0.0", "", {}, "sha512-LnQR4bZ9IADDRSkvpqMGvt/tEJWclzklNgSw48V5EAaAeDd6qGvN8ei6k5p0tvxSR171VmGyHuTiAOfxAbr8kA=="],
"execa/signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
"express/cookie": ["cookie@0.7.2", "", {}, "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w=="], "express/cookie": ["cookie@0.7.2", "", {}, "sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w=="],
"express/mime-types": ["mime-types@3.0.1", "", { "dependencies": { "mime-db": "^1.54.0" } }, "sha512-xRc4oEhT6eaBpU1XF7AjpOFD+xQmXNB5OVKwp4tqCuBpHLS/ZbBDrc07mYTDqVMg6PfxUjjNp85O6Cd2Z/5HWA=="], "express/mime-types": ["mime-types@3.0.1", "", { "dependencies": { "mime-db": "^1.54.0" } }, "sha512-xRc4oEhT6eaBpU1XF7AjpOFD+xQmXNB5OVKwp4tqCuBpHLS/ZbBDrc07mYTDqVMg6PfxUjjNp85O6Cd2Z/5HWA=="],
"fast-glob/glob-parent": ["glob-parent@5.1.2", "", { "dependencies": { "is-glob": "^4.0.1" } }, "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow=="], "fast-glob/glob-parent": ["glob-parent@5.1.2", "", { "dependencies": { "is-glob": "^4.0.1" } }, "sha512-AOIgSQCepiJYwP3ARnGx+5VnTu2HBYdzbGP45eLw1vr3zB3vZLeyed1sC9hnbcOc9/SrMyM5RPQrkGz4aS9Zow=="],
"foreground-child/signal-exit": ["signal-exit@4.1.0", "", {}, "sha512-bzyZ1e88w9O1iNJbKnOlvYTrWPDl46O1bG0D3XInv+9tkPrxrN8jUUTiFlDkkmKWgn1M6CfIA13SuGqOa9Korw=="],
"get-uri/data-uri-to-buffer": ["data-uri-to-buffer@6.0.2", "", {}, "sha512-7hvf7/GW8e86rW0ptuwS3OcBGDjIi6SZva7hCyWC0yYry2cOPmLIjXAUHI6DK2HsnwJd9ifmt57i8eV2n4YNpw=="], "get-uri/data-uri-to-buffer": ["data-uri-to-buffer@6.0.2", "", {}, "sha512-7hvf7/GW8e86rW0ptuwS3OcBGDjIi6SZva7hCyWC0yYry2cOPmLIjXAUHI6DK2HsnwJd9ifmt57i8eV2n4YNpw=="],
"glob/minimatch": ["minimatch@9.0.5", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow=="], "glob/minimatch": ["minimatch@9.0.5", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow=="],
@ -2620,6 +2638,8 @@
"prebuild-install/tar-fs": ["tar-fs@2.1.3", "", { "dependencies": { "chownr": "^1.1.1", "mkdirp-classic": "^0.5.2", "pump": "^3.0.0", "tar-stream": "^2.1.4" } }, "sha512-090nwYJDmlhwFwEW3QQl+vaNnxsO2yVsd45eTKRBzSzu+hlb1w2K9inVq5b0ngXuLVqQ4ApvsUHHnu/zQNkWAg=="], "prebuild-install/tar-fs": ["tar-fs@2.1.3", "", { "dependencies": { "chownr": "^1.1.1", "mkdirp-classic": "^0.5.2", "pump": "^3.0.0", "tar-stream": "^2.1.4" } }, "sha512-090nwYJDmlhwFwEW3QQl+vaNnxsO2yVsd45eTKRBzSzu+hlb1w2K9inVq5b0ngXuLVqQ4ApvsUHHnu/zQNkWAg=="],
"proper-lockfile/signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
"protobufjs/@types/node": ["@types/node@22.15.32", "", { "dependencies": { "undici-types": "~6.21.0" } }, "sha512-3jigKqgSjsH6gYZv2nEsqdXfZqIFGAV36XYYjf9KGZ3PSG+IhLecqPnI310RvjutyMwifE2hhhNEklOUrvx/wA=="], "protobufjs/@types/node": ["@types/node@22.15.32", "", { "dependencies": { "undici-types": "~6.21.0" } }, "sha512-3jigKqgSjsH6gYZv2nEsqdXfZqIFGAV36XYYjf9KGZ3PSG+IhLecqPnI310RvjutyMwifE2hhhNEklOUrvx/wA=="],
"proxy-agent/lru-cache": ["lru-cache@7.18.3", "", {}, "sha512-jumlc0BIUrS3qJGgIkWZsyfAM7NCWiBcCDhnd+3NNM5KbBmLTgHVfWBcg6W+rLUsIpzpERPsvwUP7CckAQSOoA=="], "proxy-agent/lru-cache": ["lru-cache@7.18.3", "", {}, "sha512-jumlc0BIUrS3qJGgIkWZsyfAM7NCWiBcCDhnd+3NNM5KbBmLTgHVfWBcg6W+rLUsIpzpERPsvwUP7CckAQSOoA=="],
@ -2648,6 +2668,12 @@
"win-export-certificate-and-key/node-addon-api": ["node-addon-api@3.2.1", "", {}, "sha512-mmcei9JghVNDYydghQmeDX8KoAm0FAiYyIcUt/N4nhyAipB17pllZQDOJD2fotxABnt4Mdz+dKTO7eftLg4d0A=="], "win-export-certificate-and-key/node-addon-api": ["node-addon-api@3.2.1", "", {}, "sha512-mmcei9JghVNDYydghQmeDX8KoAm0FAiYyIcUt/N4nhyAipB17pllZQDOJD2fotxABnt4Mdz+dKTO7eftLg4d0A=="],
"wrap-ansi/ansi-styles": ["ansi-styles@6.2.1", "", {}, "sha512-bN798gFfQX+viw3R7yrGWRqnrN2oRkEkUjjl4JNn4E8GxxbjtG3FbrEIIY3l8/hrwUwIeCZvi4QuOTP4MErVug=="],
"wrap-ansi/string-width": ["string-width@5.1.2", "", { "dependencies": { "eastasianwidth": "^0.2.0", "emoji-regex": "^9.2.2", "strip-ansi": "^7.0.1" } }, "sha512-HnLOCR3vjcY8beoNLtcjZ5/nxn2afmME6lhrDrebokqMap+XbeW8n9TXpPDOqdGK5qcI3oT0GKTW6wC7EMiVqA=="],
"wrap-ansi/strip-ansi": ["strip-ansi@7.1.0", "", { "dependencies": { "ansi-regex": "^6.0.1" } }, "sha512-iq6eVVI64nQQTRYq2KtEg2d2uU7LElhTJwsH4YzIHZshxlgZms/wIc4VoDQTlG/IvVIrBKG06CrZnp0qv7hkcQ=="],
"yauzl/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="], "yauzl/buffer-crc32": ["buffer-crc32@0.2.13", "", {}, "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ=="],
"@aws-crypto/sha256-browser/@smithy/util-utf8/@smithy/util-buffer-from": ["@smithy/util-buffer-from@2.2.0", "", { "dependencies": { "@smithy/is-array-buffer": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA=="], "@aws-crypto/sha256-browser/@smithy/util-utf8/@smithy/util-buffer-from": ["@smithy/util-buffer-from@2.2.0", "", { "dependencies": { "@smithy/is-array-buffer": "^2.2.0", "tslib": "^2.6.2" } }, "sha512-IJdWBbTcMQ6DA0gdNhh/BwrLkDR+ADW5Kr1aZmd4k3DIF6ezMV4R2NIAmT08wQJ3yUK82thHWmC/TnK/wpMMIA=="],
@ -2660,8 +2686,6 @@
"@isaacs/cliui/strip-ansi/ansi-regex": ["ansi-regex@6.1.0", "", {}, "sha512-7HSX4QQb4CspciLpVFwyRe79O3xsIZDDLER21kERQ71oaPodF8jL725AgJMFAYbooIqolJoRLuM81SpeUkpkvA=="], "@isaacs/cliui/strip-ansi/ansi-regex": ["ansi-regex@6.1.0", "", {}, "sha512-7HSX4QQb4CspciLpVFwyRe79O3xsIZDDLER21kERQ71oaPodF8jL725AgJMFAYbooIqolJoRLuM81SpeUkpkvA=="],
"@isaacs/cliui/wrap-ansi/ansi-styles": ["ansi-styles@6.2.1", "", {}, "sha512-bN798gFfQX+viw3R7yrGWRqnrN2oRkEkUjjl4JNn4E8GxxbjtG3FbrEIIY3l8/hrwUwIeCZvi4QuOTP4MErVug=="],
"@mongodb-js/oidc-plugin/express/accepts": ["accepts@1.3.8", "", { "dependencies": { "mime-types": "~2.1.34", "negotiator": "0.6.3" } }, "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw=="], "@mongodb-js/oidc-plugin/express/accepts": ["accepts@1.3.8", "", { "dependencies": { "mime-types": "~2.1.34", "negotiator": "0.6.3" } }, "sha512-PYAthTa2m2VKxuvSD3DPC/Gy+U+sOA1LAuT8mkmRuvw+NACSaeXEQ+NHcVF7rONl6qcaxV3Uuemwawk+7+SJLw=="],
"@mongodb-js/oidc-plugin/express/body-parser": ["body-parser@1.20.3", "", { "dependencies": { "bytes": "3.1.2", "content-type": "~1.0.5", "debug": "2.6.9", "depd": "2.0.0", "destroy": "1.2.0", "http-errors": "2.0.0", "iconv-lite": "0.4.24", "on-finished": "2.4.1", "qs": "6.13.0", "raw-body": "2.5.2", "type-is": "~1.6.18", "unpipe": "1.0.0" } }, "sha512-7rAxByjUMqQ3/bHJy7D6OGXvx/MMc4IqBn/X0fcM1QUcAItpZrBEYhWGem+tzXH90c+G01ypMcYJBO9Y30203g=="], "@mongodb-js/oidc-plugin/express/body-parser": ["body-parser@1.20.3", "", { "dependencies": { "bytes": "3.1.2", "content-type": "~1.0.5", "debug": "2.6.9", "depd": "2.0.0", "destroy": "1.2.0", "http-errors": "2.0.0", "iconv-lite": "0.4.24", "on-finished": "2.4.1", "qs": "6.13.0", "raw-body": "2.5.2", "type-is": "~1.6.18", "unpipe": "1.0.0" } }, "sha512-7rAxByjUMqQ3/bHJy7D6OGXvx/MMc4IqBn/X0fcM1QUcAItpZrBEYhWGem+tzXH90c+G01ypMcYJBO9Y30203g=="],
@ -2880,12 +2904,18 @@
"run-applescript/execa/onetime": ["onetime@5.1.2", "", { "dependencies": { "mimic-fn": "^2.1.0" } }, "sha512-kbpaSSGJTWdAY5KPVeMOKXSrPtr8C8C7wodJbcsd51jRnmD+GZu8Y0VoU6Dm5Z4vWr0Ig/1NKuWRKf7j5aaYSg=="], "run-applescript/execa/onetime": ["onetime@5.1.2", "", { "dependencies": { "mimic-fn": "^2.1.0" } }, "sha512-kbpaSSGJTWdAY5KPVeMOKXSrPtr8C8C7wodJbcsd51jRnmD+GZu8Y0VoU6Dm5Z4vWr0Ig/1NKuWRKf7j5aaYSg=="],
"run-applescript/execa/signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="],
"run-applescript/execa/strip-final-newline": ["strip-final-newline@2.0.0", "", {}, "sha512-BrpvfNAE3dcvq7ll3xVumzjKjZQ5tI1sEUIKr3Uoks0XUl45St3FlatVqef9prk4jRDzhW6WZg+3bk93y6pLjA=="], "run-applescript/execa/strip-final-newline": ["strip-final-newline@2.0.0", "", {}, "sha512-BrpvfNAE3dcvq7ll3xVumzjKjZQ5tI1sEUIKr3Uoks0XUl45St3FlatVqef9prk4jRDzhW6WZg+3bk93y6pLjA=="],
"send/mime-types/mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="], "send/mime-types/mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="],
"type-is/mime-types/mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="], "type-is/mime-types/mime-db": ["mime-db@1.54.0", "", {}, "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ=="],
"wrap-ansi/string-width/emoji-regex": ["emoji-regex@9.2.2", "", {}, "sha512-L18DaJsXSUk2+42pv8mLs5jJT2hqFkFE4j21wOmgbUqsZ2hL72NsUU785g9RXgo3s0ZNgVl42TiHp3ZtOv/Vyg=="],
"wrap-ansi/strip-ansi/ansi-regex": ["ansi-regex@6.1.0", "", {}, "sha512-7HSX4QQb4CspciLpVFwyRe79O3xsIZDDLER21kERQ71oaPodF8jL725AgJMFAYbooIqolJoRLuM81SpeUkpkvA=="],
"@aws-crypto/sha256-browser/@smithy/util-utf8/@smithy/util-buffer-from/@smithy/is-array-buffer": ["@smithy/is-array-buffer@2.2.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA=="], "@aws-crypto/sha256-browser/@smithy/util-utf8/@smithy/util-buffer-from/@smithy/is-array-buffer": ["@smithy/is-array-buffer@2.2.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA=="],
"@aws-crypto/util/@smithy/util-utf8/@smithy/util-buffer-from/@smithy/is-array-buffer": ["@smithy/is-array-buffer@2.2.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA=="], "@aws-crypto/util/@smithy/util-utf8/@smithy/util-buffer-from/@smithy/is-array-buffer": ["@smithy/is-array-buffer@2.2.0", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-GGP3O9QFD24uGeAXYUjwSTXARoqpZykHadOmA8G5vfJPK0/DC67qa//0qvqrJzL1xc8WQWX7/yc7fwudjPHPhA=="],

View file

@ -57,7 +57,6 @@ export class NamespacedCache implements CacheProvider {
} }
} }
getStats() { getStats() {
return this.cache.getStats(); return this.cache.getStats();
} }

View file

@ -1,9 +1,9 @@
import { join } from 'path'; import { join } from 'path';
import { z } from 'zod'; import { z } from 'zod';
import { getLogger } from '@stock-bot/logger';
import { EnvLoader } from './loaders/env.loader'; import { EnvLoader } from './loaders/env.loader';
import { FileLoader } from './loaders/file.loader'; import { FileLoader } from './loaders/file.loader';
import { ConfigError, ConfigValidationError } from './errors'; import { ConfigError, ConfigValidationError } from './errors';
import { getLogger } from '@stock-bot/logger';
import type { import type {
ConfigLoader, ConfigLoader,
ConfigManagerOptions, ConfigManagerOptions,

View file

@ -1,10 +1,10 @@
// Import necessary types // Import necessary types
import { z } from 'zod';
import { EnvLoader } from './loaders/env.loader'; import { EnvLoader } from './loaders/env.loader';
import { FileLoader } from './loaders/file.loader'; import { FileLoader } from './loaders/file.loader';
import { ConfigManager } from './config-manager'; import { ConfigManager } from './config-manager';
import type { BaseAppConfig } from './schemas'; import type { BaseAppConfig } from './schemas';
import { baseAppSchema } from './schemas'; import { baseAppSchema } from './schemas';
import { z } from 'zod';
// Legacy singleton instance for backward compatibility // Legacy singleton instance for backward compatibility
let configInstance: ConfigManager<BaseAppConfig> | null = null; let configInstance: ConfigManager<BaseAppConfig> | null = null;
@ -56,7 +56,6 @@ function loadCriticalEnvVarsSync(): void {
// Load critical env vars immediately // Load critical env vars immediately
loadCriticalEnvVarsSync(); loadCriticalEnvVarsSync();
/** /**
* Initialize configuration for a service in a monorepo. * Initialize configuration for a service in a monorepo.
* Automatically loads configs from: * Automatically loads configs from:
@ -121,8 +120,6 @@ export function getLogConfig() {
return getConfig().log; return getConfig().log;
} }
export function getQueueConfig() { export function getQueueConfig() {
return getConfig().queue; return getConfig().queue;
} }

View file

@ -1,5 +1,5 @@
import { describe, expect, it } from 'bun:test'; import { describe, expect, it } from 'bun:test';
import { unifiedAppSchema, toUnifiedConfig, getStandardServiceName } from '../unified-app.schema'; import { getStandardServiceName, toUnifiedConfig, unifiedAppSchema } from '../unified-app.schema';
describe('UnifiedAppConfig', () => { describe('UnifiedAppConfig', () => {
describe('getStandardServiceName', () => { describe('getStandardServiceName', () => {

View file

@ -1,19 +1,19 @@
import { z } from 'zod'; import { z } from 'zod';
import { environmentSchema } from './base.schema'; import { environmentSchema } from './base.schema';
import { import {
postgresConfigSchema, dragonflyConfigSchema,
mongodbConfigSchema, mongodbConfigSchema,
postgresConfigSchema,
questdbConfigSchema, questdbConfigSchema,
dragonflyConfigSchema
} from './database.schema'; } from './database.schema';
import { import {
serviceConfigSchema,
loggingConfigSchema,
queueConfigSchema,
httpConfigSchema,
webshareConfigSchema,
browserConfigSchema, browserConfigSchema,
proxyConfigSchema httpConfigSchema,
loggingConfigSchema,
proxyConfigSchema,
queueConfigSchema,
serviceConfigSchema,
webshareConfigSchema,
} from './service.schema'; } from './service.schema';
/** /**
@ -32,12 +32,14 @@ export const baseAppSchema = z.object({
log: loggingConfigSchema, log: loggingConfigSchema,
// Database configuration - apps can choose which databases they need // Database configuration - apps can choose which databases they need
database: z.object({ database: z
.object({
postgres: postgresConfigSchema.optional(), postgres: postgresConfigSchema.optional(),
mongodb: mongodbConfigSchema.optional(), mongodb: mongodbConfigSchema.optional(),
questdb: questdbConfigSchema.optional(), questdb: questdbConfigSchema.optional(),
dragonfly: dragonflyConfigSchema.optional(), dragonfly: dragonflyConfigSchema.optional(),
}).optional(), })
.optional(),
// Redis configuration (used for cache and queue) // Redis configuration (used for cache and queue)
redis: dragonflyConfigSchema.optional(), redis: dragonflyConfigSchema.optional(),

View file

@ -15,4 +15,3 @@ export type { BaseAppConfig } from './base-app.schema';
// Export unified schema for standardized configuration // Export unified schema for standardized configuration
export { unifiedAppSchema, toUnifiedConfig, getStandardServiceName } from './unified-app.schema'; export { unifiedAppSchema, toUnifiedConfig, getStandardServiceName } from './unified-app.schema';
export type { UnifiedAppConfig } from './unified-app.schema'; export type { UnifiedAppConfig } from './unified-app.schema';

View file

@ -100,8 +100,10 @@ export const proxyConfigSchema = z.object({
enabled: z.boolean().default(false), enabled: z.boolean().default(false),
cachePrefix: z.string().default('proxy:'), cachePrefix: z.string().default('proxy:'),
ttl: z.number().default(3600), ttl: z.number().default(3600),
webshare: z.object({ webshare: z
.object({
apiKey: z.string(), apiKey: z.string(),
apiUrl: z.string().default('https://proxy.webshare.io/api/v2/'), apiUrl: z.string().default('https://proxy.webshare.io/api/v2/'),
}).optional(), })
.optional(),
}); });

View file

@ -1,26 +1,31 @@
import { z } from 'zod'; import { z } from 'zod';
import { baseAppSchema } from './base-app.schema'; import { baseAppSchema } from './base-app.schema';
import { import {
postgresConfigSchema, dragonflyConfigSchema,
mongodbConfigSchema, mongodbConfigSchema,
postgresConfigSchema,
questdbConfigSchema, questdbConfigSchema,
dragonflyConfigSchema
} from './database.schema'; } from './database.schema';
/** /**
* Unified application configuration schema that provides both nested and flat access * Unified application configuration schema that provides both nested and flat access
* to database configurations for backward compatibility while maintaining a clean structure * to database configurations for backward compatibility while maintaining a clean structure
*/ */
export const unifiedAppSchema = baseAppSchema.extend({ export const unifiedAppSchema = baseAppSchema
.extend({
// Flat database configs for DI system (these take precedence) // Flat database configs for DI system (these take precedence)
redis: dragonflyConfigSchema.optional(), redis: dragonflyConfigSchema.optional(),
mongodb: mongodbConfigSchema.optional(), mongodb: mongodbConfigSchema.optional(),
postgres: postgresConfigSchema.optional(), postgres: postgresConfigSchema.optional(),
questdb: questdbConfigSchema.optional(), questdb: questdbConfigSchema.optional(),
}).transform((data) => { })
.transform(data => {
// Ensure service.serviceName is set from service.name if not provided // Ensure service.serviceName is set from service.name if not provided
if (data.service && !data.service.serviceName) { if (data.service && !data.service.serviceName) {
data.service.serviceName = data.service.name.replace(/([A-Z])/g, '-$1').toLowerCase().replace(/^-/, ''); data.service.serviceName = data.service.name
.replace(/([A-Z])/g, '-$1')
.toLowerCase()
.replace(/^-/, '');
} }
// If flat configs exist, ensure they're also in the nested database object // If flat configs exist, ensure they're also in the nested database object
@ -56,7 +61,7 @@ export const unifiedAppSchema = baseAppSchema.extend({
} }
return data; return data;
}); });
export type UnifiedAppConfig = z.infer<typeof unifiedAppSchema>; export type UnifiedAppConfig = z.infer<typeof unifiedAppSchema>;
@ -72,5 +77,8 @@ export function toUnifiedConfig(config: any): UnifiedAppConfig {
*/ */
export function getStandardServiceName(serviceName: string): string { export function getStandardServiceName(serviceName: string): string {
// Convert camelCase to kebab-case // Convert camelCase to kebab-case
return serviceName.replace(/([A-Z])/g, '-$1').toLowerCase().replace(/^-/, ''); return serviceName
.replace(/([A-Z])/g, '-$1')
.toLowerCase()
.replace(/^-/, '');
} }

View file

@ -20,6 +20,8 @@
"@stock-bot/queue": "workspace:*", "@stock-bot/queue": "workspace:*",
"@stock-bot/shutdown": "workspace:*", "@stock-bot/shutdown": "workspace:*",
"@stock-bot/handlers": "workspace:*", "@stock-bot/handlers": "workspace:*",
"@stock-bot/handler-registry": "workspace:*",
"glob": "^10.0.0",
"zod": "^3.23.8", "zod": "^3.23.8",
"hono": "^4.0.0", "hono": "^4.0.0",
"awilix": "^12.0.5" "awilix": "^12.0.5"

View file

@ -3,16 +3,16 @@
* Creates a decoupled, reusable dependency injection container * Creates a decoupled, reusable dependency injection container
*/ */
import { type AwilixContainer } from 'awilix';
import type { Browser } from '@stock-bot/browser'; import type { Browser } from '@stock-bot/browser';
import type { CacheProvider } from '@stock-bot/cache'; import type { CacheProvider } from '@stock-bot/cache';
import type { IServiceContainer } from '@stock-bot/types';
import type { Logger } from '@stock-bot/logger'; import type { Logger } from '@stock-bot/logger';
import type { MongoDBClient } from '@stock-bot/mongodb'; import type { MongoDBClient } from '@stock-bot/mongodb';
import type { PostgreSQLClient } from '@stock-bot/postgres'; import type { PostgreSQLClient } from '@stock-bot/postgres';
import type { ProxyManager } from '@stock-bot/proxy'; import type { ProxyManager } from '@stock-bot/proxy';
import type { QuestDBClient } from '@stock-bot/questdb'; import type { QuestDBClient } from '@stock-bot/questdb';
import type { QueueManager } from '@stock-bot/queue'; import type { QueueManager } from '@stock-bot/queue';
import { type AwilixContainer } from 'awilix'; import type { IServiceContainer } from '@stock-bot/types';
import type { AppConfig } from './config/schemas'; import type { AppConfig } from './config/schemas';
// Re-export for backward compatibility // Re-export for backward compatibility
@ -41,8 +41,6 @@ export interface ServiceDefinitions {
serviceContainer: IServiceContainer; serviceContainer: IServiceContainer;
} }
// Export typed container // Export typed container
export type ServiceContainer = AwilixContainer<ServiceDefinitions>; export type ServiceContainer = AwilixContainer<ServiceDefinitions>;
export type ServiceCradle = ServiceDefinitions; export type ServiceCradle = ServiceDefinitions;
@ -59,5 +57,3 @@ export interface ServiceContainerOptions {
enableBrowser?: boolean; enableBrowser?: boolean;
enableProxy?: boolean; enableProxy?: boolean;
} }

View file

@ -1,9 +1,9 @@
import { z } from 'zod'; import { z } from 'zod';
import { redisConfigSchema } from './redis.schema';
import { mongodbConfigSchema } from './mongodb.schema'; import { mongodbConfigSchema } from './mongodb.schema';
import { postgresConfigSchema } from './postgres.schema'; import { postgresConfigSchema } from './postgres.schema';
import { questdbConfigSchema } from './questdb.schema'; import { questdbConfigSchema } from './questdb.schema';
import { proxyConfigSchema, browserConfigSchema, queueConfigSchema } from './service.schema'; import { redisConfigSchema } from './redis.schema';
import { browserConfigSchema, proxyConfigSchema, queueConfigSchema } from './service.schema';
export const appConfigSchema = z.object({ export const appConfigSchema = z.object({
redis: redisConfigSchema, redis: redisConfigSchema,
@ -13,11 +13,13 @@ export const appConfigSchema = z.object({
proxy: proxyConfigSchema.optional(), proxy: proxyConfigSchema.optional(),
browser: browserConfigSchema.optional(), browser: browserConfigSchema.optional(),
queue: queueConfigSchema.optional(), queue: queueConfigSchema.optional(),
service: z.object({ service: z
.object({
name: z.string(), name: z.string(),
serviceName: z.string().optional(), // Standard kebab-case service name serviceName: z.string().optional(), // Standard kebab-case service name
port: z.number().optional(), port: z.number().optional(),
}).optional(), })
.optional(),
}); });
export type AppConfig = z.infer<typeof appConfigSchema>; export type AppConfig = z.infer<typeof appConfigSchema>;

View file

@ -4,10 +4,12 @@ export const proxyConfigSchema = z.object({
enabled: z.boolean().default(false), enabled: z.boolean().default(false),
cachePrefix: z.string().optional().default('proxy:'), cachePrefix: z.string().optional().default('proxy:'),
ttl: z.number().optional().default(3600), ttl: z.number().optional().default(3600),
webshare: z.object({ webshare: z
.object({
apiKey: z.string(), apiKey: z.string(),
apiUrl: z.string().default('https://proxy.webshare.io/api/v2/'), apiUrl: z.string().default('https://proxy.webshare.io/api/v2/'),
}).optional(), })
.optional(),
}); });
export const browserConfigSchema = z.object({ export const browserConfigSchema = z.object({
@ -21,16 +23,21 @@ export const queueConfigSchema = z.object({
concurrency: z.number().optional().default(1), concurrency: z.number().optional().default(1),
enableScheduledJobs: z.boolean().optional().default(true), enableScheduledJobs: z.boolean().optional().default(true),
delayWorkerStart: z.boolean().optional().default(false), delayWorkerStart: z.boolean().optional().default(false),
defaultJobOptions: z.object({ defaultJobOptions: z
.object({
attempts: z.number().default(3), attempts: z.number().default(3),
backoff: z.object({ backoff: z
.object({
type: z.enum(['exponential', 'fixed']).default('exponential'), type: z.enum(['exponential', 'fixed']).default('exponential'),
delay: z.number().default(1000), delay: z.number().default(1000),
}).default({}), })
.default({}),
removeOnComplete: z.number().default(100), removeOnComplete: z.number().default(100),
removeOnFail: z.number().default(50), removeOnFail: z.number().default(50),
timeout: z.number().optional(), timeout: z.number().optional(),
}).optional().default({}), })
.optional()
.default({}),
}); });
export type ProxyConfig = z.infer<typeof proxyConfigSchema>; export type ProxyConfig = z.infer<typeof proxyConfigSchema>;

View file

@ -1,15 +1,17 @@
import { createContainer, InjectionMode, asFunction, type AwilixContainer } from 'awilix'; import { asClass, asFunction, createContainer, InjectionMode, type AwilixContainer } from 'awilix';
import type { BaseAppConfig as StockBotAppConfig, UnifiedAppConfig } from '@stock-bot/config'; import type { BaseAppConfig as StockBotAppConfig, UnifiedAppConfig } from '@stock-bot/config';
import { appConfigSchema, type AppConfig } from '../config/schemas';
import { toUnifiedConfig } from '@stock-bot/config'; import { toUnifiedConfig } from '@stock-bot/config';
import { HandlerRegistry } from '@stock-bot/handler-registry';
import { appConfigSchema, type AppConfig } from '../config/schemas';
import { import {
registerCoreServices, registerApplicationServices,
registerCacheServices, registerCacheServices,
registerCoreServices,
registerDatabaseServices, registerDatabaseServices,
registerApplicationServices
} from '../registrations'; } from '../registrations';
import { HandlerScanner } from '../scanner';
import { ServiceLifecycleManager } from '../utils/lifecycle'; import { ServiceLifecycleManager } from '../utils/lifecycle';
import type { ServiceDefinitions, ContainerBuildOptions } from './types'; import type { ContainerBuildOptions, ServiceDefinitions } from './types';
export class ServiceContainerBuilder { export class ServiceContainerBuilder {
private config: Partial<AppConfig> = {}; private config: Partial<AppConfig> = {};
@ -38,7 +40,10 @@ export class ServiceContainerBuilder {
return this; return this;
} }
enableService(service: keyof Omit<ContainerBuildOptions, 'skipInitialization' | 'initializationTimeout'>, enabled = true): this { enableService(
service: keyof Omit<ContainerBuildOptions, 'skipInitialization' | 'initializationTimeout'>,
enabled = true
): this {
this.options[service] = enabled; this.options[service] = enabled;
return this; return this;
} }
@ -77,10 +82,12 @@ export class ServiceContainerBuilder {
private applyServiceOptions(config: Partial<AppConfig>): AppConfig { private applyServiceOptions(config: Partial<AppConfig>): AppConfig {
// Ensure questdb config has the right field names for DI // Ensure questdb config has the right field names for DI
const questdbConfig = config.questdb ? { const questdbConfig = config.questdb
? {
...config.questdb, ...config.questdb,
influxPort: (config.questdb as any).influxPort || (config.questdb as any).ilpPort || 9009, influxPort: (config.questdb as any).influxPort || (config.questdb as any).ilpPort || 9009,
} : { }
: {
enabled: true, enabled: true,
host: 'localhost', host: 'localhost',
httpPort: 9000, httpPort: 9000,
@ -110,9 +117,14 @@ export class ServiceContainerBuilder {
password: 'postgres', password: 'postgres',
}, },
questdb: this.options.enableQuestDB ? questdbConfig : undefined, questdb: this.options.enableQuestDB ? questdbConfig : undefined,
proxy: this.options.enableProxy ? (config.proxy || { enabled: false, cachePrefix: 'proxy:', ttl: 3600 }) : undefined, proxy: this.options.enableProxy
browser: this.options.enableBrowser ? (config.browser || { headless: true, timeout: 30000 }) : undefined, ? config.proxy || { enabled: false, cachePrefix: 'proxy:', ttl: 3600 }
queue: this.options.enableQueue ? (config.queue || { : undefined,
browser: this.options.enableBrowser
? config.browser || { headless: true, timeout: 30000 }
: undefined,
queue: this.options.enableQueue
? config.queue || {
enabled: true, enabled: true,
workers: 1, workers: 1,
concurrency: 1, concurrency: 1,
@ -123,13 +135,23 @@ export class ServiceContainerBuilder {
backoff: { type: 'exponential' as const, delay: 1000 }, backoff: { type: 'exponential' as const, delay: 1000 },
removeOnComplete: 100, removeOnComplete: 100,
removeOnFail: 50, removeOnFail: 50,
},
} }
}) : undefined, : undefined,
service: config.service, service: config.service,
}; };
} }
private registerServices(container: AwilixContainer<ServiceDefinitions>, config: AppConfig): void { private registerServices(
container: AwilixContainer<ServiceDefinitions>,
config: AppConfig
): void {
// Register handler infrastructure first
container.register({
handlerRegistry: asClass(HandlerRegistry).singleton(),
handlerScanner: asClass(HandlerScanner).singleton(),
});
registerCoreServices(container, config); registerCoreServices(container, config);
registerCacheServices(container, config); registerCacheServices(container, config);
registerDatabaseServices(container, config); registerDatabaseServices(container, config);
@ -137,9 +159,18 @@ export class ServiceContainerBuilder {
// Register service container aggregate // Register service container aggregate
container.register({ container.register({
serviceContainer: asFunction(({ serviceContainer: asFunction(
config: _config, logger, cache, globalCache, proxyManager, browser, ({
queueManager, mongoClient, postgresClient, questdbClient config: _config,
logger,
cache,
globalCache,
proxyManager,
browser,
queueManager,
mongoClient,
postgresClient,
questdbClient,
}) => ({ }) => ({
logger, logger,
cache, cache,
@ -150,21 +181,24 @@ export class ServiceContainerBuilder {
mongodb: mongoClient, // Map mongoClient to mongodb mongodb: mongoClient, // Map mongoClient to mongodb
postgres: postgresClient, // Map postgresClient to postgres postgres: postgresClient, // Map postgresClient to postgres
questdb: questdbClient, // Map questdbClient to questdb questdb: questdbClient, // Map questdbClient to questdb
})).singleton(), })
).singleton(),
}); });
} }
private transformStockBotConfig(config: UnifiedAppConfig): Partial<AppConfig> { private transformStockBotConfig(config: UnifiedAppConfig): Partial<AppConfig> {
// Unified config already has flat structure, just extract what we need // Unified config already has flat structure, just extract what we need
// Handle questdb field name mapping // Handle questdb field name mapping
const questdb = config.questdb ? { const questdb = config.questdb
? {
enabled: config.questdb.enabled || true, enabled: config.questdb.enabled || true,
host: config.questdb.host || 'localhost', host: config.questdb.host || 'localhost',
httpPort: config.questdb.httpPort || 9000, httpPort: config.questdb.httpPort || 9000,
pgPort: config.questdb.pgPort || 8812, pgPort: config.questdb.pgPort || 8812,
influxPort: (config.questdb as any).influxPort || (config.questdb as any).ilpPort || 9009, influxPort: (config.questdb as any).influxPort || (config.questdb as any).ilpPort || 9009,
database: config.questdb.database || 'questdb', database: config.questdb.database || 'questdb',
} : undefined; }
: undefined;
return { return {
redis: config.redis, redis: config.redis,

View file

@ -1,19 +1,25 @@
import type { Browser } from '@stock-bot/browser'; import type { Browser } from '@stock-bot/browser';
import type { CacheProvider } from '@stock-bot/cache'; import type { CacheProvider } from '@stock-bot/cache';
import type { IServiceContainer } from '@stock-bot/types'; import type { HandlerRegistry } from '@stock-bot/handler-registry';
import type { Logger } from '@stock-bot/logger'; import type { Logger } from '@stock-bot/logger';
import type { MongoDBClient } from '@stock-bot/mongodb'; import type { MongoDBClient } from '@stock-bot/mongodb';
import type { PostgreSQLClient } from '@stock-bot/postgres'; import type { PostgreSQLClient } from '@stock-bot/postgres';
import type { ProxyManager } from '@stock-bot/proxy'; import type { ProxyManager } from '@stock-bot/proxy';
import type { QuestDBClient } from '@stock-bot/questdb'; import type { QuestDBClient } from '@stock-bot/questdb';
import type { SmartQueueManager } from '@stock-bot/queue'; import type { SmartQueueManager } from '@stock-bot/queue';
import type { IServiceContainer } from '@stock-bot/types';
import type { AppConfig } from '../config/schemas'; import type { AppConfig } from '../config/schemas';
import type { HandlerScanner } from '../scanner';
export interface ServiceDefinitions { export interface ServiceDefinitions {
// Configuration // Configuration
config: AppConfig; config: AppConfig;
logger: Logger; logger: Logger;
// Handler infrastructure
handlerRegistry: HandlerRegistry;
handlerScanner: HandlerScanner;
// Core services // Core services
cache: CacheProvider | null; cache: CacheProvider | null;
globalCache: CacheProvider | null; globalCache: CacheProvider | null;

View file

@ -3,10 +3,7 @@ import { NamespacedCache, type CacheProvider } from '@stock-bot/cache';
import type { ServiceDefinitions } from '../container/types'; import type { ServiceDefinitions } from '../container/types';
export class CacheFactory { export class CacheFactory {
static createNamespacedCache( static createNamespacedCache(baseCache: CacheProvider, namespace: string): NamespacedCache {
baseCache: CacheProvider,
namespace: string
): NamespacedCache {
return new NamespacedCache(baseCache, namespace); return new NamespacedCache(baseCache, namespace);
} }
@ -15,7 +12,9 @@ export class CacheFactory {
serviceName: string serviceName: string
): CacheProvider | null { ): CacheProvider | null {
const baseCache = container.cradle.cache; const baseCache = container.cradle.cache;
if (!baseCache) {return null;} if (!baseCache) {
return null;
}
return this.createNamespacedCache(baseCache, serviceName); return this.createNamespacedCache(baseCache, serviceName);
} }
@ -25,7 +24,9 @@ export class CacheFactory {
handlerName: string handlerName: string
): CacheProvider | null { ): CacheProvider | null {
const baseCache = container.cradle.cache; const baseCache = container.cradle.cache;
if (!baseCache) {return null;} if (!baseCache) {
return null;
}
return this.createNamespacedCache(baseCache, `handler:${handlerName}`); return this.createNamespacedCache(baseCache, `handler:${handlerName}`);
} }
@ -35,7 +36,9 @@ export class CacheFactory {
prefix: string prefix: string
): CacheProvider | null { ): CacheProvider | null {
const baseCache = container.cradle.cache; const baseCache = container.cradle.cache;
if (!baseCache) {return null;} if (!baseCache) {
return null;
}
// Remove 'cache:' prefix if already included // Remove 'cache:' prefix if already included
const cleanPrefix = prefix.replace(/^cache:/, ''); const cleanPrefix = prefix.replace(/^cache:/, '');

View file

@ -33,3 +33,7 @@ export {
type ServiceApplicationConfig, type ServiceApplicationConfig,
type ServiceLifecycleHooks, type ServiceLifecycleHooks,
} from './service-application'; } from './service-application';
// Handler scanner
export { HandlerScanner } from './scanner';
export type { HandlerScannerOptions } from './scanner';

View file

@ -14,12 +14,16 @@ export function registerCacheServices(
const serviceName = config.service?.serviceName || config.service?.name || 'unknown'; const serviceName = config.service?.serviceName || config.service?.name || 'unknown';
// Create service-specific cache that uses the service's Redis DB // Create service-specific cache that uses the service's Redis DB
return createServiceCache(serviceName, { return createServiceCache(
serviceName,
{
host: config.redis.host, host: config.redis.host,
port: config.redis.port, port: config.redis.port,
password: config.redis.password, password: config.redis.password,
db: config.redis.db, // This will be overridden by ServiceCache db: config.redis.db, // This will be overridden by ServiceCache
}, { logger }); },
{ logger }
);
}).singleton(), }).singleton(),
// Also provide global cache for shared data // Also provide global cache for shared data
@ -27,11 +31,15 @@ export function registerCacheServices(
const { createServiceCache } = require('@stock-bot/queue'); const { createServiceCache } = require('@stock-bot/queue');
const serviceName = config.service?.serviceName || config.service?.name || 'unknown'; const serviceName = config.service?.serviceName || config.service?.name || 'unknown';
return createServiceCache(serviceName, { return createServiceCache(
serviceName,
{
host: config.redis.host, host: config.redis.host,
port: config.redis.port, port: config.redis.port,
password: config.redis.password, password: config.redis.password,
}, { global: true, logger }); },
{ global: true, logger }
);
}).singleton(), }).singleton(),
}); });
} else { } else {

View file

@ -1,7 +1,7 @@
import { asFunction, asValue, type AwilixContainer } from 'awilix';
import { MongoDBClient } from '@stock-bot/mongodb'; import { MongoDBClient } from '@stock-bot/mongodb';
import { PostgreSQLClient } from '@stock-bot/postgres'; import { PostgreSQLClient } from '@stock-bot/postgres';
import { QuestDBClient } from '@stock-bot/questdb'; import { QuestDBClient } from '@stock-bot/questdb';
import { asFunction, asValue, type AwilixContainer } from 'awilix';
import type { AppConfig } from '../config/schemas'; import type { AppConfig } from '../config/schemas';
import type { ServiceDefinitions } from '../container/types'; import type { ServiceDefinitions } from '../container/types';
@ -14,7 +14,9 @@ export function registerDatabaseServices(
container.register({ container.register({
mongoClient: asFunction(({ logger }) => { mongoClient: asFunction(({ logger }) => {
// Parse MongoDB URI to extract components // Parse MongoDB URI to extract components
const uriMatch = config.mongodb.uri.match(/mongodb:\/\/(?:([^:]+):([^@]+)@)?([^:/]+):(\d+)\/([^?]+)(?:\?authSource=(.+))?/); const uriMatch = config.mongodb.uri.match(
/mongodb:\/\/(?:([^:]+):([^@]+)@)?([^:/]+):(\d+)\/([^?]+)(?:\?authSource=(.+))?/
);
const mongoConfig = { const mongoConfig = {
host: uriMatch?.[3] || 'localhost', host: uriMatch?.[3] || 'localhost',
port: parseInt(uriMatch?.[4] || '27017'), port: parseInt(uriMatch?.[4] || '27017'),

View file

@ -60,7 +60,7 @@ export function registerApplicationServices(
// Queue Manager // Queue Manager
if (config.queue?.enabled && config.redis.enabled) { if (config.queue?.enabled && config.redis.enabled) {
container.register({ container.register({
queueManager: asFunction(({ logger }) => { queueManager: asFunction(({ logger, handlerRegistry }) => {
const { SmartQueueManager } = require('@stock-bot/queue'); const { SmartQueueManager } = require('@stock-bot/queue');
const queueConfig = { const queueConfig = {
serviceName: config.service?.serviceName || config.service?.name || 'unknown', serviceName: config.service?.serviceName || config.service?.name || 'unknown',
@ -79,7 +79,7 @@ export function registerApplicationServices(
delayWorkerStart: config.queue!.delayWorkerStart ?? false, delayWorkerStart: config.queue!.delayWorkerStart ?? false,
autoDiscoverHandlers: true, autoDiscoverHandlers: true,
}; };
return new SmartQueueManager(queueConfig, logger); return new SmartQueueManager(queueConfig, handlerRegistry, logger);
}).singleton(), }).singleton(),
}); });
} else { } else {

View file

@ -0,0 +1,201 @@
/**
* Handler Scanner
* Discovers and registers handlers with the DI container
*/
import { asClass, type AwilixContainer } from 'awilix';
import { glob } from 'glob';
import type {
HandlerConfiguration,
HandlerMetadata,
HandlerRegistry,
} from '@stock-bot/handler-registry';
import { createJobHandler } from '@stock-bot/handlers';
import { getLogger } from '@stock-bot/logger';
import type { ExecutionContext, IHandler } from '@stock-bot/types';
export interface HandlerScannerOptions {
serviceName?: string;
autoRegister?: boolean;
patterns?: string[];
}
export class HandlerScanner {
private logger = getLogger('handler-scanner');
private discoveredHandlers = new Map<string, any>();
constructor(
private registry: HandlerRegistry,
private container: AwilixContainer,
private options: HandlerScannerOptions = {}
) {}
/**
* Scan for handlers matching the given patterns
*/
async scanHandlers(patterns: string[] = this.options.patterns || []): Promise<void> {
this.logger.info('Starting handler scan', { patterns });
for (const pattern of patterns) {
const files = await glob(pattern, { absolute: true });
this.logger.debug(`Found ${files.length} files for pattern: ${pattern}`);
for (const file of files) {
try {
await this.scanFile(file);
} catch (error) {
this.logger.error('Failed to scan file', { file, error });
}
}
}
this.logger.info('Handler scan complete', {
discovered: this.discoveredHandlers.size,
patterns,
});
}
/**
* Scan a single file for handlers
*/
private async scanFile(filePath: string): Promise<void> {
try {
const module = await import(filePath);
this.registerHandlersFromModule(module, filePath);
} catch (error) {
this.logger.error('Failed to import module', { filePath, error });
}
}
/**
* Register handlers found in a module
*/
private registerHandlersFromModule(module: any, filePath: string): void {
for (const [exportName, exported] of Object.entries(module)) {
if (this.isHandler(exported)) {
this.registerHandler(exported, exportName, filePath);
}
}
}
/**
* Check if an exported value is a handler
*/
private isHandler(exported: any): boolean {
if (typeof exported !== 'function') return false;
// Check for handler metadata added by decorators
const hasHandlerName = !!(exported as any).__handlerName;
const hasOperations = Array.isArray((exported as any).__operations);
return hasHandlerName && hasOperations;
}
/**
* Register a handler with the registry and DI container
*/
private registerHandler(HandlerClass: any, exportName: string, filePath: string): void {
const handlerName = HandlerClass.__handlerName;
const operations = HandlerClass.__operations || [];
const schedules = HandlerClass.__schedules || [];
const isDisabled = HandlerClass.__disabled || false;
if (isDisabled) {
this.logger.debug('Skipping disabled handler', { handlerName });
return;
}
// Build metadata
const metadata: HandlerMetadata = {
name: handlerName,
service: this.options.serviceName,
operations: operations.map((op: any) => ({
name: op.name,
method: op.method,
})),
schedules: schedules.map((schedule: any) => ({
operation: schedule.operation,
cronPattern: schedule.cronPattern,
priority: schedule.priority,
immediately: schedule.immediately,
description: schedule.description,
})),
};
// Build configuration with operation handlers
const operationHandlers: Record<string, any> = {};
for (const op of operations) {
operationHandlers[op.name] = createJobHandler(async payload => {
const handler = this.container.resolve<IHandler>(handlerName);
const context: ExecutionContext = {
type: 'queue',
metadata: { source: 'queue', timestamp: Date.now() },
};
return await handler.execute(op.name, payload, context);
});
}
const configuration: HandlerConfiguration = {
name: handlerName,
operations: operationHandlers,
scheduledJobs: schedules.map((schedule: any) => {
const operation = operations.find((op: any) => op.method === schedule.operation);
return {
type: `${handlerName}-${schedule.operation}`,
operation: operation?.name || schedule.operation,
cronPattern: schedule.cronPattern,
priority: schedule.priority || 5,
immediately: schedule.immediately || false,
description: schedule.description || `${handlerName} ${schedule.operation}`,
};
}),
};
// Register with registry
this.registry.register(metadata, configuration);
// Register with DI container if auto-register is enabled
if (this.options.autoRegister !== false) {
this.container.register({
[handlerName]: asClass(HandlerClass).singleton(),
});
}
// Track discovered handler
this.discoveredHandlers.set(handlerName, HandlerClass);
this.logger.info('Registered handler', {
handlerName,
exportName,
filePath,
operations: operations.length,
schedules: schedules.length,
service: this.options.serviceName,
});
}
/**
* Get all discovered handlers
*/
getDiscoveredHandlers(): Map<string, any> {
return new Map(this.discoveredHandlers);
}
/**
* Manually register a handler class
*/
registerHandlerClass(HandlerClass: any, options: { serviceName?: string } = {}): void {
const serviceName = options.serviceName || this.options.serviceName;
const originalServiceName = this.options.serviceName;
// Temporarily override service name if provided
if (serviceName) {
this.options.serviceName = serviceName;
}
this.registerHandler(HandlerClass, HandlerClass.name, 'manual');
// Restore original service name
this.options.serviceName = originalServiceName;
}
}

View file

@ -0,0 +1,2 @@
export { HandlerScanner } from './handler-scanner';
export type { HandlerScannerOptions } from './handler-scanner';

View file

@ -5,12 +5,14 @@
import { Hono } from 'hono'; import { Hono } from 'hono';
import { cors } from 'hono/cors'; import { cors } from 'hono/cors';
import { getLogger, setLoggerConfig, shutdownLoggers, type Logger } from '@stock-bot/logger';
import { Shutdown } from '@stock-bot/shutdown';
import type { BaseAppConfig, UnifiedAppConfig } from '@stock-bot/config'; import type { BaseAppConfig, UnifiedAppConfig } from '@stock-bot/config';
import { toUnifiedConfig } from '@stock-bot/config'; import { toUnifiedConfig } from '@stock-bot/config';
import { getLogger, setLoggerConfig, shutdownLoggers, type Logger } from '@stock-bot/logger';
import { Shutdown } from '@stock-bot/shutdown';
import type { IServiceContainer } from '@stock-bot/types'; import type { IServiceContainer } from '@stock-bot/types';
import type { ServiceContainer } from './awilix-container'; import type { HandlerRegistry } from '@stock-bot/handler-registry';
import type { ServiceDefinitions } from './container/types';
import type { AwilixContainer } from 'awilix';
/** /**
* Configuration for ServiceApplication * Configuration for ServiceApplication
@ -71,7 +73,7 @@ export class ServiceApplication {
private hooks: ServiceLifecycleHooks; private hooks: ServiceLifecycleHooks;
private logger: Logger; private logger: Logger;
private container: ServiceContainer | null = null; private container: AwilixContainer<ServiceDefinitions> | null = null;
private serviceContainer: IServiceContainer | null = null; private serviceContainer: IServiceContainer | null = null;
private app: Hono | null = null; private app: Hono | null = null;
private server: ReturnType<typeof Bun.serve> | null = null; private server: ReturnType<typeof Bun.serve> | null = null;
@ -105,7 +107,7 @@ export class ServiceApplication {
// Initialize shutdown manager // Initialize shutdown manager
this.shutdown = Shutdown.getInstance({ this.shutdown = Shutdown.getInstance({
timeout: this.serviceConfig.shutdownTimeout timeout: this.serviceConfig.shutdownTimeout,
}); });
} }
@ -246,7 +248,7 @@ export class ServiceApplication {
* Start the service with full initialization * Start the service with full initialization
*/ */
async start( async start(
containerFactory: (config: UnifiedAppConfig) => Promise<ServiceContainer>, containerFactory: (config: UnifiedAppConfig) => Promise<AwilixContainer<ServiceDefinitions>>,
routeFactory: (container: IServiceContainer) => Hono, routeFactory: (container: IServiceContainer) => Hono,
handlerInitializer?: (container: IServiceContainer) => Promise<void> handlerInitializer?: (container: IServiceContainer) => Promise<void>
): Promise<void> { ): Promise<void> {
@ -257,7 +259,7 @@ export class ServiceApplication {
this.logger.debug('Creating DI container...'); this.logger.debug('Creating DI container...');
// Config already has service name from constructor // Config already has service name from constructor
this.container = await containerFactory(this.config); this.container = await containerFactory(this.config);
this.serviceContainer = this.container.resolve('serviceContainer'); this.serviceContainer = this.container!.resolve('serviceContainer');
this.logger.info('DI container created and initialized'); this.logger.info('DI container created and initialized');
// Call container ready hook // Call container ready hook
@ -311,7 +313,6 @@ export class ServiceApplication {
if (this.hooks.onStarted) { if (this.hooks.onStarted) {
await this.hooks.onStarted(port); await this.hooks.onStarted(port);
} }
} catch (error) { } catch (error) {
this.logger.error('DETAILED ERROR:', error); this.logger.error('DETAILED ERROR:', error);
this.logger.error('Failed to start service', { this.logger.error('Failed to start service', {
@ -332,7 +333,7 @@ export class ServiceApplication {
} }
this.logger.debug('Creating scheduled jobs from registered handlers...'); this.logger.debug('Creating scheduled jobs from registered handlers...');
const { handlerRegistry } = await import('@stock-bot/handlers'); const handlerRegistry = this.container.resolve<HandlerRegistry>('handlerRegistry');
const allHandlers = handlerRegistry.getAllHandlersWithSchedule(); const allHandlers = handlerRegistry.getAllHandlersWithSchedule();
let totalScheduledJobs = 0; let totalScheduledJobs = 0;

View file

@ -1,6 +1,6 @@
import type { AwilixContainer } from 'awilix'; import type { AwilixContainer } from 'awilix';
import type { ServiceDefinitions } from '../container/types';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { ServiceDefinitions } from '../container/types';
interface ServiceWithLifecycle { interface ServiceWithLifecycle {
connect?: () => Promise<void>; connect?: () => Promise<void>;
@ -35,7 +35,10 @@ export class ServiceLifecycleManager {
initPromises.push( initPromises.push(
Promise.race([ Promise.race([
initPromise, initPromise,
this.createTimeoutPromise(timeout, `${name} initialization timed out after ${timeout}ms`), this.createTimeoutPromise(
timeout,
`${name} initialization timed out after ${timeout}ms`
),
]) ])
); );
} }

View file

@ -0,0 +1,27 @@
{
"name": "@stock-bot/handler-registry",
"version": "1.0.0",
"type": "module",
"main": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
".": {
"import": "./dist/index.js",
"types": "./dist/index.d.ts"
}
},
"scripts": {
"build": "bun run build:clean && bun run build:tsc",
"build:clean": "rm -rf dist",
"build:tsc": "tsc",
"test": "bun test",
"clean": "rm -rf dist node_modules .turbo"
},
"dependencies": {
"@stock-bot/types": "workspace:*"
},
"devDependencies": {
"@types/bun": "*",
"typescript": "*"
}
}

View file

@ -0,0 +1,14 @@
/**
* Handler Registry Package
* Provides centralized handler registration without circular dependencies
*/
export { HandlerRegistry } from './registry';
export type {
HandlerMetadata,
OperationMetadata,
ScheduleMetadata,
HandlerConfiguration,
RegistryStats,
HandlerDiscoveryResult,
} from './types';

View file

@ -0,0 +1,226 @@
/**
* Handler Registry Implementation
* Manages handler metadata and configuration without circular dependencies
*/
import type { JobHandler, ScheduledJob } from '@stock-bot/types';
import type {
HandlerConfiguration,
HandlerMetadata,
OperationMetadata,
RegistryStats,
ScheduleMetadata,
} from './types';
export class HandlerRegistry {
private handlers = new Map<string, HandlerMetadata>();
private configurations = new Map<string, HandlerConfiguration>();
private handlerServices = new Map<string, string>();
/**
* Register handler metadata
*/
registerMetadata(metadata: HandlerMetadata): void {
this.handlers.set(metadata.name, metadata);
if (metadata.service) {
this.handlerServices.set(metadata.name, metadata.service);
}
}
/**
* Register handler configuration with operation implementations
*/
registerConfiguration(config: HandlerConfiguration): void {
this.configurations.set(config.name, config);
}
/**
* Register both metadata and configuration
*/
register(metadata: HandlerMetadata, config: HandlerConfiguration): void {
this.registerMetadata(metadata);
this.registerConfiguration(config);
}
/**
* Get handler metadata
*/
getMetadata(handlerName: string): HandlerMetadata | undefined {
return this.handlers.get(handlerName);
}
/**
* Get handler configuration
*/
getConfiguration(handlerName: string): HandlerConfiguration | undefined {
return this.configurations.get(handlerName);
}
/**
* Get a specific operation handler
*/
getOperation(handlerName: string, operationName: string): JobHandler | undefined {
const config = this.configurations.get(handlerName);
return config?.operations[operationName];
}
/**
* Get all handler metadata
*/
getAllMetadata(): Map<string, HandlerMetadata> {
return new Map(this.handlers);
}
/**
* Get all handler names
*/
getHandlerNames(): string[] {
return Array.from(this.handlers.keys());
}
/**
* Check if a handler is registered
*/
hasHandler(handlerName: string): boolean {
return this.handlers.has(handlerName);
}
/**
* Get handlers for a specific service
*/
getServiceHandlers(serviceName: string): HandlerMetadata[] {
const handlers: HandlerMetadata[] = [];
for (const [handlerName, service] of this.handlerServices) {
if (service === serviceName) {
const metadata = this.handlers.get(handlerName);
if (metadata) {
handlers.push(metadata);
}
}
}
return handlers;
}
/**
* Set service ownership for a handler
*/
setHandlerService(handlerName: string, serviceName: string): void {
this.handlerServices.set(handlerName, serviceName);
// Update metadata if it exists
const metadata = this.handlers.get(handlerName);
if (metadata) {
metadata.service = serviceName;
}
}
/**
* Get the service that owns a handler
*/
getHandlerService(handlerName: string): string | undefined {
return this.handlerServices.get(handlerName);
}
/**
* Get scheduled jobs for a handler
*/
getScheduledJobs(handlerName: string): ScheduledJob[] {
const config = this.configurations.get(handlerName);
return config?.scheduledJobs || [];
}
/**
* Get all handlers with their scheduled jobs
*/
getAllHandlersWithSchedule(): Map<
string,
{ metadata: HandlerMetadata; scheduledJobs: ScheduledJob[] }
> {
const result = new Map<string, { metadata: HandlerMetadata; scheduledJobs: ScheduledJob[] }>();
for (const [name, metadata] of this.handlers) {
const config = this.configurations.get(name);
result.set(name, {
metadata,
scheduledJobs: config?.scheduledJobs || [],
});
}
return result;
}
/**
* Get registry statistics
*/
getStats(): RegistryStats {
let operationCount = 0;
let scheduledJobCount = 0;
const services = new Set<string>();
for (const metadata of this.handlers.values()) {
operationCount += metadata.operations.length;
scheduledJobCount += metadata.schedules?.length || 0;
if (metadata.service) {
services.add(metadata.service);
}
}
return {
handlers: this.handlers.size,
operations: operationCount,
scheduledJobs: scheduledJobCount,
services: services.size,
};
}
/**
* Clear all registrations (useful for testing)
*/
clear(): void {
this.handlers.clear();
this.configurations.clear();
this.handlerServices.clear();
}
/**
* Export registry data for debugging or persistence
*/
export(): {
handlers: Array<[string, HandlerMetadata]>;
configurations: Array<[string, HandlerConfiguration]>;
services: Array<[string, string]>;
} {
return {
handlers: Array.from(this.handlers.entries()),
configurations: Array.from(this.configurations.entries()),
services: Array.from(this.handlerServices.entries()),
};
}
/**
* Import registry data
*/
import(data: {
handlers: Array<[string, HandlerMetadata]>;
configurations: Array<[string, HandlerConfiguration]>;
services: Array<[string, string]>;
}): void {
this.clear();
for (const [name, metadata] of data.handlers) {
this.handlers.set(name, metadata);
}
for (const [name, config] of data.configurations) {
this.configurations.set(name, config);
}
for (const [handler, service] of data.services) {
this.handlerServices.set(handler, service);
}
}
}

View file

@ -0,0 +1,66 @@
/**
* Handler Registry Types
* Pure types for handler metadata and registration
*/
import type { JobHandler, ScheduledJob } from '@stock-bot/types';
/**
* Metadata for a single operation within a handler
*/
export interface OperationMetadata {
name: string;
method: string;
description?: string;
}
/**
* Metadata for a scheduled operation
*/
export interface ScheduleMetadata {
operation: string;
cronPattern: string;
priority?: number;
immediately?: boolean;
description?: string;
}
/**
* Complete metadata for a handler
*/
export interface HandlerMetadata {
name: string;
service?: string;
operations: OperationMetadata[];
schedules?: ScheduleMetadata[];
version?: string;
description?: string;
}
/**
* Handler configuration with operation implementations
*/
export interface HandlerConfiguration {
name: string;
operations: Record<string, JobHandler>;
scheduledJobs?: ScheduledJob[];
}
/**
* Registry statistics
*/
export interface RegistryStats {
handlers: number;
operations: number;
scheduledJobs: number;
services: number;
}
/**
* Handler discovery result
*/
export interface HandlerDiscoveryResult {
handler: HandlerMetadata;
constructor: any;
filePath?: string;
}

View file

@ -0,0 +1,9 @@
{
"extends": "../../../tsconfig.lib.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src"
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "test/**/*", "**/*.test.ts", "**/*.spec.ts"]
}

View file

@ -10,10 +10,11 @@
"test": "bun test" "test": "bun test"
}, },
"dependencies": { "dependencies": {
"@stock-bot/cache": "workspace:*",
"@stock-bot/config": "workspace:*", "@stock-bot/config": "workspace:*",
"@stock-bot/handler-registry": "workspace:*",
"@stock-bot/logger": "workspace:*", "@stock-bot/logger": "workspace:*",
"@stock-bot/types": "workspace:*", "@stock-bot/types": "workspace:*",
"@stock-bot/cache": "workspace:*",
"@stock-bot/utils": "workspace:*", "@stock-bot/utils": "workspace:*",
"mongodb": "^6.12.0" "mongodb": "^6.12.0"
}, },

View file

@ -1,14 +1,17 @@
import type { Collection } from 'mongodb'; import type { Collection } from 'mongodb';
import { createNamespacedCache } from '@stock-bot/cache';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import type { import type {
HandlerConfigWithSchedule,
IServiceContainer,
ExecutionContext, ExecutionContext,
IHandler HandlerConfigWithSchedule,
HandlerMetadata,
IHandler,
IServiceContainer,
JobHandler,
ServiceTypes,
} from '@stock-bot/types'; } from '@stock-bot/types';
import { fetch } from '@stock-bot/utils'; import { fetch } from '@stock-bot/utils';
import { createNamespacedCache } from '@stock-bot/cache'; // Handler registry is now injected, not imported
import { handlerRegistry } from '../registry/handler-registry';
import { createJobHandler } from '../utils/create-job-handler'; import { createJobHandler } from '../utils/create-job-handler';
/** /**
@ -38,16 +41,16 @@ export interface JobScheduleOptions {
* Provides common functionality and structure for queue/event operations * Provides common functionality and structure for queue/event operations
*/ */
export abstract class BaseHandler implements IHandler { export abstract class BaseHandler implements IHandler {
// Direct service properties - flattened for cleaner access // Direct service properties - flattened for cleaner access with proper types
readonly logger; readonly logger: ServiceTypes['logger'];
readonly cache; readonly cache: ServiceTypes['cache'];
readonly globalCache; readonly globalCache: ServiceTypes['globalCache'];
readonly queue; readonly queue: ServiceTypes['queue'];
readonly proxy; readonly proxy: ServiceTypes['proxy'];
readonly browser; readonly browser: ServiceTypes['browser'];
readonly mongodb; readonly mongodb: ServiceTypes['mongodb'];
readonly postgres; readonly postgres: ServiceTypes['postgres'];
readonly questdb; readonly questdb: ServiceTypes['questdb'];
private handlerName: string; private handlerName: string;
@ -162,7 +165,7 @@ export abstract class BaseHandler implements IHandler {
* Example: handler 'webshare' creates namespace 'webshare:api' -> keys will be 'cache:data-ingestion:webshare:api:*' * Example: handler 'webshare' creates namespace 'webshare:api' -> keys will be 'cache:data-ingestion:webshare:api:*'
*/ */
protected createNamespacedCache(subNamespace: string) { protected createNamespacedCache(subNamespace: string) {
return createNamespacedCache(this.cache, `${this.handlerName}:${subNamespace}`); return createNamespacedCache(this.cache || null, `${this.handlerName}:${subNamespace}`);
} }
/** /**
@ -238,7 +241,7 @@ export abstract class BaseHandler implements IHandler {
): Promise<void> { ): Promise<void> {
return this.scheduleOperation(operation, payload, { return this.scheduleOperation(operation, payload, {
delay: delaySeconds * 1000, delay: delaySeconds * 1000,
...additionalOptions ...additionalOptions,
}); });
} }
@ -294,27 +297,45 @@ export abstract class BaseHandler implements IHandler {
// } // }
/** /**
* Register this handler using decorator metadata * Create handler configuration with job handlers
* Automatically reads @Handler, @Operation, and @QueueSchedule decorators * This is used by the scanner to create the actual handler configuration
*/ */
register(serviceName?: string): void { createHandlerConfig(): HandlerConfigWithSchedule {
const constructor = this.constructor as any; const metadata = (this.constructor as typeof BaseHandler).extractMetadata();
const handlerName = constructor.__handlerName || this.handlerName; if (!metadata) {
const operations = constructor.__operations || []; throw new Error('Handler metadata not found');
const schedules = constructor.__schedules || []; }
// Create operation handlers from decorator metadata const operationHandlers: Record<string, JobHandler> = {};
const operationHandlers: Record<string, any> = {}; for (const opName of metadata.operations) {
for (const op of operations) { operationHandlers[opName] = createJobHandler(async (payload: any) => {
operationHandlers[op.name] = createJobHandler(async payload => {
const context: ExecutionContext = { const context: ExecutionContext = {
type: 'queue', type: 'queue',
metadata: { source: 'queue', timestamp: Date.now() }, metadata: { source: 'queue', timestamp: Date.now() },
}; };
return await this.execute(op.name, payload, context); return await this.execute(opName, payload, context);
}); });
} }
return {
name: metadata.name,
operations: operationHandlers,
scheduledJobs: metadata.scheduledJobs,
};
}
/**
* Extract handler metadata from decorators
* This returns metadata only - actual handler instances are created by the scanner
*/
static extractMetadata(): HandlerMetadata | null {
const constructor = this as any;
const handlerName = constructor.__handlerName;
if (!handlerName) return null;
const operations = constructor.__operations || [];
const schedules = constructor.__schedules || [];
// Create scheduled jobs from decorator metadata // Create scheduled jobs from decorator metadata
const scheduledJobs = schedules.map((schedule: any) => { const scheduledJobs = schedules.map((schedule: any) => {
// Find the operation name from the method name // Find the operation name from the method name
@ -326,27 +347,15 @@ export abstract class BaseHandler implements IHandler {
priority: schedule.priority || 5, priority: schedule.priority || 5,
immediately: schedule.immediately || false, immediately: schedule.immediately || false,
description: schedule.description || `${handlerName} ${schedule.operation}`, description: schedule.description || `${handlerName} ${schedule.operation}`,
payload: this.getScheduledJobPayload?.(schedule.operation),
}; };
}); });
const config: HandlerConfigWithSchedule = { return {
name: handlerName, name: handlerName,
operations: operationHandlers, operations: operations.map((op: any) => op.name),
scheduledJobs, scheduledJobs,
description: constructor.__description,
}; };
handlerRegistry.registerWithSchedule(config, serviceName);
this.logger.info('Handler registered using decorator metadata', {
handlerName,
service: serviceName,
operations: operations.map((op: any) => ({ name: op.name, method: op.method })),
scheduledJobs: scheduledJobs.map((job: any) => ({
operation: job.operation,
cronPattern: job.cronPattern,
immediately: job.immediately,
})),
});
} }
/** /**

View file

@ -2,8 +2,7 @@
export { BaseHandler, ScheduledHandler } from './base/BaseHandler'; export { BaseHandler, ScheduledHandler } from './base/BaseHandler';
export type { JobScheduleOptions } from './base/BaseHandler'; export type { JobScheduleOptions } from './base/BaseHandler';
// Handler registry // Handler registry is now in a separate package
export { handlerRegistry } from './registry/handler-registry';
// Utilities // Utilities
export { createJobHandler } from './utils/create-job-handler'; export { createJobHandler } from './utils/create-job-handler';

View file

@ -6,8 +6,8 @@
import { readdirSync, statSync } from 'fs'; import { readdirSync, statSync } from 'fs';
import { join, relative } from 'path'; import { join, relative } from 'path';
import { getLogger } from '@stock-bot/logger'; import { getLogger } from '@stock-bot/logger';
import { BaseHandler } from '../base/BaseHandler';
import type { IServiceContainer } from '@stock-bot/types'; import type { IServiceContainer } from '@stock-bot/types';
import { BaseHandler } from '../base/BaseHandler';
const logger = getLogger('handler-auto-register'); const logger = getLogger('handler-auto-register');
@ -123,14 +123,13 @@ export async function autoRegisterHandlers(
} else { } else {
logger.info(`Registering handler: ${handlerName} from ${relativePath}`); logger.info(`Registering handler: ${handlerName} from ${relativePath}`);
// Create instance and register // Create instance - handlers now auto-register via decorators
const handler = new HandlerClass(services); const handler = new HandlerClass(services);
handler.register(serviceName);
// No need to set service ownership separately - it's done in register()
registered.push(handlerName); registered.push(handlerName);
logger.info(`Successfully registered handler: ${handlerName}`, { service: serviceName }); logger.info(`Successfully registered handler: ${handlerName}`, {
service: serviceName,
});
} }
} }
} catch (error) { } catch (error) {

View file

@ -14,9 +14,9 @@
"ioredis": "^5.3.0", "ioredis": "^5.3.0",
"rate-limiter-flexible": "^3.0.0", "rate-limiter-flexible": "^3.0.0",
"@stock-bot/cache": "*", "@stock-bot/cache": "*",
"@stock-bot/handler-registry": "*",
"@stock-bot/logger": "*", "@stock-bot/logger": "*",
"@stock-bot/types": "*", "@stock-bot/types": "*"
"@stock-bot/handlers": "*"
}, },
"devDependencies": { "devDependencies": {
"typescript": "^5.3.0", "typescript": "^5.3.0",

View file

@ -171,7 +171,11 @@ async function processBatched<T>(
/** /**
* Process a batch job - loads items and creates individual jobs * Process a batch job - loads items and creates individual jobs
*/ */
export async function processBatchJob(jobData: BatchJobData, queueName: string, queueManager: QueueManager): Promise<unknown> { export async function processBatchJob(
jobData: BatchJobData,
queueName: string,
queueManager: QueueManager
): Promise<unknown> {
const queue = queueManager.getQueue(queueName); const queue = queueManager.getQueue(queueName);
const logger = queue.createChildLogger('batch-job', { const logger = queue.createChildLogger('batch-job', {
queueName, queueName,
@ -304,7 +308,11 @@ async function loadPayload<T>(
} | null; } | null;
} }
async function cleanupPayload(key: string, queueName: string, queueManager: QueueManager): Promise<void> { async function cleanupPayload(
key: string,
queueName: string,
queueManager: QueueManager
): Promise<void> {
const cache = queueManager.getCache(queueName); const cache = queueManager.getCache(queueName);
await cache.del(key); await cache.del(key);
} }

View file

@ -8,11 +8,11 @@ export {
normalizeServiceName, normalizeServiceName,
generateCachePrefix, generateCachePrefix,
getFullQueueName, getFullQueueName,
parseQueueName parseQueueName,
} from './service-utils'; } from './service-utils';
// Re-export handler registry and utilities from handlers package // Re-export utilities from handlers package
export { handlerRegistry, createJobHandler } from '@stock-bot/handlers'; export { createJobHandler } from '@stock-bot/handlers';
// Batch processing // Batch processing
export { processBatchJob, processItems } from './batch-processor'; export { processBatchJob, processItems } from './batch-processor';
@ -68,6 +68,4 @@ export type {
// Smart Queue types // Smart Queue types
SmartQueueConfig, SmartQueueConfig,
QueueRoute, QueueRoute,
} from './types'; } from './types';

View file

@ -76,7 +76,8 @@ export class QueueManager {
// Prepare queue configuration // Prepare queue configuration
const workers = mergedOptions.workers ?? this.config.defaultQueueOptions?.workers ?? 1; const workers = mergedOptions.workers ?? this.config.defaultQueueOptions?.workers ?? 1;
const concurrency = mergedOptions.concurrency ?? this.config.defaultQueueOptions?.concurrency ?? 1; const concurrency =
mergedOptions.concurrency ?? this.config.defaultQueueOptions?.concurrency ?? 1;
const queueConfig: QueueWorkerConfig = { const queueConfig: QueueWorkerConfig = {
workers, workers,
@ -180,7 +181,6 @@ export class QueueManager {
return this.queues; return this.queues;
} }
/** /**
* Get statistics for all queues * Get statistics for all queues
*/ */

View file

@ -1,6 +1,7 @@
import { Queue as BullQueue, QueueEvents, Worker, type Job } from 'bullmq'; import { Queue as BullQueue, QueueEvents, Worker, type Job } from 'bullmq';
import { handlerRegistry } from '@stock-bot/handlers'; // Handler registry will be injected
import type { JobData, JobOptions, ExtendedJobOptions, QueueStats, RedisConfig } from './types'; import type { HandlerRegistry } from '@stock-bot/handler-registry';
import type { ExtendedJobOptions, JobData, JobOptions, QueueStats, RedisConfig } from './types';
import { getRedisConnection } from './utils'; import { getRedisConnection } from './utils';
// Logger interface for type safety // Logger interface for type safety
@ -17,6 +18,7 @@ export interface QueueWorkerConfig {
workers?: number; workers?: number;
concurrency?: number; concurrency?: number;
startWorker?: boolean; startWorker?: boolean;
handlerRegistry?: HandlerRegistry;
} }
/** /**
@ -30,6 +32,7 @@ export class Queue {
private queueName: string; private queueName: string;
private redisConfig: RedisConfig; private redisConfig: RedisConfig;
private readonly logger: Logger; private readonly logger: Logger;
private readonly handlerRegistry?: HandlerRegistry;
constructor( constructor(
queueName: string, queueName: string,
@ -41,6 +44,7 @@ export class Queue {
this.queueName = queueName; this.queueName = queueName;
this.redisConfig = redisConfig; this.redisConfig = redisConfig;
this.logger = logger || console; this.logger = logger || console;
this.handlerRegistry = config.handlerRegistry;
const connection = getRedisConnection(redisConfig); const connection = getRedisConnection(redisConfig);
@ -338,7 +342,10 @@ export class Queue {
try { try {
// Look up handler in registry // Look up handler in registry
const jobHandler = handlerRegistry.getOperation(handler, operation); if (!this.handlerRegistry) {
throw new Error('Handler registry not configured for worker processing');
}
const jobHandler = this.handlerRegistry.getOperation(handler, operation);
if (!jobHandler) { if (!jobHandler) {
throw new Error(`No handler found for ${handler}:${operation}`); throw new Error(`No handler found for ${handler}:${operation}`);
@ -390,5 +397,4 @@ export class Queue {
getWorkerCount(): number { getWorkerCount(): number {
return this.workers.length; return this.workers.length;
} }
} }

View file

@ -271,7 +271,12 @@ export class QueueRateLimiter {
limit, limit,
}; };
} catch (error) { } catch (error) {
this.logger.error('Failed to get rate limit status', { queueName, handler, operation, error }); this.logger.error('Failed to get rate limit status', {
queueName,
handler,
operation,
error,
});
return { return {
queueName, queueName,
handler, handler,

View file

@ -1,6 +1,6 @@
import { createCache, type CacheProvider, type CacheStats } from '@stock-bot/cache'; import { createCache, type CacheProvider, type CacheStats } from '@stock-bot/cache';
import type { RedisConfig } from './types';
import { generateCachePrefix } from './service-utils'; import { generateCachePrefix } from './service-utils';
import type { RedisConfig } from './types';
/** /**
* Service-aware cache that uses the service's Redis DB * Service-aware cache that uses the service's Redis DB
@ -132,7 +132,11 @@ export class ServiceCache implements CacheProvider {
return this.cache.set(key, value, ttl); return this.cache.set(key, value, ttl);
} }
async updateField<T = any>(key: string, updater: (current: T | null) => T, ttl?: number): Promise<T | null> { async updateField<T = any>(
key: string,
updater: (current: T | null) => T,
ttl?: number
): Promise<T | null> {
if (this.cache.updateField) { if (this.cache.updateField) {
return this.cache.updateField(key, updater, ttl); return this.cache.updateField(key, updater, ttl);
} }
@ -162,7 +166,6 @@ export class ServiceCache implements CacheProvider {
} }
} }
/** /**
* Factory function to create service cache * Factory function to create service cache
*/ */

Some files were not shown because too many files have changed in this diff Show more