358 lines
11 KiB
Markdown
358 lines
11 KiB
Markdown
# 🏗️ Stock Bot System Architecture
|
|
|
|
## System Communication Flow Diagram
|
|
|
|
```mermaid
|
|
graph TB
|
|
%% External Systems
|
|
subgraph "External APIs"
|
|
AV[Alpha Vantage API]
|
|
YF[Yahoo Finance API]
|
|
AL[Alpaca Broker API]
|
|
IB[Interactive Brokers]
|
|
NEWS[News APIs]
|
|
end
|
|
|
|
%% Core Services Layer
|
|
subgraph "Core Services"
|
|
MDG[Market Data Gateway<br/>:3001]
|
|
RG[Risk Guardian<br/>:3002]
|
|
EE[Execution Engine<br/>:3003]
|
|
PM[Portfolio Manager<br/>:3004]
|
|
end
|
|
|
|
%% Intelligence Services Layer
|
|
subgraph "Intelligence Services"
|
|
SO[Strategy Orchestrator<br/>:4001]
|
|
SG[Signal Generator<br/>:4002]
|
|
BA[Backtesting Engine<br/>:4003]
|
|
ML[ML Pipeline<br/>:4004]
|
|
end
|
|
|
|
%% Data Services Layer
|
|
subgraph "Data Services"
|
|
HDS[Historical Data Service<br/>:5001]
|
|
AS[Analytics Service<br/>:5002]
|
|
DQS[Data Quality Service<br/>:5003]
|
|
ETLS[ETL Service<br/>:5004]
|
|
end
|
|
|
|
%% Platform Services Layer
|
|
subgraph "Platform Services"
|
|
LM[Log Manager<br/>:6001]
|
|
CM[Config Manager<br/>:6002]
|
|
AM[Alert Manager<br/>:6003]
|
|
SM[Service Monitor<br/>:6004]
|
|
end
|
|
|
|
%% Integration Services Layer
|
|
subgraph "Integration Services"
|
|
BAS[Broker Adapter<br/>:7001]
|
|
DAS[Data Adapter<br/>:7002]
|
|
NS[Notification Service<br/>:7003]
|
|
WHS[Webhook Service<br/>:7004]
|
|
end
|
|
|
|
%% Interface Services Layer
|
|
subgraph "Interface Services"
|
|
TD[Trading Dashboard<br/>:5173]
|
|
API[REST API Gateway<br/>:8001]
|
|
WS[WebSocket Server<br/>Embedded]
|
|
end
|
|
|
|
%% Storage Layer
|
|
subgraph "Storage Layer"
|
|
DRAGONFLY[(Dragonfly<br/>Events & Cache)]
|
|
QDB[(QuestDB<br/>Time Series)]
|
|
PGDB[(PostgreSQL<br/>Relational)]
|
|
FS[(File System<br/>Logs & Config)]
|
|
end
|
|
|
|
%% Communication Flows
|
|
|
|
%% External to Core
|
|
AV --> MDG
|
|
YF --> MDG
|
|
AL --> BAS
|
|
IB --> BAS
|
|
NEWS --> DAS
|
|
|
|
%% Core Service Communications
|
|
MDG -->|Market Data Events| DRAGONFLY
|
|
MDG -->|Real-time Stream| WS
|
|
MDG -->|Cache| DRAGONFLY
|
|
|
|
RG -->|Risk Events| DRAGONFLY
|
|
RG -->|Risk Alerts| AM
|
|
RG -->|Position Limits| PM
|
|
|
|
EE -->|Order Events| DRAGONFLY
|
|
EE -->|Trade Execution| BAS
|
|
EE -->|Order Status| PM
|
|
|
|
PM -->|Portfolio Events| DRAGONFLY
|
|
PM -->|P&L Updates| TD
|
|
PM -->|Position Data| RG
|
|
|
|
%% Intelligence Communications
|
|
SO -->|Strategy Events| DRAGONFLY
|
|
SO -->|Signal Requests| SG
|
|
SO -->|Execution Orders| EE
|
|
SO -->|Risk Check| RG
|
|
|
|
SG -->|Trading Signals| SO
|
|
SG -->|ML Requests| ML
|
|
SG -->|Market Data| DRAGONFLY
|
|
|
|
BA -->|Backtest Results| SO
|
|
BA -->|Historical Data| HDS
|
|
|
|
ML -->|Predictions| SG
|
|
ML -->|Training Data| HDS
|
|
|
|
%% Data Service Communications
|
|
HDS -->|Store Data| QDB
|
|
HDS -->|Query Data| QDB
|
|
HDS -->|Data Events| DRAGONFLY
|
|
|
|
AS -->|Analytics| QDB
|
|
AS -->|Metrics| SM
|
|
AS -->|Reports| TD
|
|
|
|
DQS -->|Data Quality| DRAGONFLY
|
|
DQS -->|Alerts| AM
|
|
|
|
ETLS -->|Raw Data| DAS
|
|
ETLS -->|Processed Data| HDS
|
|
|
|
%% Platform Communications
|
|
LM -->|Logs| FS
|
|
LM -->|Log Events| DRAGONFLY
|
|
|
|
CM -->|Config| FS
|
|
CM -->|Config Updates| DRAGONFLY
|
|
|
|
AM -->|Alerts| NS
|
|
AM -->|Alert Events| DRAGONFLY
|
|
|
|
SM -->|Health Checks| DRAGONFLY
|
|
SM -->|Metrics| QDB
|
|
|
|
%% Integration Communications
|
|
BAS -->|Orders| AL
|
|
BAS -->|Orders| IB
|
|
BAS -->|Order Updates| EE
|
|
|
|
DAS -->|Data Feed| MDG
|
|
DAS -->|External Data| HDS
|
|
|
|
NS -->|Notifications| WHS
|
|
NS -->|Alerts| TD
|
|
|
|
WHS -->|Webhooks| External
|
|
|
|
%% Interface Communications
|
|
TD -->|API Calls| API
|
|
TD -->|WebSocket| WS
|
|
TD -->|Dashboard Data| PM
|
|
|
|
API -->|Service Calls| SO
|
|
API -->|Data Queries| HDS
|
|
API -->|System Status| SM
|
|
|
|
WS -->|Real-time Data| TD
|
|
WS -->|Events| DRAGONFLY
|
|
|
|
%% Storage Access DRAGONFLY -.->|Events| SO
|
|
DRAGONFLY -.->|Events| RG
|
|
DRAGONFLY -.->|Events| PM
|
|
DRAGONFLY -.->|Cache| MDG
|
|
|
|
QDB -.->|Time Series| HDS
|
|
QDB -.->|Analytics| AS
|
|
QDB -.->|Metrics| SM
|
|
|
|
PGDB -.->|Relational| PM
|
|
PGDB -.->|Config| CM
|
|
PGDB -.->|Users| API
|
|
|
|
%% Styling
|
|
classDef external fill:#ff9999
|
|
classDef core fill:#99ccff
|
|
classDef intelligence fill:#99ff99
|
|
classDef data fill:#ffcc99
|
|
classDef platform fill:#cc99ff
|
|
classDef integration fill:#ffff99
|
|
classDef interface fill:#ff99cc
|
|
classDef storage fill:#cccccc
|
|
|
|
class AV,YF,AL,IB,NEWS external
|
|
class MDG,RG,EE,PM core
|
|
class SO,SG,BA,ML intelligence
|
|
class HDS,AS,DQS,ETLS data
|
|
class LM,CM,AM,SM platform
|
|
class BAS,DAS,NS,WHS integration
|
|
class TD,API,WS interface
|
|
class DRAGONFLY,QDB,PGDB,FS storage
|
|
```
|
|
|
|
## Communication Patterns
|
|
|
|
### 1. **Event-Driven Architecture (Dragonfly Streams)**
|
|
```
|
|
┌─────────────┐ Dragonfly Events ┌─────────────┐
|
|
│ Service │ ─────────────────→ │ Service │
|
|
│ A │ │ B │
|
|
└─────────────┘ └─────────────┘
|
|
```
|
|
|
|
**Event Types:**
|
|
- `MARKET_DATA` - Real-time price updates
|
|
- `ORDER_CREATED/FILLED/CANCELLED` - Order lifecycle
|
|
- `SIGNAL_GENERATED` - Trading signals
|
|
- `RISK_ALERT` - Risk threshold violations
|
|
- `PORTFOLIO_UPDATE` - Position changes
|
|
- `STRATEGY_START/STOP` - Strategy lifecycle
|
|
|
|
### 2. **Request-Response (HTTP/REST)**
|
|
```
|
|
┌─────────────┐ HTTP Request ┌─────────────┐
|
|
│ Client │ ─────────────────→ │ Service │
|
|
│ │ ←───────────────── │ │
|
|
└─────────────┘ HTTP Response └─────────────┘
|
|
```
|
|
|
|
**API Endpoints:**
|
|
- `/api/market-data/:symbol` - Current market data
|
|
- `/api/portfolio/positions` - Portfolio positions
|
|
- `/api/strategies` - Strategy management
|
|
- `/api/orders` - Order management
|
|
- `/api/health` - Service health checks
|
|
|
|
### 3. **Real-time Streaming (WebSocket)**
|
|
```
|
|
┌─────────────┐ WebSocket ┌─────────────┐
|
|
│ Client │ ←═════════════════→ │ Server │
|
|
│ │ Bidirectional │ │
|
|
└─────────────┘ └─────────────┘
|
|
```
|
|
|
|
**WebSocket Messages:**
|
|
- Market data subscriptions
|
|
- Portfolio updates
|
|
- Trading signals
|
|
- Risk alerts
|
|
- System notifications
|
|
|
|
### 4. **Data Persistence**
|
|
```
|
|
┌─────────────┐ Store/Query ┌─────────────┐
|
|
│ Service │ ─────────────────→ │ Database │
|
|
│ │ ←───────────────── │ │
|
|
└─────────────┘ └─────────────┘
|
|
```
|
|
|
|
**Storage Types:**
|
|
- **Dragonfly**: Events, cache, sessions
|
|
- **QuestDB**: Time-series data, metrics
|
|
- **PostgreSQL**: Configuration, users, metadata
|
|
- **File System**: Logs, configurations
|
|
|
|
## Service Communication Matrix
|
|
|
|
| Service | Publishes Events | Subscribes to Events | HTTP APIs | WebSocket | Storage |
|
|
|---------|-----------------|---------------------|-----------|-----------|---------|
|
|
| Market Data Gateway | ✅ Market Data | - | ✅ REST | ✅ Server | Dragonfly Cache |
|
|
| Risk Guardian | ✅ Risk Alerts | ✅ All Events | ✅ REST | - | PostgreSQL |
|
|
| Strategy Orchestrator | ✅ Strategy Events | ✅ Market Data, Signals | ✅ REST | - | PostgreSQL |
|
|
| Execution Engine | ✅ Order Events | ✅ Strategy Events | ✅ REST | - | PostgreSQL |
|
|
| Portfolio Manager | ✅ Portfolio Events | ✅ Order Events | ✅ REST | - | PostgreSQL |
|
|
| Trading Dashboard | - | ✅ All Events | ✅ Client | ✅ Client | - |
|
|
|
|
## Data Flow Example: Trade Execution
|
|
|
|
```mermaid
|
|
sequenceDiagram
|
|
participant TD as Trading Dashboard
|
|
participant SO as Strategy Orchestrator
|
|
participant SG as Signal Generator
|
|
participant RG as Risk Guardian
|
|
participant EE as Execution Engine
|
|
participant BAS as Broker Adapter
|
|
participant PM as Portfolio Manager
|
|
participant DRAGONFLY as Dragonfly Events
|
|
|
|
Note over TD,DRAGONFLY: User starts a trading strategy
|
|
|
|
TD->>SO: POST /api/strategies/start
|
|
SO->>DRAGONFLY: Publish STRATEGY_START event
|
|
|
|
Note over SO,SG: Strategy generates signals
|
|
|
|
SO->>SG: Request signals for AAPL
|
|
SG->>SO: Return BUY signal
|
|
SO->>DRAGONFLY: Publish SIGNAL_GENERATED event
|
|
|
|
Note over SO,RG: Risk check before execution
|
|
|
|
SO->>RG: Check risk limits
|
|
RG->>SO: Risk approved
|
|
|
|
Note over SO,EE: Execute the trade
|
|
|
|
SO->>EE: Submit order
|
|
EE->>DRAGONFLY: Publish ORDER_CREATED event
|
|
EE->>BAS: Send order to broker
|
|
BAS->>EE: Order filled
|
|
EE->>DRAGONFLY: Publish ORDER_FILLED event
|
|
|
|
Note over PM,TD: Update portfolio and notify user
|
|
|
|
PM->>DRAGONFLY: Subscribe to ORDER_FILLED
|
|
PM->>PM: Update positions PM->>DRAGONFLY: Publish PORTFOLIO_UPDATE
|
|
TD->>DRAGONFLY: Subscribe to PORTFOLIO_UPDATE
|
|
TD->>TD: Update dashboard
|
|
```
|
|
|
|
## Port Allocation
|
|
|
|
| Service Category | Port Range | Services |
|
|
|-----------------|------------|----------|
|
|
| Core Services | 3001-3099 | Market Data Gateway (3001), Risk Guardian (3002) |
|
|
| Intelligence Services | 4001-4099 | Strategy Orchestrator (4001), Signal Generator (4002) |
|
|
| Data Services | 5001-5099 | Historical Data (5001), Analytics (5002) |
|
|
| Platform Services | 6001-6099 | Log Manager (6001), Config Manager (6002) |
|
|
| Integration Services | 7001-7099 | Broker Adapter (7001), Data Adapter (7002) |
|
|
| Interface Services | 8001-8099 | API Gateway (8001), Dashboard (5173-Vite) |
|
|
|
|
## Security & Authentication
|
|
|
|
```mermaid
|
|
graph LR
|
|
subgraph "Security Layer"
|
|
JWT[JWT Tokens]
|
|
API_KEY[API Keys]
|
|
TLS[TLS/HTTPS]
|
|
RBAC[Role-Based Access]
|
|
end
|
|
|
|
subgraph "External Security"
|
|
BROKER_AUTH[Broker Authentication]
|
|
DATA_AUTH[Data Provider Auth]
|
|
WEBHOOK_SIG[Webhook Signatures]
|
|
end
|
|
|
|
JWT --> API_KEY
|
|
API_KEY --> TLS
|
|
TLS --> RBAC
|
|
RBAC --> BROKER_AUTH
|
|
BROKER_AUTH --> DATA_AUTH
|
|
DATA_AUTH --> WEBHOOK_SIG
|
|
```
|
|
|
|
This architecture provides:
|
|
- **Scalability**: Services can be scaled independently
|
|
- **Reliability**: Event-driven communication with retry mechanisms
|
|
- **Maintainability**: Clear separation of concerns
|
|
- **Observability**: Centralized logging and monitoring
|
|
- **Security**: Multiple layers of authentication and authorization
|