272 lines
8.1 KiB
Markdown
272 lines
8.1 KiB
Markdown
# 🐳 Trading Bot Docker Infrastructure
|
|
|
|
This directory contains the Docker Compose configuration for the Trading Bot infrastructure, including databases, caching, monitoring, and admin tools.
|
|
|
|
## 🏗️ Architecture Overview
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────┐
|
|
│ Trading Bot Infrastructure │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ 🐉 Dragonfly (Redis) │ 🐘 PostgreSQL │ 📊 QuestDB │
|
|
│ Events & Caching │ Operational DB │ Time-series Data │
|
|
│ Port: 6379 │ Port: 5432 │ Port: 9000/8812 │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ 🔧 Redis Insight │ 🛠️ PgAdmin │ 📈 Prometheus │
|
|
│ Dragonfly GUI │ PostgreSQL GUI │ Metrics Collection │
|
|
│ Port: 8001 │ Port: 8080 │ Port: 9090 │
|
|
├─────────────────────────────────────────────────────────────────┤
|
|
│ 📊 Grafana Dashboard │
|
|
│ Visualization & Alerting │
|
|
│ Port: 3000 │
|
|
└─────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
## 🚀 Quick Start
|
|
|
|
### 1. Start Core Infrastructure
|
|
```powershell
|
|
# Start essential services (Dragonfly, PostgreSQL, QuestDB)
|
|
npm run infra:up
|
|
|
|
# Or use the management script
|
|
npm run docker:start
|
|
```
|
|
|
|
### 2. Start Admin Interfaces
|
|
```powershell
|
|
# Start Redis Insight and PgAdmin
|
|
npm run docker:admin
|
|
```
|
|
|
|
### 3. Start Full Development Environment
|
|
```powershell
|
|
# Start infrastructure + admin + your trading services
|
|
npm run dev:full
|
|
```
|
|
|
|
## 📊 Service Access Points
|
|
|
|
| Service | URL | Credentials | Purpose |
|
|
|---------|-----|-------------|---------|
|
|
| **Dragonfly** | `localhost:6379` | None | Redis-compatible cache & events |
|
|
| **PostgreSQL** | `localhost:5432` | `trading_user` / `trading_pass_dev` | Operational database |
|
|
| **QuestDB Console** | http://localhost:9000 | None | Time-series database GUI |
|
|
| **QuestDB Wire** | `localhost:8812` | `admin` / `quest` | PostgreSQL protocol access |
|
|
| **Redis Insight** | http://localhost:8001 | None | Dragonfly/Redis management |
|
|
| **PgAdmin** | http://localhost:8080 | `admin@tradingbot.local` / `admin123` | PostgreSQL management |
|
|
| **Prometheus** | http://localhost:9090 | None | Metrics collection |
|
|
| **Grafana** | http://localhost:3000 | `admin` / `admin123` | Dashboards & alerts |
|
|
|
|
## 🛠️ Management Commands
|
|
|
|
### Using NPM Scripts (Recommended)
|
|
```powershell
|
|
# Infrastructure management
|
|
npm run docker:start # Start core services
|
|
npm run docker:stop # Stop all services
|
|
npm run docker:restart # Restart services
|
|
npm run docker:status # Show service status
|
|
npm run docker:logs # Show logs
|
|
npm run docker:reset # Reset all data (destructive)
|
|
|
|
# Additional services
|
|
npm run docker:admin # Start admin interfaces
|
|
npm run docker:monitoring # Start monitoring stack
|
|
|
|
# Quick development
|
|
npm run dev:full # Infrastructure + admin + services
|
|
npm run dev:clean # Reset + full restart
|
|
```
|
|
|
|
### Using PowerShell Script Directly
|
|
```powershell
|
|
# Basic operations
|
|
./scripts/docker.ps1 start
|
|
./scripts/docker.ps1 stop
|
|
./scripts/docker.ps1 status
|
|
|
|
# View logs for specific service
|
|
./scripts/docker.ps1 logs -Service dragonfly
|
|
|
|
# Development mode (lighter configuration)
|
|
./scripts/docker.ps1 start -Dev
|
|
|
|
# Get help
|
|
./scripts/docker.ps1 help
|
|
```
|
|
|
|
### Using Docker Compose Directly
|
|
```powershell
|
|
# Start core services
|
|
docker-compose up -d dragonfly postgres questdb
|
|
|
|
# Start with development overrides
|
|
docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d
|
|
|
|
# Start monitoring (optional)
|
|
docker-compose --profile monitoring up -d
|
|
|
|
# View logs
|
|
docker-compose logs -f
|
|
|
|
# Stop everything
|
|
docker-compose down
|
|
|
|
# Reset all data
|
|
docker-compose down -v
|
|
```
|
|
|
|
## 🗃️ Database Schemas
|
|
|
|
### PostgreSQL Structure
|
|
```sql
|
|
trading.* -- Orders, positions, executions, accounts
|
|
strategy.* -- Strategies, signals, performance
|
|
risk.* -- Risk limits, events, monitoring
|
|
audit.* -- System events, health, configuration
|
|
```
|
|
|
|
### Initial Data
|
|
The PostgreSQL database is automatically initialized with:
|
|
- Common stock symbols (AAPL, GOOGL, etc.)
|
|
- Demo trading account with $100k
|
|
- Sample mean reversion strategy
|
|
- Basic risk limits
|
|
- Audit triggers for data tracking
|
|
|
|
## 📈 Monitoring & Observability
|
|
|
|
### Prometheus Metrics
|
|
The setup includes Prometheus configuration to monitor:
|
|
- Trading bot services (ports 3001, 3002, 4001)
|
|
- Dragonfly performance
|
|
- PostgreSQL health
|
|
- QuestDB metrics
|
|
|
|
### Grafana Dashboards
|
|
Access Grafana at http://localhost:3000 to view:
|
|
- Trading system performance
|
|
- Database metrics
|
|
- Service health
|
|
- Risk monitoring
|
|
|
|
## 🔧 Configuration
|
|
|
|
### Environment Variables
|
|
Services use these environment variables (configured in docker-compose.yml):
|
|
|
|
```yaml
|
|
# Dragonfly
|
|
DRAGONFLY_HOST=localhost
|
|
DRAGONFLY_PORT=6379
|
|
|
|
# PostgreSQL
|
|
POSTGRES_HOST=localhost
|
|
POSTGRES_PORT=5432
|
|
POSTGRES_DB=trading_bot
|
|
POSTGRES_USER=trading_user
|
|
POSTGRES_PASSWORD=trading_pass_dev
|
|
|
|
# QuestDB
|
|
QUESTDB_HOST=localhost
|
|
QUESTDB_PORT=8812
|
|
QUESTDB_DB=qdb
|
|
```
|
|
|
|
### Development vs Production
|
|
- **Development**: Use `docker-compose.dev.yml` for lighter resource usage
|
|
- **Production**: Use base `docker-compose.yml` with proper secrets management
|
|
|
|
## 🚨 Troubleshooting
|
|
|
|
### Common Issues
|
|
|
|
**Port Conflicts**
|
|
```powershell
|
|
# Check what's using a port
|
|
netstat -ano | findstr :6379
|
|
|
|
# Kill process if needed
|
|
taskkill /PID <process_id> /F
|
|
```
|
|
|
|
**Permission Issues**
|
|
```powershell
|
|
# Ensure Docker is running as administrator
|
|
# Check Docker Desktop settings
|
|
```
|
|
|
|
**Data Corruption**
|
|
```powershell
|
|
# Reset all data and restart fresh
|
|
npm run docker:reset
|
|
```
|
|
|
|
**Service Won't Start**
|
|
```powershell
|
|
# Check logs for specific service
|
|
npm run docker:logs
|
|
./scripts/docker.ps1 logs -Service dragonfly
|
|
```
|
|
|
|
### Health Checks
|
|
All services include health checks. Monitor with:
|
|
```powershell
|
|
npm run docker:status
|
|
docker ps --filter "name=trading-bot"
|
|
```
|
|
|
|
## 🔒 Security Notes
|
|
|
|
### Development Security
|
|
- Default passwords are for development only
|
|
- No SSL/TLS in development mode
|
|
- Services exposed on localhost only
|
|
|
|
### Production Considerations
|
|
- Change all default passwords
|
|
- Enable SSL/TLS encryption
|
|
- Use Docker secrets for sensitive data
|
|
- Implement proper backup strategies
|
|
- Set up log rotation
|
|
- Configure firewall rules
|
|
|
|
## 📋 Maintenance
|
|
|
|
### Backup Data
|
|
```powershell
|
|
# Backup PostgreSQL
|
|
docker exec trading-bot-postgres pg_dump -U trading_user trading_bot > backup.sql
|
|
|
|
# Backup Dragonfly (if persistence enabled)
|
|
docker exec trading-bot-dragonfly redis-cli BGSAVE
|
|
```
|
|
|
|
### Update Services
|
|
```powershell
|
|
# Pull latest images
|
|
docker-compose pull
|
|
|
|
# Restart with new images
|
|
npm run docker:restart
|
|
```
|
|
|
|
### Clean Up
|
|
```powershell
|
|
# Remove unused containers and images
|
|
docker system prune
|
|
|
|
# Remove all trading-bot related containers and volumes
|
|
docker-compose down -v --remove-orphans
|
|
```
|
|
|
|
## 🎯 Next Steps
|
|
|
|
1. **Start the infrastructure**: `npm run docker:start`
|
|
2. **Open admin interfaces**: `npm run docker:admin`
|
|
3. **Configure your services**: Update .env files to use Docker services
|
|
4. **Start development**: `npm run dev:full`
|
|
5. **Monitor health**: Check http://localhost:8001 and http://localhost:8080
|
|
|
|
For more advanced configuration, see the individual service documentation in the `database/` and `monitoring/` directories.
|