initial setup

DOCKER.md

@@ -0,0 +1,272 @@
+# 🐳 Trading Bot Docker Infrastructure
+
+This directory contains the Docker Compose configuration for the Trading Bot infrastructure, including databases, caching, monitoring, and admin tools.
+
+## 🏗️ Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    Trading Bot Infrastructure                    │
+├─────────────────────────────────────────────────────────────────┤
+│  🐉 Dragonfly (Redis)  │  🐘 PostgreSQL  │  📊 QuestDB         │
+│  Events & Caching      │  Operational DB  │  Time-series Data   │
+│  Port: 6379           │  Port: 5432      │  Port: 9000/8812    │
+├─────────────────────────────────────────────────────────────────┤
+│  🔧 Redis Insight     │  🛠️ PgAdmin      │  📈 Prometheus      │
+│  Dragonfly GUI        │  PostgreSQL GUI  │  Metrics Collection │
+│  Port: 8001          │  Port: 8080      │  Port: 9090         │
+├─────────────────────────────────────────────────────────────────┤
+│                    📊 Grafana Dashboard                         │
+│                    Visualization & Alerting                     │
+│                         Port: 3000                             │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## 🚀 Quick Start
+
+### 1. Start Core Infrastructure
+```powershell
+# Start essential services (Dragonfly, PostgreSQL, QuestDB)
+npm run infra:up
+
+# Or use the management script
+npm run docker:start
+```
+
+### 2. Start Admin Interfaces
+```powershell
+# Start Redis Insight and PgAdmin
+npm run docker:admin
+```
+
+### 3. Start Full Development Environment
+```powershell
+# Start infrastructure + admin + your trading services
+npm run dev:full
+```
+
+## 📊 Service Access Points
+
+| Service | URL | Credentials | Purpose |
+|---------|-----|-------------|---------|
+| **Dragonfly** | `localhost:6379` | None | Redis-compatible cache & events |
+| **PostgreSQL** | `localhost:5432` | `trading_user` / `trading_pass_dev` | Operational database |
+| **QuestDB Console** | http://localhost:9000 | None | Time-series database GUI |
+| **QuestDB Wire** | `localhost:8812` | `admin` / `quest` | PostgreSQL protocol access |
+| **Redis Insight** | http://localhost:8001 | None | Dragonfly/Redis management |
+| **PgAdmin** | http://localhost:8080 | `admin@tradingbot.local` / `admin123` | PostgreSQL management |
+| **Prometheus** | http://localhost:9090 | None | Metrics collection |
+| **Grafana** | http://localhost:3000 | `admin` / `admin123` | Dashboards & alerts |
+
+## 🛠️ Management Commands
+
+### Using NPM Scripts (Recommended)
+```powershell
+# Infrastructure management
+npm run docker:start          # Start core services
+npm run docker:stop           # Stop all services
+npm run docker:restart        # Restart services
+npm run docker:status         # Show service status
+npm run docker:logs          # Show logs
+npm run docker:reset         # Reset all data (destructive)
+
+# Additional services
+npm run docker:admin          # Start admin interfaces
+npm run docker:monitoring     # Start monitoring stack
+
+# Quick development
+npm run dev:full              # Infrastructure + admin + services
+npm run dev:clean             # Reset + full restart
+```
+
+### Using PowerShell Script Directly
+```powershell
+# Basic operations
+./scripts/docker.ps1 start
+./scripts/docker.ps1 stop
+./scripts/docker.ps1 status
+
+# View logs for specific service
+./scripts/docker.ps1 logs -Service dragonfly
+
+# Development mode (lighter configuration)
+./scripts/docker.ps1 start -Dev
+
+# Get help
+./scripts/docker.ps1 help
+```
+
+### Using Docker Compose Directly
+```powershell
+# Start core services
+docker-compose up -d dragonfly postgres questdb
+
+# Start with development overrides
+docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d
+
+# Start monitoring (optional)
+docker-compose --profile monitoring up -d
+
+# View logs
+docker-compose logs -f
+
+# Stop everything
+docker-compose down
+
+# Reset all data
+docker-compose down -v
+```
+
+## 🗃️ Database Schemas
+
+### PostgreSQL Structure
+```sql
+trading.*          -- Orders, positions, executions, accounts
+strategy.*         -- Strategies, signals, performance
+risk.*             -- Risk limits, events, monitoring
+audit.*            -- System events, health, configuration
+```
+
+### Initial Data
+The PostgreSQL database is automatically initialized with:
+- Common stock symbols (AAPL, GOOGL, etc.)
+- Demo trading account with $100k
+- Sample mean reversion strategy
+- Basic risk limits
+- Audit triggers for data tracking
+
+## 📈 Monitoring & Observability
+
+### Prometheus Metrics
+The setup includes Prometheus configuration to monitor:
+- Trading bot services (ports 3001, 3002, 4001)
+- Dragonfly performance
+- PostgreSQL health
+- QuestDB metrics
+
+### Grafana Dashboards
+Access Grafana at http://localhost:3000 to view:
+- Trading system performance
+- Database metrics
+- Service health
+- Risk monitoring
+
+## 🔧 Configuration
+
+### Environment Variables
+Services use these environment variables (configured in docker-compose.yml):
+
+```yaml
+# Dragonfly
+DRAGONFLY_HOST=localhost
+DRAGONFLY_PORT=6379
+
+# PostgreSQL
+POSTGRES_HOST=localhost
+POSTGRES_PORT=5432
+POSTGRES_DB=trading_bot
+POSTGRES_USER=trading_user
+POSTGRES_PASSWORD=trading_pass_dev
+
+# QuestDB
+QUESTDB_HOST=localhost
+QUESTDB_PORT=8812
+QUESTDB_DB=qdb
+```
+
+### Development vs Production
+- **Development**: Use `docker-compose.dev.yml` for lighter resource usage
+- **Production**: Use base `docker-compose.yml` with proper secrets management
+
+## 🚨 Troubleshooting
+
+### Common Issues
+
+**Port Conflicts**
+```powershell
+# Check what's using a port
+netstat -ano | findstr :6379
+
+# Kill process if needed
+taskkill /PID <process_id> /F
+```
+
+**Permission Issues**
+```powershell
+# Ensure Docker is running as administrator
+# Check Docker Desktop settings
+```
+
+**Data Corruption**
+```powershell
+# Reset all data and restart fresh
+npm run docker:reset
+```
+
+**Service Won't Start**
+```powershell
+# Check logs for specific service
+npm run docker:logs
+./scripts/docker.ps1 logs -Service dragonfly
+```
+
+### Health Checks
+All services include health checks. Monitor with:
+```powershell
+npm run docker:status
+docker ps --filter "name=trading-bot"
+```
+
+## 🔒 Security Notes
+
+### Development Security
+- Default passwords are for development only
+- No SSL/TLS in development mode
+- Services exposed on localhost only
+
+### Production Considerations
+- Change all default passwords
+- Enable SSL/TLS encryption
+- Use Docker secrets for sensitive data
+- Implement proper backup strategies
+- Set up log rotation
+- Configure firewall rules
+
+## 📋 Maintenance
+
+### Backup Data
+```powershell
+# Backup PostgreSQL
+docker exec trading-bot-postgres pg_dump -U trading_user trading_bot > backup.sql
+
+# Backup Dragonfly (if persistence enabled)
+docker exec trading-bot-dragonfly redis-cli BGSAVE
+```
+
+### Update Services
+```powershell
+# Pull latest images
+docker-compose pull
+
+# Restart with new images
+npm run docker:restart
+```
+
+### Clean Up
+```powershell
+# Remove unused containers and images
+docker system prune
+
+# Remove all trading-bot related containers and volumes
+docker-compose down -v --remove-orphans
+```
+
+## 🎯 Next Steps
+
+1. **Start the infrastructure**: `npm run docker:start`
+2. **Open admin interfaces**: `npm run docker:admin`
+3. **Configure your services**: Update .env files to use Docker services
+4. **Start development**: `npm run dev:full`
+5. **Monitor health**: Check http://localhost:8001 and http://localhost:8080
+
+For more advanced configuration, see the individual service documentation in the `database/` and `monitoring/` directories.