169 lines
5 KiB
Markdown
169 lines
5 KiB
Markdown
# Loki Logging for Stock Bot
|
|
|
|
This document outlines how to use the Loki logging system integrated with the Stock Bot platform (Updated June 2025).
|
|
|
|
## Overview
|
|
|
|
Loki provides centralized logging for all Stock Bot services with:
|
|
|
|
1. **Centralized logging** for all microservices
|
|
2. **Log aggregation** and filtering by service, level, and custom labels
|
|
3. **Grafana integration** for visualization and dashboards
|
|
4. **Query capabilities** using LogQL for log analysis
|
|
5. **Alert capabilities** for critical issues
|
|
|
|
## Getting Started
|
|
|
|
### Starting the Logging Stack
|
|
|
|
```cmd
|
|
# Start the monitoring stack (includes Loki and Grafana)
|
|
scripts\docker.ps1 monitoring
|
|
```
|
|
|
|
Or start services individually:
|
|
|
|
```cmd
|
|
# Start Loki service only
|
|
docker-compose up -d loki
|
|
|
|
# Start Loki and Grafana
|
|
docker-compose up -d loki grafana
|
|
```
|
|
|
|
### Viewing Logs
|
|
|
|
Once started:
|
|
|
|
1. Access Grafana at http://localhost:3000 (login with admin/admin)
|
|
2. Navigate to the "Stock Bot Logs" dashboard
|
|
3. View and query your logs
|
|
|
|
## Using the Logger in Your Services
|
|
|
|
The Stock Bot logger automatically sends logs to Loki using the updated pattern:
|
|
|
|
```typescript
|
|
import { getLogger } from '@stock-bot/logger';
|
|
|
|
// Create a logger for your service
|
|
const logger = getLogger('your-service-name');
|
|
|
|
// Log at different levels
|
|
logger.debug('Detailed information for debugging');
|
|
logger.info('General information about operations');
|
|
logger.warn('Potential issues that don\'t affect operation');
|
|
logger.error('Critical errors that require attention');
|
|
|
|
// Log with structured data (searchable in Loki)
|
|
logger.info('Processing trade', {
|
|
symbol: 'MSFT',
|
|
price: 410.75,
|
|
quantity: 50
|
|
});
|
|
```
|
|
|
|
## Configuration Options
|
|
|
|
Logger configuration is managed through the `@stock-bot/config` package and can be set in your `.env` file:
|
|
|
|
```bash
|
|
# Logging configuration
|
|
LOG_LEVEL=debug # debug, info, warn, error
|
|
LOG_CONSOLE=true # Log to console in addition to Loki
|
|
LOKI_HOST=localhost # Loki server hostname
|
|
LOKI_PORT=3100 # Loki server port
|
|
LOKI_RETENTION_DAYS=30 # Days to retain logs
|
|
LOKI_LABELS=environment=development,service=stock-bot # Default labels
|
|
LOKI_BATCH_SIZE=100 # Number of logs to batch before sending
|
|
LOKI_BATCH_WAIT=5 # Max time to wait before sending logs
|
|
```
|
|
|
|
## Useful Loki Queries
|
|
|
|
Inside Grafana, you can use these LogQL queries to analyze your logs:
|
|
|
|
1. **All logs from a specific service**:
|
|
```
|
|
{service="market-data-gateway"}
|
|
```
|
|
|
|
2. **All error logs across all services**:
|
|
```
|
|
{level="error"}
|
|
```
|
|
|
|
3. **Logs containing specific text**:
|
|
```
|
|
{service="market-data-gateway"} |= "trade"
|
|
```
|
|
|
|
4. **Count of error logs by service over time**:
|
|
```
|
|
sum by(service) (count_over_time({level="error"}[5m]))
|
|
```
|
|
|
|
## Testing the Logging Integration
|
|
|
|
Test the logging integration using Bun:
|
|
|
|
```cmd
|
|
# Run from project root using Bun (current runtime)
|
|
bun run tools/test-loki-logging.ts
|
|
```
|
|
|
|
## Architecture
|
|
|
|
Our logging implementation follows this architecture:
|
|
|
|
```
|
|
┌─────────────────┐ ┌─────────────────┐
|
|
│ Trading Services│────►│ @stock-bot/logger│
|
|
└─────────────────┘ │ getLogger() │
|
|
└────────┬────────┘
|
|
│
|
|
▼
|
|
┌────────────────────────────────────────┐
|
|
│ Loki │
|
|
└────────────────┬───────────────────────┘
|
|
│
|
|
▼
|
|
┌────────────────────────────────────────┐
|
|
│ Grafana │
|
|
└────────────────────────────────────────┘
|
|
```
|
|
|
|
## Adding New Dashboards
|
|
|
|
To create new Grafana dashboards for log visualization:
|
|
|
|
1. Build your dashboard in the Grafana UI
|
|
2. Export it to JSON
|
|
3. Add it to `monitoring/grafana/provisioning/dashboards/json/`
|
|
4. Restart the monitoring stack
|
|
|
|
## Troubleshooting
|
|
|
|
If logs aren't appearing in Grafana:
|
|
|
|
1. Run the status check script to verify Loki and Grafana are working:
|
|
```cmd
|
|
tools\check-loki-status.bat
|
|
```
|
|
|
|
2. Check that Loki and Grafana containers are running:
|
|
```cmd
|
|
docker ps | findstr "loki grafana"
|
|
```
|
|
|
|
3. Verify .env configuration for Loki host and port:
|
|
```cmd
|
|
type .env | findstr "LOKI_"
|
|
```
|
|
|
|
4. Ensure your service has the latest @stock-bot/logger package
|
|
|
|
5. Check for errors in the Loki container logs:
|
|
```cmd
|
|
docker logs stock-bot-loki
|
|
```
|