Loki Logging for Stock Bot

This document outlines how to use the Loki logging system integrated with the Stock Bot platform (Updated June 2025).

Overview

Loki provides centralized logging for all Stock Bot services with:

Centralized logging for all microservices
Log aggregation and filtering by service, level, and custom labels
Grafana integration for visualization and dashboards
Query capabilities using LogQL for log analysis
Alert capabilities for critical issues

Getting Started

Starting the Logging Stack

# Start the monitoring stack (includes Loki and Grafana)
scripts\docker.ps1 monitoring

Or start services individually:

# Start Loki service only
docker-compose up -d loki

# Start Loki and Grafana
docker-compose up -d loki grafana

Viewing Logs

Once started:

Access Grafana at http://localhost:3000 (login with admin/admin)
Navigate to the "Stock Bot Logs" dashboard
View and query your logs

Using the Logger in Your Services

The Stock Bot logger automatically sends logs to Loki using the updated pattern:

import { getLogger } from '@stock-bot/logger';

// Create a logger for your service
const logger = getLogger('your-service-name');

// Log at different levels
logger.debug('Detailed information for debugging');
logger.info('General information about operations');
logger.warn('Potential issues that don\'t affect operation');
logger.error('Critical errors that require attention');

// Log with structured data (searchable in Loki)
logger.info('Processing trade', { 
  symbol: 'MSFT', 
  price: 410.75, 
  quantity: 50 
});

Configuration Options

Logger configuration is managed through the @stock-bot/config package and can be set in your .env file:

# Logging configuration
LOG_LEVEL=debug           # debug, info, warn, error
LOG_CONSOLE=true          # Log to console in addition to Loki
LOKI_HOST=localhost       # Loki server hostname
LOKI_PORT=3100            # Loki server port
LOKI_RETENTION_DAYS=30    # Days to retain logs
LOKI_LABELS=environment=development,service=stock-bot  # Default labels
LOKI_BATCH_SIZE=100       # Number of logs to batch before sending
LOKI_BATCH_WAIT=5         # Max time to wait before sending logs

Useful Loki Queries

Inside Grafana, you can use these LogQL queries to analyze your logs:

All logs from a specific service:
```
{service="market-data-gateway"}
```
All error logs across all services:
```
{level="error"}
```

Logs containing specific text:

{service="market-data-gateway"} |= "trade"

Count of error logs by service over time:

sum by(service) (count_over_time({level="error"}[5m]))

Testing the Logging Integration

Test the logging integration using Bun:

# Run from project root using Bun (current runtime)
bun run tools/test-loki-logging.ts

Architecture

Our logging implementation follows this architecture:

┌─────────────────┐     ┌─────────────────┐
│ Trading Services│────►│ @stock-bot/logger│
└─────────────────┘     │   getLogger()   │
                        └────────┬────────┘
                                 │
                                 ▼
┌────────────────────────────────────────┐
│               Loki                     │
└────────────────┬───────────────────────┘
                 │
                 ▼
┌────────────────────────────────────────┐
│              Grafana                   │
└────────────────────────────────────────┘

Adding New Dashboards

To create new Grafana dashboards for log visualization:

Build your dashboard in the Grafana UI
Export it to JSON
Add it to monitoring/grafana/provisioning/dashboards/json/
Restart the monitoring stack

Troubleshooting

If logs aren't appearing in Grafana:

Run the status check script to verify Loki and Grafana are working:
```
tools\check-loki-status.bat
```
Check that Loki and Grafana containers are running:
```
docker ps | findstr "loki grafana"
```
Verify .env configuration for Loki host and port:
```
type .env | findstr "LOKI_"
```
Ensure your service has the latest @stock-bot/logger package
Check for errors in the Loki container logs:
```
docker logs stock-bot-loki
```

5 KiB Raw Permalink Blame History