# Loki Logging for Stock Bot

This document outlines how to use the Loki logging system integrated with the Stock Bot platform (Updated June 2025).

## Overview

Loki provides centralized logging for all Stock Bot services with:

1. **Centralized logging** for all microservices
2. **Log aggregation** and filtering by service, level, and custom labels
3. **Grafana integration** for visualization and dashboards
4. **Query capabilities** using LogQL for log analysis
5. **Alert capabilities** for critical issues

## Getting Started

### Starting the Logging Stack

```cmd
# Start the monitoring stack (includes Loki and Grafana)
scripts\docker.ps1 monitoring
```

Or start services individually:

```cmd
# Start Loki service only
docker-compose up -d loki

# Start Loki and Grafana
docker-compose up -d loki grafana
```

### Viewing Logs

Once started:

1. Access Grafana at http://localhost:3000 (login with admin/admin)
2. Navigate to the "Stock Bot Logs" dashboard
3. View and query your logs

## Using the Logger in Your Services

The Stock Bot logger automatically sends logs to Loki using the updated pattern:

```typescript
import { getLogger } from '@stock-bot/logger';

// Create a logger for your service
const logger = getLogger('your-service-name');

// Log at different levels
logger.debug('Detailed information for debugging');
logger.info('General information about operations');
logger.warn('Potential issues that don\'t affect operation');
logger.error('Critical errors that require attention');

// Log with structured data (searchable in Loki)
logger.info('Processing trade', { 
  symbol: 'MSFT', 
  price: 410.75, 
  quantity: 50 
});
```

## Configuration Options

Logger configuration is managed through the `@stock-bot/config` package and can be set in your `.env` file:

```bash
# Logging configuration
LOG_LEVEL=debug           # debug, info, warn, error
LOG_CONSOLE=true          # Log to console in addition to Loki
LOKI_HOST=localhost       # Loki server hostname
LOKI_PORT=3100            # Loki server port
LOKI_RETENTION_DAYS=30    # Days to retain logs
LOKI_LABELS=environment=development,service=stock-bot  # Default labels
LOKI_BATCH_SIZE=100       # Number of logs to batch before sending
LOKI_BATCH_WAIT=5         # Max time to wait before sending logs
```

## Useful Loki Queries

Inside Grafana, you can use these LogQL queries to analyze your logs:

1. **All logs from a specific service**:
   ```
   {service="market-data-gateway"}
   ```

2. **All error logs across all services**:
   ```
   {level="error"}
   ```

3. **Logs containing specific text**:
   ```
   {service="market-data-gateway"} |= "trade"
   ```

4. **Count of error logs by service over time**:
   ```
   sum by(service) (count_over_time({level="error"}[5m]))
   ```

## Testing the Logging Integration

Test the logging integration using Bun:

```cmd
# Run from project root using Bun (current runtime)
bun run tools/test-loki-logging.ts
```

## Architecture

Our logging implementation follows this architecture:

```
┌─────────────────┐     ┌─────────────────┐
│ Trading Services│────►│ @stock-bot/logger│
└─────────────────┘     │   getLogger()   │
                        └────────┬────────┘
                                 │
                                 ▼
┌────────────────────────────────────────┐
│               Loki                     │
└────────────────┬───────────────────────┘
                 │
                 ▼
┌────────────────────────────────────────┐
│              Grafana                   │
└────────────────────────────────────────┘
```

## Adding New Dashboards

To create new Grafana dashboards for log visualization:

1. Build your dashboard in the Grafana UI
2. Export it to JSON
3. Add it to `monitoring/grafana/provisioning/dashboards/json/`
4. Restart the monitoring stack

## Troubleshooting

If logs aren't appearing in Grafana:

1. Run the status check script to verify Loki and Grafana are working:
   ```cmd
   tools\check-loki-status.bat
   ```

2. Check that Loki and Grafana containers are running:
   ```cmd
   docker ps | findstr "loki grafana"
   ```

3. Verify .env configuration for Loki host and port:
   ```cmd
   type .env | findstr "LOKI_"
   ```

4. Ensure your service has the latest @stock-bot/logger package

5. Check for errors in the Loki container logs:
   ```cmd
   docker logs stock-bot-loki
   ```