# Loki Logging for Stock Bot

This document outlines how to use the Loki logging system integrated with the Stock Bot platform.

## Overview

Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. Unlike other logging systems, Loki is built around the idea of only indexing metadata about your logs (labels), not the full text. This makes Loki more resource-efficient than traditional log storage systems.

For Stock Bot, Loki provides:

1. Centralized logging for all services
2. Log aggregation and filtering by service, level, and custom labels
3. Integration with Grafana for visualization
4. Query capabilities for log analysis
5. Alert capabilities for critical issues

## Getting Started

### Starting the Logging Stack

```cmd
# Start the monitoring stack (includes Loki and Grafana)
scripts\docker.ps1 monitoring
```

You can also start Loki directly using Docker Compose:

```cmd
# Start Loki service only
docker-compose up -d loki

# Start Loki and Grafana
docker-compose up -d loki grafana
```

### Viewing Logs

Once started:

1. Access Grafana at http://localhost:3000 (login with admin/admin)
2. Navigate to the "Stock Bot Logs" dashboard
3. View and query your logs

## Using the Logger in Your Services

The Stock Bot logger has been enhanced to automatically send logs to Loki. Here's how to use it:

```typescript
import { Logger, LogLevel } from '@stock-bot/utils';

// Create a logger for your service
const logger = new Logger('your-service-name', LogLevel.INFO);

// Log at different levels
logger.debug('Detailed information for debugging');
logger.info('General information about operations');
logger.warn('Potential issues that don't affect operation');
logger.error('Critical errors that require attention');

// Log with structured data (will be searchable in Loki)
logger.info('Processing trade', { 
  symbol: 'MSFT', 
  price: 410.75, 
  quantity: 50 
});
```

## Configuration Options

Logger configuration is managed through the `@stock-bot/config` package and can be set in your `.env` file:

```bash
# Logging configuration
LOG_LEVEL=debug           # debug, info, warn, error
LOG_CONSOLE=true          # Log to console in addition to Loki
LOKI_HOST=localhost       # Loki server hostname
LOKI_PORT=3100            # Loki server port
LOKI_RETENTION_DAYS=30    # Days to retain logs
LOKI_LABELS=environment=development,service=stock-bot  # Default labels
LOKI_BATCH_SIZE=100       # Number of logs to batch before sending
LOKI_BATCH_WAIT=5         # Max time to wait before sending logs
```

## Useful Loki Queries

Inside Grafana, you can use these LogQL queries to analyze your logs:

1. **All logs from a specific service**:
   ```
   {service="market-data-gateway"}
   ```

2. **All error logs across all services**:
   ```
   {level="error"}
   ```

3. **Logs containing specific text**:
   ```
   {service="market-data-gateway"} |= "trade"
   ```

4. **Count of error logs by service over time**:
   ```
   sum by(service) (count_over_time({level="error"}[5m]))
   ```

## Testing the Logging Integration

A test script is provided to verify the logging integration:

```bash
# Run from project root
bun run tools/test-loki-logging.ts
```

## Architecture

Our logging implementation follows this architecture:

```
┌─────────────────┐     ┌─────────────────┐
│ Trading Services│────►│ @stock-bot/utils│
└─────────────────┘     │     Logger      │
                        └────────┬────────┘
                                 │
                                 ▼
┌────────────────────────────────────────┐
│               Loki                     │
└────────────────┬───────────────────────┘
                 │
                 ▼
┌────────────────────────────────────────┐
│              Grafana                   │
└────────────────────────────────────────┘
```

## Adding New Dashboards

To create new Grafana dashboards for log visualization:

1. Build your dashboard in the Grafana UI
2. Export it to JSON
3. Add it to `monitoring/grafana/provisioning/dashboards/json/`
4. Restart the monitoring stack

## Troubleshooting

If logs aren't appearing in Grafana:

1. Run the status check script to verify Loki and Grafana are working:
   ```cmd
   tools\check-loki-status.bat
   ```

2. Check that Loki and Grafana containers are running:
   ```cmd
   docker ps | findstr "loki grafana"
   ```

3. Verify .env configuration for Loki host and port:
   ```cmd
   type .env | findstr "LOKI_"
   ```

4. Ensure your service has the latest @stock-bot/utils package

5. Check for errors in the Loki container logs:
   ```cmd
   docker logs trading-bot-loki
   ```