linxus fs fixes

This commit is contained in:
Boki 2025-06-09 22:55:51 -04:00
parent ac23b70146
commit 0b7846fe67
292 changed files with 41947 additions and 41947 deletions

File diff suppressed because it is too large Load diff

View file

@ -1,169 +1,169 @@
# Loki Logging for Stock Bot
This document outlines how to use the Loki logging system integrated with the Stock Bot platform (Updated June 2025).
## Overview
Loki provides centralized logging for all Stock Bot services with:
1. **Centralized logging** for all microservices
2. **Log aggregation** and filtering by service, level, and custom labels
3. **Grafana integration** for visualization and dashboards
4. **Query capabilities** using LogQL for log analysis
5. **Alert capabilities** for critical issues
## Getting Started
### Starting the Logging Stack
```cmd
# Start the monitoring stack (includes Loki and Grafana)
scripts\docker.ps1 monitoring
```
Or start services individually:
```cmd
# Start Loki service only
docker-compose up -d loki
# Start Loki and Grafana
docker-compose up -d loki grafana
```
### Viewing Logs
Once started:
1. Access Grafana at http://localhost:3000 (login with admin/admin)
2. Navigate to the "Stock Bot Logs" dashboard
3. View and query your logs
## Using the Logger in Your Services
The Stock Bot logger automatically sends logs to Loki using the updated pattern:
```typescript
import { getLogger } from '@stock-bot/logger';
// Create a logger for your service
const logger = getLogger('your-service-name');
// Log at different levels
logger.debug('Detailed information for debugging');
logger.info('General information about operations');
logger.warn('Potential issues that don\'t affect operation');
logger.error('Critical errors that require attention');
// Log with structured data (searchable in Loki)
logger.info('Processing trade', {
symbol: 'MSFT',
price: 410.75,
quantity: 50
});
```
## Configuration Options
Logger configuration is managed through the `@stock-bot/config` package and can be set in your `.env` file:
```bash
# Logging configuration
LOG_LEVEL=debug # debug, info, warn, error
LOG_CONSOLE=true # Log to console in addition to Loki
LOKI_HOST=localhost # Loki server hostname
LOKI_PORT=3100 # Loki server port
LOKI_RETENTION_DAYS=30 # Days to retain logs
LOKI_LABELS=environment=development,service=stock-bot # Default labels
LOKI_BATCH_SIZE=100 # Number of logs to batch before sending
LOKI_BATCH_WAIT=5 # Max time to wait before sending logs
```
## Useful Loki Queries
Inside Grafana, you can use these LogQL queries to analyze your logs:
1. **All logs from a specific service**:
```
{service="market-data-gateway"}
```
2. **All error logs across all services**:
```
{level="error"}
```
3. **Logs containing specific text**:
```
{service="market-data-gateway"} |= "trade"
```
4. **Count of error logs by service over time**:
```
sum by(service) (count_over_time({level="error"}[5m]))
```
## Testing the Logging Integration
Test the logging integration using Bun:
```cmd
# Run from project root using Bun (current runtime)
bun run tools/test-loki-logging.ts
```
## Architecture
Our logging implementation follows this architecture:
```
┌─────────────────┐ ┌─────────────────┐
│ Trading Services│────►│ @stock-bot/logger│
└─────────────────┘ │ getLogger() │
└────────┬────────┘
┌────────────────────────────────────────┐
│ Loki │
└────────────────┬───────────────────────┘
┌────────────────────────────────────────┐
│ Grafana │
└────────────────────────────────────────┘
```
## Adding New Dashboards
To create new Grafana dashboards for log visualization:
1. Build your dashboard in the Grafana UI
2. Export it to JSON
3. Add it to `monitoring/grafana/provisioning/dashboards/json/`
4. Restart the monitoring stack
## Troubleshooting
If logs aren't appearing in Grafana:
1. Run the status check script to verify Loki and Grafana are working:
```cmd
tools\check-loki-status.bat
```
2. Check that Loki and Grafana containers are running:
```cmd
docker ps | findstr "loki grafana"
```
3. Verify .env configuration for Loki host and port:
```cmd
type .env | findstr "LOKI_"
```
4. Ensure your service has the latest @stock-bot/logger package
5. Check for errors in the Loki container logs:
```cmd
docker logs stock-bot-loki
```
# Loki Logging for Stock Bot
This document outlines how to use the Loki logging system integrated with the Stock Bot platform (Updated June 2025).
## Overview
Loki provides centralized logging for all Stock Bot services with:
1. **Centralized logging** for all microservices
2. **Log aggregation** and filtering by service, level, and custom labels
3. **Grafana integration** for visualization and dashboards
4. **Query capabilities** using LogQL for log analysis
5. **Alert capabilities** for critical issues
## Getting Started
### Starting the Logging Stack
```cmd
# Start the monitoring stack (includes Loki and Grafana)
scripts\docker.ps1 monitoring
```
Or start services individually:
```cmd
# Start Loki service only
docker-compose up -d loki
# Start Loki and Grafana
docker-compose up -d loki grafana
```
### Viewing Logs
Once started:
1. Access Grafana at http://localhost:3000 (login with admin/admin)
2. Navigate to the "Stock Bot Logs" dashboard
3. View and query your logs
## Using the Logger in Your Services
The Stock Bot logger automatically sends logs to Loki using the updated pattern:
```typescript
import { getLogger } from '@stock-bot/logger';
// Create a logger for your service
const logger = getLogger('your-service-name');
// Log at different levels
logger.debug('Detailed information for debugging');
logger.info('General information about operations');
logger.warn('Potential issues that don\'t affect operation');
logger.error('Critical errors that require attention');
// Log with structured data (searchable in Loki)
logger.info('Processing trade', {
symbol: 'MSFT',
price: 410.75,
quantity: 50
});
```
## Configuration Options
Logger configuration is managed through the `@stock-bot/config` package and can be set in your `.env` file:
```bash
# Logging configuration
LOG_LEVEL=debug # debug, info, warn, error
LOG_CONSOLE=true # Log to console in addition to Loki
LOKI_HOST=localhost # Loki server hostname
LOKI_PORT=3100 # Loki server port
LOKI_RETENTION_DAYS=30 # Days to retain logs
LOKI_LABELS=environment=development,service=stock-bot # Default labels
LOKI_BATCH_SIZE=100 # Number of logs to batch before sending
LOKI_BATCH_WAIT=5 # Max time to wait before sending logs
```
## Useful Loki Queries
Inside Grafana, you can use these LogQL queries to analyze your logs:
1. **All logs from a specific service**:
```
{service="market-data-gateway"}
```
2. **All error logs across all services**:
```
{level="error"}
```
3. **Logs containing specific text**:
```
{service="market-data-gateway"} |= "trade"
```
4. **Count of error logs by service over time**:
```
sum by(service) (count_over_time({level="error"}[5m]))
```
## Testing the Logging Integration
Test the logging integration using Bun:
```cmd
# Run from project root using Bun (current runtime)
bun run tools/test-loki-logging.ts
```
## Architecture
Our logging implementation follows this architecture:
```
┌─────────────────┐ ┌─────────────────┐
│ Trading Services│────►│ @stock-bot/logger│
└─────────────────┘ │ getLogger() │
└────────┬────────┘
┌────────────────────────────────────────┐
│ Loki │
└────────────────┬───────────────────────┘
┌────────────────────────────────────────┐
│ Grafana │
└────────────────────────────────────────┘
```
## Adding New Dashboards
To create new Grafana dashboards for log visualization:
1. Build your dashboard in the Grafana UI
2. Export it to JSON
3. Add it to `monitoring/grafana/provisioning/dashboards/json/`
4. Restart the monitoring stack
## Troubleshooting
If logs aren't appearing in Grafana:
1. Run the status check script to verify Loki and Grafana are working:
```cmd
tools\check-loki-status.bat
```
2. Check that Loki and Grafana containers are running:
```cmd
docker ps | findstr "loki grafana"
```
3. Verify .env configuration for Loki host and port:
```cmd
type .env | findstr "LOKI_"
```
4. Ensure your service has the latest @stock-bot/logger package
5. Check for errors in the Loki container logs:
```cmd
docker logs stock-bot-loki
```

View file

@ -1,65 +1,65 @@
# Testing with Bun in Stock Bot Platform
The Stock Bot platform uses [Bun Test](https://bun.sh/docs/cli/test) as the primary testing framework (Updated June 2025). Bun Test provides fast, modern testing with Jest-like API compatibility.
## Getting Started
Run tests using these commands:
```cmd
# Run all tests (using Turbo)
bun test
# Run tests in watch mode
bun test:watch
# Run tests with coverage
bun test:coverage
# Run specific test types
bun test:unit
bun test:integration
bun test:e2e
```
## Library-specific Testing
Each library has its own testing configuration in a `bunfig.toml` file. This allows for library-specific test settings while maintaining consistent patterns across the codebase.
### Example bunfig.toml:
```toml
[test]
preload = ["./test/setup.ts"]
timeout = 5000
[test.env]
NODE_ENV = "test"
[bun]
paths = {
"@/*" = ["./src/*"]
}
```
## Migration from Jest
This project has been fully migrated from Jest to Bun Test. Some key differences:
1. **Import statements**: Use `import { describe, it, expect } from 'bun:test'` instead of Jest imports
2. **Mocking**: Use Bun's built-in mocking utilities (see global `spyOn` helper)
3. **Configuration**: Use `bunfig.toml` instead of Jest config files
4. **Test helpers**: Test helpers are available globally via `global.testHelpers`
## Best Practices
- Use `describe` and `it` for test organization
- Use relative imports (`../src/`) in test files
- Keep test setup clean with proper `beforeEach` and `afterEach` handlers
- For complex test scenarios, create dedicated setup files
## Test Environment
- All tests run with `NODE_ENV=test`
- Console output is silenced by default (restore with `testHelpers.restoreConsole()`)
- Default timeout is 30 seconds for integration tests, 5 seconds for unit tests
# Testing with Bun in Stock Bot Platform
The Stock Bot platform uses [Bun Test](https://bun.sh/docs/cli/test) as the primary testing framework (Updated June 2025). Bun Test provides fast, modern testing with Jest-like API compatibility.
## Getting Started
Run tests using these commands:
```cmd
# Run all tests (using Turbo)
bun test
# Run tests in watch mode
bun test:watch
# Run tests with coverage
bun test:coverage
# Run specific test types
bun test:unit
bun test:integration
bun test:e2e
```
## Library-specific Testing
Each library has its own testing configuration in a `bunfig.toml` file. This allows for library-specific test settings while maintaining consistent patterns across the codebase.
### Example bunfig.toml:
```toml
[test]
preload = ["./test/setup.ts"]
timeout = 5000
[test.env]
NODE_ENV = "test"
[bun]
paths = {
"@/*" = ["./src/*"]
}
```
## Migration from Jest
This project has been fully migrated from Jest to Bun Test. Some key differences:
1. **Import statements**: Use `import { describe, it, expect } from 'bun:test'` instead of Jest imports
2. **Mocking**: Use Bun's built-in mocking utilities (see global `spyOn` helper)
3. **Configuration**: Use `bunfig.toml` instead of Jest config files
4. **Test helpers**: Test helpers are available globally via `global.testHelpers`
## Best Practices
- Use `describe` and `it` for test organization
- Use relative imports (`../src/`) in test files
- Keep test setup clean with proper `beforeEach` and `afterEach` handlers
- For complex test scenarios, create dedicated setup files
## Test Environment
- All tests run with `NODE_ENV=test`
- Console output is silenced by default (restore with `testHelpers.restoreConsole()`)
- Default timeout is 30 seconds for integration tests, 5 seconds for unit tests

View file

@ -1,118 +1,118 @@
# TypeScript Configuration Structure
This document explains the TypeScript configuration structure used in the Stock Bot trading platform.
## Root Configuration
The root `tsconfig.json` at the project root establishes common settings for all projects in the monorepo:
```json
{
"$schema": "https://json.schemastore.org/tsconfig",
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"noImplicitThis": true,
"alwaysStrict": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"sourceMap": false,
"declaration": true,
"baseUrl": ".",
"paths": {
"@stock-bot/*": ["libs/*/src"]
}
},
"exclude": [
"node_modules",
"dist"
]
}
```
## Template Configurations
We provide two template configurations:
1. `tsconfig.lib.json` - For library projects:
```json
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"declaration": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts", "**/*.spec.ts"]
}
```
2. `tsconfig.app.json` - For application projects:
```json
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"module": "ESNext",
"moduleResolution": "bundler",
"types": ["bun-types"]
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
```
## Project-Specific Configurations
Each project in the monorepo extends from the root configuration and adds its own specific settings:
### Library Projects
Library projects extend the root configuration with a relative path:
```json
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"declaration": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts", "**/*.spec.ts"]
}
```
### Application Projects
Application projects also extend the root configuration with a relative path:
```json
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"module": "ESNext",
"moduleResolution": "bundler",
"types": ["bun-types"]
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
```
## Special Configurations
Some projects have special needs:
1. **Trading Dashboard (Angular)**: Uses an extended configuration structure with separate files for app and testing.
2. **Projects with TypeScript imports from extensions**: These projects set `"allowImportingTsExtensions": true` and `"noEmit": true`.
# TypeScript Configuration Structure
This document explains the TypeScript configuration structure used in the Stock Bot trading platform.
## Root Configuration
The root `tsconfig.json` at the project root establishes common settings for all projects in the monorepo:
```json
{
"$schema": "https://json.schemastore.org/tsconfig",
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"noImplicitAny": true,
"strictNullChecks": true,
"noImplicitThis": true,
"alwaysStrict": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"sourceMap": false,
"declaration": true,
"baseUrl": ".",
"paths": {
"@stock-bot/*": ["libs/*/src"]
}
},
"exclude": [
"node_modules",
"dist"
]
}
```
## Template Configurations
We provide two template configurations:
1. `tsconfig.lib.json` - For library projects:
```json
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"declaration": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts", "**/*.spec.ts"]
}
```
2. `tsconfig.app.json` - For application projects:
```json
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"module": "ESNext",
"moduleResolution": "bundler",
"types": ["bun-types"]
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
```
## Project-Specific Configurations
Each project in the monorepo extends from the root configuration and adds its own specific settings:
### Library Projects
Library projects extend the root configuration with a relative path:
```json
{
"extends": "../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"declaration": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist", "**/*.test.ts", "**/*.spec.ts"]
}
```
### Application Projects
Application projects also extend the root configuration with a relative path:
```json
{
"extends": "../../../tsconfig.json",
"compilerOptions": {
"outDir": "./dist",
"rootDir": "./src",
"module": "ESNext",
"moduleResolution": "bundler",
"types": ["bun-types"]
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}
```
## Special Configurations
Some projects have special needs:
1. **Trading Dashboard (Angular)**: Uses an extended configuration structure with separate files for app and testing.
2. **Projects with TypeScript imports from extensions**: These projects set `"allowImportingTsExtensions": true` and `"noEmit": true`.