80 lines
No EOL
2.4 KiB
Markdown
80 lines
No EOL
2.4 KiB
Markdown
# QM Spider Symbol Search Test
|
|
|
|
## How the Spider Search Works
|
|
|
|
The spider search is a recursive symbol discovery system that explores the QM API systematically:
|
|
|
|
### 1. Root Job (Weekly Schedule)
|
|
```typescript
|
|
// Triggered every Saturday at midnight
|
|
@ScheduledOperation('spider-symbol-search', '0 0 * * 6', {...})
|
|
|
|
// Creates 26 jobs: A, B, C, ... Z
|
|
input: { prefix: null, depth: 0, maxDepth: 4 }
|
|
```
|
|
|
|
### 2. Level 1 - Single Letters (A-Z)
|
|
```typescript
|
|
// Each job searches for symbols starting with that letter
|
|
input: { prefix: 'A', depth: 1, maxDepth: 4 }
|
|
|
|
// If 10+ symbols found, creates: AA, AB, AC, ... AZ
|
|
```
|
|
|
|
### 3. Level 2 - Two Letters (AA-ZZ)
|
|
```typescript
|
|
// Searches for symbols starting with two letters
|
|
input: { prefix: 'AA', depth: 2, maxDepth: 4 }
|
|
|
|
// If 10+ symbols found, creates: AAA, AAB, AAC, ... AAZ
|
|
```
|
|
|
|
### 4. Level 3 & 4 - Deeper Search
|
|
Continues until `maxDepth` is reached or fewer than 10 symbols are found.
|
|
|
|
## Testing the Spider
|
|
|
|
### Manual Test - Root Job
|
|
```bash
|
|
# Trigger spider search root job
|
|
curl -X POST http://localhost:3000/api/handlers/qm/operations/spider-symbol-search \
|
|
-H "Content-Type: application/json" \
|
|
-d '{}'
|
|
```
|
|
|
|
### Manual Test - Specific Prefix
|
|
```bash
|
|
# Search for symbols starting with "APP"
|
|
curl -X POST http://localhost:3000/api/handlers/qm/operations/spider-symbol-search \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"prefix": "APP", "depth": 3, "maxDepth": 4}'
|
|
```
|
|
|
|
### Direct Symbol Search
|
|
```bash
|
|
# Search for specific symbols
|
|
curl -X POST http://localhost:3000/api/handlers/qm/operations/search-symbols \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"query": "AAPL"}'
|
|
```
|
|
|
|
## Optimization Features
|
|
|
|
1. **Intelligent Branching**: Only creates child jobs if 10+ symbols found
|
|
2. **Priority Management**: Deeper searches get lower priority
|
|
3. **Staggered Execution**: Jobs are delayed to avoid API rate limits
|
|
4. **Session Management**: Uses QM sessions with failure tracking
|
|
5. **Caching**: Results cached for 30 minutes to avoid duplicate API calls
|
|
|
|
## MongoDB Collections
|
|
|
|
- `qm_symbols`: Stores discovered symbols with metadata
|
|
- `qm_exchanges`: Stores unique exchanges found during searches
|
|
|
|
## Monitoring
|
|
|
|
Check logs for:
|
|
- "Spider symbol search" - Job execution
|
|
- "Created root spider jobs" - Initial A-Z creation
|
|
- "Stored symbols from spider search" - Successful symbol storage
|
|
- "Found X symbols for Y, queued Z child jobs" - Branching decisions |