# Nexus.Data & Nexus.Memory — The Data Pipeline

## Overview

Two-layer architecture: **Memory** reads raw bytes from the game process; **Data** interprets, classifies, and caches them. This separation keeps memory reading logic free of business rules and makes the data layer testable independently.

```
Game Process (RPM)
    │
    ▼
Nexus.Memory (raw reads, no business logic)
    │  GameMemoryReader → hierarchical state tree
    │  EntityList → red-black tree traversal
    │  ComponentReader → ECS component extraction
    │
    ▼
Nexus.Data (interpretation, classification, caching)
    │  MemoryPoller → two-tier event loop
    │  EntityMapper → Memory.Entity → Core.EntitySnapshot
    │  EntityClassifier → path + components → EntityCategory
    │  GameStateEnricher → derived threat/danger metrics
    │
    ▼
GameDataCache (volatile references, lock-free)
    │
    ▼
Bot Systems (read-only consumers)
```

---

## GameDataCache — Single Source of Truth

Thread-safe, volatile reference holder. Writer: MemoryPoller thread. Readers: bot systems.

**Hot fields** (updated at 60Hz — 4 lightweight RPM calls):
- CameraMatrix (64 bytes)
- PlayerPosition (X, Y, Z)
- PlayerVitals (HP, mana, ES current/max)
- IsLoading, IsEscapeOpen

**Cold fields** (updated at 10Hz — full hierarchical read):
- Entities, HostileMonsters, NearbyLoot
- Terrain (WalkabilitySnapshot)
- AreaHash, AreaLevel, CurrentAreaName
- Quest data (linked lists, UI groups, state entries)
- LatestState (complete GameState)

**Slow fields** (updated at 1Hz):
- Character name
- Quest linked lists, quest state entries

No locks — relies on volatile reference semantics for atomic swaps.

---

## MemoryPoller — Two-Tier Event Loop

Owns the memory-reading background thread.

### Hot Tick (60Hz)

4 pre-resolved RPM calls using cached addresses:
1. Camera matrix (64 bytes from cached address)
2. Player position (12 bytes: X, Y, Z)
3. Player vitals (24 bytes: HP, mana, ES)
4. Loading/escape state (pointer dereference + int)

No allocations, no GC. Targeting ~3-5ms per tick.

### Cold Tick (10Hz, every 6th hot tick)

Full hierarchical read:
1. `GameMemoryReader.ReadSnapshot()` — cascades through state tree
2. Re-resolve hot addresses via `ResolveHotAddresses()`
3. `BuildGameState()` — map entities, filter lists, process quests
4. `GameStateEnricher.Enrich()` — compute derived fields
5. Update all cache fields

### BuildGameState() Flow

```
ReadSnapshot() → GameStateSnapshot (raw)
    │
    ├── Entity mapping:
    │   for each entity in snapshot:
    │     EntityMapper.MapEntity(entity, playerPos) → EntitySnapshot
    │
    ├── Filter into:
    │   - HostileMonsters (Category==Monster && IsAlive)
    │   - NearbyLoot (Category==WorldItem)
    │   - All entities
    │
    ├── Quest processing:
    │   - Filter active (StateId > 0)
    │   - Resolve state text via QuestStateLookup
    │   - Convert to QuestProgress, QuestInfo
    │
    └── Returns GameState
```

---

## GameMemoryReader — Hierarchical State Tree

Top-level orchestrator. Creates sub-readers on `Attach()`:

```
GameStates (top)
└── InGameState
    ├── AreaInstance
    │   ├── EntityList (MSVC std::map red-black tree)
    │   ├── PlayerSkills (Actor component)
    │   ├── QuestStates (dat file entries)
    │   └── Terrain (walkability grid)
    ├── UIElements (quest linked lists, UI tree)
    └── WorldData (camera matrix)
```

Each `RemoteObject` caches its data and depends on parent for context. Single `Update()` call cascades through tree.

### Infrastructure

- **ProcessMemory**: P/Invoke wrapper for ReadProcessMemory. Tracks reads/sec and KB/sec.
- **MemoryContext**: Shared state — process handle, offsets, module base, pattern scanner.
- **ComponentReader**: Reads ECS components (Life, Render, Mods, etc.) from entities.
- **MsvcStringReader**: Reads MSVC std::wstring (SSO-aware: inline if capacity ≤ 8, heap pointer otherwise).
- **PatternScanner**: AOB scan for resolving base addresses.

---

## EntityList — Tree Traversal

Reads entities from AreaInstance's MSVC std::map (red-black tree, in-order traversal).

**Tree node layout:**
```
+0x00: left child ptr
+0x08: parent ptr
+0x10: right child ptr
+0x28: entity pointer
```

**Optimization**: Tree order is cached — re-walked only when entity count changes.

**Per-entity reads:**
1. Path (EntityDetails → std::wstring)
2. Skip low-priority types (effects, terrain, critters — no components read)
3. Position (Render component: X, Y, Z)
4. Component lookup (STL hash map: name → index)
5. Component data:
   - Targetable (bool flag)
   - Mods/ObjectMagicProperties (rarity)
   - Life (HP, dynamic — re-read every frame for monsters)
   - Actor (action ID)
   - WorldItem (inner entity for ground loot)
   - AreaTransition (destination area)

**Caching strategy:**
- Stable per entity: path, component list, targetable, rarity, transition name
- Dynamic (re-read every frame): monster HP, action ID

---

## EntityClassifier — Path + Components → Category

Single source of truth for entity classification.

1. **Path-based** (primary): Parses `Metadata/[Category]/...` path segments
2. **Component override**: Monster, Chest, Shrine, Waypoint, AreaTransition, Portal, TownPortal, NPC, Player

Output: `EntityCategory` (Core enum, 17 types)

---

## EntityMapper — Memory.Entity → Core.EntitySnapshot

Transforms raw memory data to enriched snapshots:

```
Memory.Entity (raw)
    │
    ├── Copy: ID, path, metadata, position, Z, vitals, components, mods
    ├── Classify: EntityClassifier.Classify(path, components) → EntityCategory
    ├── Threat level: Rarity → MonsterThreatLevel
    ├── Area name: AreaNameLookup.Resolve() for transitions
    ├── Distance: Vector2.Distance(position, playerPos)
    └── Alive state: HasVitals ? LifeCurrent > 0 : true
    │
    ▼
Core.EntitySnapshot (public, classified, enriched)
```

---

## GameStateEnricher — Derived Metrics

Computed once per cold tick, before systems run.

**NearestEnemies**: HostileMonsters sorted by distance to player.

**ThreatMap**:
- TotalHostiles, CloseRange (<300u), MidRange (300-600u), FarRange (600-1200u)
- ClosestDistance, ThreatCentroid (position average), HasRareOrUnique

**DangerLevel** — Weighted threat score:
```
score = Σ (distance_weight × rarity_multiplier)

Distance weights:  <200u = 3×,  <400u = 2×,  else = 1×
Rarity multipliers: Unique=5×, Rare=3×, Magic=1.5×, White=1×

Life override:  HP < 30% → Critical,  HP < 50% → High
Score thresholds: ≥15 = Critical, ≥8 = High, ≥4 = Medium, >0 = Low, 0 = Safe
```

---

## Key Architectural Patterns

| Pattern | Implementation |
|---------|---------------|
| Lock-free cross-thread | Volatile references in GameDataCache; no locks needed |
| Two-tier polling | Hot (4 RPM, 60Hz) + Cold (full read, 10Hz) |
| Hierarchical caching | Each RemoteObject caches data, re-reads only on change |
| Entity caching | Stable data cached per entity/zone; dynamic data (HP) re-read per frame |
| Separation of concerns | Memory: raw bytes. Data: interpretation + classification |
| Area name resolution | AreaNameLookup loads areas.json, caches ID → display name |
| Area graph | BFS pathfinding for quest progression ordering |