207 lines
7.7 KiB
Markdown
207 lines
7.7 KiB
Markdown
# Infrastructure & Game Interaction Projects
|
||
|
||
## Nexus.GameOffsets — Memory Layout Definitions
|
||
|
||
Pure offset structs for POE2 game memory. No logic, no reading — just struct layouts.
|
||
|
||
**Contents:**
|
||
- **Entities/**: `EntityStruct`, `EntityDetails`, `ComponentLookup`, `ComponentNameAndIndex`, `ItemStruct`, `EntityTreeNode`
|
||
- **Components/** (22 structs): Actor, Animated, Buffs, Chest, Life, Mods, Player, Positioned, Render, Stats, Targetable, Transitionable, WorldItem, etc.
|
||
- **States/**: `InGameState`, `AreaInstance`, `AreaLoading`, `ServerData`, `WorldData`, `Inventory`, `ImportantUiElements`
|
||
- **Natives/**: C++ STL memory layouts — `StdVector`, `StdMap`, `StdList`, `StdBucket`, `StdWString`, `StdTuple`
|
||
|
||
**Key offsets:**
|
||
- Actor skills: 0xB00 (ActiveSkillsVector), 0xB18 (CooldownsVector)
|
||
- UIElement: 0x10 (Children), 0x98 (StringId), 0x180 (Flags), 0x448 (Text)
|
||
- Entity: 0x80 (ID), 0x84 (Flags), 0x08 (EntityDetails)
|
||
|
||
**Dependencies**: None. Used by Memory and Data layers.
|
||
|
||
---
|
||
|
||
## Nexus.Screen — Screen Capture, OCR & Detection
|
||
|
||
Screen capture, OCR, image processing, grid/item detection, loot label detection.
|
||
|
||
### Core Components
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| ScreenReader | Main facade — OCR, template matching, screenshot, diff OCR |
|
||
| IScreenCapture | Desktop duplication or GDI capture |
|
||
| IOcrEngine | Interface for OCR backends (Win native, EasyOCR, OneOCR, WinOCR) |
|
||
| PythonOcrBridge | Calls Python script via subprocess for EasyOCR/YOLO |
|
||
|
||
### Grid & Item Detection
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| GridReader | Reads stash/inventory grids (12-col 70×70px or 24-col 35×35px) |
|
||
| GridHandler | Template matching for cell occupancy, item size detection |
|
||
| TemplateMatchHandler | NCC-based visual matching (find identical items in grid) |
|
||
| DetectGridHandler | Edge detection to find grid boundaries |
|
||
|
||
### Detection Systems
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| EnemyDetector | YOLO/ONNX object detection for enemy positions |
|
||
| BossDetector | Boss-specific recognition |
|
||
| HudReader | HUD element OCR (HP bar, mana, buffs) |
|
||
| GameStateDetector | Main menu vs in-game state |
|
||
| ScreenReader.DetectLootLabels() | Three-pass loot detection (polygon, contour, yellow text) |
|
||
|
||
### Frame Pipeline
|
||
|
||
Pub-sub for screen frames: `FramePipeline` distributes captured frames to multiple `IFrameConsumer` implementations (GameState, Enemy, Boss detectors, Minimap, Navigation).
|
||
|
||
**Used by**: Bot, Navigation, Inventory, Ui
|
||
|
||
---
|
||
|
||
## Nexus.Game — Game Interaction
|
||
|
||
Low-level game control — window focus, input sending, clipboard operations.
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| GameController | Main facade — focus, chat, input, shortcuts |
|
||
| InputSender | Win32 SendInput (scan codes), Bézier mouse movement, Ctrl+click |
|
||
| WindowManager | SetForegroundWindow (with alt-key trick), GetWindowRect |
|
||
| ClipboardHelper | System clipboard read/write |
|
||
|
||
**Key operations:**
|
||
- `FocusWindow()` — SetForegroundWindow + alt-key trick (required for background processes)
|
||
- `CtrlRightClick()` — buying from seller stash
|
||
- `MoveMouse()` — Bézier curve smooth move
|
||
- `MoveMouseInstant()` — direct teleport (no interpolation)
|
||
- `TypeText()`, `SelectAll()`, `Paste()` — clipboard operations
|
||
|
||
**Used by**: Inventory, Trade, Items, Navigation, Bot
|
||
|
||
---
|
||
|
||
## Nexus.Log — Game Log Watcher
|
||
|
||
Parses Client.txt game log at 200ms poll intervals.
|
||
|
||
| Event | Pattern |
|
||
|-------|---------|
|
||
| AreaEntered | `[SCENE] Set Source [AreaName]` or `You have entered AreaName` |
|
||
| WhisperReceived | Incoming whisper messages |
|
||
| WhisperSent | Outgoing whisper messages |
|
||
| TradeAccepted | Trade completion |
|
||
| PartyJoined/Left | Party state changes |
|
||
| LineReceived | Raw log lines |
|
||
|
||
`CurrentArea` detected from log tail on startup. Used by Bot (reset navigation on area change), Inventory (wait for area transitions), Navigation.
|
||
|
||
---
|
||
|
||
## Nexus.Trade — Trade Daemon IPC
|
||
|
||
Manages trade search monitoring via external Node.js Playwright daemon.
|
||
|
||
### TradeDaemonBridge
|
||
|
||
Spawns `node tools/trade-daemon/daemon.mjs`, communicates via stdin/stdout JSON.
|
||
|
||
**Commands (→ daemon):**
|
||
- `start`, `addSearch`, `addDiamondSearch`
|
||
- `pauseSearch`, `clickTravel`
|
||
- `openScrapPage`, `reloadScrapPage`, `closeScrapPage`
|
||
|
||
**Events (← daemon):**
|
||
- `newListings` → `NewListings(searchId, items[])`
|
||
- `diamondListings` → `DiamondListings(searchId, pricedItems[])`
|
||
- `wsClose` → websocket disconnection
|
||
|
||
**Trade flow**: Website "Travel to Hideout" button → stash opens → Ctrl+right-click to buy → `/hideout` to go home → store items
|
||
|
||
---
|
||
|
||
## Nexus.Items — Item Parsing
|
||
|
||
Parse item text from clipboard (Ctrl+C) using Sidekick item parser library.
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| ItemReader | Move to item → Ctrl+C → read clipboard → parse |
|
||
| SidekickBootstrapper | Initialize Sidekick parser on first use |
|
||
|
||
**Used by**: Bot (identify items during scraping)
|
||
|
||
---
|
||
|
||
## Nexus.Inventory — Stash & Grid Management
|
||
|
||
Scan player inventory, track item placement, deposit to stash.
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| InventoryManager | Main interface — scan, deposit, clear |
|
||
| InventoryTracker | Cell occupancy matrix + item metadata |
|
||
| StashCalibrator | Grid boundary calibration via edge detection |
|
||
|
||
**Key operations:**
|
||
- `ScanInventory()` → screenshot + grid scan → populate tracker
|
||
- `DepositItemsToStash()` → find stash NPC → click items with Shift+Ctrl
|
||
- `DepositAllToOpenStash()` → scan → click first occupied → repeat
|
||
- `ClearToStash()` → scan → deposit all → return to hideout
|
||
- `EnsureAtOwnHideout()` → `/hideout` command if needed
|
||
|
||
**Grid calibration (2560×1440):**
|
||
- Cell sizes: 70×70px (12-col) or 35×35px (24-col), all 840px wide
|
||
- Inventory (12×5): origin (1696, 788)
|
||
- Stash 12×12: origin (23, 169) or (23, 216) in folder
|
||
|
||
---
|
||
|
||
## Nexus.Navigation — Minimap-Based Movement
|
||
|
||
Real-time navigation using minimap image matching + pathfinding. Separate from Nexus.Pathfinding (which is grid-based A*).
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| NavigationExecutor | State machine: Capture → Process → Plan → Move → Stuck |
|
||
| MinimapCapture | Frame pipeline consumer — wall color detection, checkpoint detection |
|
||
| WorldMap | Position matching via cross-correlation, canvas stitching |
|
||
| StuckDetector | No-progress detection |
|
||
| WallColorTracker | Learns wall palette from initial spawn |
|
||
|
||
**Flow**: Capture minimap → detect position via wall color stitching → pathfind → send WASD keys
|
||
|
||
---
|
||
|
||
## Nexus.Bot — Top-Level Orchestration
|
||
|
||
Central coordinator that wires everything together.
|
||
|
||
| Class | Purpose |
|
||
|-------|---------|
|
||
| BotOrchestrator | DI container, state machine, frame pipeline management |
|
||
| TradeExecutor | Single trade flow (navigate → buy → deposit) |
|
||
| MappingExecutor | Map exploration (navigate + loot) |
|
||
| KulemakExecutor | Boss fight with arena mechanics |
|
||
| CraftingExecutor | Crafting bench operations |
|
||
| DiamondExecutor | Diamond trade handling |
|
||
| ScrapExecutor | Vendor scrapping |
|
||
| TradeQueue | FIFO queue of trade tasks |
|
||
| LinkManager | Trade search management |
|
||
|
||
**Bot modes**: Trading, Mapping, Crafting (via BotMode enum)
|
||
|
||
---
|
||
|
||
## Nexus.Ui — Avalonia Desktop Application
|
||
|
||
Entry point executable. Avalonia 11.2 + CommunityToolkit.MVVM + FluentTheme.
|
||
|
||
**App.xaml.cs** wires all DI:
|
||
- Services: ConfigStore, GameController, ScreenReader, ClientLogWatcher, TradeMonitor, InventoryManager
|
||
- Bot: FramePipelineService, LinkManager, TradeExecutor, TradeQueue, BotOrchestrator, ModPoolService
|
||
- ViewModels: Main, Debug, Settings, Mapping, Atlas, Crafting, Memory, Nexus, ObjectBrowser
|
||
|
||
**Additional dependencies**: Vortice.Direct2D1 (overlay rendering), Microsoft.Extensions.DependencyInjection
|
||
|
||
**Views**: MainWindow, DebugWindow, SettingsWindow, MappingWindow, etc. with MVVM bindings.
|