123 lines
3.6 KiB
Markdown
123 lines
3.6 KiB
Markdown
# YOLO Detection Pipeline
|
|
|
|
Capture frames → annotate → train → detect live in-game.
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
cd tools/python-detect
|
|
python -m venv .venv
|
|
.venv/Scripts/activate # Windows
|
|
pip install ultralytics opencv-python
|
|
# GPU training (NVIDIA):
|
|
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128 --force-reinstall
|
|
```
|
|
|
|
## 1. Capture Frames
|
|
|
|
In the app Debug tab, click **Burst Capture** during a boss fight. Frames save to `training-data/raw/` every 200ms. Click **Stop Capture** when done.
|
|
|
|
## 2. Annotate
|
|
|
|
```bash
|
|
python annotate.py # defaults to ../../training-data/raw
|
|
python annotate.py ../../training-data/raw
|
|
|
|
tools\python-detect\.venv\Scripts\python.exe tools\python-detect\annotate.py training-data\raw
|
|
```
|
|
|
|
### Toolbar
|
|
|
|
Clickable buttons at the top of the window:
|
|
|
|
| Button | Hotkey | Action |
|
|
|---|---|---|
|
|
| Predict | **P** | Run YOLO model on current image to auto-generate boxes |
|
|
| Save+Next | **Space** | Save labels and advance to next image |
|
|
| Filter | **F** | Cycle filter: All → Unlabeled → Labeled → per-class |
|
|
| Undo | **Z** | Undo last change |
|
|
| Del Image | **X** | Delete current image file and its label |
|
|
|
|
### Controls
|
|
|
|
| Action | Input |
|
|
|---|---|
|
|
| Draw new box | Left-drag on empty area |
|
|
| Select box | Left-click on box |
|
|
| Move box | Left-drag box body |
|
|
| Resize box | Left-drag corner handle |
|
|
| Cycle class | Right-click on box |
|
|
| Set class | 1-9 keys (selected box or new-box default) |
|
|
| Delete box | Delete / Backspace |
|
|
| Navigate | Left / Right arrow |
|
|
| Deselect | E |
|
|
| Quit | Q / Escape (auto-saves) |
|
|
|
|
### Predict (P)
|
|
|
|
Press **P** or click the Predict button to run the current YOLO model (`models/boss-v1.pt`) on the image. It replaces existing boxes with model predictions (undoable with Z). The model lazy-loads on first use.
|
|
|
|
Workflow: press P to auto-predict → adjust/delete bad boxes → Space to save+next.
|
|
|
|
### Classes
|
|
|
|
Edit the `CLASSES` list at the top of `annotate.py` to add new boss types:
|
|
|
|
```python
|
|
CLASSES = ["kulemak", "arbiter"]
|
|
```
|
|
|
|
## 3. Split Dataset
|
|
|
|
Create a train/valid split (85/15) for training. Example structure:
|
|
|
|
```
|
|
training-data/boss-dataset/
|
|
├── data.yaml
|
|
├── train/
|
|
│ ├── images/
|
|
│ └── labels/
|
|
└── valid/
|
|
├── images/
|
|
└── labels/
|
|
```
|
|
|
|
`data.yaml`:
|
|
|
|
```yaml
|
|
train: C:/Users/boki/repos/poe2trade/training-data/boss-dataset/train/images
|
|
val: C:/Users/boki/repos/poe2trade/training-data/boss-dataset/valid/images
|
|
|
|
nc: 2
|
|
names: ['kulemak', 'arbiter']
|
|
```
|
|
|
|
Copy images and their `.txt` label files into the appropriate split directories.
|
|
|
|
## 4. Train
|
|
|
|
```bash
|
|
python train.py --data C:/Users/boki/repos/poe2trade/training-data/boss-dataset/data.yaml --name boss-v1 --epochs 100
|
|
```
|
|
|
|
Options: `--batch 16`, `--imgsz 640`, `--device 0` (GPU) or `--device cpu`.
|
|
|
|
Best weights are auto-copied to `models/boss-v1.pt`. If ultralytics auto-increments the run name (e.g. `boss-v12`), copy manually:
|
|
|
|
```bash
|
|
cp runs/detect/boss-v1*/weights/best.pt models/boss-v1.pt
|
|
```
|
|
|
|
## 5. Live Detection
|
|
|
|
The C# app loads `models/boss-v1.pt` via `daemon.py` (JSON stdin/stdout protocol). Enable detection in the Mapping tab. Boss bounding boxes render on the Direct2D overlay in cyan.
|
|
|
|
## Files
|
|
|
|
| File | Purpose |
|
|
|---|---|
|
|
| `annotate.py` | Interactive annotator with predict, select/move/resize, tag, filter |
|
|
| `train.py` | Train YOLOv11n, copies best weights to `models/` |
|
|
| `daemon.py` | Persistent inference daemon (managed by C# `PythonDetectBridge`) |
|
|
| `fix_labels.py` | One-time fix for labels generated with the old buggy annotator |
|
|
| `models/` | Trained `.pt` weight files |
|