# YOLO Detection Pipeline

Capture frames → annotate → train → detect live in-game.

## Setup

```bash
cd tools/python-detect
python -m venv .venv
.venv/Scripts/activate        # Windows
pip install ultralytics opencv-python
# GPU training (NVIDIA):
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu128 --force-reinstall
```

## 1. Capture Frames

In the app Debug tab, click **Burst Capture** during a boss fight. Frames save to `training-data/raw/` every 200ms. Click **Stop Capture** when done.

## 2. Annotate

```bash
python annotate.py                              # defaults to ../../training-data/raw
python annotate.py ../../training-data/raw

tools\python-detect\.venv\Scripts\python.exe tools\python-detect\annotate.py training-data\raw
```

### Toolbar

Clickable buttons at the top of the window:

| Button | Hotkey | Action |
|---|---|---|
| Predict | **P** | Run YOLO model on current image to auto-generate boxes |
| Save+Next | **Space** | Save labels and advance to next image |
| Filter | **F** | Cycle filter: All → Unlabeled → Labeled → per-class |
| Undo | **Z** | Undo last change |
| Del Image | **X** | Delete current image file and its label |

### Controls

| Action | Input |
|---|---|
| Draw new box | Left-drag on empty area |
| Select box | Left-click on box |
| Move box | Left-drag box body |
| Resize box | Left-drag corner handle |
| Cycle class | Right-click on box |
| Set class | 1-9 keys (selected box or new-box default) |
| Delete box | Delete / Backspace |
| Navigate | Left / Right arrow |
| Deselect | E |
| Quit | Q / Escape (auto-saves) |

### Predict (P)

Press **P** or click the Predict button to run the current YOLO model (`models/boss-v1.pt`) on the image. It replaces existing boxes with model predictions (undoable with Z). The model lazy-loads on first use.

Workflow: press P to auto-predict → adjust/delete bad boxes → Space to save+next.

### Classes

Edit the `CLASSES` list at the top of `annotate.py` to add new boss types:

```python
CLASSES = ["kulemak", "arbiter"]
```

## 3. Split Dataset

Create a train/valid split (85/15) for training. Example structure:

```
training-data/boss-dataset/
├── data.yaml
├── train/
│   ├── images/
│   └── labels/
└── valid/
    ├── images/
    └── labels/
```

`data.yaml`:

```yaml
train: C:/Users/boki/repos/poe2trade/training-data/boss-dataset/train/images
val: C:/Users/boki/repos/poe2trade/training-data/boss-dataset/valid/images

nc: 2
names: ['kulemak', 'arbiter']
```

Copy images and their `.txt` label files into the appropriate split directories.

## 4. Train

```bash
python train.py --data C:/Users/boki/repos/poe2trade/training-data/boss-dataset/data.yaml --name boss-v1 --epochs 100
```

Options: `--batch 16`, `--imgsz 640`, `--device 0` (GPU) or `--device cpu`.

Best weights are auto-copied to `models/boss-v1.pt`. If ultralytics auto-increments the run name (e.g. `boss-v12`), copy manually:

```bash
cp runs/detect/boss-v1*/weights/best.pt models/boss-v1.pt
```

## 5. Live Detection

The C# app loads `models/boss-v1.pt` via `daemon.py` (JSON stdin/stdout protocol). Enable detection in the Mapping tab. Boss bounding boxes render on the Direct2D overlay in cyan.

## Files

| File | Purpose |
|---|---|
| `annotate.py` | Interactive annotator with predict, select/move/resize, tag, filter |
| `train.py` | Train YOLOv11n, copies best weights to `models/` |
| `daemon.py` | Persistent inference daemon (managed by C# `PythonDetectBridge`) |
| `fix_labels.py` | One-time fix for labels generated with the old buggy annotator |
| `models/` | Trained `.pt` weight files |