# Edge network watchdog

**One script:** `wan-watch.sh` — deploy everywhere, read one log, get a verdict every 5 minutes.

## Deploy

```bash
sudo cp examples/network/wan-watch.sh /usr/local/bin/wan-watch.sh
sudo chmod +x /usr/local/bin/wan-watch.sh
sudo cp examples/network/wan-watch.env.example /etc/conf.d/wan-watch
# edit WAN_WATCH_ROLE=pi|router|fe
```

Remove old cron entries for `wan-snapshot.sh` / `monstro-netdiag.sh`. **One cron only:**

```cron
*/5 * * * * root . /etc/conf.d/wan-watch 2>/dev/null; /usr/local/bin/wan-watch.sh >>/var/log/wan-watch.cron.log 2>&1
```

Log: `/var/log/wan-watch.log`

## Every run (5 min) captures

| Layer | What |
|-------|------|
| **Ping matrix** | wan, fe, be, gw, router — loss/min/avg/max/mdev |
| **Verdict** | `kind=verdict` — immediate conclusion + action hint |
| **Tunnels** | GRE links/modules, PPTP (1723/pppd/pptpd), **tinc** (tincctl status), **WireGuard**, **iodine/dns0/tun0** |
| **Edge services** | named, squid, nginx, iodined — proc count + CPU |
| **Host** | load, cpu, conntrack, softnet drops, ss summary |
| **Network** | routes, neigh, iface drops, qdisc backlog, iptables hot rules (router) |
| **On anomaly** | `kind=alert` + `kind=deep` burst pings + full probe repeat (5 min cooldown) |

## Commands

```bash
wan-watch.sh                    # cron default
wan-watch.sh --full             # instant human dump (when slow NOW)
wan-watch.sh --analyze          # final conclusion from log (last 24h) — no waiting
wan-watch.sh --analyze 6        # last 6 hours
wan-watch.sh --timeline         # hourly good/bad bars
wan-watch.sh --correlate /var/log/wan-watch.log path/to/other/log
wan-watch.sh --force-deep       # full capture even if OK
wan-watch.sh --tail 30
```

## Verdicts (automatic)

| verdict | Meaning |
|---------|---------|
| `OK` | Within thresholds |
| `MONSTRO_WAN_BLOAT` | Router WAN slow, FE local OK — **your main issue** |
| `UPSTREAM_GW` | WAN + gw 10.7.16.228 both slow |
| `PATH_TO_FE` | Pi ISP OK, path to 10.7.0.10 slow |
| `CONNTRACK_PRESSURE` | nf_conntrack > 70% |
| `TUNNEL_DNS0_LOAD` | iodine dns0 drops elevated |
| `WAN_JITTER` | High mdev, bufferbloat signal |

Cron stdout example:

```text
WAN-WATCH [router/monstro] verdict=MONSTRO_WAN_BLOAT confidence=high summary="WAN leg slow..." action="SQM/fq_codel on xenbr0..."
```

## `--analyze` example output

```text
CONCLUSION: MONSTRO WAN EDGE QUEUEING (confidence=HIGH)
  Fix order: 1) fq_codel/SQM on xenbr0  2) gw 10.7.16.228  3) squid/named  4) iodine dns0
```

Run this **any time** — no need to wait 24h after deploy.

## Host config (`/etc/conf.d/wan-watch`)

```sh
WAN_WATCH_LOG=/var/log/wan-watch.log
WAN_WATCH_ROLE=pi              # or router on monstro
ROUTER_PING_TARGET=10.7.6.228  # Pi only
WAN_WATCH_DEEP=auto
WAN_WATCH_COOLDOWN_S=300
```

## Legacy

- `wan-snapshot.sh` → wrapper to `wan-watch.sh`
- `wan-deep-diag.sh` → merged into `wan-watch.sh`
- `monstro-netdiag.sh` → use `wan-watch.sh --full` instead

## Known conclusion (from your logs)

**MONSTRO WAN EDGE QUEUEING** on `xenbr0` → upstream `10.7.16.228`. Not FE/BE, not Pi ISP. Contributing load: **named**, **squid**, **dns0** (iodine). Fix: **SQM/fq_codel** on WAN iface first.