Edge network probes
Scripts for the Debian router (134.17.26.161 / 10.7.6.228) — not FE/BE app hosts.
wan-snapshot.sh
Periodic ping comparison:
| Target | Default | Meaning |
|---|---|---|
| WAN | 1.1.1.1 |
ISP/upstream leg (outbound from router) |
| FE | 10.7.0.10 |
LAN leg router → Gentoo FE |
| BE | 10.7.16.128 |
Optional deeper LAN (disable: BE_PING_TARGET=none) |
Install (router)
sudo cp examples/network/wan-snapshot.sh /usr/local/bin/wan-snapshot.sh
sudo chmod +x /usr/local/bin/wan-snapshot.sh
Cron (every 15 minutes)
*/15 * * * * root /usr/local/bin/wan-snapshot.sh >>/var/log/wan-quality.cron.log 2>&1
Log: /var/log/wan-quality.log (override with WAN_SNAPSHOT_LOG).
Usage
# one line appended + printed
sudo /usr/local/bin/wan-snapshot.sh
# compare tonight vs yesterday
/usr/local/bin/wan-snapshot.sh --report
# recent samples
/usr/local/bin/wan-snapshot.sh --tail 48
Example log line:
ts=2026-06-04T21:15:01+02:00 wan=1.1.1.1 wan_loss=0% wan_avg_ms=11.2 fe=10.7.0.10 fe_loss=0% fe_avg_ms=0.3 be=10.7.16.128 be_loss=0% be_avg_ms=0.5
Reading results
wan_avg_mshigh,fe_avg_mslow → ISP/upstream (your “rocketship vs gap” bet).- Both high → router CPU, conntrack, or local link.
- Ping good, HTTPS/git slow → DNAT or service layer (not measured here).
From outside LAN, still run occasionally:
mtr -rwzc 50 134.17.26.161
That catches path issues ping from the router cannot see.
monstro-netdiag.sh (bottleneck + iptables + processes)
Run on the Debian edge router (10.7.6.228, same host as wan-snapshot.sh) when infra feels stuck.
sudo cp examples/network/monstro-netdiag.sh /usr/local/bin/
sudo chmod +x /usr/local/bin/monstro-netdiag.sh
Cron (pair with wan-snapshot):
*/15 * * * * root /usr/local/bin/wan-snapshot.sh >>/var/log/wan-quality.cron.log 2>&1
*/15 * * * * root /usr/local/bin/monstro-netdiag.sh --snapshot >>/var/log/monstro-netdiag.cron.log 2>&1
When sluggish (run immediately, compare to a “good” capture):
sudo monstro-netdiag.sh --full # ping + conntrack + iptables -v + top CPU
sudo monstro-netdiag.sh --watch 30 # stream one-liners every 30s
monstro-netdiag.sh --tail 48
One-line log (/var/log/monstro-netdiag.log) includes: load1, cpu_busy, conntrack=cur/max (%), top=process, fe_avg_ms, wan_avg_ms, iface drops.
Why intermittent (excluding ISP)?
Your data showed WAN ~9 ms but FE ~230 ms — classic intra-LAN flapping:
| Cause | What to look for in --full |
|---|---|
| conntrack table full | conntrack >80%, new flows stall |
| Router CPU / softirq | high cpu_busy, softnet dropped |
| iptables rule cost | FORWARD/NAT rules with huge packet counts |
| FE/BE VM host busy | FE ping high from router, low load on router |
| ARP/route flap | ip neigh INCOMPLETE/FAILED for 10.7.0.10 |
| Bufferbloat on LAN | rising fe_avg_ms under load, 0% loss |
Good vs bad: save one --full output on a rocketship evening and diff when stuck.
Safe checks on monstro (read-only)
# conntrack pressure
cat /proc/sys/net/netfilter/nf_conntrack_count
cat /proc/sys/net/netfilter/nf_conntrack_max
# hottest iptables rules (no changes)
iptables -L FORWARD -n -v --line-numbers | sort -k1 -rn | head
iptables -t nat -L PREROUTING -n -v --line-numbers | head -20
# who eats CPU during slowness
ps -eo pcpu,pmem,comm --sort=-pcpu | head
Do not change iptables/NAT on monstro without a rollback plan (per infra policy).