# Cluster setup journal (`/shared/journal/`)

Human-readable audit trail for populate / verify runs. **Read from FE** when VMs are down or after reboot failure — no SSH to cast nodes required.

## Files (per cluster)

| File | Purpose |
|------|---------|
| `{CLUSTER_NAME}.journal.tsv` | Append-only machine log (tab-separated) |
| `{CLUSTER_NAME}.summary.md` | Regenerated table with **OK** / **NOK** / **WARN** |
| `README.md` | This file |

Example for cluster0:

```
/shared/journal/cluster0.journal.tsv
/shared/journal/cluster0.summary.md
```

## TSV columns

```
timestamp_utc  host  actor  phase  action  status  detail
```

**status:** `START` → work begun; `OK` / `NOK` / `WARN` → outcome.

**actor:** script name (`populate`, `verify_lab`, `verify_global`, `baseline`, …).

## Who writes entries

All cluster scripts call `journal_*` helpers in `scripts/lib/common.sh`:

- `populate_lab_setup.sh` — each phase and sub-script
- `verify_lab_setup.sh` / `verify_global_setup.sh` — pass/fail
- `run_script` — every `scripts/*.sh` invocation

## Read from FE (10.7.0.10)

```sh
less /mnt/raid0/xendomains/domU_cast_cluster_0/shared/journal/cluster0.summary.md
grep NOK /mnt/raid0/xendomains/domU_cast_cluster_0/shared/journal/cluster0.journal.tsv
```

## Rebuild summary manually

```sh
sudo sh /shared/cluster/scripts/journal-rebuild-summary.sh
```

## Snapshots

If reboot breaks a node and journal shows last **OK** before power-loss, ask sysop for **cluster snapshot restore** (XEN/HVM snapshot — out of band, not stored on NFS).

Do **not** store VM disk images or MariaDB datadirs in `/shared/`.