cluster0: lab seeds, Gitea mirrors, compose mail, verify tooling

Freeze lab-seeds backend for COMPOSE_SKIP_MONOLITH; fix Gitea mirror scripts for 1.25 API; add secrets.lab.env template, verify-mail-lab, LAB_PUBLIC_ORIGIN, and credentials pointers for mirror token recovery. Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-29 05:59:05 +03:00 · 2026-06-23 17:07:46 +02:00
parent 98f30786b7
commit c554dc761d
209 changed files with 31249 additions and 6 deletions
--- a/sim/cluster0/lab-seeds/backend/README.md
+++ b/sim/cluster0/lab-seeds/backend/README.md
@@ -0,0 +1,343 @@
+# Android Cast — crash reporter backend (PHP)
+
+Anonymous crash JSON receiver and web console for  
+`https://f0xx.org/app/androidcast_project/crashes/`
+
+## Requirements
+
+- PHP 8.1+ with **PDO** (`pdo_sqlite` for default, `pdo_mysql` for MariaDB)
+- Extensions: **session**, **json** (usually core), **pdo**, **pdo_sqlite** or **pdo_mysql**
+
+### Alpine Linux (php-fpm 8.1)
+
+Minimal `php81-fpm` does **not** include sessions or PDO. Install modules explicitly:
+
+```sh
+# SQLite only (dev / small installs)
+apk add php81-fpm php81-session php81-pdo php81-pdo_sqlite php81-sqlite3
+
+# MariaDB (recommended on BE)
+apk add mariadb mariadb-client php81-pdo_mysql
+
+rc-service php-fpm81 restart
+# package names may be php82-* on newer Alpine; adjust to your php -v
+php -m | grep -E 'session|pdo_sqlite|pdo_mysql|PDO'
+```
+
+If `session_name()` is undefined, `php81-session` is missing or FPM was not restarted after install.
+
+**CLI vs FPM:** `php` on the shell may be a different major version than `php-fpm81` (e.g. `php82` CLI + `php81-fpm`). Install `phpNN-session` for **both** SAPIs you use, or run cron with the matching binary: `php81 scripts/purge_remote_access.php`. Maintenance scripts skip sessions on CLI when the extension is absent (web/FPM still require it).
+
+## Quick start (SQLite, development)
+
+```bash
+cd examples/crash_reporter/backend
+cp config/config.example.php config/config.php
+# For `php -S`, set 'base_path' => '' in config.php
+
+mkdir -p data storage
+sqlite3 data/crashes.sqlite < sql/schema.sqlite.sql
+
+php -S 127.0.0.1:8080 -t public
+```
+
+Open http://127.0.0.1:8080/ — login **admin** / **admin**.
+
+## Google Analytics 4 (optional)
+
+In `config/config.php`:
+
+```php
+'analytics' => [
+    'enabled' => true,
+    'measurement_id' => 'G-XXXXXXXXXX',
+    'debug' => false,
+],
+```
+
+Loads `public/assets/js/analytics.js` from `layout.php` and `login.php`. The project hub at  
+`/app/androidcast_project/` uses the same script; configure `examples/app_hub/assets/analytics.config.js`.
+
+Events include `page_view`, `console_view` (home, tickets, reports, …), and hub `hub_nav_click` / `hub_open_docs`.
+
+## RBAC (phase 1 + admin UI)
+
+Multi-tenant foundation:
+
+- **Global roles** on `users.role`: `root`, `platform_admin` (legacy `admin` works), `viewer`
+- **Companies** + **company_memberships** (`owner` / `admin` / `member`) — **slug admin** = `owner` or `admin` on a company (graphs company scope)
+- **Privilege sets** on `company_memberships.permissions_json`: `remote_access_user`, `remote_access_admin`, `short_links_viewer`, `short_links_user` (extra grants for `member` rows)
+- **Devices** auto-registered on upload; reports get `company_id` + `device_id`
+- Default company slug `default` — existing rows migrate on first request (`Database::ensureRbacSchema()`)
+
+**Admin UI:** `?view=rbac` (nav **Access**) — company owner/admin and global admins. **Root only** can change global roles. API: `GET/POST …/api/rbac.php` (`panel`, `set_global_role`, `set_company_role`, `apply_privilege_set`). Smoke: `./scripts/test_rbac_api.sh`.
+
+Config (`config.php`):
+
+```php
+'rbac' => [
+    'default_company_slug' => 'default',
+    'default_company_id' => 1,
+],
+```
+
+Permission checks: `Rbac::can('edit_tags', $companyId)` (used by `Auth::canEditTags()`).  
+Manual SQL (MariaDB, **as root** — `androidcast` has no `ALTER`):
+
+```sh
+mysql -u root -p androidcast_crashes < sql/migrations/001_rbac_companies.sql
+mysql -u root -p androidcast_crashes < sql/migrations/002_reports_company_columns.sql
+mysql -u root -p androidcast_crashes < sql/migrations/003_tickets.sql
+```
+
+Or from `backend/`:
+
+```sh
+chmod +x scripts/migrate-003-tickets.sh
+./scripts/migrate-003-tickets.sh
+```
+
+The PHP app user (`androidcast`) **cannot** create `tickets` on MariaDB — auto-migrate only works for SQLite dev. If Tickets shows *table missing*, run **003** as root once.
+
+Re-login after deploy so session includes company memberships.
+
+## UI languages
+
+JSON catalogs under `public/assets/i18n/` (`en.json`, `ru.json`) — same key style as Android `strings.xml`.  
+Default: English. User choice is stored in `localStorage` (`crash_console_lang`). Language selector is in the reports toolbar next to **Theme**, and in the nav on other pages (login included).
+
+Upload test payload:
+
+```bash
+curl -X POST http://127.0.0.1:8080/api/upload.php \
+  -H 'Content-Type: application/json' \
+  -d @../sample_crash_java.json
+```
+
+**Gzip ingest** (Android sends `Content-Encoding: gzip`; plain JSON still works):
+
+```bash
+./scripts/test_gzip_upload.sh
+# BASE=https://apps.f0xx.org/app/androidcast_project/crashes ./scripts/test_gzip_upload.sh
+```
+
+On the **BE nginx** upload `location`, enable gunzip so PHP receives plain JSON (see `nginx.vm.conf`):
+
+```nginx
+gunzip on;
+gunzip_types application/json;
+```
+
+PHP also decodes gzip if the header reaches FPM (`read_crash_upload_body()` in `bootstrap.php`). After deploy, sync `public/` + `src/` and reload nginx.
+
+## Production (nginx + PHP-FPM)
+
+**Deploy topology (FE/BE, ports, sshfs):** [docs/INFRA.md](../../../docs/INFRA.md).  
+**Canonical BE vhost for FE traffic:** `nginx.apps-port80.fragment` (`listen 80`). Port **8089** is not androidcast.
+
+0. Ensure PHP modules above are installed for the FPM SAPI (not only CLI `php -m`).
+1. Deploy the git tree so this path exists on the VM (no symlinks required):
+   `.../androidcast_project/android_cast/examples/crash_reporter/backend/`
+2. Point nginx `SCRIPT_FILENAME` at `.../backend/public/` (see `nginx.vm.conf`). Do not mix `proxy_*`, `try_files`, and broken alias/try_files combos in one block.
+3. Set `base_path` in `config.php` to match the URL prefix exactly.
+3. Choose DB:
+   - **SQLite (default):** `sqlite3 data/crashes.sqlite < sql/schema.sqlite.sql`, `db.driver` => `sqlite`
+   - **MariaDB (fresh, recommended on BE):**
+     ```sh
+     rc-service mariadb start
+     mysql -u root -p < sql/schema.mariadb.sql
+     # optional: CRASH_DB_PASS='…' ./scripts/init-mariadb.sh  # creates app user
+     ```
+     In `config/config.php`: `db.driver` => `mysql`, fill `db.mysql` (host `127.0.0.1`).
+     PHP auto-creates missing tables on first request if schema was not applied manually.
+4. `chown -R www-data:www-data data storage` (SQLite only needs writable `data/`; MariaDB uses mysqld)
+5. TLS terminate at nginx; restrict `/api/upload.php` by firewall if needed
+
+## Android app configuration
+
+In app-private `files/settings.json` (see `examples/settings.json`):
+
+```json
+"crash": {
+  "enabled": "true",
+  "upload_url": "https://f0xx.org/app/androidcast_project/crashes/api/upload.php",
+  "prune_after_upload": "true",
+  "upload_interval": "5m"
+}
+```
+
+Enable **Send anonymous crash logs** in the app settings drawer.
+
+## Report JSON schema (v1)
+
+See `sample_crash_java.json`. Key fields:
+
+| Field | Purpose |
+|-------|---------|
+| `schema_version` | Always `1` for now |
+| `report_id` | UUID from device |
+| `generated_at_epoch_ms` | Device timestamp |
+| `crash_type` | `java` or `native` |
+| `fingerprint` | Grouping hash (top frames) |
+| `device` | Manufacturer, model, SDK, ABIs |
+| `app` | Package, version |
+| `java` | Exception + `stack_frames[]` |
+| `native` | Signal + `backtrace[]` |
+
+Backend stores full JSON in `reports.payload_json` for faithful replay in the UI.
+
+## Tickets (tasks / QA)
+
+Web UI: `/?view=tickets` — list columns **Issue**, **Opened**, **App/OS**, **Rating**, **Tags**.  
+Detail: `/?view=ticket&id=N` — same layout family as crash reports (tags block, device/app/timing cards, collapsible raw JSON). Editable when you are **owner** or company **admin+**: caption, summary, body, owner, verified by, rating, tags.
+
+**Upload API** (no session; same as crashes):
+
+```http
+POST /api/ticket_upload.php
+Content-Type: application/json
+```
+
+```json
+{
+  "schema_version": 1,
+  "ingest_kind": "ticket",
+  "ticket_id": "unique-id",
+  "title": "Short caption",
+  "brief": "One-line summary for the list",
+  "body": "Full description",
+  "opened_at_epoch_ms": 1710000000000,
+  "rating": 3,
+  "owner": "admin",
+  "tags": [
+    {"id": "ticket", "label": "ticket", "bg": "#6366f1"},
+    {"id": "open", "label": "open", "bg": "#22c55e"}
+  ],
+  "device": { "model": "BL6000 Pro", "sdk_int": 29, "release": "10" },
+  "app": { "package": "com.foxx.androidcast", "version_name": "0.1.0" }
+}
+```
+
+Tags must include **`ticket`** plus at least one status tag (`open`, `in-progress`, `closed`, `rejected`, `confirmed`, `postponed`, `urgent`).
+
+**Ingest alpha smoke results** from repo `tmp/alpha_smoke_round1.txt`:
+
+```bash
+php scripts/ingest_alpha_smoke_tickets.php --url=https://apps.f0xx.org/app/androidcast_project/crashes
+```
+
+MariaDB (**required on existing servers**): `mysql -u root -p androidcast_crashes < sql/migrations/003_tickets.sql` or `./scripts/migrate-003-tickets.sh`. Fresh installs: included in `sql/schema.mariadb.sql` and `scripts/init-mariadb.sh`.
+
+**Workflow / assignees / comments (004):** `mysql -u root -p androidcast_crashes < sql/migrations/004_ticket_workflow.sql` — see [docs/TICKETS_ROADMAP.md](../../../docs/TICKETS_ROADMAP.md).
+
+**Attachments (005):** `mysql -u root -p androidcast_crashes < sql/migrations/005_ticket_attachments.sql` — files + external links (Google URLs auto-labeled).
+
+## Graphs dashboard (`/graphs`)
+
+Custom Grafana-style BE dashboards (direct MariaDB reads, no ETL):
+
+- `https://apps.f0xx.org/app/androidcast_project/graphs/`
+- Same PHP session/RBAC as crashes, tickets, and builder (`session_cookie_path` `/app/androidcast_project`).
+- Hub card: **AndroidCast graphs** (quick links to crashes, tickets, builder, git).
+
+**MariaDB (existing server):** run once as root:
+
+```sh
+mysql -u root -p androidcast_crashes < sql/migrations/006_graph_sessions.sql
+```
+
+Table auto-creates on first ingest if missing (SQLite dev); production MariaDB should use migration **006**.
+
+APIs:
+
+- `GET …/graphs/api/graphs.php?days=7` — role-scoped metrics (`user`, `slug_admin`, `platform_admin`)
+- `POST …/graphs/api/graph_upload.php` — device/session ingest (`ingest_kind=graph_session`, no session cookie)
+
+Smoke test:
+
+```bash
+./scripts/test_graphs_api.sh
+BASE=http://localhost:8082/app/androidcast_project/crashes ./scripts/test_graphs_api.sh
+```
+
+Fake data:
+
+```bash
+php scripts/generate_fake_graph_sessions.php --count=240 --days=30
+php scripts/upload_fake_graph_sessions.php --url=https://apps.f0xx.org/app/androidcast_project/graphs
+```
+
+Payload (`schema_version=1`): `session_id`, `company_id`, `user_id`, `device_id`, `app_version`,
+`sdk_int`, timing, `completed`, `direction`, `transport`, `avg_kbps`, `recv_kbps`,
+`packet_loss_pct`, `ntp_source`, `ntp_correction_s`, `install_source`, optional `meta`.
+
+NTP heartbeat (`POST …/api/heartbeat.php`, `type=ntp`) returns `time_source` and `ntp_correction_s`
+for clients that sync clocks before upload.
+
+## Remote access (WireGuard v1)
+
+On-demand debug tunnels — **same crashes vhost**, no new nginx HTTP locations. Full notes: [docs/REMOTE_ACCESS_IMPL.md](../../../docs/REMOTE_ACCESS_IMPL.md).
+
+- Dashboard: `https://apps.f0xx.org/app/androidcast_project/crashes/?view=remote_access`
+- Hub card: **Remote access**
+- Device poll: `POST …/api/heartbeat.php` with `heartbeat.type = ra`
+- Admin API: `…/api/remote_access.php` (session cookie + RBAC)
+
+**MariaDB (existing server):** run once as root if not already applied:
+
+```sh
+mysql -u root -p androidcast_crashes < sql/migrations/007_remote_access.sql
+```
+
+**Production `config.php`** (not in git) — set `remote_access.wg_server_public_key` from `wg show wg0 public-key`, `require_wg_tools => true`, and `wg_endpoint` (public UDP DNAT hostname). See `config.example.php`.
+
+**Cron:** hourly purge — on Alpine use **php81** if default `php` is 8.x without session (see § Requirements):
+
+```cron
+0 * * * * cd /var/www/localhost/htdocs/apps/app/androidcast_project/android_cast/examples/crash_reporter/backend && php81 scripts/purge_remote_access.php --hours=24
+# Prunes orphan wg0 peers each run; manual: php81 scripts/sync_wg_peers.php --dry-run
+```
+
+Smoke / verify:
+
+```bash
+chmod +x scripts/ra_*.sh scripts/test_remote_access_api.sh scripts/verify_remote_access_prod.sh
+./scripts/test_rbac_api.sh
+./scripts/test_remote_access_api.sh
+BASE=https://apps.f0xx.org/app/androidcast_project/crashes ./scripts/test_remote_access_api.sh
+./scripts/verify_remote_access_prod.sh
+BASE=https://apps.f0xx.org/app/androidcast_project/crashes ./scripts/ra_e2e_cli.sh   # device + operator E2E (jq)
+./scripts/ra_device_sim.sh poll --until connect   # after open session in UI
+```
+
+**CLI validation (Linux device simulator):** [docs/REMOTE_ACCESS_VALIDATION.md](../../../docs/REMOTE_ACCESS_VALIDATION.md).
+
+Android: Developer settings → **Remote access** → **RSSH** (alpha target; WireGuard lab-only). RSSH avoids VPN permission. Polls BE every 1–7 min while enabled. See [REMOTE_ACCESS_IMPL.md § VPN consent](../../../docs/REMOTE_ACCESS_IMPL.md#android-vpn-consent-wireguard) and [ROADMAP § RSSH](../../../docs/ROADMAP.md#remote-access-alpha--rssh).
+
+## Short links (URL shortener admin)
+
+Operator UI for `https://s.f0xx.org` — mint/revoke bearer tokens, list links, shorten from the console. SPEC §20.
+
+- Dashboard: `…/crashes/?view=short_links` (RBAC: `short_links_view`; company owner/admin or privilege sets `short_links_viewer` / `short_links_user`)
+- Hub nav + console home card: **Short links**
+- Admin API: `…/api/short_links.php` (session cookie — not on `s.f0xx.org`)
+- Permanent links: mint signer key in UI, bearer with **Permanent links** cap, shorten with **Permanent** + signer
+- Public shorten API stays on `https://s.f0xx.org/api/v1/shorten` (bearer token only)
+
+**Production `config.php`:** set `url_shortener.enabled => true` and `url_shortener.db.mysql` to the same `androidcast` user + `url_shortener` database as [backend/url-shortener/config/config.php](../../../backend/url-shortener/config/config.example.php). See `config.example.php`.
+
+Smoke: `./scripts/test_short_links_api.sh` (needs admin/owner session cookie). Purge expired links: platform admin only (UI button or `backend/url-shortener/scripts/purge_url_shortener.php` cron).
+
+## Third-party licenses (BE / landing)
+
+- [THIRD_PARTY_LICENSES.md](THIRD_PARTY_LICENSES.md) — server stack, optional WireGuard tools, link to mobile app list.
+- Mobile APK list: in-app **Licenses** button → `app/src/main/assets/licenses/combined.html`.
+- Suggested public URL after sync: `…/crashes/THIRD_PARTY_LICENSES.md` (or HTML excerpt on hub legal page).
+
+## Default accounts
+
+| User | Password | Role |
+|------|----------|------|
+| admin | admin | root (all permissions) |
+
+Change the admin password in production (update `users.password_hash`).