mirror of
git://f0xx.org/ac/ac-docs
synced 2026-07-29 08:38:53 +03:00
initial
This commit is contained in:
Anton Afanasyeu
394
DRs/20100611_3_url_shortener.md
Normal file
394
DRs/20100611_3_url_shortener.md
Normal file
@@ -0,0 +1,394 @@
|
||||
# URL shortener service — design review (DR)
|
||||
|
||||
<!-- doc-meta:start -->
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Author | Anton Afanasyeu |
|
||||
| Revision | R1 |
|
||||
| Creation date | 2026-06-11 |
|
||||
| Last modification date | 2026-06-11 |
|
||||
| Co-authored | |
|
||||
| Severity | medium |
|
||||
| State | pending review |
|
||||
| Document type | DR |
|
||||
<!-- doc-meta:end -->
|
||||
|
||||
\newpage
|
||||
\newpage
|
||||
|
||||
---
|
||||
|
||||
|
||||
**Document type:** DR (Design Review)
|
||||
**Source draft:** [docs/drafts/20100611_3_url_shortener.txt](../drafts/20100611_3_url_shortener.txt)
|
||||
**Specification:** [docs/specs/20100611_3_url_shortener.md](../specs/20100611_3_url_shortener.md)
|
||||
**Date:** 2026-06-11
|
||||
**Status:** Reviewed — ready for post-alpha implementation planning
|
||||
**Severity:** Medium
|
||||
|
||||
**Documentation index:** [README.md](../README.md)
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Table of contents
|
||||
|
||||
<!-- toc -->
|
||||
- [1. Executive summary](#1-executive-summary)
|
||||
- [2. Resolved open questions](#2-resolved-open-questions)
|
||||
- [3. Authentication and trust](#3-authentication-and-trust)
|
||||
- [3.1 Bearer token](#31-bearer-token)
|
||||
- [3.2 Signer](#32-signer)
|
||||
- [3.3 Shared session (operators)](#33-shared-session-operators)
|
||||
- [3.4 Trust flow (normative)](#34-trust-flow-normative)
|
||||
- [4. Slug algorithm](#4-slug-algorithm)
|
||||
- [4.1 Decision: SHA-256 truncated](#41-decision-sha-256-truncated)
|
||||
- [4.2 Collision policy](#42-collision-policy)
|
||||
- [4.3 Draft example (preserved for comparison)](#43-draft-example-preserved-for-comparison)
|
||||
- [5. API and flow corrections](#5-api-and-flow-corrections)
|
||||
- [6. Data store](#6-data-store)
|
||||
- [7. Deployment and infra](#7-deployment-and-infra)
|
||||
- [8. QR codes](#8-qr-codes)
|
||||
- [9. Security model](#9-security-model)
|
||||
- [10. Integration with androidcast](#10-integration-with-androidcast)
|
||||
- [11. Estimates](#11-estimates)
|
||||
- [12. Task dependency graph](#12-task-dependency-graph)
|
||||
- [13. Suggestions (PO / dev discretion)](#13-suggestions-po-dev-discretion)
|
||||
- [14. Remaining open items](#14-remaining-open-items)
|
||||
- [15. Changelog](#15-changelog)
|
||||
<!-- /toc -->
|
||||
|
||||
**Documentation index:** [README.md](README.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
The draft defines a **standalone URL shortener** at `https://s.f0xx.org`, separate from the crash-reporter BE app, with bearer/signer trust, TTL/permalinks, QR support, and a JSON shorten API.
|
||||
|
||||
**DR outcome:**
|
||||
|
||||
| Area | Decision |
|
||||
|------|----------|
|
||||
| Priority | **Post-alpha**; medium severity; may move up if BE load stays low |
|
||||
| Slug hash | **SHA-256 truncated (12 hex)** — reject draft interleave + Java `hashCode` |
|
||||
| Auth | **Bearer tokens** for API; **signer** for permanent TTL; operator **session RBAC** only for token admin UI |
|
||||
| Placement | `backend/url-shortener/` scaffold in repo; dedicated nginx vhost; **opt-in** deploy |
|
||||
| Admin UI | Hub **Short links** card (not separate admin skin) — SPEC §20 |
|
||||
| Infra | Git remote + FE TLS **live**; nginx snippets in repo |
|
||||
| Stack | PHP + nginx + MariaDB per draft |
|
||||
| OpenAPI | **Required** in v1 |
|
||||
|
||||
---
|
||||
|
||||
## 2. Resolved open questions
|
||||
|
||||
Questions from draft §“TBD and open questions list” and inline `TBD` markers.
|
||||
|
||||
| # | Draft question / gap | DR answer |
|
||||
|---|----------------------|-----------|
|
||||
| Q1 | Token signing, trust models, sessions, security | See [§3](#3-authentication-and-trust) and [§9](#9-security-model). Machine clients use **bearer** (+ optional **signer**). Humans use existing BE session only to **mint** bearers. Optional HMAC for GET left as [suggestion](#13-suggestions-po-dev-discretion). |
|
||||
| Q2 | `... TBD — open part` | Filled in SPEC: audit log, rate limits, SSRF blocklist, normalization rules, collision policy, admin tables, smoke tests. |
|
||||
| Q3 | Database `*... TBD` columns | Full schema in [SPEC §9](../specs/20100611_3_url_shortener.md#9-database-schema): `origin_hash`, `expires_at`, `access_count`, FK tables for bearer/signer. |
|
||||
| Q4 | Java hash vs SHA-256 | **SHA-256** (see [§4](#4-slug-algorithm)). Java hash is 32-bit, non-cryptographic, JVM-stable but poor for URL slugs. |
|
||||
| Q5 | Interleaved URL+TOKEN string | **Rejected** for production. Deterministic `SHA256(bearer_id ‖ normalized_url)` instead. Draft interleave kept as historical note only. |
|
||||
| Q6 | Authentication: “app-wide shared session, project-wise token” | **Split roles:** `curl`/services → **project bearer token**; console operators → **shared `ac_crash_sess`** to create/revoke tokens (same RBAC as crashes/tickets). Shorten API does **not** accept PHP session cookies. |
|
||||
| Q7 | Submodule remote | **Done** — `git://f0xx.org/androidcast_project/url-shortener`; pinned at `backend/url-shortener/` submodule (`next`). |
|
||||
| Q8 | OTA-style “invalid URL” on unrelated services | N/A here; shorten service validates URL scheme/host and returns `INVALID_URL`. |
|
||||
| Q9 | Step 9.1.1 / 9.1.2 trusted sub-capabilities | `can_temporary` on bearer; `can_permanent` on bearer **and** valid signer required for `ttl=0`. |
|
||||
| Q10 | Cached hit when TTL “exceeds given limits” | If stored link **expired** → error `TTL_EXPIRED`, not silent renew. If valid → return same slug with **remaining** `ttl` in response (draft step 8). |
|
||||
| Q11 | Estimations (draft empty) | See [§11](#11-estimates). |
|
||||
|
||||
---
|
||||
|
||||
## 3. Authentication and trust
|
||||
|
||||
### 3.1 Bearer token
|
||||
|
||||
- Opaque string (≥32 chars random), stored as **SHA-256 hash** in `bearer_tokens`.
|
||||
- **`trusted=1`** required to **create** new links (draft steps 6–7).
|
||||
- **Demo token:** single `trusted=0` token that can only resolve existing slugs in docs/examples — cannot create (403).
|
||||
|
||||
### 3.2 Signer
|
||||
|
||||
- Separate long-lived secret for **elevated** operations.
|
||||
- Required when `ttl=0` (permanent permalink).
|
||||
- Validated against `signer_keys` where `can_permanent=1` and not revoked.
|
||||
|
||||
### 3.3 Shared session (operators)
|
||||
|
||||
- **Frozen:** hub **Short links** card (SPEC §20) — same nav shell as Remote access / Builder.
|
||||
- Actions: create bearer, revoke bearer, list links for bearer, purge expired.
|
||||
- Uses same RBAC roles as [REMOTE_ACCESS_IMPL](../REMOTE_ACCESS_IMPL.md) pattern (`admin`, `operator`).
|
||||
|
||||
### 3.4 Trust flow (normative)
|
||||
|
||||
Matches draft low-level flow with SPEC clarifications:
|
||||
|
||||
```text
|
||||
Request → bearer present? ─no→ 401 MISSING_BEARER
|
||||
→ lookup (bearer, url)
|
||||
→ hit & valid TTL → 200 cached (remaining ttl)
|
||||
→ hit & expired → 409 TTL_EXPIRED
|
||||
→ miss → bearer trusted? ─no→ 403 ACCESS_DENIED
|
||||
→ ttl=0 → signer ok? ─no→ 403 SIGNER_REQUIRED
|
||||
→ compute slug, insert, return 200
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Slug algorithm
|
||||
|
||||
### 4.1 Decision: SHA-256 truncated
|
||||
|
||||
**Chosen:**
|
||||
|
||||
```text
|
||||
slug = hex(SHA256(bearer_id + "\x1f" + normalized_url))[0:12]
|
||||
```
|
||||
|
||||
| Criterion | Java `hashCode` + interleave (draft) | SHA-256 truncated (chosen) |
|
||||
|-----------|--------------------------------------|----------------------------|
|
||||
| Collision resistance | Poor (~32 bits) | ~48 bits with 12 hex chars |
|
||||
| Determinism | JVM-specified but weak | Standard crypto |
|
||||
| Reversibility | No (good) | No (good) |
|
||||
| Debuggability | Opaque interleave | Simple test vectors |
|
||||
|
||||
### 4.2 Collision policy
|
||||
|
||||
- Unique index on `slug`.
|
||||
- On duplicate insert: append `-1`…`-f` (4 chars max) **or** re-hash with `:\x1e{counter}` appended to material.
|
||||
- Log collision to `audit_log` at `WARN` level.
|
||||
|
||||
### 4.3 Draft example (preserved for comparison)
|
||||
|
||||
Draft interleaved `https://t.co` + token → `htta5bpesf:4/3/3td.2c3oa` → Java hash or SHA slice `4623b436f`.
|
||||
**Do not implement interleave** — kept only to show original intent (per-token namespacing). Namespacing is achieved via `bearer_id` in hash input instead.
|
||||
|
||||
---
|
||||
|
||||
## 5. API and flow corrections
|
||||
|
||||
| Draft item | Correction |
|
||||
|------------|------------|
|
||||
| `GET https://s.f0xx.org?url=...` | Use **`/api/v1/shorten`** path so root `/{slug}` stays redirect-only |
|
||||
| Response `ttl` as string | Keep string in JSON for draft compatibility; document integer seconds |
|
||||
| `signer` on GET | Optional query param; omit from success body (security) |
|
||||
| Fail body echoes `bearer` | Truncate in response to first 6 + `…` in production logs; full echo in JSON only for client debugging (SPEC) |
|
||||
|
||||
**Routing suggestion:** nginx sends `/api/` → PHP front controller; slug paths match regex (quoted on BE: `location ~ "^/[a-f0-9]{6,16}$"`) → redirect handler.
|
||||
|
||||
---
|
||||
|
||||
## 6. Data store
|
||||
|
||||
| Option | Pros | Cons | DR pick |
|
||||
|--------|------|------|---------|
|
||||
| A. Same MariaDB instance, new schema `url_shortener` | Simple ops, one backup | Shared failure domain | **Recommended v1** |
|
||||
| B. Separate MariaDB database | Blast radius isolation | More admin | If PO wants hard isolation |
|
||||
| C. SQLite on BE | Zero MariaDB migration | Poor fit for BE pattern | **Reject** |
|
||||
|
||||
Retention cron: `scripts/purge_url_shortener.php` (mirror `purge_remote_access.php` style) — delete expired rows + audit >90d.
|
||||
|
||||
---
|
||||
|
||||
## 7. Deployment and infra
|
||||
|
||||
Per [INFRA.md](../INFRA.md) topology:
|
||||
|
||||
```text
|
||||
Internet → FE (s.f0xx.org TLS) → proxy_pass BE :80 → url-shortener php-fpm
|
||||
```
|
||||
|
||||
| Step | Owner | Notes |
|
||||
|------|-------|-------|
|
||||
| DNS `s.f0xx.org` → FE | PO / infra | **Done** (with TLS) |
|
||||
| FE TLS cert | PO / infra | **Done** — `/etc/letsencrypt/live/s.f0xx.org/` |
|
||||
| FE nginx `server_name s.f0xx.org` | DEV | [nginx.fe-s.f0xx.org.snippet](../../backend/url-shortener/deploy/nginx.fe-s.f0xx.org.snippet) |
|
||||
| BE vhost fragment | DEV | [nginx.be-url-shortener.fragment](../../backend/url-shortener/deploy/nginx.be-url-shortener.fragment) |
|
||||
| Git remote | PO | **Done** — `git://f0xx.org/androidcast_project/url-shortener` |
|
||||
| Licenses | DEV | Add QR library to BE license inventory when impl starts |
|
||||
|
||||
**Safe deploy:** nginx fragment only; no topology changes to router/FE↔BE path. Validate with `nginx -t` before reload.
|
||||
|
||||
**Not alpha-blocking:** omit from `validate_be_services.sh` until first deploy.
|
||||
|
||||
---
|
||||
|
||||
## 8. QR codes
|
||||
|
||||
| Decision | Detail |
|
||||
|----------|--------|
|
||||
| Must-have | Yes (draft) |
|
||||
| Format | PNG via `/api/v1/qr/{slug}.png` |
|
||||
| Library | `endroid/qr-code` (MIT) — **suggestion**; PO approves license addition |
|
||||
| Inline in JSON | Optional `qr` URL field in shorten success — **suggestion** for v1.1 |
|
||||
|
||||
---
|
||||
|
||||
## 9. Security model
|
||||
|
||||
| Threat | Mitigation |
|
||||
|--------|------------|
|
||||
| Token leak | Revoke bearer; rotate signer; rate limit |
|
||||
| SSRF via shorten | Block private/metadata IPs at validation |
|
||||
| Slug enumeration | 48+ bits entropy; rate limit redirects (suggestion) |
|
||||
| Open redirect abuse | Only whitelisted bearers create links |
|
||||
| Session fixation | N/A — API is stateless |
|
||||
|
||||
**Trust model summary:**
|
||||
|
||||
```text
|
||||
┌─────────────────┐
|
||||
│ Operator (web) │
|
||||
│ ac_crash_sess │
|
||||
└────────┬────────┘
|
||||
│ mint/revoke
|
||||
▼
|
||||
┌─────────────────┐
|
||||
signer ───────►│ bearer_tokens │◄────── curl / services
|
||||
(permanent) └────────┬────────┘
|
||||
│ trusted create
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ links │
|
||||
└─────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Integration with androidcast
|
||||
|
||||
| Consumer | Use case | When |
|
||||
|----------|----------|------|
|
||||
| BE remote-access UI | Short “open session” links for operators | Post-alpha |
|
||||
| OTA / docs | Stable short links in printed QR | Post-alpha |
|
||||
| Android app | None v1 — use HTTPS from BE | Deferred |
|
||||
| Crash reporter | **No** embed — standalone service | Normative |
|
||||
|
||||
Submodule lives beside `examples/crash_reporter/backend/` conceptually but **not** inside it.
|
||||
|
||||
---
|
||||
|
||||
## 11. Estimates
|
||||
|
||||
Engineering person-days (single DEV, familiar with BE stack):
|
||||
|
||||
| Task | Days |
|
||||
|------|------|
|
||||
| Repo scaffold + OpenAPI + DB migrations | 1.5 |
|
||||
| Shorten API + auth + TTL logic | 2 |
|
||||
| Redirect handler + permalink cache | 1 |
|
||||
| QR endpoint + license | 0.5 |
|
||||
| Admin token UI (minimal) | 1.5 |
|
||||
| nginx FE/BE fragments + deploy doc | 1 |
|
||||
| PHPUnit + smoke script | 1 |
|
||||
| **Total** | **~8.5 days** |
|
||||
|
||||
**Severity:** Medium — no alpha path dependency.
|
||||
|
||||
**Reorder triggers (PO):** high demand for shareable debug links; BE CPU <50%; remote-access operator UX needs short URLs.
|
||||
|
||||
---
|
||||
|
||||
## 12. Task dependency graph
|
||||
|
||||
Legend: **1** = highest priority within this feature.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
T1["1. Submodule scaffold + openapi.yaml"]
|
||||
T2["2. MariaDB schema + migrations"]
|
||||
T3["3. Bearer/signer auth layer"]
|
||||
T4["4. Shorten API (POST/GET)"]
|
||||
T5["5. Redirect /{slug} + TTL"]
|
||||
T6["6. QR PNG endpoint"]
|
||||
T7["7. Admin token UI (RBAC)"]
|
||||
T8["8. nginx FE/BE + DNS s.f0xx.org"]
|
||||
T9["9. Tests + smoke + licenses"]
|
||||
|
||||
T1 --> T2
|
||||
T2 --> T3
|
||||
T3 --> T4
|
||||
T4 --> T5
|
||||
T4 --> T6
|
||||
T3 --> T7
|
||||
T5 --> T8
|
||||
T4 --> T9
|
||||
T6 --> T9
|
||||
T7 --> T9
|
||||
T8 --> T9
|
||||
```
|
||||
|
||||
| # | Task | Blocker | Assignee |
|
||||
|---|------|---------|----------|
|
||||
| 1 | Submodule + OpenAPI | — | DEV |
|
||||
| 2 | Schema | 1 | DEV |
|
||||
| 3 | Auth | 2 | DEV |
|
||||
| 4 | Shorten API | 3 | DEV |
|
||||
| 5 | Redirect | 4 | DEV |
|
||||
| 6 | QR | 4 | DEV |
|
||||
| 7 | Admin UI | 3 | DEV |
|
||||
| 8 | Infra deploy | 5, PO DNS | DEV + PO |
|
||||
| 9 | Tests/licenses | 4–7 | DEV |
|
||||
|
||||
---
|
||||
|
||||
## 13. Suggestions (PO / dev discretion)
|
||||
|
||||
Items intentionally **not** frozen in SPEC — recommended but optional:
|
||||
|
||||
1. **HMAC signed GET** — `&sig=HMAC-SHA256(url|ttl, bearer_secret)` to prevent query tampering on shared links; skip for v1 if all clients use POST.
|
||||
2. ~~**Console integration**~~ — **Done (spec):** hub **Short links** card (SPEC §20).
|
||||
3. **Response field `qr`** — include ready-made QR URL in shorten JSON.
|
||||
4. **Click analytics** — increment `access_count` only (already in schema); defer graphs to Grafana/nginx logs.
|
||||
5. **Custom slug** — `&alias=` for operators; increases collision/abuse risk — post-v1.
|
||||
6. **Separate PHP-FPM pool** — if redirect traffic is high; share pool initially.
|
||||
7. **FE rate limit** — `limit_req` on `/api/v1/shorten` at nginx.
|
||||
8. **Draft demo token** — ship `examples/url-shortener/demo_curl.sh` with documented read-only token for CI smoke.
|
||||
9. **PDF export** — run `python3 docs/_pdf_build/md_to_pdf.py` on SPEC+DR when PO wants signed-off copies.
|
||||
10. **Atomic commits on `next`** — per [bottomline_reminder.txt](../../bottomline_reminder.txt): one commit per task row above for easy revert.
|
||||
|
||||
---
|
||||
|
||||
## 14. Remaining open items
|
||||
|
||||
Requires PO / infra input (hard blockers for **prod** only):
|
||||
|
||||
| ID | Item | Owner | Status |
|
||||
|----|------|-------|--------|
|
||||
| O1 | Git remote `git://f0xx.org/androidcast_project/url-shortener` | PO | **Done** |
|
||||
| O2 | DNS + TLS for `s.f0xx.org` on FE | PO / infra | **Done** |
|
||||
| O3 | BE docroot `/var/www/localhost/htdocs/apps/s/` | DEV + PO | **Default in SPEC**; confirm on first sync |
|
||||
| O4 | Approve QR library license addition | PO | Open (before QR impl) |
|
||||
| O5 | First production bearer token issuance policy | PO | Open |
|
||||
| O6 | PHP shorten/redirect handlers + hub Short links UI | DEV | Open (scaffold only) |
|
||||
|
||||
Soft blockers cleared for BE nginx + MariaDB local dev. Hard blocker for prod traffic: **O6**.
|
||||
|
||||
---
|
||||
|
||||
## 15. Changelog
|
||||
|
||||
| Date | Rev | Change |
|
||||
|------|-----|--------|
|
||||
| 2026-06-11 | 1 | Draft → SPEC + DR; resolved hash, auth, schema, API paths; task graph + estimates |
|
||||
| 2026-06-11 | 2 | Scaffold `backend/url-shortener/`; openapi.yaml; nginx snippets; curl appendix; O1/O2 done per draft |
|
||||
|
||||
**Source linkage:** [docs/drafts/20100611_3_url_shortener.txt](../drafts/20100611_3_url_shortener.txt)
|
||||
565
DRs/20100611_3_url_shortener.pdf
Normal file
565
DRs/20100611_3_url_shortener.pdf
Normal file
File diff suppressed because one or more lines are too long
676
DRs/20100612_1_scaling.md
Normal file
676
DRs/20100612_1_scaling.md
Normal file
@@ -0,0 +1,676 @@
|
||||
# Platform scaling and multi-environment architecture — design review (DR)
|
||||
|
||||
<!-- doc-meta:start -->
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Author | Anton Afanasyeu |
|
||||
| Revision | R1 |
|
||||
| Creation date | 2026-06-12 |
|
||||
| Last modification date | 2026-06-12 |
|
||||
| Co-authored | |
|
||||
| Severity | medium |
|
||||
| State | pending review |
|
||||
| Document type | DR |
|
||||
<!-- doc-meta:end -->
|
||||
|
||||
\newpage
|
||||
\newpage
|
||||
|
||||
---
|
||||
|
||||
|
||||
**Document type:** DR (Design Review)
|
||||
**Source draft:** [docs/drafts/20100612_1_scaling.txt](../drafts/20100612_1_scaling.txt)
|
||||
**Specification:** [docs/specs/20100612_1_scaling.md](../specs/20100612_1_scaling.md)
|
||||
**PDF:** [20100612_1_scaling.pdf](20100612_1_scaling.pdf) · Regenerate: `bash scripts/build-all-docs-pdf.sh` (Mermaid → PNG diagrams; requires `npx`)
|
||||
**Date:** 2026-06-12
|
||||
**Status:** Reviewed — planning / brainstorming (no implementation sprint yet)
|
||||
**Severity:** Strategic
|
||||
|
||||
**Documentation index:** [README.md](../README.md)
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
|
||||
**Documentation index:** [README.md](../README.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Table of contents
|
||||
|
||||
<!-- toc -->
|
||||
- [1. Executive summary](#1-executive-summary)
|
||||
- [2. Problem statement](#2-problem-statement)
|
||||
- [3. Architecture options](#3-architecture-options)
|
||||
- [3.1 Option A — Dev cluster (draft default)](#31-option-a-dev-cluster-draft-default)
|
||||
- [3.2 Option B — Tighter cost (minimal split)](#32-option-b-tighter-cost-minimal-split)
|
||||
- [3.3 Option C — Cloud-first hybrid](#33-option-c-cloud-first-hybrid)
|
||||
- [3.4 Comparison matrix](#34-comparison-matrix)
|
||||
- [4. DR decision — recommended path](#4-dr-decision-recommended-path)
|
||||
- [5. Current vs target diagrams](#5-current-vs-target-diagrams)
|
||||
- [Today (single BE)](#today-single-be)
|
||||
- [Target phase B (minimal split + HA web)](#target-phase-b-minimal-split-ha-web)
|
||||
- [6. Service split rationale](#6-service-split-rationale)
|
||||
- [7. Environment promotion model](#7-environment-promotion-model)
|
||||
- [8. RSSH and connection capacity](#8-rssh-and-connection-capacity)
|
||||
- [Assumptions](#assumptions)
|
||||
- [Estimates](#estimates)
|
||||
- [8b. Network bandwidth (DR)](#8b-network-bandwidth-dr)
|
||||
- [Baseline risk (today)](#baseline-risk-today)
|
||||
- [DR sizing decisions](#dr-sizing-decisions)
|
||||
- [Planning numbers (single-site prod, until CDN)](#planning-numbers-single-site-prod-until-cdn)
|
||||
- [8c. Global availability and GEO segments (DR)](#8c-global-availability-and-geo-segments-dr)
|
||||
- [Policy model (DR decision)](#policy-model-dr-decision)
|
||||
- [Phasing](#phasing)
|
||||
- [Example use cases](#example-use-cases)
|
||||
- [9. Future capabilities — priority weighting](#9-future-capabilities-priority-weighting)
|
||||
- [10. Simulation lab design](#10-simulation-lab-design)
|
||||
- [11. Cloud provider notes (GCP / Azure)](#11-cloud-provider-notes-gcp-azure)
|
||||
- [12. RBAC evolution](#12-rbac-evolution)
|
||||
- [13. Risks and mitigations](#13-risks-and-mitigations)
|
||||
- [14. Estimates](#14-estimates)
|
||||
- [15. Task dependency graph](#15-task-dependency-graph)
|
||||
- [16. Resolved open questions](#16-resolved-open-questions)
|
||||
- [17. Remaining open items (PO)](#17-remaining-open-items-po)
|
||||
- [18. Suggestions (PO / dev discretion)](#18-suggestions-po-dev-discretion)
|
||||
- [19. Changelog](#19-changelog)
|
||||
- [Related docs](#related-docs)
|
||||
<!-- /toc -->
|
||||
|
||||
**Documentation index:** [README.md](README.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
The project outgrew **one Alpine BE VM** that hosts consoles, APIs, MariaDB, Gitea, OTA, WireGuard, RSSH, URL shortener, and Docker builds. The draft asks for **clusters**, **dev/staging/prod**, **zero-downtime deploys**, optional **cloud**, and headroom for **RT cast**, **VPN commercialization**, **payments**, and **ffmpeg** pipelines.
|
||||
|
||||
**DR outcome:**
|
||||
|
||||
| Area | Decision |
|
||||
|------|----------|
|
||||
| **Timing** | **Post-alpha** strategic track; does not block alpha soak |
|
||||
| **Architecture** | **Option B → A** phased: minimal VM split first, then full dev cluster simulation |
|
||||
| **Edge** | Keep **Gentoo FE** as TLS/DNAT anchor; add LB upstream to N BE web nodes |
|
||||
| **Data** | **Dedicated MariaDB VM** before scaling anything else CPU-heavy |
|
||||
| **Build** | **First split candidate** — Docker off shared PHP VM |
|
||||
| **Cloud** | **Hybrid** — object store + staging dress rehearsal before prod migration |
|
||||
| **RSSH capacity** | Plan **100 concurrent SSH** / **5k heartbeats** per bastion pair before commercial VPN |
|
||||
| **Bandwidth** | Document **WAN / FE→BE / per-service** Mbps & GB-month budgets ([SPEC §12.3](../specs/20100612_1_scaling.md#123-network-bandwidth-and-egress)); CDN before horizontal web scale for OTA |
|
||||
| **Global + GEO** | **Worldwide reach** default; **deny segments** per content (e.g. no serve in `ZM`) via layered enforcement ([SPEC §14](../specs/20100612_1_scaling.md#14-global-availability-and-geo-policy)) |
|
||||
| **Future slots** | Weighted backlog in [§9](#9-future-capabilities-priority-weighting); SFU before ffmpeg farm; **F10 GEO** with F3/F5 |
|
||||
|
||||
---
|
||||
|
||||
## 2. Problem statement
|
||||
|
||||
| Pain | Evidence today |
|
||||
|------|----------------|
|
||||
| **Blast radius** | Submodule sync wiped gitignored configs; one BE restart affects all services |
|
||||
| **Resource contention** | Docker build spikes CPU/RAM during PHP console use |
|
||||
| **No promotion path** | Single prod-like env; staging OTA path still pending ([20260608_BE_SERVICES_and_infra.md](../20260608_BE_SERVICES_and_infra.md)) |
|
||||
| **Scaling ceiling** | RSSH + WG + future SFU on same VM as MariaDB |
|
||||
| **Bandwidth ceiling** | Single FE WAN + FE→BE path; OTA/APK egress competes with consoles ([SPEC §12.3](../specs/20100612_1_scaling.md#123-network-bandwidth-and-egress)) |
|
||||
| **GEO gap** | No per-content region policy yet; required before global recordings/commerce ([SPEC §14](../specs/20100612_1_scaling.md#14-global-availability-and-geo-policy)) |
|
||||
| **Operational risk** | Topology changes on FE/router are hard to roll back ([bottomline_reminder.txt](../../bottomline_reminder.txt)) |
|
||||
|
||||
**Success criteria:** PO can deploy unstable work to **dev**, beta-test on **staging**, and promote to **prod** with rollback per service — without rewriting mobile apps or public URLs each time.
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture options
|
||||
|
||||
### 3.1 Option A — Dev cluster (draft default)
|
||||
|
||||
Draft proposal: **N VMs per environment**, each role class (web+OTA, DB, object store, WG, url shortener+Gitea, build, +N for new features).
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph Cluster["One environment cluster (e.g. dev)"]
|
||||
W["Web + OTA VM(s)"]
|
||||
D["Database VM(s)"]
|
||||
O["Object store VM(s)"]
|
||||
V["WireGuard VM(s)"]
|
||||
U["URL shortener + Gitea VM"]
|
||||
B["Build VM(s)"]
|
||||
X["Future service VM(s)"]
|
||||
end
|
||||
W --> D
|
||||
W --> O
|
||||
U --> D
|
||||
B --> O
|
||||
V --> W
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Clear isolation; matches future SFU/ffmpeg | Highest VM count / cost |
|
||||
| Easy to simulate on FE XEN | Ops overhead for small team |
|
||||
| Aligns with cloud MIG later | Overkill before alpha exit |
|
||||
|
||||
### 3.2 Option B — Tighter cost (minimal split)
|
||||
|
||||
**Phase 1 prod shape** (recommended start):
|
||||
|
||||
| VM | Bundled services |
|
||||
|----|------------------|
|
||||
| **be-web** | nginx, PHP (hub, crashes, graphs, builder **UI**, OTA **metadata**) |
|
||||
| **be-data** | MariaDB only |
|
||||
| **be-ops** | Gitea, url-shortener |
|
||||
| **be-build** | Docker builder |
|
||||
| **be-vpn** | WireGuard + RSSH bastion |
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
FE["Gentoo FE"] --> WEB["be-web"]
|
||||
WEB --> DATA["be-data"]
|
||||
WEB --> VPN["be-vpn"]
|
||||
OPS["be-ops"] --> DATA
|
||||
BUILD["be-build"] --> DATA
|
||||
```
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| ~5 VMs vs 7+; fits “money-wise” draft | Gitea + shortener still coupled |
|
||||
| Unblocks build/DB contention immediately | Second web node needed later for true HA |
|
||||
| Low risk to current URLs | WG still single point until pool |
|
||||
|
||||
### 3.3 Option C — Cloud-first hybrid
|
||||
|
||||
Move **staging** entirely to GCP/Azure; keep **prod** on FE→BE until hybrid LB proven.
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Managed LB, DB, object store day one | Egress cost; UDP/WG complexity |
|
||||
| Dress rehearsal for full cloud | PO learning curve; vendor lock-in mild |
|
||||
| ffmpeg/SFU elastic scale | **Violates** “safest topology” unless staged |
|
||||
|
||||
### 3.4 Comparison matrix
|
||||
|
||||
Scores 1 (poor) – 5 (excellent) for **current team size** and **alpha proximity**:
|
||||
|
||||
| Criterion | A — Full cluster | B — Minimal split | C — Cloud-first |
|
||||
|-----------|------------------|-------------------|-----------------|
|
||||
| Cost now | 2 | **4** | 3 |
|
||||
| Time to first value | 2 | **5** | 3 |
|
||||
| Zero-downtime readiness | 4 | 3 | **5** |
|
||||
| Topology safety | 4 | **5** | 2 |
|
||||
| SFU/ffmpeg fit | **5** | 3 | **5** |
|
||||
| Simulation fidelity | **5** | 4 | 2 |
|
||||
| **Global CDN / GEO** | 3 | 3 | **5** |
|
||||
| **Weighted total** | 25 | **27** | 25 |
|
||||
|
||||
**Weights used:** safety and cost now > cloud elasticity pre-alpha.
|
||||
|
||||
---
|
||||
|
||||
## 4. DR decision — recommended path
|
||||
|
||||
```text
|
||||
Now ──► Option B (minimal split) on-prem HVMs
|
||||
│
|
||||
├── Simulate Option A as DEV cluster (3–7 Alpine HVMs)
|
||||
│
|
||||
├── Add be-web-2 + FE upstream when metrics say so
|
||||
│
|
||||
└── Option C for staging only → then hybrid prod
|
||||
```
|
||||
|
||||
| Step | Action | Owner |
|
||||
|------|--------|-------|
|
||||
| 1 | Document service boundaries (this SPEC/DR) | Agent / PO review |
|
||||
| 2 | Move Docker builder to **be-build-1** | Sysop |
|
||||
| 3 | Move MariaDB to **be-data-1** (hardest cutover) | Sysop |
|
||||
| 4 | Stand up **dev cluster** simulation on intra network | Sysop |
|
||||
| 5 | Staging promotion runbook + DNS prefix | PO + DEV |
|
||||
| 6 | Cloud object store for OTA/APK | Sysop |
|
||||
| 7 | **CDN + bandwidth dashboards** (SPEC §12.3) | Sysop |
|
||||
| 8 | SFU SPEC + media workers (separate track) | PO spec |
|
||||
| 9 | **GEO policy schema + edge rules** (SPEC §14) | PO + DEV |
|
||||
|
||||
**Not chosen now:** full Option A in prod; full Option C prod cutover.
|
||||
|
||||
---
|
||||
|
||||
## 5. Current vs target diagrams
|
||||
|
||||
### Today (single BE)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
R["Router 134.17.26.161"] --> FE["FE Gentoo 10.7.0.10"]
|
||||
FE --> BE["Single Alpine BE 10.7.16.128"]
|
||||
BE --> S1["PHP consoles + APIs"]
|
||||
BE --> S2["MariaDB"]
|
||||
BE --> S3["Gitea + url-shortener"]
|
||||
BE --> S4["Docker build"]
|
||||
BE --> S5["WG + RSSH"]
|
||||
FE -->|"UDP 45340 DNAT"| S5
|
||||
```
|
||||
|
||||
### Target phase B (minimal split + HA web)
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
R["Router"] --> FE["Gentoo FE"]
|
||||
FE --> LB["nginx upstream / cloud LB"]
|
||||
LB --> W1["be-web-1"]
|
||||
LB --> W2["be-web-2"]
|
||||
W1 --> DB["be-data MariaDB"]
|
||||
W2 --> DB
|
||||
OPS["be-ops Gitea+shortener"] --> DB
|
||||
BUILD["be-build Docker"] --> DB
|
||||
VPN1["be-vpn-1"] --> DB
|
||||
VPN2["be-vpn-2 optional"]
|
||||
FE -->|"UDP DNAT pool"| VPN1
|
||||
FE --> VPN2
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Service split rationale
|
||||
|
||||
Draft asked to balance **cost vs CPU/RAM needs**. DR sizing rules:
|
||||
|
||||
| Workload | CPU | RAM | Disk | Pin to |
|
||||
|----------|-----|-----|------|--------|
|
||||
| PHP consoles | Medium | 2–4 GiB | Small | be-web* |
|
||||
| MariaDB | Medium | **8+ GiB** | **Fast SSD** | be-data |
|
||||
| Gitea mirrors | Low | 2 GiB | Medium | be-ops |
|
||||
| URL shortener | Low | 512 MiB | Tiny | be-ops |
|
||||
| Docker build | **Burst high** | **16 GiB** | Large ephemeral | be-build |
|
||||
| WireGuard | Low | 1 GiB | Tiny | be-vpn |
|
||||
| RSSH sshd | Low–med | 2 GiB | Tiny | be-vpn |
|
||||
| ffmpeg (future) | **Sustained high** | 8–32 GiB | Temp SSD | be-media* |
|
||||
| SFU (future) | **High** | 8+ GiB | Low | be-media* |
|
||||
| Recordings (future) | Low | — | **Large** | object store |
|
||||
|
||||
**Anti-pattern rejected:** MariaDB + Docker build + ffmpeg on one VM (draft implied bundle risk).
|
||||
|
||||
---
|
||||
|
||||
## 7. Environment promotion model
|
||||
|
||||
| Gate | dev → staging | staging → prod |
|
||||
|------|---------------|----------------|
|
||||
| Automated tests | `./gradlew :app:testDebugUnitTest` + BE smoke | + soak checklist |
|
||||
| Manual | PO feature acceptance | PO sign-off + tag |
|
||||
| Data | Synthetic | Anonymized copy optional |
|
||||
| Rollback | Re-deploy previous image | Git revert + DB restore runbook |
|
||||
|
||||
Align app releases with [GIT_FLOW.md](../GIT_FLOW.md); infra releases may cadence independently with **config version pins** per environment.
|
||||
|
||||
---
|
||||
|
||||
## 8. RSSH and connection capacity
|
||||
|
||||
Draft question: *how many simultaneous connections per instance?*
|
||||
|
||||
### Assumptions
|
||||
|
||||
- Heartbeat poll **1–7 min** ([REMOTE_ACCESS_IMPL](../REMOTE_ACCESS_IMPL.md))
|
||||
- Average SSH session **15 min** operator work
|
||||
- Alpine **be-vpn-1**: 2 vCPU, 2 GiB RAM
|
||||
|
||||
### Estimates
|
||||
|
||||
| Resource | Limiting factor | Conservative | With tuning |
|
||||
|----------|-----------------|--------------|-------------|
|
||||
| **Concurrent SSH** | `sshd` `MaxSessions` / CPU | 50 | **100** |
|
||||
| **Devices polling heartbeat** | PHP-FPM + MariaDB writes | 2 000 | **5 000** |
|
||||
| **New SSH/min** | `MaxStartups` | 10 | 30 |
|
||||
| **WG peers (lab)** | kernel + UDP | 200 | 500 / 2 nodes |
|
||||
|
||||
**Tuning knobs:** separate **heartbeat** PHP pool to **be-web** (off bastion); keep sshd on **be-vpn** only; connection pooling to MariaDB; read replicas for heartbeat read-heavy phases.
|
||||
|
||||
**DR recommendation:** Instrument **active SSH** and **heartbeat QPS** in graphs before selling commercial VPN (draft F4).
|
||||
|
||||
---
|
||||
|
||||
## 8b. Network bandwidth (DR)
|
||||
|
||||
Draft rev.2 adds **throughput** alongside connection counts.
|
||||
|
||||
### Baseline risk (today)
|
||||
|
||||
| Path | Risk |
|
||||
|------|------|
|
||||
| Single **WAN** uplink | OTA release or viral short link saturates all services |
|
||||
| **FE→BE** proxy | All HTTPS doubles through FE NIC |
|
||||
| No **CDN** | Every APK download hits origin BE |
|
||||
| **Media future** | SFU + ffmpeg on same uplink = hard fail without PoP/CDN |
|
||||
|
||||
### DR sizing decisions
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| **CDN before be-web-N** for OTA/static | Fixes egress Mbps cheaper than more PHP VMs |
|
||||
| **Separate media NIC/subnet** when SFU lands | Isolate UDP/WebRTC from MariaDB LAN |
|
||||
| **Egress budget alerts** at 80% ISP/cloud quota | PO notified before hard cap |
|
||||
| **Simulate release day** on staging | e.g. 1000 concurrent OTA downloads |
|
||||
|
||||
### Planning numbers (single-site prod, until CDN)
|
||||
|
||||
| Traffic | Peak Mbps (conservative) | Mitigation |
|
||||
|---------|--------------------------|------------|
|
||||
| Steady HTTPS (API + consoles) | 5–20 | Normal |
|
||||
| OTA drop (1000 devices × 40 MB APK) | **200–400** burst 10 min | **CDN required** |
|
||||
| Recording download (F3) | 50–200 per hot file | Object store + CDN signed URLs |
|
||||
| SFU fan-out 100 viewers | 50–500+ | Regional PoP; not on FE WAN |
|
||||
|
||||
Full table: [SPEC §12.3](../specs/20100612_1_scaling.md#123-network-bandwidth-and-egress).
|
||||
|
||||
---
|
||||
|
||||
## 8c. Global availability and GEO segments (DR)
|
||||
|
||||
Draft rev.2: **global reach** + **segment isolation** (example: deny publish/serve in **Zambia `ZM`**).
|
||||
|
||||
### Policy model (DR decision)
|
||||
|
||||
| Rule | Choice |
|
||||
|------|--------|
|
||||
| **Default** | **Allow** worldwide for control plane and public APIs |
|
||||
| **Content objects** | Attach optional `geo_policy` with `deny_publish` / `deny_serve` / `allow_only` |
|
||||
| **Enforcement order** | CDN/WAF → BE API (trusted geo header) → object signed URL → app entitlement |
|
||||
| **Operator UI** | Hub/console editor for geo segments when F3/F10 ship — not alpha |
|
||||
| **Sanctions** | PO-maintained deny lists; not auto-synced from external API in v1 |
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
REQ["Client request"] --> EDGE["CDN / FE edge"]
|
||||
EDGE --> GEO{"Country in deny_serve?"}
|
||||
GEO -->|yes| DENY["451 / blocked manifest"]
|
||||
GEO -->|no| BE["BE + geo_policy DB"]
|
||||
BE --> OK["200 / stream / redirect"]
|
||||
```
|
||||
|
||||
### Phasing
|
||||
|
||||
| Phase | GEO capability |
|
||||
|-------|----------------|
|
||||
| **Now** | Implicit worldwide; no per-content rules |
|
||||
| **CDN phase** | Edge geo block for **coarse** country bans |
|
||||
| **F3 storage** | Per-recording `geo_policy` in MariaDB + object metadata |
|
||||
| **F5 payments** | Entitlement includes `geo_allowed` |
|
||||
| **Multi-PoP** | GeoDNS routes to nearest **allowed** PoP only |
|
||||
|
||||
### Example use cases
|
||||
|
||||
| Use case | Policy |
|
||||
|----------|--------|
|
||||
| OTA `stable` globally | `default: allow` |
|
||||
| Beta APK EU-only | `allow_only: ["DE","FR",…]` |
|
||||
| Recording licensed EU/US only | `deny_serve: ["ZM",…]` + `allow_only` optional |
|
||||
| Short link to blocked catalog | BE returns 451 at redirect time |
|
||||
|
||||
**Open for PO:** whether **VPN exit** in denied country circumvents policy (DR: log + ToS; technical block on **VPN product** separate from HTTP geo).
|
||||
|
||||
---
|
||||
|
||||
## 9. Future capabilities — priority weighting
|
||||
|
||||
Weighted for **platform dependency** and **revenue enablement** (PO may override):
|
||||
|
||||
| Rank | Slot | Weight | Rationale |
|
||||
|------|------|--------|-----------|
|
||||
| **1** | F1 Multipoint cast (SFU/MCU) | Critical path | Unblocks WAN cast; already on [OPEN_TASKS_GRAPH.md](../OPEN_TASKS_GRAPH.md) |
|
||||
| **2** | F2 ffmpeg transcode | High | Shared worker pool for F3, F9 |
|
||||
| **3** | F3 Cast storage | High | Needs F1+F2 + object store |
|
||||
| **4** | F5 Payments | Medium | Enables F3, F4, F7 |
|
||||
| **5** | F4 Commercial VPN | Medium | RSSH/WG scale must be proven first |
|
||||
| **6** | F9 3rd-party streaming | Medium | Uses F2 cast handlers |
|
||||
| **7** | F6 Mobile add-ons | Lower | Can ship on-device first |
|
||||
| **8** | F7 In-app transfers | Lower | Compliance heavy |
|
||||
| **9** | F8 DNS / subdomains | Exploratory | Largest scope creep |
|
||||
| **10** | F10 GEO enforcement | Medium | Required for F3/F5 global commerce; draft rev.2 |
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
SFU["F1 SFU / multipoint"] --> STORE["F3 Cast storage"]
|
||||
FF["F2 ffmpeg workers"] --> STORE
|
||||
FF --> EXT["F9 3rd-party streams"]
|
||||
SFU --> EXT
|
||||
PAY["F5 Payments"] --> STORE
|
||||
PAY --> VPN["F4 Commercial VPN"]
|
||||
PAY --> XFER["F7 In-app transfers"]
|
||||
PAY --> ADD["F6 Add-ons"]
|
||||
DNS["F8 DNS / site builder"] --> PAY
|
||||
GEO["F10 GEO policy"] --> STORE
|
||||
GEO --> PAY
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Simulation lab design
|
||||
|
||||
Sysop simulates **Option A** under dev without WAN changes:
|
||||
|
||||
| HVM | Alpine packages | Role |
|
||||
|-----|-----------------|------|
|
||||
| `dev-web-1` | nginx, php81-fpm | Web tier |
|
||||
| `dev-data-1` | mariadb | DB |
|
||||
| `dev-ops-1` | git, gitea (optional) | Gitea + shortener |
|
||||
| `dev-build-1` | docker | Builder |
|
||||
| `dev-vpn-1` | wireguard-tools, openssh | VPN lab |
|
||||
|
||||
**Network:** `10.7.0.0/8` only; FE routes `dev.apps.f0xx.org` to dev-web when ready.
|
||||
|
||||
**Deliverable suggestion:** `docs/SIMULATION_CLUSTER.md` + scripts in phase 1 — not blocking this DR.
|
||||
|
||||
---
|
||||
|
||||
## 11. Cloud provider notes (GCP / Azure)
|
||||
|
||||
| Concern | GCP | Azure |
|
||||
|---------|-----|-------|
|
||||
| **Managed LB** | ILB + NEG | App Gateway / Front Door |
|
||||
| **Object store** | GCS | Blob |
|
||||
| **Managed MariaDB** | Cloud SQL | Azure Database for MariaDB |
|
||||
| **UDP / WG** | harder | harder — keep on-prem FE DNAT longer |
|
||||
| **ffmpeg burst** | MIG (CPU) | VMSS |
|
||||
| **Egress cost** | Watch OTA/APK | Same |
|
||||
| **CDN + geo WAF** | Cloud CDN / Cloud Armor | Azure Front Door + rules |
|
||||
| **Multi-region** | GCS dual-region; Cloud CDN PoPs | Blob + AFD PoPs |
|
||||
|
||||
**DR:** Use cloud for **staging cluster first** (Option C partial) — validates IaC without prod risk. **CDN geo rules** prototype on staging before prod F3.
|
||||
|
||||
---
|
||||
|
||||
## 12. RBAC evolution
|
||||
|
||||
| Stage | Who | Access |
|
||||
|-------|-----|--------|
|
||||
| **Alpha (now)** | Developers = operators | Max privileges; shared session |
|
||||
| **Beta** | Split **developer** vs **support operator** | Tickets/RA without build admin |
|
||||
| **Commercial** | Customer admin + platform admin | Tenant isolation; audit export |
|
||||
|
||||
New roles MUST NOT require mobile app changes until F5 entitlements API exists.
|
||||
|
||||
---
|
||||
|
||||
## 13. Risks and mitigations
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
|------|--------|------------|
|
||||
| MariaDB migration downtime | Prod outage | Replica + cutover window; practice on dev |
|
||||
| Config loss on sync | 503 (seen url-shortener) | `ensure_*` scripts; secrets outside tree |
|
||||
| FE upstream misconfig | Partial outage | `nginx -t`; weighted rollout |
|
||||
| WG UDP cloud move | VPN breaks | Keep DNAT path until PO signs UDP plan |
|
||||
| Over-scoping F5–F9 | Never ship SFU | [§9](#9-future-capabilities-priority-weighting) order |
|
||||
| WAN saturation on OTA | Global outage perception | CDN + [§8b](#8b-network-bandwidth-dr) budgets |
|
||||
| Geo policy bypass via VPN | Compliance / licensing | Separate VPN egress policy; audit logs |
|
||||
| Wrong deny list | Legitimate users blocked | Staging test + `X-Debug-Country`; PO approval workflow |
|
||||
| Simulation sprawl | Sysop time | Cap dev HVMs at 7; destroy scripts |
|
||||
|
||||
---
|
||||
|
||||
## 14. Estimates
|
||||
|
||||
Person-days (**single sysop + DEV support**), planning accuracy ±40%:
|
||||
|
||||
| Phase | Work | Days |
|
||||
|-------|------|------|
|
||||
| **P0** | SPEC/DR + simulation design (this doc) | 2 |
|
||||
| **P1** | Split Docker build VM | 2 |
|
||||
| **P2** | MariaDB dedicated VM + migration | 3 |
|
||||
| **P3** | be-ops split (Gitea + shortener) | 1 |
|
||||
| **P4** | Dev cluster HVM simulation (5 VMs) | 4 |
|
||||
| **P5** | Staging env + promotion runbook | 3 |
|
||||
| **P6** | be-web-2 + FE upstream HA | 2 |
|
||||
| **P7** | Cloud object store for OTA | 2 |
|
||||
| **P7b** | CDN + bandwidth metrics on graphs | 2 |
|
||||
| **P8** | GEO policy schema + staging CDN geo test | 3 |
|
||||
| **P9** | SFU/media (separate SPEC) | TBD |
|
||||
| **Total (through P7b)** | | **~21 days** |
|
||||
|
||||
**Severity:** Strategic — parallel to alpha; no alpha gate.
|
||||
|
||||
---
|
||||
|
||||
## 15. Task dependency graph
|
||||
|
||||
Legend: **1** = highest priority (PO: post-alpha infra). Assignee **Sysop** = PO today; **Agent** = documentation/automation; **DEV** = app/config changes.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
T1["1. SPEC+DR scaling docs<br/>Agent ✓"]
|
||||
T2["2. Split be-build Docker<br/>Sysop"]
|
||||
T3["3. Dedicated be-data MariaDB<br/>Sysop"]
|
||||
T4["4. ensure_* config hardening all services<br/>Agent"]
|
||||
T5["5. Dev cluster simulation HVMs<br/>Sysop"]
|
||||
T6["6. Staging env + DNS prefix<br/>Sysop + PO"]
|
||||
T7["7. Promotion runbook dev→stg→prod<br/>PO"]
|
||||
T8["8. be-web-2 + FE upstream<br/>Sysop"]
|
||||
T9["9. Cloud object store OTA<br/>Sysop"]
|
||||
T9b["9b. CDN + bandwidth metrics<br/>Sysop"]
|
||||
T10["10. GEO policy + edge rules<br/>PO+DEV"]
|
||||
T11["11. SFU owner SPEC<br/>PO"]
|
||||
T12["12. ffmpeg worker pool<br/>DEV post-SFU"]
|
||||
T13["13. Payments / entitlements SPEC<br/>PO"]
|
||||
|
||||
T1 --> T2
|
||||
T1 --> T4
|
||||
T2 --> T3
|
||||
T3 --> T5
|
||||
T4 --> T6
|
||||
T5 --> T6
|
||||
T6 --> T7
|
||||
T7 --> T8
|
||||
T3 --> T8
|
||||
T8 --> T9
|
||||
T9 --> T9b
|
||||
T9b --> T10
|
||||
T10 --> T11
|
||||
T11 --> T12
|
||||
T12 --> T13
|
||||
|
||||
style T1 fill:#86efac,color:#14532d
|
||||
style T11 fill:#e5e7eb,color:#374151
|
||||
style T12 fill:#e5e7eb,color:#374151
|
||||
style T13 fill:#e5e7eb,color:#374151
|
||||
```
|
||||
|
||||
| ID | Task | Assignee | Blocks |
|
||||
|----|------|----------|--------|
|
||||
| 1 | Scaling SPEC/DR | Agent | — |
|
||||
| 2 | Docker → be-build | Sysop | DB split |
|
||||
| 3 | MariaDB → be-data | Sysop | HA web |
|
||||
| 4 | Post-sync config scripts (all services) | Agent | staging |
|
||||
| 5 | Dev HVM simulation | Sysop | staging realism |
|
||||
| 6 | Staging cluster | Sysop | promotion |
|
||||
| 7 | Promotion runbook | PO | prod multi-node |
|
||||
| 8 | Second web node | Sysop | cloud OTA |
|
||||
| 9 | Object store OTA | Sysop | CDN |
|
||||
| 9b | CDN + bandwidth dashboards | Sysop | GEO test |
|
||||
| 10 | GEO policy + edge | PO+DEV | F3 commerce |
|
||||
| 11 | SFU SPEC | PO | media workers |
|
||||
| 12 | ffmpeg pool | DEV | commercial storage |
|
||||
| 13 | Payments SPEC | PO | F6–F8 |
|
||||
|
||||
**Parallel now (non-blocking):** alpha soak, url-shortener ops, Gitea mirrors, ticket **SVC-OTA**.
|
||||
|
||||
---
|
||||
|
||||
## 16. Resolved open questions
|
||||
|
||||
| # | Draft question | DR answer |
|
||||
|---|----------------|-----------|
|
||||
| Q1 | How many VMs in a cluster? | **Option B:** 5 prod VMs to start; **dev simulation:** 3–7 ([SPEC §6.3](../specs/20100612_1_scaling.md#63-reference-cluster-layout-prod)) |
|
||||
| Q2 | “Other” architecture approaches? | **Option C** hybrid for staging; not prod-first ([§3.3](#33-option-c-cloud-first-hybrid)) |
|
||||
| Q3 | Cost vs CPU/RAM balance? | Split by [§6](#6-service-split-rationale) profile table |
|
||||
| Q4 | Seamless scaling / LB? | FE upstream → N PHP nodes; cloud managed LB in phase 5 ([SPEC §9](../specs/20100612_1_scaling.md#9-zero-downtime-and-load-balancing)) |
|
||||
| Q5 | Simultaneous connections / RSSH? | [§8](#8-rssh-and-connection-capacity) — 100 SSH, 5k heartbeats per bastion pair (tuned) |
|
||||
| Q6 | Alpine simulation? | [§10](#10-simulation-lab-design) — intra `10.7.0.0/8` HVMs only |
|
||||
| Q7 | Future MCU/SFU/ffmpeg/storage/VPN/payments? | Platform **slots** F1–F9; priority [§9](#9-future-capabilities-priority-weighting) |
|
||||
| Q8 | RBAC model change? | [§12](#12-rbac-evolution) three-stage |
|
||||
| Q9 | Keep current topology safe? | **Yes** — [SPEC §18](../specs/20100612_1_scaling.md#18-safety-constraints-current-topology) |
|
||||
| Q10 | Network bandwidth metrics? | [§8b](#8b-network-bandwidth-dr) + [SPEC §12.3](../specs/20100612_1_scaling.md#123-network-bandwidth-and-egress) — CDN before web horizontal scale |
|
||||
| Q11 | Global availability + GEO deny (e.g. Zambia)? | [§8c](#8c-global-availability-and-geo-segments-dr) + [SPEC §14](../specs/20100612_1_scaling.md#14-global-availability-and-geo-policy) — default allow, per-content deny |
|
||||
|
||||
---
|
||||
|
||||
## 17. Remaining open items (PO)
|
||||
|
||||
| ID | Question | Needed for |
|
||||
|----|----------|------------|
|
||||
| O1 | Target **monthly infra budget** for phase B (5 VMs)? | Option A vs B timing |
|
||||
| O2 | **Staging URL** pattern: `staging.apps.f0xx.org` vs path prefix? | DNS + TLS |
|
||||
| O3 | **Cloud vendor** preference GCP vs Azure vs none for 12 months? | P7–P9 |
|
||||
| O4 | Accept **maintenance window** for MariaDB cutover (minutes of read-only)? | P2 schedule |
|
||||
| O5 | **Commercial VPN** — same WG infra or separate product stack? | F4 design |
|
||||
| O6 | **SFU** — Janus on `:8089` vs greenfield mediasoup? | F1 SPEC ([20260608_BE_SERVICES_and_infra.md](../20260608_BE_SERVICES_and_infra.md)) |
|
||||
| O7 | **Payments** — Stripe-only vs multi-provider? | F5 SPEC |
|
||||
| O8 | **ISP/cloud egress cap** (GB/month) on FE WAN? | [§8b](#8b-network-bandwidth-dr) budgets |
|
||||
| O9 | **CDN provider** (Cloudflare vs cloud-native vs none 12 mo)? | OTA + GEO WAF |
|
||||
| O10 | **GEO deny lists** — PO-owned static JSON vs admin UI first? | F10 rollout |
|
||||
| O11 | Recordings **451** vs silent 404 for denied regions? | UX + compliance |
|
||||
|
||||
*When blocked on O1–O7, append to `tmp/YYYYMMDD_implementation_summary.pdf` per bottomline_reminder.*
|
||||
|
||||
---
|
||||
|
||||
## 18. Suggestions (PO / dev discretion)
|
||||
|
||||
| # | Suggestion |
|
||||
|---|------------|
|
||||
| S1 | Add **infra smoke** to `validate_be_services.sh` per VM role after split |
|
||||
| S2 | Version **nginx upstream** fragments in `examples/crash_reporter/backend/deploy/` |
|
||||
| S3 | Use **Docker** for PHP-FPM pools only after MariaDB split — avoid double complexity |
|
||||
| S4 | Keep **Gitea** on be-ops until mirror count > 10 repos |
|
||||
| S5 | Document **connection pool** sizes when RSSH device count > 500 |
|
||||
| S6 | Re-read this DR when **SFU SPEC** lands — revise [§9](#9-future-capabilities-priority-weighting) |
|
||||
| S7 | Add **WAN Mbps** graph series on FE before next OTA wide release |
|
||||
| S8 | Staging test matrix: `X-Debug-Country: ZM` for geo deny paths |
|
||||
|
||||
---
|
||||
|
||||
## 19. Changelog
|
||||
|
||||
| Date | Change |
|
||||
|------|--------|
|
||||
| 2026-06-12 | Initial DR from draft `20100612_1_scaling.txt`; options matrix; task graph |
|
||||
| 2026-06-12 | Rev. 2 — bandwidth (§8b), global/GEO (§8c), F10, task graph + open items |
|
||||
|
||||
---
|
||||
|
||||
## Related docs
|
||||
|
||||
- [SPEC — Platform scaling](../specs/20100612_1_scaling.md)
|
||||
- [INFRA.md](../INFRA.md)
|
||||
- [OPEN_TASKS_GRAPH.md](../OPEN_TASKS_GRAPH.md)
|
||||
- [20260608_ALPHA_PRIORITIES.md](../20260608_ALPHA_PRIORITIES.md)
|
||||
945
DRs/20100612_1_scaling.pdf
Normal file
945
DRs/20100612_1_scaling.pdf
Normal file
File diff suppressed because one or more lines are too long
426
DRs/20260616_rssh_routed_egress.md
Normal file
426
DRs/20260616_rssh_routed_egress.md
Normal file
@@ -0,0 +1,426 @@
|
||||
# RSSH routed egress (dev-only) — design review (DR)
|
||||
|
||||
<!-- doc-meta:start -->
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Author | Anton Afanasyeu |
|
||||
| Revision | R0 |
|
||||
| Creation date | 2026-06-16 |
|
||||
| Last modification date | 2026-06-16 |
|
||||
| Co-authored | Cursor Agent (project assistant) |
|
||||
| Severity | low |
|
||||
| State | postponed |
|
||||
| Document type | DR |
|
||||
| Pre-requisite to | Finish WireGuard remote-access E2E ([REMOTE_ACCESS_VALIDATION.md](../REMOTE_ACCESS_VALIDATION.md), [VPN_DEMO_GAPS_20260613.md](../VPN_DEMO_GAPS_20260613.md)) |
|
||||
<!-- doc-meta:end -->
|
||||
|
||||
\newpage
|
||||
\newpage
|
||||
|
||||
---
|
||||
|
||||
**Document type:** DR (Design Review)
|
||||
**Source draft:** [docs/drafts/20260616_rssh_routed_egress.txt](../drafts/20260616_rssh_routed_egress.txt)
|
||||
**PDF:** [20260616_rssh_routed_egress.pdf](20260616_rssh_routed_egress.pdf) · Regenerate: `bash scripts/build-all-docs-pdf.sh`
|
||||
**Status:** Draft for PO review — **no implementation** until WireGuard track is closed
|
||||
**Scope:** Developer settings only; optional future path; does **not** replace alpha RSSH (reverse tunnel, no VPN)
|
||||
|
||||
**Related:** [REMOTE_ACCESS_IMPL.md](../REMOTE_ACCESS_IMPL.md) · [20260602_REVERSE_SSH_proposals_summary.md](../20260602_REVERSE_SSH_proposals_summary.md) · [ROADMAP.md](../ROADMAP.md) (RSSH alpha)
|
||||
|
||||
**Documentation index:** [README.md](../README.md)
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Table of contents
|
||||
|
||||
<!-- toc -->
|
||||
- [1. Executive summary](#1-executive-summary)
|
||||
- [2. Problem statement](#2-problem-statement)
|
||||
- [3. Vocabulary and scope](#3-vocabulary-and-scope)
|
||||
- [4. Current state](#4-current-state)
|
||||
- [5. Goals and non-goals](#5-goals-and-non-goals)
|
||||
- [6. Architecture options](#6-architecture-options)
|
||||
- [7. DR recommendation](#7-dr-recommendation)
|
||||
- [8. Reference design (if pursued later)](#8-reference-design-if-pursued-later)
|
||||
- [9. Developer settings UX](#9-developer-settings-ux)
|
||||
- [10. Backend and bastion requirements](#10-backend-and-bastion-requirements)
|
||||
- [11. App components (sketch)](#11-app-components-sketch)
|
||||
- [12. Third-party and licenses](#12-third-party-and-licenses)
|
||||
- [13. Risks](#13-risks)
|
||||
- [14. Effort estimate (order of magnitude)](#14-effort-estimate-order-of-magnitude)
|
||||
- [15. Task dependency graph](#15-task-dependency-graph)
|
||||
- [16. Open questions (PO)](#16-open-questions-po)
|
||||
- [17. Changelog](#17-changelog)
|
||||
<!-- /toc -->
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
**Question:** Can we add a developer-only, system-wide VPN-style path that routes part or all device traffic through the **RSSH** (reverse SSH) session?
|
||||
|
||||
**Short answer:** **Yes, technically** — but it requires a **new data plane** (`VpnService` + IP-to-SOCKS or TUN-over-SSH), not an extension of today’s RSSH `-R` forward. It **reintroduces VPN permission and the status-bar key**, and overlaps **WireGuard full-tunnel**, which already implements route/app scope in the app.
|
||||
|
||||
**DR decision (R0):**
|
||||
|
||||
| Item | Decision |
|
||||
|------|----------|
|
||||
| **Timing** | **Postponed** until WireGuard remote-access work is finished and validated E2E |
|
||||
| **Alpha RSSH** | Unchanged — outbound reverse SSH for operator → device; **no** device routing |
|
||||
| **Near-term routing** | Use **WireGuard** + `RemoteAccessVpnRouting` (hub-only or full-tunnel) |
|
||||
| **If revived** | Dev-only sub-mode **“RSSH + route traffic (experimental)”** via **SSH dynamic forward (-D) + tun2socks + existing route/app prefs** |
|
||||
| **SPEC** | Do **not** write implementation SPEC until PO confirms after WG closure |
|
||||
|
||||
---
|
||||
|
||||
## 2. Problem statement
|
||||
|
||||
Developers may want:
|
||||
|
||||
1. **Operator access** to the device (shell/SFTP) — **RSSH today**.
|
||||
2. **Device traffic** to egress via the hub (e.g. Minsk NAT) or reach hub-private IPs — **WireGuard today** (split or full tunnel).
|
||||
3. In some networks, **UDP WireGuard** may be blocked while **TCP 443 SSH** works — motivation to explore routing **over RSSH**.
|
||||
|
||||
Without a clear DR, it is easy to assume `vpn-route-scope` / `vpn-app-scope` apply to RSSH; they currently apply to **WireGuard only** ([REMOTE_ACCESS_IMPL.md](../REMOTE_ACCESS_IMPL.md)).
|
||||
|
||||
---
|
||||
|
||||
## 3. Vocabulary and scope
|
||||
|
||||
| Term | Meaning here |
|
||||
|------|----------------|
|
||||
| **RSSH** | Device-initiated reverse SSH; `-R bastion:port → device:8022` for operator access |
|
||||
| **Routed egress** | Device IP traffic (some or all) sent through bastion/hub NAT to the Internet or private nets |
|
||||
| **VpnService** | Android API for app-provided VPN; required for system-wide or per-app routing **from our app** |
|
||||
| **Route scope** | Hub-only (`172.200.2.1/32`) vs full-tunnel (`0.0.0.0/0`) — `RemoteAccessVpnRouting` |
|
||||
| **App scope** | All apps vs this app only — `IncludedApplications` / `addAllowedApplication` |
|
||||
|
||||
**Not in scope for this DR:** Commercial VPN product (F4), always-on VPN in Android Settings, routing for non-developer builds.
|
||||
|
||||
---
|
||||
|
||||
## 4. Current state
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph phone [Device]
|
||||
RA[RemoteAccessService]
|
||||
WG[WireGuard :vpn]
|
||||
RSSH[ReverseSshTunnelBridge]
|
||||
LOCAL[RsshLocalSshServer :8022]
|
||||
end
|
||||
subgraph be [BE / bastion]
|
||||
WG0[wg0 UDP 45340]
|
||||
SSHD[sshd TCP 443]
|
||||
end
|
||||
RA -->|mode wireguard| WG
|
||||
WG -->|UDP| WG0
|
||||
RA -->|mode rssh| RSSH
|
||||
RSSH -->|TCP -R| SSHD
|
||||
SSHD -->|operator| LOCAL
|
||||
```
|
||||
|
||||
| Mode | Transport | Device routing | VPN icon |
|
||||
|------|-----------|----------------|----------|
|
||||
| **WireGuard** | UDP → `wg0` | Yes (hub-only or full) | Yes |
|
||||
| **RSSH** | TCP → sshd | **No** | **No** |
|
||||
|
||||
Existing prefs (`dev_remote_access_vpn_route_scope`, `dev_remote_access_vpn_app_scope`) are sent on heartbeat for WG; RSSH ignores them.
|
||||
|
||||
---
|
||||
|
||||
## 5. Goals and non-goals
|
||||
|
||||
### Goals (if implemented later)
|
||||
|
||||
| ID | Goal |
|
||||
|----|------|
|
||||
| G1 | Optional **dev-only** routing of device traffic via **TCP 443** SSH path |
|
||||
| G2 | Reuse **route scope** and **app scope** semantics from `RemoteAccessVpnRouting` |
|
||||
| G3 | Network self-test shows bastion, bind ports, egress expectation (like WG `vpn-ip`) |
|
||||
| G4 | Clear separation from **alpha RSSH** (reverse access only) |
|
||||
|
||||
### Non-goals
|
||||
|
||||
| ID | Non-goal |
|
||||
|----|----------|
|
||||
| NG1 | Replace WireGuard for production/lab routing |
|
||||
| NG2 | Alpha release blocker |
|
||||
| NG3 | Avoid `VpnService.prepare()` — routed mode **will** need it |
|
||||
| NG4 | SSH `PermitTunnel` / `ssh -w` as v1 (too heavy for Alpine bastion v1) |
|
||||
| NG5 | Third-party “VPN provider” in Android Settings (no such API; we use `VpnService`) |
|
||||
|
||||
---
|
||||
|
||||
## 6. Architecture options
|
||||
|
||||
### 6.1 Option A — WireGuard full-tunnel (existing)
|
||||
|
||||
**Use when:** Hub egress or full WAN via Minsk NAT is needed **now**.
|
||||
|
||||
- Dev settings → WireGuard → route scope **All traffic** → reconnect session.
|
||||
- BE `wg0` PostUp MASQUERADE already documented.
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Implemented | UDP DNAT + VPN consent |
|
||||
| Kernel/userspace WG | Not RSSH transport |
|
||||
|
||||
**Verdict:** **Default for routing** until WG track is done.
|
||||
|
||||
### 6.2 Option B — RSSH + SSH `-D` (SOCKS) + `VpnService` + tun2socks
|
||||
|
||||
**Use when:** PO explicitly needs **TCP-only** routing and accepts VPN icon + extra stack.
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph device [Device]
|
||||
TUN[VpnService TUN]
|
||||
T2S[tun2socks userspace]
|
||||
SOCKS[127.0.0.1:SOCKS]
|
||||
JSch[JSch session]
|
||||
end
|
||||
subgraph bastion [Bastion / BE]
|
||||
SSD[sshd]
|
||||
NAT[NAT egress eth0]
|
||||
end
|
||||
TUN --> T2S --> SOCKS --> JSch --> SSD --> NAT
|
||||
```
|
||||
|
||||
1. Extend `ReverseSshTunnelBridge`: keep `-R` for operator; add **`-D 127.0.0.1:<socks_port>`** on same session.
|
||||
2. New **RsshRoutedVpnService** (or branch in `:vpn`) establishes TUN with routes from `RemoteAccessVpnRouting`.
|
||||
3. **tun2socks** forwards IP packets → local SOCKS → SSH stream → bastion → Internet.
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| TCP 443 friendly | VPN permission + icon |
|
||||
| Reuses SSH session | tun2socks dep + licenses |
|
||||
| Reuses route/app prefs | TCP-over-TCP, MTU, battery |
|
||||
| | Bastion egress policy + capacity |
|
||||
| | JSch dynamic forwarding QA |
|
||||
|
||||
**Verdict:** **Reference design if PO revives** routed RSSH.
|
||||
|
||||
### 6.3 Option C — SSH layer-3 `tun` (`ssh -w`)
|
||||
|
||||
Both ends need TUN, `PermitTunnel`, root/capabilities on bastion, custom Android client.
|
||||
|
||||
**Verdict:** **Reject** for dev-only v1 (operational cost).
|
||||
|
||||
### 6.4 Option D — Hybrid (proposal doc)
|
||||
|
||||
**WG for packets + RSSH (or SSH over WG) for operator files.**
|
||||
|
||||
**Verdict:** Valid **long-term**; does not require RSSH alone to carry IP routing.
|
||||
|
||||
### 6.5 Comparison matrix
|
||||
|
||||
| Criterion | A WG | B RSSH+SOCKS | C ssh -w | D hybrid |
|
||||
|-----------|------|--------------|----------|----------|
|
||||
| Ready today | Yes | No | No | Partial |
|
||||
| TCP 443 only | No | Yes | Yes | Mixed |
|
||||
| VPN icon | Yes | Yes | Yes | Yes |
|
||||
| Alpha RSSH purity | N/A | Weakens if default | Weakens | OK |
|
||||
| Impl effort | Done | High | Very high | Medium |
|
||||
|
||||
---
|
||||
|
||||
## 7. DR recommendation
|
||||
|
||||
1. **Close WireGuard E2E** (peer reconcile, self-test `vpn-ip`, hub→phone diagnostics, full-tunnel egress check) — **current priority**.
|
||||
2. **Keep alpha RSSH** as reverse tunnel only ([ROADMAP.md](../ROADMAP.md)).
|
||||
3. **Park Option B** in this DR; PO reviews §16 and decides **go / no-go** after WG.
|
||||
4. If **go:** spawn SPEC `rssh_routed_egress` (dev-only feature flag), not a change to core RSSH connect payload for alpha.
|
||||
|
||||
---
|
||||
|
||||
## 8. Reference design (if pursued later)
|
||||
|
||||
### 8.1 Control plane
|
||||
|
||||
Unchanged: `heartbeat.php` `type: ra`, `tunnel: ssh_reverse`, credentials for bastion + `-R` port.
|
||||
|
||||
Optional dev flag in heartbeat meta: `rssh_route_experimental: true` (telemetry only; no BE requirement for v0).
|
||||
|
||||
### 8.2 Data plane
|
||||
|
||||
| Step | Component |
|
||||
|------|-----------|
|
||||
| 1 | `ReverseSshTunnelBridge` connects; `-R` as today |
|
||||
| 2 | Same session: `session.setPortForwardingD("127.0.0.1", socksPort)` (JSch) |
|
||||
| 3 | `RsshRoutedVpnEngine` starts TUN via `VpnService` |
|
||||
| 4 | tun2socks reads TUN, connects to `127.0.0.1:socksPort` |
|
||||
| 5 | Bastion sshd forwards; **iptables MASQUERADE** on bastion for egress |
|
||||
|
||||
### 8.3 Routing semantics
|
||||
|
||||
Reuse `RemoteAccessVpnRouting.resolveAllowedIps()`:
|
||||
|
||||
| Route scope | TUN routes |
|
||||
|-------------|------------|
|
||||
| **HUB_ONLY** | `172.200.2.1/32` (or configured hub CIDR) |
|
||||
| **FULL_TUNNEL** | `0.0.0.0/0`, `::/0` |
|
||||
|
||||
Reuse `appendInterfaceExtras()` for **THIS_APP_ONLY**.
|
||||
|
||||
**Note:** Hub reachability over RSSH may need **explicit route to hub IP via TUN** or SOCKS; validate that hub-only does not assume WG interface addresses.
|
||||
|
||||
### 8.4 Failure modes
|
||||
|
||||
| Symptom | Likely cause |
|
||||
|---------|----------------|
|
||||
| SOCKS up, no Internet | Bastion NAT / forwarding disabled |
|
||||
| Tunnel up, wrong egress IP | Route scope hub-only (expected) |
|
||||
| Intermittent stalls | TCP-over-TCP congestion |
|
||||
| VPN works, operator SFTP dead | `-R` forward dropped; session limits |
|
||||
|
||||
---
|
||||
|
||||
## 9. Developer settings UX
|
||||
|
||||
Proposed (not implemented):
|
||||
|
||||
```
|
||||
Remote access mode: [ Disabled | WireGuard | RSSH ]
|
||||
|
||||
— When WireGuard —
|
||||
VPN route scope: Hub only | All traffic
|
||||
VPN app scope: All apps | This app only
|
||||
|
||||
— When RSSH —
|
||||
[ ] Route device traffic via bastion (experimental)
|
||||
(shows route/app scope only when checked)
|
||||
Hint: Reverse tunnel for operator access is always on in RSSH mode.
|
||||
Routing requires VPN permission (same as WireGuard).
|
||||
```
|
||||
|
||||
Network self-test (RSSH + experimental): `rssh-bastion`, `rssh-remote-bind`, `rssh-socks`, `wan-egress-via-vpn`, `rssh-connected`.
|
||||
|
||||
---
|
||||
|
||||
## 10. Backend and bastion requirements
|
||||
|
||||
| Requirement | Notes |
|
||||
|-------------|--------|
|
||||
| `AllowTcpForwarding yes` | Already for `-R` |
|
||||
| Dynamic forwarding | Confirm sshd `GatewayPorts` / no restrictive `Match` block for `-D` |
|
||||
| Egress NAT | iptables/nft MASQUERADE on bastion (similar to `wg0` PostUp) |
|
||||
| Capacity | Routed dev traffic adds bandwidth; separate from §12.1 RSSH SSH counts in scaling DR |
|
||||
| Audit | Log experimental flag; no change to alpha session schema required |
|
||||
|
||||
No new nginx HTTP paths; optional stream unchanged.
|
||||
|
||||
---
|
||||
|
||||
## 11. App components (sketch)
|
||||
|
||||
| Component | Process | Role |
|
||||
|-----------|---------|------|
|
||||
| `ReverseSshTunnelBridge` | main | SSH session; `-R` + `-D` |
|
||||
| `RsshRoutedVpnService` | `:vpn` | TUN + lifecycle |
|
||||
| `RsshTun2SocksEngine` | `:vpn` | Packet relay |
|
||||
| `RemoteAccessVpnRouting` | shared | Route/app scope |
|
||||
| `DevVpnStatusProbe` | main | Diagnostics |
|
||||
|
||||
Do **not** fold into `WireGuardVpnEngine`; separate engine behind common `VpnServiceClient` interface if refactored later.
|
||||
|
||||
---
|
||||
|
||||
## 12. Third-party and licenses
|
||||
|
||||
| Candidate | License | Action if adopted |
|
||||
|-----------|---------|-------------------|
|
||||
| **badvpn** / **tun2socks** | Various | Evaluate; add to `third-party/`, mobile + BE license files |
|
||||
| **JSch** | BSD | Already in app; verify `-D` support |
|
||||
| **Apache MINA SSHD** | Apache-2.0 | Local server only today |
|
||||
|
||||
Update [COMMERCIAL.md](../COMMERCIAL.md) and `app/src/main/assets/licenses/` before merge.
|
||||
|
||||
---
|
||||
|
||||
## 13. Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Duplicates WG | PO go/no-go; default to WG |
|
||||
| Alpha scope creep | Dev-only flag; off in release if needed |
|
||||
| VPN consent UX regression | Document in FR; separate from “no VPN” RSSH story |
|
||||
| Bastion abuse (open egress) | Dev whitelist + session TTL + rate limits |
|
||||
| Maintenance burden | Two routing stacks |
|
||||
|
||||
---
|
||||
|
||||
## 14. Effort estimate (order of magnitude)
|
||||
|
||||
| Phase | Work | Duration (indicative) |
|
||||
|-------|------|------------------------|
|
||||
| P0 | WG E2E closure | **Current sprint** (PO priority) |
|
||||
| P1 | Spike: JSch `-D` + manual SOCKS browser test | 2–3 d |
|
||||
| P2 | tun2socks + `VpnService` + route prefs | 5–8 d |
|
||||
| P3 | Bastion NAT + self-test + docs | 2–3 d |
|
||||
| P4 | Unit/integration tests, soak | 3–5 d |
|
||||
|
||||
**Total if pursued:** ~3–4 weeks **after** WG done — not parallelized with WG closure unless PO overrides.
|
||||
|
||||
---
|
||||
|
||||
## 15. Task dependency graph
|
||||
|
||||
Legend: **blocked by** upstream task.
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
WG1["1. WG peer reconcile on BE deploy"]
|
||||
WG2["2. WG E2E phone ↔ hub"]
|
||||
WG3["3. WG full-tunnel egress validation"]
|
||||
PO["4. PO review this DR"]
|
||||
GO{"5. Go routed RSSH?"}
|
||||
SPIKE["6. JSch -D spike"]
|
||||
SPEC["7. SPEC rssh_routed_egress"]
|
||||
IMPL["8. Implement dev experimental mode"]
|
||||
|
||||
WG1 --> WG2 --> WG3 --> PO
|
||||
PO --> GO
|
||||
GO -->|no| PARK["Park — use WG only"]
|
||||
GO -->|yes| SPIKE --> SPEC --> IMPL
|
||||
```
|
||||
|
||||
| Priority | Task | Owner | State |
|
||||
|----------|------|-------|-------|
|
||||
| **1** | Finish WG (peer sync, ping/handshake, self-test, full-tunnel) | Dev/PO | In progress |
|
||||
| **2** | Review DR R0 | PO | Pending |
|
||||
| **3** | RSSH routed egress | — | **Postponed** |
|
||||
|
||||
---
|
||||
|
||||
## 16. Open questions (PO)
|
||||
|
||||
| # | Question | Default if silent |
|
||||
|---|----------|-------------------|
|
||||
| Q1 | After WG E2E, is TCP-only routing still a requirement? | **No** — stay WG-only for routing |
|
||||
| Q2 | Accept VPN icon for an experimental RSSH routing mode? | Required for Option B |
|
||||
| Q3 | Should hub-only over RSSH target `172.200.2.1` without WG? | Clarify use case |
|
||||
| Q4 | Bastion egress NAT for dev traffic — allowed on prod `be-vpn`? | Lab-only |
|
||||
| Q5 | Single SSH session for `-R` + `-D` or two sessions? | Prefer **one** session |
|
||||
|
||||
---
|
||||
|
||||
## 17. Changelog
|
||||
|
||||
| Rev | Date | Change |
|
||||
|-----|------|--------|
|
||||
| R0 | 2026-06-16 | Initial DR from PO question; postponed pending WG closure |
|
||||
|
||||
---
|
||||
|
||||
## Related docs
|
||||
|
||||
- [REMOTE_ACCESS_IMPL.md](../REMOTE_ACCESS_IMPL.md) — WG vs RSSH modes
|
||||
- [REMOTE_ACCESS_VALIDATION.md](../REMOTE_ACCESS_VALIDATION.md) — CLI/mobile WG checks
|
||||
- [VPN_DEMO_GAPS_20260613.md](../VPN_DEMO_GAPS_20260613.md) — WG demo gaps
|
||||
- [20260602_REVERSE_SSH_proposals_summary.md](../20260602_REVERSE_SSH_proposals_summary.md) — original transport comparison
|
||||
- [specs/20100612_1_scaling.md](../specs/20100612_1_scaling.md) — `be-vpn`, RSSH capacity
|
||||
675
DRs/20260616_rssh_routed_egress.pdf
Normal file
675
DRs/20260616_rssh_routed_egress.pdf
Normal file
File diff suppressed because one or more lines are too long
375
DRs/20260617_codec2_ulbw_voice.md
Normal file
375
DRs/20260617_codec2_ulbw_voice.md
Normal file
@@ -0,0 +1,375 @@
|
||||
# Codec2 / ultra-low-bandwidth voice — design review (DR)
|
||||
|
||||
<!-- doc-meta:start -->
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Author | Anton Afanasyeu |
|
||||
| Revision | R0 |
|
||||
| Creation date | 2026-06-17 |
|
||||
| Last modification date | 2026-06-17 |
|
||||
| Co-authored | Cursor Agent (project assistant) |
|
||||
| Severity | low |
|
||||
| State | approved for planning (implementation post-alpha) |
|
||||
| Document type | DR |
|
||||
| Pre-requisite to | [ROADMAP.md](../ROADMAP.md) §6.1 Opus/Speex JNI encode/decode; F1 SFU SPEC (post-alpha) |
|
||||
<!-- doc-meta:end -->
|
||||
|
||||
\newpage
|
||||
\newpage
|
||||
|
||||
---
|
||||
|
||||
**Document type:** DR (Design Review)
|
||||
**Source draft:** [docs/drafts/20260617_codec2_ulbw_voice_integration.txt](../drafts/20260617_codec2_ulbw_voice_integration.txt)
|
||||
**PDF:** [20260617_codec2_ulbw_voice.pdf](20260617_codec2_ulbw_voice.pdf) · Regenerate: `bash scripts/build-all-docs-pdf.sh`
|
||||
**Status:** PO-reviewed (inline draft comments incorporated) — **no native implementation** until Opus/Speex JNI sinks land
|
||||
**Scope:** Voice-only / ULBW audio cast on P2P (LAN, BT, mesh); future MCU/SFU relay hooks
|
||||
|
||||
**Related:** [OPUS_SPEEX_VALIDATION.md](../OPUS_SPEEX_VALIDATION.md) · [ndk/README.md](../../ndk/README.md) · [20260608_BE_SERVICES_and_infra.md](../20260608_BE_SERVICES_and_infra.md) § SFU · [specs/20100612_1_scaling.md](../specs/20100612_1_scaling.md) F1
|
||||
|
||||
**Documentation index:** [README.md](../README.md)
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Table of contents
|
||||
|
||||
<!-- toc -->
|
||||
- [1. Executive summary](#1-executive-summary)
|
||||
- [2. Problem statement](#2-problem-statement)
|
||||
- [3. Vocabulary](#3-vocabulary)
|
||||
- [4. Current state (mobile + BE)](#4-current-state-mobile--be)
|
||||
- [5. Codec landscape and PO decisions](#5-codec-landscape-and-po-decisions)
|
||||
- [6. Goals and non-goals](#6-goals-and-non-goals)
|
||||
- [7. Mobile peer negotiation (today)](#7-mobile-peer-negotiation-today)
|
||||
- [8. Proposed mobile integration](#8-proposed-mobile-integration)
|
||||
- [9. Future backend: MCU / SFU](#9-future-backend-mcu--sfu)
|
||||
- [10. Protocol extensions](#10-protocol-extensions)
|
||||
- [11. Third-party and licenses](#11-third-party-and-licenses)
|
||||
- [12. Testing and validation](#12-testing-and-validation)
|
||||
- [13. Task dependency graph](#13-task-dependency-graph)
|
||||
- [14. Risks](#14-risks)
|
||||
- [15. Open questions (remaining)](#15-open-questions-remaining)
|
||||
- [16. Changelog](#16-changelog)
|
||||
<!-- /toc -->
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
**Question:** Should androidcast integrate **Codec2** for ultra-low-bandwidth (ULBW) **voice-only** casts when normal AAC/Opus paths are too heavy for the link?
|
||||
|
||||
**Short answer:** **Yes, as a post-alpha product slice** — not a lab experiment. End users on marginal links (BT P2P, slow mesh, very poor LTE) should get **intelligible audio** (“something rather than nothing”). **Codec2 @ 3200 bit/s** (and lower modes where needed) fits that goal better than Opus narrowband @ ~6 kbps + FEC in PO’s practical experience (lower sustained bandwidth, less audible noise on those links).
|
||||
|
||||
**DR decision (R0):**
|
||||
|
||||
| Item | Decision |
|
||||
|------|----------|
|
||||
| **Primary use case** | **Voice-only** ULBW cast; not the audio leg of full screen cast (8 kHz telephony; no lip-sync with HD video) |
|
||||
| **Codec2 modes** | **3200 bit/s** default target; **700C** for extreme BT/mesh; user/dev sub-mode selection |
|
||||
| **MELP / TWELP** | **Out of scope** (patent / slim OSS path) |
|
||||
| **AAC** | Keep as platform MediaCodec path; no license conflict for screen cast |
|
||||
| **Opus / Speex** | Finish **JNI encode/decode** first ([ROADMAP.md](../ROADMAP.md) — cross-compile done, sinks **TODO**); negotiation/scaffolding exists today but wire path is **stub → AAC** without `libopus.a` / encode JNI |
|
||||
| **Codec2 timing** | After **6.1** JNI baseline; behind dev flag until E2E validated |
|
||||
| **SFU / MCU** | Plan **transcode at ingress** (Codec2 → Opus) for mixed WebRTC rooms; separate F1 SPEC — signaling reuses extended audio caps |
|
||||
| **MSG_AUDIO_SELECTED** | Add in handshake when Codec2 implementation starts (SETTINGS_VERSION bump) — aligns P2P multicast and SFU data channel |
|
||||
|
||||
---
|
||||
|
||||
## 2. Problem statement
|
||||
|
||||
Screen cast defaults to **AAC** (32–128 kbps class). Negotiation already advertises **Opus** and **Speex**, but without native encode/decode the sender often still ships **AAC** while labeling Opus in diagnostics.
|
||||
|
||||
For **ULBW** links:
|
||||
|
||||
- Opus @ 6 kbps + FEC/NACK can exceed the sustained budget PO has seen on BT/mesh and adds noise versus Codec2-class vocoders.
|
||||
- Users still want **audio cast** (commentary, coordination, voice relay) when video is impossible — a deliberate **voice-only** mode.
|
||||
|
||||
Future **multipoint** (F1 SFU/MCU) cannot rely on pairwise discovery bitmasks alone; room-level **negotiated_audio** must be defined before Codec2 legs join a relay.
|
||||
|
||||
---
|
||||
|
||||
## 3. Vocabulary
|
||||
|
||||
| Term | Meaning |
|
||||
|------|---------|
|
||||
| **ULBW** | Ultra-low bandwidth; sustained **< 3–4 kbps** audio payload target on worst links |
|
||||
| **Voice-only cast** | No video track; discovery may send `videoCodecs = 0` |
|
||||
| **Radio mode** | Dev/user profile preferring Codec2 in AUTO when both peers advertise `AUDIO_CODEC2` |
|
||||
| **P2P negotiation** | `CastProtoHeader.audioCodecs` bitmask + `AudioNegotiator` (today) |
|
||||
| **Room negotiation** | Signaling API + server-published codec (SFU future) |
|
||||
| **SFU** | Selective forwarding unit — relay without per-subscriber transcode |
|
||||
| **MCU** | Mix decode → PCM → encode; needed for heterogeneous legs |
|
||||
|
||||
---
|
||||
|
||||
## 4. Current state (mobile + BE)
|
||||
|
||||
### 4.1 Audio codecs in app
|
||||
|
||||
| Layer | Opus / Speex today | Notes |
|
||||
|-------|-------------------|-------|
|
||||
| `CastCodecFlags` | `AUDIO_OPUS=4`, `AUDIO_SPEEX=8` | Masks advertised in discovery |
|
||||
| `AudioNegotiator` | Works | AUTO scoring via `CodecPriorityCatalog` |
|
||||
| `NativeCodecBridge` | **Probe only** (`nativeIsOpusAvailable`) | No encode/decode JNI in `opus_bridge.c` / `speex_bridge.c` |
|
||||
| Wire path | **OPUS_STUB / SPEEX_STUB → AAC** | `PassthroughCodecPolicy`, `AudioCodecFactory` |
|
||||
| Build | `scripts/build-native-codecs.sh` can link `.a` | [ROADMAP.md](../ROADMAP.md): JNI sinks **TODO** |
|
||||
|
||||
**Clarification for PO:** Prior agent work completed **negotiation, settings UI, priority catalog, and native autolink scaffolding** — not full Opus/Speex on-the-wire encoding. Task **6.1** remains open for JNI sinks + E2E per [OPUS_SPEEX_VALIDATION.md](../OPUS_SPEEX_VALIDATION.md).
|
||||
|
||||
### 4.2 Video vs audio negotiation asymmetry
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant S as Sender
|
||||
participant R as Receiver
|
||||
Note over S,R: Discovery: audioCodecs bitmask in header
|
||||
S->>R: MSG_HELLO, CAST_SETTINGS
|
||||
S->>R: MSG_CODEC_CAPS (video MIME list)
|
||||
R->>S: MSG_RECEIVER_CAST_SETTINGS
|
||||
R->>S: MSG_CODEC_CAPS
|
||||
R->>S: MSG_CODEC_SELECTED (video MIME only)
|
||||
Note over S: AudioNegotiator.resolveForSender(peerMask)
|
||||
Note over R: No MSG_AUDIO_SELECTED today
|
||||
```
|
||||
|
||||
### 4.3 Backend (gray)
|
||||
|
||||
- `SfuRelayGate.PREVIEW_ENABLED = false`; `api/sfu_health.php` returns disabled.
|
||||
- Janus on **:8089** is **other stack** — androidcast signaling stays on `apps.f0xx.org` PHP ([INFRA.md](../INFRA.md)).
|
||||
|
||||
---
|
||||
|
||||
## 5. Codec landscape and PO decisions
|
||||
|
||||
| Codec | Bitrate | Role in androidcast | DR decision |
|
||||
|-------|---------|---------------------|-------------|
|
||||
| **Codec2** | 700–3200 bps | ULBW voice-only | **Integrate** (post-6.1) |
|
||||
| **Opus SILK NB** | ~6–12 kbps | General narrowband when link allows | Finish JNI; keep in AUTO above Codec2 only when bandwidth sufficient |
|
||||
| **Speex NB** | ~2–15 kbps | Legacy / fallback | Finish JNI |
|
||||
| **AAC-LC** | 32+ kbps | Screen cast default | Unchanged |
|
||||
| **MELP/TWELP** | ~2.4 kbps | — | **Rejected** |
|
||||
|
||||
**PO rationale (2026-06-17):** Opus @ 6k + FEC is **heavier and noisier** than Codec2 on target links; Codec2 **3200 bps** meets product goals; intent is **end-user value**, not codec shootout lab work.
|
||||
|
||||
**Maintainer risk:** Codec2 upstream is in active re-development; pin submodule tag (e.g. 1.2.0) and mirror on Gitea like `third-party/opus`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Goals and non-goals
|
||||
|
||||
### Goals
|
||||
|
||||
- Voice-only cast mode with **Codec2 encode/decode** on Android (NDK).
|
||||
- Extend **peer negotiation** (`AUDIO_CODEC2 = 16`, optional `MSG_AUDIO_SELECTED`).
|
||||
- Document **SFU ingress transcode** and room caps for F1 SPEC.
|
||||
- LGPL compliance + license files.
|
||||
|
||||
### Non-goals (R0)
|
||||
|
||||
- MELP/TWELP integration.
|
||||
- Codec2 as default for **screen cast with video**.
|
||||
- SFU passthrough-only design (closed Android mesh edge case only).
|
||||
- Alpha ship — hidden until E2E + PO QA.
|
||||
|
||||
---
|
||||
|
||||
## 7. Mobile peer negotiation (today)
|
||||
|
||||
| Step | Mechanism | Codec2 impact |
|
||||
|------|-----------|---------------|
|
||||
| Advertise | `CastCodecFlags.localAudioMask()` in discovery | Add bit 16 when native probe passes |
|
||||
| Resolve | `AudioNegotiator.negotiate(senderMask, receiverMask, pref, settings)` | Add `CODEC2` to enum + `CodecPriorityId` |
|
||||
| Score | `codecs.json` + `CodecPriorityCatalog` | ULBW profile: CODEC2 lowest `bitrate_score` |
|
||||
| Wire | Audio frames (today AAC ADTS) | New codec tag or `MSG_AUDIO_FRAME` subtype |
|
||||
|
||||
**Gap:** Receiver does not receive explicit audio selection — sender-resolved only. **Must fix** before multicast + SFU (§10).
|
||||
|
||||
---
|
||||
|
||||
## 8. Proposed mobile integration
|
||||
|
||||
### 8.1 Settings and enum
|
||||
|
||||
```text
|
||||
CastSettings.AudioCodec.CODEC2
|
||||
CastSettings.codec2Mode: 3200 | 1300 | 700C (default 3200 per PO)
|
||||
CastSettings.VoiceOnlyMode: boolean
|
||||
```
|
||||
|
||||
### 8.2 Native stack
|
||||
|
||||
1. `third-party/codec2` submodule + `scripts/init-third-party-submodules.sh`
|
||||
2. `scripts/build-native-codecs.sh` → `libcodec2.a`, `ANDROIDCAST_HAVE_CODEC2`
|
||||
3. JNI: `isCodec2Available()`, encode/decode **8 kHz mono** frames
|
||||
4. Link via existing `libandroidcast_codecs.so`
|
||||
|
||||
### 8.3 Priority (ULBW profile)
|
||||
|
||||
Suggested AUTO order when `NetworkAdoption.ESTIMATED` or **radio mode** on:
|
||||
|
||||
**CODEC2 → OPUS → SPEEX → AAC → PCM**
|
||||
|
||||
### 8.4 Milestones
|
||||
|
||||
| # | Deliverable | Depends on |
|
||||
|---|-------------|------------|
|
||||
| M1 | Opus/Speex JNI encode/decode + E2E | 6.1 |
|
||||
| M2 | Codec2 probe + round-trip unit test | M1 |
|
||||
| M3 | Voice-only UI slice (dev gate) | M2 |
|
||||
| M4 | `MSG_AUDIO_SELECTED` + SETTINGS_VERSION | M2 |
|
||||
| M5 | SFU signaling fields in F1 SPEC | Owner SFU spec |
|
||||
|
||||
---
|
||||
|
||||
## 9. Future backend: MCU / SFU
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph clients [Android clients]
|
||||
A[Sender A Codec2]
|
||||
B[Sender B Opus]
|
||||
C[Receiver WebRTC]
|
||||
end
|
||||
subgraph be [BE post-alpha]
|
||||
SIG[Signaling API PHP]
|
||||
SFU[Janus / media worker]
|
||||
end
|
||||
A --> SIG
|
||||
B --> SIG
|
||||
SIG --> SFU
|
||||
SFU -->|transcode| C
|
||||
SFU -->|fan-out Opus| B
|
||||
```
|
||||
|
||||
### 9.1 Negotiation shift (P2P → room)
|
||||
|
||||
| Layer | P2P today | SFU future |
|
||||
|-------|-----------|------------|
|
||||
| Caps | Header bitmask | `join` JSON: `audio_caps.bitmask`, `codec2_modes[]` |
|
||||
| Pick | `AudioNegotiator` local | Server `negotiated_audio` |
|
||||
| Auth | PIN / LAN | Session RBAC + room token |
|
||||
|
||||
### 9.2 Codec2 on media plane (DR preference)
|
||||
|
||||
| Mode | When | CPU on be-media-N |
|
||||
|------|------|-------------------|
|
||||
| **Transcode ingress** | Mixed room (Android + browser) | **Recommended** — decode Codec2 → Opus egress |
|
||||
| Passthrough | All Codec2 clients | Low CPU; no WebRTC viewers |
|
||||
| MCU mix | Conference voice | High CPU; radio bridge legs only |
|
||||
|
||||
**Signaling sketch** (F1 SPEC — not implemented):
|
||||
|
||||
- `POST /api/sfu/room` — `ingress_caps: [opus, codec2]`, `egress_caps: [opus]`
|
||||
- `POST /api/sfu/room/{id}/join` — returns transport params + `negotiated_audio`
|
||||
|
||||
Mobile: `SfuSignalingClient` fills `CastSettings.resolvedAudioCodec` from server; **LAN fallback** if signaling down.
|
||||
|
||||
---
|
||||
|
||||
## 10. Protocol extensions
|
||||
|
||||
### 10.1 `CastCodecFlags`
|
||||
|
||||
```text
|
||||
AUDIO_CODEC2 = 16 // next free bit after SPEEX=8
|
||||
```
|
||||
|
||||
### 10.2 `MSG_AUDIO_SELECTED` (recommended at M4)
|
||||
|
||||
```text
|
||||
payload: uint32 audio_flag (little-endian) + uint8 codec2_mode_id
|
||||
```
|
||||
|
||||
Symmetric with `MSG_CODEC_SELECTED`; reusable on WebRTC data channel.
|
||||
|
||||
### 10.3 `CastSettings.SETTINGS_VERSION`
|
||||
|
||||
Bump when adding `sfu_room_id`, `relay_mode`, `server_negotiated_audio` (SFU phase).
|
||||
|
||||
---
|
||||
|
||||
## 11. Third-party and licenses
|
||||
|
||||
| Component | License | Action |
|
||||
|-----------|---------|--------|
|
||||
| codec2 (libcodec2) | LGPL-2.1 | Submodule + Gitea mirror; dynamic link in `.so` |
|
||||
| Opus / Speex | BSD | Existing |
|
||||
| ffmpeg (SFU transcode) | LGPL/GPL | BE media worker SPEC only |
|
||||
|
||||
Update mobile `LICENSES` / notices when submodule added ([bottomline_reminder.txt](../../bottomline_reminder.txt)).
|
||||
|
||||
---
|
||||
|
||||
## 12. Testing and validation
|
||||
|
||||
| Phase | Tests |
|
||||
|-------|-------|
|
||||
| Unit | `AudioNegotiatorTest` CODEC2 mask; `CastProtoHeader` round-trip |
|
||||
| Native | JNI c2enc/c2dec parity arm64-v8a |
|
||||
| E2E | Extend [OPUS_SPEEX_VALIDATION.md](../OPUS_SPEEX_VALIDATION.md) — voice-only, UDP/TCP, diagnostics kbps |
|
||||
| SFU | Post-alpha: 2× Codec2 Android + 1× WebRTC viewer via transcode |
|
||||
|
||||
---
|
||||
|
||||
## 13. Task dependency graph
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
A61["6.1 Opus/Speex JNI sinks<br/>Developer · P5"]
|
||||
C2["Codec2 NDK + voice-only<br/>Agent · post-alpha"]
|
||||
AUDIO_HS["MSG_AUDIO_SELECTED<br/>Agent"]
|
||||
SFU["F1 SFU SPEC + signaling<br/>PO · post-alpha"]
|
||||
TRANS["SFU Codec2 transcode worker<br/>BE media"]
|
||||
|
||||
A61 --> C2
|
||||
C2 --> AUDIO_HS
|
||||
AUDIO_HS --> SFU
|
||||
SFU --> TRANS
|
||||
|
||||
style A61 fill:#b45309,color:#fff
|
||||
style C2 fill:#7c3aed,color:#fff
|
||||
style SFU fill:#e5e7eb,color:#374151
|
||||
```
|
||||
|
||||
**Numbered priority (PO overrides AI):**
|
||||
|
||||
1. **6.1** — Opus/Speex JNI (alpha LAN quality path; unblocks honest ULBW comparison)
|
||||
2. **Codec2 ULBW** — this DR (parallel after M1 green)
|
||||
3. **F1 SFU SPEC** — room negotiation + transcode
|
||||
4. Alpha blockers **5.1 / 5.2 / 4.rssh** remain above on [OPEN_TASKS_GRAPH.md](../OPEN_TASKS_GRAPH.md)
|
||||
|
||||
---
|
||||
|
||||
## 14. Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Codec2 upstream churn | Pin tag; Gitea mirror |
|
||||
| 8 kHz-only confuses users expecting music | Voice-only mode UX; no video leg |
|
||||
| SFU CPU cost | Ingress transcode budget in scaling SPEC |
|
||||
| Opus/Speex “done” confusion | ROADMAP + this DR state JNI sinks explicitly |
|
||||
| BT latency 150–300 ms | Document; jitter buffer 80–120 ms |
|
||||
|
||||
---
|
||||
|
||||
## 15. Open questions (remaining)
|
||||
|
||||
| # | Question | Owner |
|
||||
|---|----------|-------|
|
||||
| 1 | SFU commercial story: always transcode vs optional passthrough mesh? | PO |
|
||||
| 2 | Ship Codec2 in AUTO on ULBW profile or explicit “radio mode” only? | PO + UX |
|
||||
| 3 | `700C` exposed to end users or dev-only? | PO |
|
||||
|
||||
**Resolved by PO (2026-06-17):** voice-only ULBW product intent; Codec2 3200 OK; skip MELP/TWELP; Opus @ 6k insufficient for target links; AAC platform use OK.
|
||||
|
||||
---
|
||||
|
||||
## 16. Changelog
|
||||
|
||||
| Rev | Date | Change |
|
||||
|-----|------|--------|
|
||||
| R0 | 2026-06-17 | Initial DR from draft; PO inline comments; Opus/Speex JNI status clarified |
|
||||
669
DRs/20260617_codec2_ulbw_voice.pdf
Normal file
669
DRs/20260617_codec2_ulbw_voice.pdf
Normal file
File diff suppressed because one or more lines are too long
1043
DRs/20260618_repos_reorganizing.md
Normal file
1043
DRs/20260618_repos_reorganizing.md
Normal file
File diff suppressed because it is too large
Load Diff
1084
DRs/20260618_repos_reorganizing.pdf
Normal file
1084
DRs/20260618_repos_reorganizing.pdf
Normal file
File diff suppressed because one or more lines are too long
40
DRs/20260620_live_cast_control_plane.md
Normal file
40
DRs/20260620_live_cast_control_plane.md
Normal file
@@ -0,0 +1,40 @@
|
||||
# Live Cast Control Plane (P0-P2 kickoff)
|
||||
|
||||
## Scope implemented
|
||||
|
||||
- New BE API `POST/GET .../api/live_cast.php`:
|
||||
- `create_intent`, `heartbeat` (60s KA), `join_event`
|
||||
- `GET action=active|analytics|session|list`
|
||||
- Public (no login): `session`, `join_event`, education `create_intent`/`heartbeat`
|
||||
- New BE repository `LiveCastRepository` with auto-schema:
|
||||
- `live_cast_sessions`, `live_cast_join_events`
|
||||
- Expiry rule: sessions without heartbeat for 60s become `expired`.
|
||||
- Short links + QR per session (join + direct) via internal short-links service.
|
||||
- Analytics payload includes `live_cast` section; shell label **Analytics**.
|
||||
- Web pages:
|
||||
- `/live/join` — viewer entry, join_event tracking, player placeholder
|
||||
- `/live/education` — 5-minute browser screen-share demo
|
||||
- `?view=live_sessions` — session tree (read-only, auto-refresh)
|
||||
- Cookie consent banner (`cookie_consent.js`) gates GA4 until user accepts.
|
||||
- Mobile:
|
||||
- `LiveCastControlPlane` — start/heartbeat/stop from sender/receiver services
|
||||
- `LiveCastPeerTracker` — polls `active` sessions; green border on discovery list
|
||||
|
||||
## Non-goals (remaining)
|
||||
|
||||
- WebRTC/SFU media plane and real HTML5 playback.
|
||||
- Push notification delivery (Web Push / FCM).
|
||||
- Gravatar / refined avatar chrome on web lists.
|
||||
- Short-link click counter migration in url-shortener submodule.
|
||||
|
||||
## Data model notes
|
||||
|
||||
`live_cast_sessions` stores owner/device/platform, state transitions, short URLs, codec/BW hints.
|
||||
`live_cast_join_events` stores open/join/leave events with platform/channel/direct flags and basic anon dimensions.
|
||||
|
||||
## Follow-up phases
|
||||
|
||||
1. Integrate SFU/web viewer when media-plane branch is ready.
|
||||
2. Browser/mobile notification fan-out on cast start.
|
||||
3. Gravatar + green-border on web peer lists.
|
||||
4. Shortener click stats joined into live session analytics.
|
||||
178
DRs/20260620_live_cast_control_plane.pdf
Normal file
178
DRs/20260620_live_cast_control_plane.pdf
Normal file
@@ -0,0 +1,178 @@
|
||||
%PDF-1.4
|
||||
%“Œ‹ž ReportLab Generated PDF document (opensource)
|
||||
1 0 obj
|
||||
<<
|
||||
/F1 2 0 R /F2 3 0 R /F3 8 0 R /F4 9 0 R
|
||||
>>
|
||||
endobj
|
||||
2 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica /Encoding /WinAnsiEncoding /Name /F1 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
3 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica-Bold /Encoding /WinAnsiEncoding /Name /F2 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
4 0 obj
|
||||
<<
|
||||
/Border [ 0 0 0 ] /Contents () /Dest [ 10 0 R /Fit ] /Rect [ 45.68504 762.0394 116.381 773.0394 ] /Subtype /Link /Type /Annot
|
||||
>>
|
||||
endobj
|
||||
5 0 obj
|
||||
<<
|
||||
/Border [ 0 0 0 ] /Contents () /Dest [ 10 0 R /Fit ] /Rect [ 45.68504 751.0394 124.813 762.0394 ] /Subtype /Link /Type /Annot
|
||||
>>
|
||||
endobj
|
||||
6 0 obj
|
||||
<<
|
||||
/Border [ 0 0 0 ] /Contents () /Dest [ 10 0 R /Fit ] /Rect [ 45.68504 740.0394 108.381 751.0394 ] /Subtype /Link /Type /Annot
|
||||
>>
|
||||
endobj
|
||||
7 0 obj
|
||||
<<
|
||||
/Border [ 0 0 0 ] /Contents () /Dest [ 10 0 R /Fit ] /Rect [ 45.68504 729.0394 108.373 740.0394 ] /Subtype /Link /Type /Annot
|
||||
>>
|
||||
endobj
|
||||
8 0 obj
|
||||
<<
|
||||
/BaseFont /Helvetica-Oblique /Encoding /WinAnsiEncoding /Name /F3 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
9 0 obj
|
||||
<<
|
||||
/BaseFont /Courier /Encoding /WinAnsiEncoding /Name /F4 /Subtype /Type1 /Type /Font
|
||||
>>
|
||||
endobj
|
||||
10 0 obj
|
||||
<<
|
||||
/Contents 21 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
11 0 obj
|
||||
<<
|
||||
/Annots [ 4 0 R 5 0 R 6 0 R 7 0 R ] /Contents 22 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0
|
||||
/Trans <<
|
||||
|
||||
>> /Type /Page
|
||||
>>
|
||||
endobj
|
||||
12 0 obj
|
||||
<<
|
||||
/Contents 23 0 R /MediaBox [ 0 0 595.2756 841.8898 ] /Parent 20 0 R /Resources <<
|
||||
/Font 1 0 R /ProcSet [ /PDF /Text /ImageB /ImageC /ImageI ]
|
||||
>> /Rotate 0 /Trans <<
|
||||
|
||||
>>
|
||||
/Type /Page
|
||||
>>
|
||||
endobj
|
||||
13 0 obj
|
||||
<<
|
||||
/Outlines 15 0 R /PageMode /UseNone /Pages 20 0 R /Type /Catalog
|
||||
>>
|
||||
endobj
|
||||
14 0 obj
|
||||
<<
|
||||
/Author (Android Cast project) /CreationDate (D:20260623115606+02'00') /Creator (\(unspecified\)) /Keywords () /ModDate (D:20260623115606+02'00') /Producer (ReportLab PDF Library - \(opensource\))
|
||||
/Subject (\(unspecified\)) /Title (Live Cast Control Plane \(P0-P2 kickoff\)) /Trapped /False
|
||||
>>
|
||||
endobj
|
||||
15 0 obj
|
||||
<<
|
||||
/Count 4 /First 16 0 R /Last 19 0 R /Type /Outlines
|
||||
>>
|
||||
endobj
|
||||
16 0 obj
|
||||
<<
|
||||
/Dest [ 10 0 R /Fit ] /Next 17 0 R /Parent 15 0 R /Title (Scope implemented)
|
||||
>>
|
||||
endobj
|
||||
17 0 obj
|
||||
<<
|
||||
/Dest [ 10 0 R /Fit ] /Next 18 0 R /Parent 15 0 R /Prev 16 0 R /Title (Non-goals \(remaining\))
|
||||
>>
|
||||
endobj
|
||||
18 0 obj
|
||||
<<
|
||||
/Dest [ 10 0 R /Fit ] /Next 19 0 R /Parent 15 0 R /Prev 17 0 R /Title (Data model notes)
|
||||
>>
|
||||
endobj
|
||||
19 0 obj
|
||||
<<
|
||||
/Dest [ 10 0 R /Fit ] /Parent 15 0 R /Prev 18 0 R /Title (Follow-up phases)
|
||||
>>
|
||||
endobj
|
||||
20 0 obj
|
||||
<<
|
||||
/Count 3 /Kids [ 10 0 R 11 0 R 12 0 R ] /Type /Pages
|
||||
>>
|
||||
endobj
|
||||
21 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 350
|
||||
>>
|
||||
stream
|
||||
Gas309kseb&-h(iregH[(*#.+hJ6skH*ASb@?K(l<nD1gqYN?Oqk95ea?YoVH'0kG$pMp1#M^\c-4JHr/_H4TRc6%:mfd&n>":b;AJ%6OAL]3UZ:uf`isa".?A/Xl:03adIjq0?J%#_TS[?aOQWn-o)_`4O;6gqb''466Z,1a*3+C\3Nr+9`fljW)@:%7/E2KmC2k.NukCWf`/(,Gmf*0+i,i^PaB6$4KVmu;VgN[M"g1N9U^:]BH$WE:RIGa#Bq(U5hHf_Z8[mI8;U"@PW?GC&4FiuH(5<'HMa)JoW!u*;-n@4Hj?Im&rWQ8F1kni'A5TJl-Zm3U2:DYDs,FBteMGNHDF+j~>endstream
|
||||
endobj
|
||||
22 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 491
|
||||
>>
|
||||
stream
|
||||
Gatn"?#SI?'Sc)J/)JHk;CMj0Z$/3N\XKBio4=_R3]-Aa_)%n_qnl0.D/Co"W]Cb`S-.)a4s9?21p:a,HtdQ/,*5(]#V&r-jr;#mMa%S17i$uIQ/VtSer7e`aS3!0nOc=u_.6hl?%H]9l@L_X(AU[N_%9,cB8/B%KU<%g`]Q5=n;'!tA:9=oBRZ`P1pMOB]>QAu48AB"-WG/A1t-r^@Betl_:_#imIgm5;1B;YHgnMV?sLWr3SN>3^@Su0bnV.-ohcXq9etqMeiNSflo!l@mqMejpaS6>NjjcE_qr>ti5DGR1r1gM&80NR+KJG2lB:X_J"r%'P<K:9lf6I11a(a2[e;K[(F:^<hmi+-DH[_&AfB;V\BWrj\#eIr/VHU#D<c_.lhj]"0oSol4FZ(hnD)]l*@'"h,fISSKJ@%FZD0>g`jbh!7eW\^9G^+.`9[Ad_Fm=5<h=g-5D^hg)-+3t68:o.bt0MkmNDPi@.>&E$M(k~>endstream
|
||||
endobj
|
||||
23 0 obj
|
||||
<<
|
||||
/Filter [ /ASCII85Decode /FlateDecode ] /Length 1744
|
||||
>>
|
||||
stream
|
||||
Gatm<D/\-!&H:Nn0d"Lpf\"gn;H9V?Qh7:NHYGgs01aLE*q6W3Y3WLSOS\!Uei&UL&DVet0]66_1G@'1ZWPSLQW:a)$,FLsb\IS;)obg\*5d?t%:"nMr-W#).im4Y%,IKWT'%!)rlCd]FIP0llXs?9N"`F4^"L',bO<pND@a#jFRB7!,4Hk;0gUq3GQQpo0`FFDqEhB&ecrSADR2f/l&N2bp%7pSFC=mni5G1Kpl#Tj\:X>,O&4OdYN`<<?0Cq=qm#tUC(,i*Qn7?2biea40U3SkPh(cYLN^sHJ9C8HTkP%GJM>0,jnQV?=<_Y`3c-L0OCbsu)]B)mdM_Zkr,Y%^H+)nJjPg4'X/E`!Smre0MhZ?pHPY[IMYHs&a'VNEGa\.grDKR.h=q_+eHT783]-K<Kcfo:JgD\\ElQ)ZjK&Zt#.6)tNMo$IM.$uMmIBY@g3bX(0*fMb*Jm$!VGcqhB$2On)p6q^Z0#r;U03$TSk$SR>]s*Y@D^`[M-q?j9P\f9e7hnZo!mAXaf*o(I4uQ[)Y:$0;GMegMAXZI"h+o8/94kU+/Udq9l;$TR:$[mlDN,e]5c;5iPBL*./5</gKdrHGi_lc_0mT8*+COL.!Q,,mNBJbjU%&C#F'ZJ$"&ci:12?bkh=?R!'oe^h.j.1CsE-!"54\8)YWcfr'8DjZ","d$K<6t[Bh]YV67$:[!1R6`\GEUa*>#FXr"4b]HT.aiLu\8L-E.,6e-dB"4oW?k*JfneEMJ6VJP"^3ktn6$5"joT#-5*OCp->_8YaTd[2V#$!dUkA/o(0_@L;"e8h3(CH:;;N8%(:-uoGTP7+5QT2Zk#=%+Z-#VhGnOS8uMcG6&5`p_(PhB@OeqA!.S_G#obrY0OPLaapPP%Qc!:dpS3(-^B'6_f,[jU[@_&6N3I4qiP-3U&B4BY!K'?=nS'H:CL_UGf12_mol'aE\2J:(.L39QSa=J<p'\+0QmH-g>^#obOu^mZFPneZ(8K&PU3R0b_&[q)Pi"F5:M'Q-0l=R7sP1"B!Gih*u]m[M%d>IF0&q7kWng41C%!=:7O!"\io5)9:&K]G`f&lZYdSG;.KN"^bm&/eAO[Xo_090;72qlL$^FlouqIQD-I)N%f&(-&`40^,AZ+SR]$^]CD_CJ,L)p%&khV\k%@.O@DKMc@:m>h709eq\8N8N,eaOTo?N6<!5PMI*1d>3CGH,P`&+'pTnmk$.kN9-7<XQg"rE2qN.sB,Ee-g;F2uiK_gHHb4I5S+;]])f1HufENR'N;7^+q9b>ZKL>HL87k.()/??WW6W]Dhh/^:tXe;]`',f5cf`Im(gqhnH"oT6+#07A#6BoJJ@c;Pnk5X:ZXf*,N0t2AV-[h3.]g+1u#\Jlr_+^lnS-J9><3%*`D&PD;SS.TgdqU4`YBMqt3?UfnoAU]W1pXP]Y2eq0&cA4Vbu;\!Io7!BE+R&1jp9MslMS&1cO;#R4_(f:^0qqdR4A%"1IW[!-;(O9:m@)Xj.<qo9&*bjf;-W2d4J6e,GnWZL=P2!h;3%@=9a-u8n^7.OMmQUU@la@Le-r,am2ujk>VA4#e8="(G=GFU1`p<nI*uc""pT6d7h[hiMkg2RHWqR++;YX:j(t;R//')p$q;d)khU_Fi<^F,O43`e9Y>W>^pG#5!G!(.9riVQ,%+*Wq)C)qP]]Qct$9TY3US$SjP:ra0N>Y1p7X_[IKA6W;d3QKMaZbCZl5DVGY^GL*b\OT/o:<M&gj45>cB;I/~>endstream
|
||||
endobj
|
||||
xref
|
||||
0 24
|
||||
0000000000 65535 f
|
||||
0000000061 00000 n
|
||||
0000000122 00000 n
|
||||
0000000229 00000 n
|
||||
0000000341 00000 n
|
||||
0000000488 00000 n
|
||||
0000000635 00000 n
|
||||
0000000782 00000 n
|
||||
0000000929 00000 n
|
||||
0000001044 00000 n
|
||||
0000001149 00000 n
|
||||
0000001355 00000 n
|
||||
0000001597 00000 n
|
||||
0000001803 00000 n
|
||||
0000001890 00000 n
|
||||
0000002206 00000 n
|
||||
0000002280 00000 n
|
||||
0000002379 00000 n
|
||||
0000002497 00000 n
|
||||
0000002608 00000 n
|
||||
0000002706 00000 n
|
||||
0000002781 00000 n
|
||||
0000003222 00000 n
|
||||
0000003804 00000 n
|
||||
trailer
|
||||
<<
|
||||
/ID
|
||||
[<c4fec4b99072874c8420e5c9ea80366d><c4fec4b99072874c8420e5c9ea80366d>]
|
||||
% ReportLab generated PDF document -- digest (opensource)
|
||||
|
||||
/Info 14 0 R
|
||||
/Root 13 0 R
|
||||
/Size 24
|
||||
>>
|
||||
startxref
|
||||
5640
|
||||
%%EOF
|
||||
Reference in New Issue
Block a user