ac/ac-docs
1
0
Fork 0
mirror of git://f0xx.org/ac/ac-docs synced 2026-07-29 05:39:40 +03:00
Code Issues Packages Projects Releases Wiki Activity

initial

Browse Source
This commit is contained in:
Anton Afanasyeu
2026-06-23 12:20:43 +02:00
commit 69a448f156
151 changed files with 33372 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
drafts/20100611_1_net_obfuscation.txt Normal file
View File

@@ -0,0 +1,6 @@
Network traffic obfuscation
Definitions:
- E2E - E2E encryption
- P2P - P2P encryption
- POLY - Polymorphic ciphers, persistent and temporary keys

12
drafts/20100611_1_net_topology_glossary.txt Normal file
View File

@@ -0,0 +1,12 @@
network topology glossary (to be used as a code names within the product):
1. SIRENE - P2P-alike topology
2. SPIDER - MCU-alike topology
3. SPIDERWEB - SFU-alike topology
4. MINE - mesh-alike topology
5. SLY - E2E/VPN/hidden/any
6. BLUEMOON - Bluetooth-transported any available (mesh, P2P)
7. SLOTH - passive
and the combinations of them: SIRENE SLOTH - slow-to-respond passive P2P networking; SPIDER SLY - hidden non-transparent network activity using MCU-alike topology; MINE BLUEMOON, etc.

17
drafts/20100611_2_av_conversion_and_non_rt.txt Normal file
View File

@@ -0,0 +1,17 @@
Audio/Video RT and non-RT
linked drafts:
- Network obfuscation
- Transport layers
- Network topology
- Paid services
- End-user services
The service (standalone or the current one, depends on max. capacity, bandwidth, storage trade-offs, CPU load) should provide:
1. RT AV
- RT streaming and casting using well-known codecs and transports, including store recordings services for offline / non-RT usage
2. non-RT AV
- non-RT on-demand conversion routines with additional services, like:
** reproducible (playable) streams
** ability to download/delete streams

112
drafts/20100611_3_url_shortener.txt Normal file
View File

@@ -0,0 +1,112 @@
URL shortener service (to review / full SPEC / DR)
URL shortener service aims to provide short encoded URLs on demand based on given output. it aims to be a helper to some other service and should act as a part of post-alpha (next to alpha) stages, but the implementation priority may be reordered depending on:
- the BE capacity
- the BE load
- the product urgency
Basic requirements:
- URL shortener shouldn't be a part of the BE application - the development should be separated out of it and act like a standalone opt-in service
- should include REST API (Open API) - as a must
- stack: PHP, NGINX, MariaDB, any free / open-source licensed 3rd parties if needed
- the placement inside the project structure: backend/url-shortener submodule
- the development placement: git://f0xx.org/androidcast_project/url-shortener (will be available soon)
- QR codes support: must
- authentication: app-wide (shared session), project-wise token
- permalink urls: must
- time-restricted urls: must
- token signing, trust models, sessions, security, other options: TBD or described later on
- ... TBD - open part
The high-level flow:
- some app (not requirely androidcast - for example, curl) requests url shortener to shorten the long url
- some app provides the authentication on the way we delegate to it (temporary time-restricted or persistent)
- url shortener generates JSON with response of the operation
The request:
1) GET: https://s.f0xx.org?url=[URL]&bearer=[TOKEN]&signer=[SIGNER]&ttl=TTL
URL - url-encoded string to shorten
TOKEN - app-wise, demo or personal token to attach the shortened url to into the database
SIGNER - trusted signer token
TTL - time-to-live for shortened URL
2) POST: JSON-based body with the same fields as for 1) GET
The response:
1) fail: {"code": "ERRORCODE", "url": "URL", "bearer": "TOKEN", "desc": "ERROR_DESCRIPTION"}
ERRORCODE - internal error code
URL - the URL to shorten
TOKEN - auth token
DESC - error description (for example "access denied")
2) success: {"code": "0", "url": "ORIGINAL_URL", "u": "SHORTENED_URL", "bearer": "TOKEN", "ttl": "TTL"}
ORIGINAL_URL - the original URL
SHORTENED_URL - short URL
TOKEN - auth token
TTL - time to live for this SHORTENED_URL or 0 if forever
The low-level flow:
1) web app receives the Request to shorten url
2) web app checks if the TOKEN is already active (may be demo token or pre-shared special)
3) if TOKEN is absent - return the error response
4) if the TOKEN persists, do a lookup for the URL (already shortened) through the database that matches this TOKEN, return cached one if exists and TTL is good
5) if shortened URL found but TTL exceeds given limits - return error response
6) if not exists, check if the TOKEN is in whitelist (trusted)
7) if not trusted - return error response
8) if trusted token and the shortened URL exists and TTL is ok - return Ok response with corrected TTL in the response body (remaining)
9) if trusted token and URL doesn't exist in the database,
9.1) check if the TOKEN is really trusted to
9.1.1) create infinite TTL shortened URLs
9.1.2) create temporary TTL shortened URLs
9.2) if ok - shorten the URL using our ALGO (below), record everything in the database (token, ttl, original url, shortened url, creation time, modification time, etc)
10. return error response otherwise
ALGO for shorten urls:
considering the shortened url must be in format like https://[s.f0xx.org]/[3dfac0] where
[s.f0xx.org] - the domain name of url shortener
[3dfac0] - the shortened URL part
it must be resolved finally in, for example, https://defender.net/very/long/link#with_anchors&param1=value1&paramN=valueN
the database should keep the original url (https://defender.net/...) as well as the shortened (https://s.f0xx.org/3dfac0), so suggested database/table structure is (simplified):
* id - auto-incremental int
* token - token record that created this url, varchar(128)
* signer - varchar(128) - trusted signer key
* creation timestamp - datetime
* modification timestamp - datetime
* access timestamp - last access timestamp, datetime
* origin - original url to shorten, text
* shortened - text, self-explanatory
* hash - text, self-explanatory (full hash, not a part of)
*... TBD
the hash for shorten suggested to be calculated as Java String hash (4-byte value in hex) taked the intermixed string as an input from URL and TOKEN, for example
- URL is 'https://defender.net/very/long/link#with_anchors&param1=value1&paramN=valueN', TOKEN=4abef433d23ab5x&&83^c6
- intermix, take 1 symbol from URL and 1 symbol from TOKEN, paste them:
h t t p s : / / d e f e n d e r . n e t / v e r y / l o n g /
4 a b e f 4 3 3 d 2 3 a b 5 x & & 8 3 ^ c 6 4 a b e f 4 3 3 d
h4tatbpesf:4/3/3dde2f3eanbd5exr&.&n8e3t^/cv6e4rayb/elfo4n3g3/d - the string to calculate hash from
^ here we have the interleaving of URL and TOKEN so starting the TOKEN from the beginning
the similar logic with the URLs shorter than TOKENs:
h t t p s : / / t . c o
4 a b e f 4 3 3 d 2 3 a - taking just a part of TOKEN
htta5bpesf:4/3/3td.2c3oa - the string to calculate hash from
then we take 'the string to calculate hash from' and compute 4-8 bytes hex hash using Java string hashing algo. alternatively we can compute sha256 over it an cut only some part of sha256 result, like:
htta5bpesf:4/3/3td.2c3oa | sha256 -> 0c7d8fd69bc54623b436fd9543489230c27f177a56383ea6a9aace9f87f04004
take 12-20 bytes from '0c7d8fd69bc54623b436fd9543489230c27f177a56383ea6a9aace9f87f04004' -> 4623b436f
so the shortened url would be https://s.f0xx.org/4623b436f
The reverse lookup:
is straightforward
- BE takes a part of the request, 4623b436f for this example, and performs reverse lookup through the database, depending on hashing algo used. once found, returns the response with the full URL, error otherwise
TBD and open questions list:
TODO/DONE list:
- git OK: git://f0xx.org/androidcast_project/url-shortener
- SSL OK: (FE) /etc/letsencrypt/live/s.f0xx.org/fullchain.pem, /etc/letsencrypt/live/s.f0xx.org/privkey.pem
- docs OK: docs/specs/20100611_3_url_shortener.md, docs/DRs/20100611_3_url_shortener.md
- scaffold OK: backend/url-shortener/ (openapi.yaml, nginx snippets, sql, demo_curl.sh)
- impl TODO: PHP handlers, hub Short links card, BE sync
Estimations:
Severity: Medium

62
drafts/20100612_1_scaling.txt Normal file
View File

@@ -0,0 +1,62 @@
# Draft — source for SPEC/DR (rev. 2 — 2026-06-12)
# SPEC: docs/specs/20100612_1_scaling.md
# DR: docs/DRs/20100612_1_scaling.md
#
The problem:
Now we have a small VM instance which serves not so much of things. A short list of services include:
- own web service (nginx+php) for UI/UX and some control panels, serving issues, tickets, graphs, build control (orchestration)
- git UI (gitea)
- database service - data store
- minimalistic OTA service for mobile app
- minimalistic WireGuard provider
- url shortener (nginx+php)
- build service (docker, orchestration)
The problem is, sooner or later we'll need to expand the amount of VM instances, delegate some features to separate virtual machines, most probably migrate to the clouds (GCP, Azure...), proper scaling with zero downtime with the ability of seamlessdeploy/update, add new services - and all this bundle should work ultra-stable, durable environments, splitting everything to the clusters of dev, staging, prod in a way that, for example, one deploys the unstable features to dev cluster, then the initial testing occurs, after decision making - move to the staging for beta tests, and finally to production once the decision is that staging behaves stable.
The how:
The proposed way is:
- one, let's say, cluster (let's call it dev cluster for example), which has N virtual machines, each of:
** web frontent VM(s) + OTA (linked together with common datastore VMs)
** database VM(s)
** data store VM(s)
** WireGuard VM(s)
** URL shortener VM + gitea for developers (1 small instance should be enough I believe)
** build service VM(s)
** N VMs for new services and features (starting from 0 and depend on the features)
- more tighter (money-wise) approach
- other
TODO: consult with AI for the others and the best practices of building scalable systems like proposed one depend on the current services and future needs
The simulation:
In theory, sysop (system operator aka project owner as for now) may simulate any (meaningful) number of HVMs for cluster simulation based on something lightweight like alpine linux distro plus sets of packages pre-installed.
The solution:
Ask AI to collect the information, make its decision based on all that we currently have plus look deeper into the future. The solution might be balanced between cost and providing services, for example, we don't need CPU or RAM resources for data storage but CPU/RAM is important for build service, etc.
Also, the matter of seamless scaling is super important, better to have the load balancing ready to use just from cloud provider.
Also, let AI think of how many simulatltaneous connections can we have per instance - the question is for RSSH implementation and future improvements
Network bandwidth (missing in rev.1 — add to SPEC/DR):
- we scale not only by CPU/RAM/connections but by **network throughput**: ingress/egress on FE WAN, FE→BE backhaul, per-VM NIC, CDN/object-store egress, SFU/ffmpeg media pipes, OTA/APK download storms, WireGuard UDP
- need **budget metrics** per service (Mbps sustained, peak, GB/month egress) and alerts before link saturation
- single FE uplink + single BE path is a hidden bottleneck today; cloud/hybrid phase must model **egress cost** (OTA, recordings, transcodes)
Global availability and GEO (missing in rev.1 — add to SPEC/DR):
- long-term: **the world can reach our services** (consoles, APIs, OTA, cast/signaling, short links, future commercial VPN) with acceptable latency — multi-region or CDN edge as we grow
- simultaneously: **segment isolation by policy** — not all content or features in all countries; example: certain recordings, OTA builds, or published cast archives must **not** be offered in Zambia (ZM) or other denylisted regions unless PO overrides
- enforcement should be layered: DNS/Geo routing, CDN/WAF geo rules, app/BE **geo_policy** on content objects, tenant RBAC — default allow globally, explicit **deny segments** per resource or catalog
- dev/staging may stay single-region; prod gains GEO rules when F3 cast storage / F5 payments / F9 streaming go live
The future key role requirements:
- multipoint casting and routing outside the intra network, usage of 3rdparty or own multipoint solutions on the backends(MCU, SFU...)
- new RBAC control, new roles - now we're thinking as developers and deliver the max of services to developers, but we will migrate to the other model sooner or later
- both realtime and non-realtime data streaming solutions, including streams re-compression (from any input to any output that ffmpeg can do for us, so ffmpeg will be involved)
- casts storage - we'll keep the records on the backends and provide them on demand if the user pays for it
- VPN capacities (question for all the resources), anonymous realtime VPN provider services (payed)
- payment systems integration (should cost near 0 for the developer's point of view of scaling/growth, it's more about marketing)
- potential payed add-on services (not decided yet on which side, on the backends or on mobiles) - user face masks, background extraction and replacement, etc.
- in-app money transfers between the app users (payed service)
- DNS control if possible - meaning selling the subdomains to the users, provide simplified own site builders accessible by own domains...
- 3rd party streaming providers integrations: realtime and non-realtime, let them stream and cast using our platform (I see ffmpeg active usage there plus cast handlers: low delay SFU-alike, mid-high delay using m3u playlists or similar hls-alike technologies)
AI should take everything described there plus the current project structure into account, weight everything, prioritize, use beautiful graphs and visual diagrams, and create SPEC/DR/ documents out of this draft using our common templates.

157
drafts/20260616_VOD_and_restreaming_service.txt Normal file
View File

@@ -0,0 +1,157 @@
VOD and restreaming service
R0-pre draft
severity: medium
milestone: V.2x demo
[AI comment]: Strong brainstorming capture — codec matrix, nginx/ffmpeg snippets, and UI intent are all useful pre-DR material. Before DR/spec split, add one paragraph up front: **product boundary** (standalone “media hub” vs extension of android cast ingest), **alpha vs post-alpha** (scaling spec already slots F2 transcode + F9 3rd-party streaming as post-alpha), and **success criteria** for the first demo (e.g. “one m3u8 in → HLS out + BE config row + web play”).
Problem statement:
- near realtime restreaming service
[AI comment]: Define “near realtime” numerically for this doc — e.g. glass-to-glass target (3 s? 10 s? 30 s?) and whether that applies to **live ingest** only. ffmpeg transcode + HLS segment duration (often 2–6 s) + player buffer usually means **not** sub-second; that is fine if labeled **low-latency VOD/restream**, not WebRTC-class cast.
[foxx answers]: "near realtime" means from 3s delay to infinite (in case of VOD); refine better: let's say 2-10seconds delay from start processing to final destination delivery
- sources: raw audio/video (h264, m4a, mp4, mkv), OBS, VLC, ffmpeg, live and VOD
[AI comment]: Good source list. Add explicit **ingest protocols**: file/URL pull, RTMP push (OBS), optional SRT/RTSP later. Cross-link Janus `:8089` ([INFRA.md](../INFRA.md)) — is live phone cast ever an ingest here, or only file/RTMP/m3u?
[foxx answers]: explicit ingest protocols would be defined as:
- file/URL pull
- OBS RTMP
others to come later (they sill be supported under the hood but not exposed). live phone cast is out of this document, we'll use other technologies for that i suppose.
Requirements:
- limited CPU and RAM on transcode VM
[AI comment]: Tie to [specs/20100612_1_scaling.md](../specs/20100612_1_scaling.md) §12.2 / P7 — transcode must not share the primary DB VM. Document **concurrency model**: N parallel jobs × preset (e.g. 1080p medium) → vCPU/RAM budget; what happens when queue is full (reject, shed resolution, delay).
[foxx answers]: transcode may share DB VM, see no reason not to do it, but the way we expose the DB is debatable; meaning, using DB just via OpenAPI looks way too slow, but exposing the DB as we use it today looks risky and improper. although, MariaDB can be exposed by socket listening which looks like a compromise for demo, I do believe that the real cloud providers have much better and flexible solution to expose a single DB interface for all our service and we don't really care what's behind the hood there and how many terabytes distributed somewhere across the world we're using - we pay money for DB service, our job is just to utilize/integrate its interface. about transcoding - I don't really understand what's the reference between transcoding and db usage. We just need to record some metadata, not so big amount of; like who's the owner of that video, processing pipeline parameters and so on, shareable between different VMs the same way as we'll use other kind of information. if you think otherwise, please share your vision.
- CDN, storage and bandwidth for production
[AI comment]: Needs a **non-goals** line for R0: “no multi-region CDN in v1” vs “must use existing FE/BFF + object store”. Name candidate backends (S3-compatible, local NFS on `be-media`, or FE static) and **egress** ownership (FE vs media worker).[foxx answers]: * no multi-region CDN in v1, local NFS on `BE-media` for feature alpha (demo) release; not really intended to be full-featured, we just need lots of disk space to store videos (~200Gb f0r demo would be enough)
- m3u and youtube as input sources for demo
[AI comment]: **Legal/ToS**: yt-dlp + YouTube as demo input is fine for internal lab; flag as **non-prod** or rights-cleared content only for anything public/commercial. DR should separate “URL ingest” from “YouTube-specific adapter”.
[foxx answers]: good point. any ways to legally grab youtube videos? tagging it as **non-prod** for now, let's see what can we do. rephrasing in general words: not exactly youtube, but any video source that provides pull-alike link (together with access parameters and, probably, in some rare cases - cookies) is acceptable, at least, for lab testing or demo, unless we'll close the gap with legal notes. the mindset there is - better to have the feature and keep it disabled rather not to have it but suddenly being requested to implement it just yesterday; thus we need to hurry up and whole infra is almost ready, we don't have to do any refactoring, etc.
- m3u as output for demo
[AI comment]: Clarify m3u vs m3u8 — playlist **master** vs **media** playlist, and whether output is **HLS** (`.m3u8` + `.ts`/fMP4) served by nginx or plain redirect URLs.
[foxx answers]: see the examples - assuming .m3u8+ts as the most easier way to deliver the content, but AFAIK there should be a master playlist anyway, no? this looks like the implementation details, not a design, correct me if i'm wrong
- nginx and ffmpeg as core
[AI comment]: Reasonable for Alpine lab. Note **nginx-rtmp-module** maintenance/long-term: acceptable for demo; DR should compare **nginx-rtmp + ffmpeg** vs **ffmpeg-only** (publish HLS to disk/object store without RTMP hop) vs dedicated packager (e.g. MediaMTX, SRS) for prod.
[foxx answers]: this is what i'm expecting from you - to close the gaps like that. i don't have deep enough knowledge in that area.
- watermarks
[AI comment]: Distinguish **static burn-in** (ffmpeg overlay, cheap) vs **per-session forensic** (section below). R0 should scope static/logo + optional text; forensic = later phase.
[foxx answers]: totally agree. I will even simplify the things:
Milestone | Protection feature
-----------------------------------------------------------------------------------------------------------
V.2x demo (unstable) | No watermarks but keeping this feature in mind to be implemented in near future
V.2x alpha | Assuming we already published V.1x releases - basic watermarking as a PoC (static burn-in)
V.2x beta | Introducing per-session forensic watermarking
V.2x | All alpha watermarking features plus beta (per-session) as trial feature
-----------------------------------------------------------------------------------------------------------
- stream protection
[AI comment]: Large surface — recommend **phased DR**: (1) signed URLs + short TTL, (2) session-bound tokens via existing BE auth, (3) per-user watermark, (4) ML pirate detection. Do not block core pipeline on (3)–(4).
[foxx comments]: agree, each method deserves its own design. now, since it doesn't require any implementation before V2.x alpha stage. i believe there we'll have feature-rich documentation for stream protection methods we'd like to see.
- parallel disk recording
[AI comment]: Add retention, quota, encryption-at-rest, and whether recordings are **mezzanine** (pre-CDN) or **compliance archive**. Who triggers delete?
[foxx comments]: let's define some details, although, they might change due to experiments and probably the values will be incorrectly established as for now, but:
Key | Value (alpha, free of charge) | Value (alpha, pay as you go) | Value (alpha, subscription paid service emulation)
-----------------------------------------------------------------------------------------------------------------------------------------------------
retention time: | 5 days | per-month payments, so 1 mo | per-subscription payments, may be up to infinite
quota: | 3Gb | 50Gb/mo, infinite(golden payments) | infinite till subsription is active
encryption-at-rest | n/a | ? | ?
pre-CDN or... | not sure I get the question correctly, I think there shouldn't be something before CDN
remove trigger | cron+script each 5 days | cron+script, based on pay type | cron+script, based on subscription/pay type
-----------------------------------------------------------------------------------------------------------------------------------------------------
- BE UI integration, standalone transcode service later
[AI comment]: Good split. Specify **control plane** (PHP/DB: pipeline defs, schedules, ACL) vs **data plane** (ffmpeg workers, nginx). Matches scaling “media worker pool” pattern.
control plane: UI/UX similar page to what we have today, but it will be a bit updated after RBAC review. features:
Type | Value (dev/demo) | Value (staging/prod, v.2x)
dev lang PHP PHP
web FE nginx nginx
DB baseline basic metadata and accounting storage (RBAC-linked) full metadata, accounting info (RBAC-linked), shorthand recording key features, etc.
DB type mariadb cloud providers distributed DB
DB type mariadb cloud providers distributed DB
Data plane
UI/UX:
- pipeline config in DB
[AI comment]: Align schema sketch with crash-reporter DB patterns (SQLite/MariaDB?) and whether pipeline state lives in same cluster or dedicated `be-media` DB.
- probe input, select output
[AI comment]: ffprobe JSON → normalized “source profile” table (resolution, codecs, fps, audio channels). Document validation errors for unsupported combos.
- Nuevo Video.js player
[AI comment]: Confirm license (commercial use) and whether Nuevo stays or **Video.js + standard plugins** for alpha. Mobile: native ExoPlayer path vs WebView player — one line each.
- mobile and web playback
[AI comment]: HLS on Android/iOS may need **LL-HLS** or accept standard HLS latency; note **Safari** native HLS vs Chrome MSE.
- popup announcements
[AI comment]: Seems orthogonal to VOD/restream — move to separate “catalog/marketing” DR or mark optional so core media DR stays focused.
Research:
- ffmpeg and nginx on Alpine, 8vcpu, 16gb ram
[AI comment]: Run a **capacity table** (even estimates): e.g. 1× 1080p30 transcode ~X vCPU; max concurrent jobs on 8 vCPU; RAM per ffmpeg (~500MB–2GB). Without this, “limited CPU” requirement is not testable.
- alternatives: cvlc, mediapipe, janus
[AI comment]: **Janus** = WebRTC SFU (F1), not a VOD packager — good for live cast, different from HLS restream. **MediaPipe** = analysis/ML, not a replacement for ffmpeg transcode. **cvlc** = playback/relay edge cases. Keep ffmpeg+nginx as primary; Janus cross-ref for “phone as live source” only.
- ffmpeg build: libx264, libfdk-aac, overlay, ...
[AI comment]: **libfdk-aac is nonfree** — distribution and store policy risk for a commercial app/service. DR must pick: (a) build `--enable-nonfree` for internal lab only, (b) **aac** (native/ffmpeg) or **libopus** for prod, (c) ship codecs only server-side. Document in [COMMERCIAL.md](../COMMERCIAL.md) / license appendix.
- yt-dlp for youtube
[AI comment]: Pin version, update cadence, and failure modes (bot detection, format churn). Wrapper service with timeouts, not shell from PHP directly.
- FFdynamic
[AI comment]: Worth a spike doc: dynamic ffmpeg graphs vs static CLI. If rejected, say why (operational debuggability).
Limitations:
- no dash in Alpine ffmpeg, side load needed
[AI comment]: For R0 **non-goal**: DASH unless side-loaded. **HLS-only** output is enough for demo + most mobile/web if documented. If LL-HLS needed later, separate spec.
How:
- nginx config (rtmp, hls, exec ffmpeg on publish, ...)
[AI comment]: Examples are valuable. Production DR should replace hardcoded IPs (`192.168.122.40`) with templated deploy vars. **exec ffmpeg on publish** couples nginx lifecycle to transcode — prefer **supervisor/systemd** or job queue driving ffmpeg, nginx only serves HLS/RTMP.
- ffmpeg tee + unix pipe multistream
[AI comment]: Clever for 1→N outputs; document **failure modes** (one broken pipe stalls tee?), **ordering** (start nginx-rtmp before publisher), and ops runbook. Simpler v1: single output HLS to disk, add tee in v2.
- 1080p encoding presets
[AI comment]: Good defaults. Add **audio-only** and **copy video + transcode audio** paths to save CPU when source is already H.264.
- Nuevo player HTML fragment
[AI comment]: Fine for spike. Spec should reference **where** this mounts in BE (new vhost vs `apps.f0xx.org` path) — [INFRA.md](../INFRA.md): public apps on FE→BE `:80`, not `:8089`.
Stream protection:
- token/key exchange
[AI comment]: Reuse existing **session/JWT** patterns from 2FA/auth flow where possible; avoid a second ad-hoc crypto system. Short-lived **HLS segment URL signing** (CDN or nginx `secure_link`) is the usual pattern.
- per-user watermarking
[AI comment]: Viable via ffmpeg drawtext/overlay per job; cost = **one transcode per viewer** unless watermark is client-side (weak). DR: forensic vs “good enough” deterrent.
- pirate detection (forensic watermarks, ML vision)
[AI comment]: Mark **R2+ / research** — high cost, false positives, legal workflow. “AI questionable” is fair; do not merge into R0 acceptance criteria.
Open questions:
(awaiting Rev.R0)
[AI comment]: Suggested questions to resolve before DR:
Q1. Same product as android cast or separate “f0xx media” service?
Q2. First milestone: live RTMP in, HLS out, or file/VOD only?
Q3. AuthZ: public unlisted links, logged-in only, or paywall (F5)?
Q4. Codec policy for prod without libfdk-aac?
Q5. Object store vs local disk for HLS segments in prod?
Q6. Observability: per-pipeline metrics (ffmpeg exit code, segment age, bitrate) — Prometheus/Grafana vs logs only?
Q7. Relationship to scaling **F9** (3rd-party ingest) — is this doc the F9 implementation parent?
[AI comment]: **Suggested DR/spec split** when promoting:
- DR: architecture choice (control vs data plane, HLS-only, worker sizing, security phases, nonfree codec decision).
- SPEC-1: pipeline model + DB + BE API + ffprobe.
- SPEC-2: worker runtime (ffmpeg graphs, nginx-rtmp or alternative, deploy on Alpine).
- SPEC-3 (optional/post-alpha): stream protection + forensic watermarking.
[AI comment]: **Global verdict** — Yes, the draft is **useful**: it captures real config experiments, constraints (Alpine, CPU), and UI intent. It is **not yet** “well enough described” for implementation without the clarifications above (latency definition, prod codec policy, legal ingest, capacity math, service boundaries, phased security). Recommend **Rev.R0** = answer open questions + trim orthogonal bits (popup announcements) + one demo milestone; then convert to DR + 1–2 specs aligned with `20100612_1_scaling` F2/F9 slots.

31
drafts/20260616_rssh_routed_egress.txt Normal file
View File

@@ -0,0 +1,31 @@
RSSH routed egress (dev-only VPN over SSH) — source draft
R0-pre
severity: low
state: postponed (finish WireGuard track first)
Problem:
- PO asked whether RSSH can get a system-wide VPN provider that routes part or all device traffic through the RSSH tunnel (split or full), dev settings only.
- Today RSSH is reverse SSH (-R) for operator → device shell/SFTP; no VpnService, no WAN egress change.
- WireGuard already supports hub-only / full-tunnel + all-apps / this-app-only via RemoteAccessVpnRouting.
Why consider RSSH routing at all:
- TCP 443 only path (no UDP WG) in hostile networks.
- Single transport for alpha if WG UDP blocked.
Why defer:
- Alpha RSSH goal = no VPN permission, no key icon, file/shell access.
- WG full-tunnel solves "egress via hub" for lab now.
- RSSH+VPN is new subsystem (SOCKS or tun2socks + VpnService + bastion NAT), not a setting.
Options sketched:
A) Use WG full-tunnel (existing) — recommended near-term for routing.
B) RSSH + ssh -D SOCKS + VpnService + tun2socks — dev experimental.
C) ssh -w tun both ends — heavy, skip.
D) Hybrid: WG packets + RSSH operator — already in proposal doc.
DR should decide: postpone until WG E2E closed; if revived, dev-only sub-mode under RSSH.
Open for PO:
- Is TCP-only routing worth duplicating WG?
- Accept VPN icon/consent again for RSSH-routed mode?
- Bastion egress NAT policy (Minsk) for dev traffic?

278
drafts/20260617_codec2_ulbw_voice_integration.txt Normal file
View File

@@ -0,0 +1,278 @@
Codec2 / ultra-low-bandwidth (ULBW) voice — research draft
R0-pre
severity: low
state: research (post-alpha candidate; blocked on Opus/Speex JNI baseline)
author: PO + agent
date: 2026-06-17
Problem:
- PO asked whether Codec2 (or MELP/TWELP-class codecs) should be integrated for voice-only or
audio-primary paths on very slow links: mesh/BT P2P, marginal LTE, future multipoint relay.
- Today screen cast defaults to AAC; Opus/Speex are advertised in discovery masks but native
encode/decode JNI is still TODO (stub → AAC on wire). See ndk/README.md, OPUS_SPEEX_VALIDATION.md.
- Future F1 multipoint cast (MCU/SFU on BE) will need a single negotiated audio format per room
or per leg — P2P bitmask negotiation alone will not scale.
[foxx comment]: it's surprise to me to hear that Opus/Speex is still TODO - reported to be finished by AI
[AI note]: Negotiation/UI/autolink scaffolding is done; ROADMAP.md lists "Opus/Speex JNI encode/decode sinks" as TODO.
opus_bridge.c only exposes nativeIsOpusAvailable — wire path is OPUS_STUB→AAC without libopus.a + encode JNI.
See DR docs/DRs/20260617_codec2_ulbw_voice.md §4.1.
Scope of this draft:
- Research + integration sketch only. No Codec2 until Opus/Speex JNI encode/decode lands (ROADMAP 6.1).
- Covers mobile peer negotiation (today + extensions) and backend MCU/SFU hooks (gray / post-alpha).
──────────────────────────────────────────────────────────────────────────────
1. Codec landscape (skeptical first)
──────────────────────────────────────────────────────────────────────────────
| Codec | Bitrate | Sample rate | Quality / role | License | Android fit |
|--------------|-------------|-------------|-----------------------------|-----------|--------------------|
| Opus SILK NB | ~6–12 kbps | 8 kHz | Good comms; FEC; wide use | BSD | Submodule exists |
| Speex NB | ~2–15 kbps | 8 kHz | Legacy narrowband | BSD | Submodule exists |
| Codec2 | 700–3200 bps| 8 kHz | HF radio; intelligible only | LGPL-2.1 | New submodule |
| MELP / TWELP | ~2400 bps | 8 kHz | Govt vocoder; patent encumbrance | varies | Poor OSS story |
| AAC-LC | 32–128 kbps | 44.1/48 kHz | Screen cast default | patent | MediaCodec HW |
Skeptical PO questions (answer before building Codec2):
- Is Codec2 needed if Opus @ 6 kbps + existing FEC/NACK already fits the slow-link budget?
- Is the use case **voice-only cast** (no video) or **audio leg of screen cast**? Codec2 is 8 kHz
telephony — downmixing stereo app audio loses music/UI cues; fine for push-to-talk, poor for
“share my meeting audio”.
- Codec2 maintainer note (2023+): major re-dev; many modes “not actively maintained”; only 3200 for
M17 called out as exception. Fork/maintenance risk vs stable Opus.
- LGPL-2.1: dynamic link via libandroidcast_codecs.so is OK; document in mobile + BE licenses;
Gitea mirror like opus/speex.
[foxx comment]: from my past practical experience, Opus@6k+FEC gives a lot more hearable noise and network bandwidth rather than Codec2, so everything just fits its hole and goals; we use AAC legally as a part of Android platform, no conflict there; Codec2 3200bps looks also OK for the goals we want to achieve; MELP/TWELP - skipping that, out of scope because of many slim points using it
When Codec2 still wins:
- Sub-3 kbps **sustained** (not burst): BT Classic SPP mesh, APRS-style relay, satellite backhaul.
- Symmetric **voice-only** sessions where both peers opt into “radio mode” (dev/PO setting).
- Lab demos comparing Speex vs Codec2 700C on identical packet loss (after Opus baseline exists).
[foxx comment]: the intent of using Codec2 is not the lab comparison but the desire from end-users to have audio casts on ultra-low bandwidth (better to have something than have nothing, let's say)
──────────────────────────────────────────────────────────────────────────────
2. Current mobile negotiation (P2P LAN / TCP / UDP)
──────────────────────────────────────────────────────────────────────────────
Two parallel paths today — video and audio are **not** negotiated the same way.
A) Discovery + wire header (both peers, pre-session):
- CastProtoHeader.audioCodecs / videoCodecs bitmasks (48-byte header, every packet).
- CastCodecFlags: AUDIO_PCM=1, AAC=2, OPUS=4, SPEEX=8 (room for 16, 32, …).
- DiscoveryManager advertises localAudioMask() from CastCodecFlags.localAudioMask().
- Receiver list in UI carries peer audioCodecs → sender picks target.
B) TCP handshake (video only, CastSession):
- MSG_CODEC_CAPS → MIME lists (H.264, VP8, …).
- MSG_CODEC_SELECTED → single negotiated video MIME.
- Receiver runs CodecNegotiator; sender logs CodecPriorityCatalog decision.
- **No MSG_AUDIO_SELECTED** — audio is not on this wire sequence.
C) Audio resolution (sender side, after target known):
- AudioNegotiator.resolveForSender(settings, peerAudioMask):
negotiate(localMask, peerMask, user preference, settings).
- AUTO → CodecPriorityCatalog.decideBestAudioCodec (scores from codecs.json).
- Explicit pref → mask intersection; fallback to AUTO → AAC.
- Result stored in CastSettings.resolvedAudioCodec (transient).
- Wired from ScreenCastService / MultiCastCoordinator.
D) Stream protection (UDP):
- Separate post-handshake negotiation (FEC/NACK); independent of codec choice.
Implications for Codec2:
- Adding Codec2 requires a new flag bit (e.g. AUDIO_CODEC2 = 16) in CastCodecFlags + proto docs.
- Must update: localAudioMask(), flagForAudioPreference(), isLocallyAvailable(), CodecPriorityId,
CastSettings.AudioCodec enum, codecs.json, manualAudioPreferences() order.
- **Do not** advertise CODEC2 in mask until NativeCodecBridge probe passes (same rule as Opus).
- Unit tests: AudioNegotiatorTest, CastProtoHeaderTest round-trip, mask intersection cases.
Gap (intentional for alpha, painful for SFU):
- Audio choice is **sender-resolved** from discovery masks; receiver is not sent MSG_AUDIO_SELECTED.
- Receiver must infer from CastSettings in session or mirror sender logic on first audio frame
(today: implicit via shared settings blob / same negotiation inputs). Document before SFU.
──────────────────────────────────────────────────────────────────────────────
3. Proposed Codec2 mobile integration (minimal diff)
──────────────────────────────────────────────────────────────────────────────
Enum / settings:
- CastSettings.AudioCodec.CODEC2 (or CODEC2_1300 / CODEC2_700C sub-modes in dev settings).
- Optional CastSettings.VoiceOnlyMode flag (no video track; skips video handshake).
Native:
- third-party/codec2 submodule → scripts/init-third-party-submodules.sh + Gitea mirror.
- Extend scripts/build-native-codecs.sh + ndk CMake: ANDROIDCAST_HAVE_CODEC2.
- JNI in NativeCodecBridge: isCodec2Available(), encode/decode 20 ms frames @ 8 kHz mono.
- Frame format on wire: length-prefixed Codec2 blobs (mode id in session meta or first frame);
**not** AAC ADTS — new CastProtocol MSG_AUDIO_FRAME subtype or codec tag in existing payload.
Priority catalog (codecs.json):
- Default order suggestion if ULBW profile enabled: CODEC2 < OPUS < SPEEX < AAC < PCM.
- bitrate_score: CODEC2 lowest; internal_score: higher delay penalty (see §5).
- Profile hook: CastSettings.NetworkAdoption.ESTIMATED or new ULBW_PRESET gates AUTO toward CODEC2
only when both masks include it AND user/dev “radio mode” on.
Voice-only cast (optional product slice):
- Capture: AudioRecord 8 kHz mono (or resample from 48 kHz graph).
- Skip video encoder; discovery still sends videoCodecs=0 or VIDEO_UNKNOWN.
- BT / mesh: same negotiation masks over RFCOMM or UDP wrapper — framing spec TBD (§6).
Milestone order:
1. Opus + Speex JNI encode/decode (ROADMAP 6.1) — validates pipeline end-to-end.
2. Codec2 probe + encode/decode behind dev flag.
3. Voice-only mode + E2E on two devices.
4. SFU signaling extensions (§4) — only when F1 spec approved.
──────────────────────────────────────────────────────────────────────────────
4. Future backend: MCU / SFU integration
──────────────────────────────────────────────────────────────────────────────
References:
- docs/20260608_BE_SERVICES_and_infra.md § Planned SFU relay (Janus-class)
- docs/specs/20100612_1_scaling.md F1 Multipoint cast, be-media-N tier
- docs/INFRA.md: port 8089 = Janus/other — **not** androidcast PHP vhost; do not mix nginx paths
- App: SfuRelayGate.PREVIEW_ENABLED = false; CastConfig.ALPHA_FEATURE_FREEZE
Architecture (target, gray):
[Sender A]──┐ ┌──[Receiver B]
├──► Signaling (PHP) ──►├── Janus/mediasoup SFU (:8089 or dedicated)
[Sender C]──┘ │ └── transcode / fan-out
RBAC/session
room lifecycle
Negotiation shifts from pairwise bitmask to **room capability contract**:
| Layer | P2P today | SFU future |
|--------------------|------------------------------|-------------------------------------------------|
| Capability advert | Discovery header masks | Join-room: client caps JSON + bitmask |
| Audio pick | AudioNegotiator local | Server publishes room_audio_codec or per-leg |
| Video pick | MSG_CODEC_SELECTED | WebRTC SDP / simulcast layers |
| Auth | PIN / LAN | Existing session/RBAC + room token |
| Transport | UDP/TCP cast | WebRTC (+ optional TURN); LAN cast unchanged |
Codec2 on SFU — three realistic modes:
1) **Passthrough (hardest):** SFU relays Codec2 packets without decode. All clients must speak
Codec2; no WebRTC browser viewers. Fits closed Android mesh only.
2) **Transcode at SFU (likely):** Android sends Codec2 or Opus; media worker (ffmpeg/Janus plugin)
decodes → Opus/VP8 for WebRTC subscribers. PO must budget CPU on be-media-N (scaling SPEC).
Negotiation: room advertises sfu_audio_ingress: [opus, codec2], egress: opus.
3) **MCU mix (voice conference):** Decode all legs → PCM mix → encode Opus (or Codec2 for radio
leg). Codec2 only on “radio bridge” participants, not main room.
Signaling API sketch (owner spec TBD — not implemented):
- POST /api/sfu/room — create; body includes max_participants, ingress_caps, egress_caps.
- POST /api/sfu/room/{id}/join — returns Janus session/handle or mediasoup transport params.
- Client payload extension (parallel to CastSettings):
audio_caps: { bitmask, codec2_modes: ["700C","1300"], max_bitrate_bps }
video_caps: { bitmask, mime_list }
preference: { audio: "AUTO", network_profile: "ULBW" }
- Server response:
negotiated_audio: "opus" | "codec2_1300" | ...
negotiated_video: mime or null for voice-only
relay_mode: "p2p" | "sfu" | "mcu"
Mobile changes when SFU lands:
- New transport enum value TRANSPORT_WEBRTC (frozen in alpha UI).
- Before cast: if SfuRelayGate.isAvailable() && room URL → skip LAN discovery masks;
run SfuSignalingClient.negotiate(roomCaps) → populate CastSettings.resolvedAudioCodec from server.
- Fallback: LAN P2P unchanged if signaling unreachable (local-first).
- AudioNegotiator.negotiate() reused for **pre-signaling** pairwise preview; server may override.
Janus / 8089 caution:
- FE nginx location /janus/ exists for other stack — androidcast signaling stays on apps.f0xx.org
PHP API; media UDP ports on dedicated media subnet per scaling DR. No coupling in alpha.
MCU vs SFU choice (for PO):
- SFU: lower server CPU if passthrough/simulcast; N subscribers see one encoded stream.
- MCU: needed for true multiparty voice mix or heterogeneous codecs (Codec2 leg + Opus leg).
- F1 backlog item — separate SFU SPEC before any Codec2-on-BE work.
──────────────────────────────────────────────────────────────────────────────
5. Peer negotiation — extensions (mobile + protocol)
──────────────────────────────────────────────────────────────────────────────
Short term (Codec2 on P2P, no proto version bump if possible):
- Reserve CastCodecFlags.AUDIO_CODEC2 = 16.
- Add optional CastSettings.codec2Mode (1300, 700C, 3200) — must match on both sides; if mismatch,
AudioNegotiator falls back to next scored codec (same as explicit OPUS unavailable).
- Consider MSG_AUDIO_SELECTED in TCP handshake (symmetric with video):
payload: uint32 audio_flag + uint8 mode
Benefits: receiver logs definitive choice; multi-receiver multicast gets same codec;
SFU migration reuses message type over WebRTC data channel.
Cost: proto doc + CastSession serverHandshake/clientHandshake change — schedule with SETTINGS_VERSION bump.
Medium term (multipoint / SFU):
- Extend CastSettings SETTINGS_VERSION when adding sfu_room_id, relay_mode, server_negotiated_audio.
- Discovery header insufficient for 3+ peers — room state lives on signaling server.
- Per-peer legs: senderMask/receiverMask become N-vectors or server-issued single codec.
- AudioNegotiator refactor: negotiateRoom(RoomCaps, List<PeerCaps>, preference) → server or elected
“master” receiver (first joiner) picks; align with CodecNegotiator pattern for video.
BT / mesh P2P (no BE):
- Same bitmask in link-local discovery (mDNS or BLE advertisement) — reuse CastProtoHeader layout.
- Frame timing: Codec2 1600-mode ~ 40 ms frames; BT SPP throughput ~1–3 kbps practical — profile
700C only; add jitter buffer 80–120 ms (document in AV_QUALITY_QA_SESSION follow-up).
Delay budget (manage expectations):
- Codec2 encode+decode + BT latency → 150–300 ms one-way plausible; unacceptable for lip-sync
with video — reinforce voice-only or audio-only cast for ULBW profile.
──────────────────────────────────────────────────────────────────────────────
6. Testing & validation (when implemented)
──────────────────────────────────────────────────────────────────────────────
Automated:
- AudioNegotiatorTest: CODEC2 in both masks → selected when scores favor; one side missing → fallback.
- CastCodecFlags round-trip AUDIO_CODEC2 bit.
- Native round-trip: c2enc/c2dec parity via JNI on arm64-v8a CI (optional, after submodule).
Manual E2E (extend OPUS_SPEEX_VALIDATION.md matrix):
- Voice-only, CODEC2 explicit, two devices, UDP then TCP.
- AUTO + ESTIMATED network preset on marginal Wi‑Fi (throttle via dev settings).
- Record: resolvedAudioCodec in diagnostics, audio kbps, subjective MOS notes.
SFU (post-alpha):
- Room with 2 Android + 1 ingress Codec2 → verify transcode to Opus for third WebRTC viewer.
- sfu_health.php enabled; SfuRelayGate preview on dev builds only.
──────────────────────────────────────────────────────────────────────────────
7. Open questions for PO
──────────────────────────────────────────────────────────────────────────────
1. Primary use case: voice-only P2P, or audio leg of screen cast on ULBW?
2. Accept 8 kHz telephony quality for “radio mode”, or require wideband Opus for anything with video?
3. Codec2 mode default: 1300 (balance) vs 700C (minimum bits) vs user-selectable?
4. Proceed with Codec2 at all after Opus @ 6 kbps measured on target BT/mesh hardware?
5. SFU: passthrough Codec2 vs transcode-only at ingress — which matches commercial story?
6. MSG_AUDIO_SELECTED in handshake: worth SETTINGS_VERSION bump now, or defer until SFU spec?
7. MELP/TWELP: drop — insufficient OSS path unless PO has licensed binaries.
──────────────────────────────────────────────────────────────────────────────
8. Recommendation (agent opinion)
──────────────────────────────────────────────────────────────────────────────
Defer Codec2 implementation until:
- Opus/Speex JNI is real on wire (not stub).
- PO picks voice-only vs AV-leg scope.
- One measured session on slow link proves Opus+FEC insufficient.
If proceeding:
- Treat Codec2 as dev/ULBW profile codec, not AUTO default.
- Design bitmask + optional MSG_AUDIO_SELECTED now so SFU room negotiation reuses the same types.
- Plan BE transcode path (ffmpeg decode Codec2 → Opus) in F1 SFU SPEC — do not assume SFU passthrough.
Next doc steps:
- DR: docs/DRs/20260617_codec2_ulbw_voice.md + PDF (done 2026-06-17)
- SPEC slice in scaling or dedicated docs/specs/…_sfu_audio_negotiation.md when F1 unblocks
- Ticket: “ULBW voice” → implementation epic after 6.1 green
Related files:
- app/.../CastCodecFlags.java, AudioNegotiator.java, CodecPriorityCatalog.java
- app/.../network/CastSession.java, CastProtoHeader.java, CastProtocol.java
- app/.../sfu/SfuRelayGate.java
- ndk/README.md, app/src/main/assets/codecs.json
- third-party/opus, third-party/speex (pattern for codec2 submodule)

556
drafts/20260618_repos_reorganizing.txt Normal file
View File

@@ -0,0 +1,556 @@
Project repository and structure reorganization draft
milestone: R0-pre
severity: high
state: pre-implementation
author: Anton Afanasyeu
date: 2026-06-18
[AI comment]: Updated 2026-06-18 — PO chose org **`ac`**, **full microservice** end-state, separate **ac-scripts**, URL path migration (§L), optional **ac-workspace** (not build-all-in-one), VM-first cloud (§N). See §G master repo table.
The problem:
The project looks like monolith. For better and more atomic support we need to reorganize the whole project repo structure
[AI comment]: Agree on the problem, but **atomic** should mean: independent release cadence, CI, deploy, and rollback — not “one git repo per PHP class.” Today’s real coupling is **one MariaDB**, **one PHP session** (`ac_crash_sess`, [BUILD_DEPLOY.md](../BUILD_DEPLOY.md)), and **one nginx vhost tree** on BE. Split repos before extracting shared platform code will multiply broken deploys.
──────────────────────────────────────────────────────────────────────────────
A. Current codebase map (monolith → draft names)
──────────────────────────────────────────────────────────────────────────────
| Draft name / intent | Actual path today | Coupling notes |
|---------------------|-------------------|----------------|
| ac-mobile | `app/`, `ndk/`, `gradle/`, `build.gradle`, `settings.gradle` | WireGuard tunnel module; `third-party/*` submodules |
| ac-session-studio | `desktop/session-studio/` | Standalone; low coupling — **easy first extract** |
| ac-docs | `docs/` (+ `docs/_pdf_build/`) | No runtime deps — **easy extract** |
| ac-scripts | `scripts/` (pdf, native codecs, OTA, CI helpers) | Used by mobile + orchestration; keep with **ac-deploy** or meta until split clear |
| ac-be-console (crashes/issues/tickets/graphs/rbac/ra) | `examples/crash_reporter/backend/` | **32 PHP classes**, 24 `public/api/*.php`, shared `Auth.php`, `Database.php`, `Rbac.php` — **one modular monolith** |
| ac-be-builder | `examples/build_console/backend/` | Separate app; shares auth pattern via `examples/platform/shared_session.php` + same DB users table |
| ac-be-landing | `examples/app_hub/` | Static hub + deploy nginx snippets |
| ac-be-ota (artifacts) | `examples/ota/v0/...` | Files on disk + nginx; logic also in builder + `scripts/generate-ota-v0.sh` |
| ac-ms-url-shortener | `backend/url-shortener/` | **Only live submodule today** — `git://f0xx.org/androidcast_project/url-shortener` |
| ac-deploy / orchestration | `orchestration/` (docker, cluster0, nginx, runtime) | Local stack + **cast01–03** lab ([orchestration/sim/cluster0/ARCHITECTURE.md](../../orchestration/sim/cluster0/ARCHITECTURE.md)) |
| ac-ms-vpn / WG / RSSH | Mobile: `app/.../remoteaccess/`; BE: `RemoteAccessRepository.php`, `WireGuard*`, `Rssh*` provisioners | **Split by plane**: client in mobile, control API in BE console — not two deployables yet |
| ac-ms-vcs (gitea) | `examples/crash_reporter/backend/scripts/gitea/` | **Ops scripts only** — Gitea is external service on BE, not app source |
| ac-ms-tracker / analytics | `Ticket*`, `Graph*` repos in crash_reporter | Same DB + session — **logical modules**, not separate services today |
| SFU stub | `examples/sfu/`, `api/sfu_health.php` | Post-alpha; stay in console until F1 SPEC |
| Experimental / lab | Nested `.git` under `examples/` (not `third-party/` or `backend/url-shortener`) | **Out of scope** — omit from migration docs and manifests |
| third-party codecs | `third-party/libvpx|opus|speex|wireguard-android` | Stay **submodules of ac-mobile** (upstream URLs unchanged); Gitea mirrors per existing migrate script |
[AI comment]: **Correction for draft “how” section:** `be-console` is **not** `./examples/build_console` — that is **builder**. Primary operator console is **`examples/crash_reporter/backend`** (crashes + tickets + graphs + RBAC + remote access APIs). Builder is separate product surface under `/build/`.
──────────────────────────────────────────────────────────────────────────────
B. Infrastructure & cloud fit (combine lab + prod + future cloud)
──────────────────────────────────────────────────────────────────────────────
Principles (from [specs/20100612_1_scaling.md](../specs/20100612_1_scaling.md), [INFRA.md](../INFRA.md), cluster0):
1. **Deploy unit = VM role or container image**, not git repo count. Phase A prod: `be-web-1` runs nginx + PHP-FPM for hub + console + graphs UI; `be-data-1` MariaDB; `be-ops-1` url-shortener + Gitea; `be-build-1` builder.
2. **cluster0 pattern** (NFS config only, app on local disk, GTID MariaDB) maps to cloud as:
- NFS `/shared/cluster/` → **ConfigMaps / object storage** for nginx + SQL seeds + scripts
- Per-node `/var/www/...` → **container image** or **git pull on VM** per service
- Managed DB → **connection string in secret** (PO: “pay for DB service”) — must **not** live in `ms-common` git
3. **PHP session + RBAC**: Shared cookie path `/app/androidcast_project` across hub/crashes/build ([BUILD_DEPLOY.md](../BUILD_DEPLOY.md)). Until true SSO/service tokens exist, **splitting BE UI repos is fine**; splitting **API auth** into N repos without `ac-platform-php` Composer package is painful.
4. **OpenAPI**: PO wants MS OpenAPI-compatible — today APIs are PHP files, not generated spec. Recommend **`ac-platform-openapi`** (or `ac-docs/openapi/`) as **contract-first** artifacts; implement in console first, extract services later.
5. **CI**: `build.config.yml` + CircleCI shape assume **monorepo checkout**. Meta-repo needs **per-repo pipelines** + `ac-deploy` triggering builds with `git_ref` across submodules (builder already does this for `android_cast`).
6. **URL / paths:** Under **discussion** — PO wants path-per-service (§M), not frozen `?view=` under `/crashes/`. nginx + mobile `BackendEndpoints` must migrate with redirects.
[AI comment]: Shared cookie path may become `/app/androidcast_project/` at **edge** only while backends move to `/issues/`, `/tickets/`, etc.
──────────────────────────────────────────────────────────────────────────────
C. Target structure — **full microservice** (PO decision 2026-06-18)
──────────────────────────────────────────────────────────────────────────────
PO chose **full microservice** end-state (§K comparison). Delivery remains **phased** (§D).
```text
Platform (libraries — Composer, not runtime MS)
ac-platform-php, ac-platform-db, ac-platform-web, ac-scripts, ac-docs
Infrastructure
ac-deploy submodules ac-scripts; cluster0/docker/nginx
ac-workspace optional clone manifest only (§N) — NOT unified build
Clients
ac-mobile-android (+ future ac-mobile-ios)
ac-session-studio
Microservices (OpenAPI APIs / workers)
ac-ms-identity, ac-ms-rbac, ac-ms-devices, ac-ms-issues, ac-ms-tickets,
ac-ms-graphs, ac-ms-remote-access, ac-ms-url-shortener, ac-ms-build,
ac-ms-ota, ac-ms-notifications (later)
Backend UI (thin)
ac-be-hub, ac-be-issues, ac-be-tickets, ac-be-graphs, ac-be-remote-access,
ac-be-access, ac-be-builder, ac-be-auth (optional)
Edge
ac-platform-edge nginx BFF — recommended for cookie + URL migration (§M)
```
[AI]: Collapse PO’s **ms-vpn-rssh** + **ms-vpn-wireguard** → **ac-ms-remote-access**. Drop **ac-ms-vcs** as code repo (Gitea = infra). Drop **ac-ms-tracker** as separate MS — split into **ac-ms-issues** + **ac-ms-tickets** (different URLs PO wants).
**Risk mitigation:** Composer semver for platform libs (avoid @common submodule diamond in 20+ repos).
──────────────────────────────────────────────────────────────────────────────
D. Phased migration (safe order — no big-bang)
──────────────────────────────────────────────────────────────────────────────
| Phase | Action | Alpha blocker? | Rollback |
|-------|--------|----------------|----------|
| **P0** | Create Gitea org **`ac`**; mirror legacy → **ac-workspace** (optional) | No | Keep `android_cast` mirror |
| **P1** | Extract **ac-docs**, **ac-session-studio** submodules; meta points SHAs | No | Revert submodule |
| **P2** | Extract **ac-ms-url-shortener** path to `ac/ac-ms-url-shortener` (already submodule — change URL only) | No | Old URL works via mirror |
| **P3** | **ac-platform-php** + **ac-platform-web** + **ac-ms-identity** first (everything depends on auth) | **Yes** for MS split | Shim monolith |
| **P4** | Extract **ac-ms-issues**, **ac-ms-tickets**, **ac-ms-rbac**, **ac-ms-devices** + matching **ac-be-*** UI repos | Medium | Edge proxy to monolith |
| **P5** | **ac-mobile-android**, **ac-deploy** (+ submodule **ac-scripts**), **ac-ms-build**, **ac-ms-ota** | Medium | Per-repo CI |
| **P6** | **ac-platform-edge** nginx routes (§M); retire `/crashes/?view=` | Medium | 301 redirects |
| **P7** | cluster0 + cloud VMs (§O); optional **ac-ms-graphs**, **ac-ms-remote-access** | No | Per scaling SPEC |
[AI comment]: Do **not** start P4 until P3 — otherwise Auth/Database duplicates diverge in days.
**ms-common sensitive data:** Split into:
- `ac-platform-php` — code (session helpers, RBAC classes, DB adapter interfaces)
- **Runtime config** — `config.php` / env / K8s secrets per deploy (never in git)
- Optional `ac-platform-db` — **SQL migrations only** (shared schema), consumed by console + shortener + builder
──────────────────────────────────────────────────────────────────────────────
E. The what (original table — annotated)
──────────────────────────────────────────────────────────────────────────────
Current project structure
Project | Placement | Repo | State | Description
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
AndroidCast all in one | repo root | git://f0xx.org/android_cast | monolithic | all-in-one project, needs to be splitted
[AI]: → becomes **ac-meta** umbrella; keep **read-only mirror** at old URL ≥ 1 release
libvpx | third-party/libvpx | https://chromium.googlesource.com/webm/libvpx | 3rdparty exported submodule | VPx video codecs
[AI]: stays upstream URL in .gitmodules; Gitea mirror `AndroidCast/libvpx` unchanged
opus | third-party/opus | https://github.com/xiph/opus | 3rdparty exported submodule | Opus voice codec
speex | third-party/speex | https://github.com/xiph/speex/ | 3rdparty exported submodule | Speex voice codec
wireguard | third-party/wireguard-android | https://git.zx2c4.com/wireguard-android | 3rdparty exported submodule | wireguard for android
url-shortener | backend/url-shortener | git://f0xx.org/androidcast_project/url-shortener | part of AndroidCast backend | url shortener standalone backend
[AI]: migrate to **git://f0xx.org/ac/ac-ms-url-shortener**; alias old URL
──────────────────────────────────────────────────────────────────────────────
F. The how (original target — annotated + fixes)
──────────────────────────────────────────────────────────────────────────────
Next hope project structure
AndroidCast project - all in one | git://f0xx.org/ac/ac-meta
|-- AndroidCast - common | @ac-platform-php [AI: Composer, not submodule diamond]
|-- AndroidCast - mobile | @ac-mobile
|-- AndroidCast - BE | @ac-be-console (+ builder, hub as siblings)
... TBD
AndroidCast project | Placement (@ - submodule) | Description | Repo
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
AndroidCast - common | @common | project all common | git://f0xx.org/ac/ac-common
[AI]: **Rename → ac-platform-php** (Composer). Reserve **ac-common** for non-PHP only: LICENSE templates, OpenAPI, semver constants — or drop repo and fold into ac-docs.
|-- AndroidCast - docs | @docs | project docs, BE, MS, mobile, web, everything | git://f0xx.org/ac/ac-docs
|-- AndroidCast - scripts | @scripts | common project scripts | git://f0xx.org/ac/ac-scripts
[AI]: merge ac-scripts into **ac-deploy** unless mobile-only scripts dominate
AndroidCast - mobile | @mobile | mobile app sources | git://f0xx.org/ac/ac-mobile
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
[AI]: mobile should NOT submodule ac-common PHP — only third-party natives
AndroidCast - Session Studio | @session-studio | offline analytic tools | git://f0xx.org/ac/ac-session-studio
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
[AI]: session-studio has no PHP dep — drop @common edge
AndroidCast - BE - MS | @backend | backend microservices common base (UI) | git://f0xx.org/ac/ac-be-microservice
[AI]: **Overloaded name.** Split: **ac-be-console** (main UI+API), **ac-platform-php** (lib). “ac-be-microservice” as umbrella meta for BE apps optional.
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
AndroidCast - MS | @microservices | microservice commons | git://f0xx.org/ac/ac-microservice
[AI]: defer **ac-microservice** meta-repo until ≥3 independently deployed APIs exist (today: 1 = url-shortener)
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
... TBD
AndroidCast BE microservices and their derivatives (OpenAPI providers, any REST API services for web and mobile apps)
AndroidCast - MS - template <- AndroidCast - MS (@ means suibmodule) |git://f0xx.org/ac/ac-microservice/ac-ms-template
[AI]: good template for **future** services; seed from url-shortener layout (public/, src/, sql/, tests/)
| -- AndroidCast - common | @common | project all common | git://f0xx.org/ac/ac-common
| -- AndroidCast - MS - common | @ms-common | microservices common | git://f0xx.org/ac/ac-microservice/ac-ms-common
[AI]: **ms-common = ac-platform-php + openapi + migration runner** — NOT DB passwords. Fix typo **@ms-commmon** in remote-access row below.
|-- AndroidCast - MS - builder | @ms-builder | builder microservice | git://f0xx.org/ac/ac-microservice/ac-ms-builder
[AI]: today = **ac-be-builder** (UI+worker in one); split worker when be-build-1 is isolated CI agent only
|-- AndroidCast - MS - VPN | @ms-vpn | VPN microservice | git://f0xx.org/ac/ac-microservice/ac-ms-vpn
[AI]: **Phase 6+** — API surface = `remote_access.php` + provisioners; client stays in ac-mobile
|-- AndroidCast - MS - VPN - RSSH | @ms-vpn-rssh | VPN RSSH microservice | git://f0xx.org/ac/ac-microservice/ac-ms-vpn-rssh
|-- AndroidCast - MS - VPN - WireGuard | @ms-vpn-wireguard | VPN WG microservice | git://f0xx.org/ac/ac-microservice/ac-ms-vpn-wireguard
[AI]: package by directory inside console first: `src/RemoteAccess/Rssh/`, `src/RemoteAccess/WireGuard/`
|-- AndroidCast - MS - VCS | @ms-vcs | VCS microservice (gitea) | git://f0xx.org/ac/ac-microservice/ac-ms-vcs
[AI]: **Not a code repo** — Gitea is infra; keep **ac-deploy** scripts + docs for Gitea ops
|-- AndroidCast - MS - URL shortener | @ms-url-shortener | URL shortener microservice | git://f0xx.org/ac/ac-microservice/ac-ms-url-shortener
[AI]: **P2** — flatten path to `git://f0xx.org/ac/ac-ms-url-shortener` (no nested ac-microservice parent)
|-- AndroidCast - MS - analytics | @ms-analytics | analytics microservice | git://f0xx.org/ac/ac-microservice/ac-ms-analytics
[AI]: graphs ingest + GA config — module in console until read load warrants split
|-- AndroidCast - MS - orchestration | @ms-orchestration | orchestration microservice | git://f0xx.org/ac/ac-microservice/ac-ms-orchestration
[AI]: today = **ac-deploy** + builder API; name collision with folder `orchestration/`
|-- AndroidCast - MS - tracker | @ms-tracker | issues & tickets tracker microservice | git://f0xx.org/ac/ac-microservice/ac-ms-tracker
|-- AndroidCast - MS - RBAC | @ms-rbac | RBAC microservice | git://f0xx.org/ac/ac-microservice/ac-ms-rbac
... TBD
Sub-project of AndroidCast BE/MS, mostly Web UI stuff, but also should be OpenAPI-compatible and provide services to the mobile app (@ means suibmodule)
AndroidCast - BE - common <- AndroidCast - BE - MS | git://f0xx.org/ac/ac-backend
[AI]: prefer **per-app repos** (ac-be-console, ac-be-builder, …) over deep `ac-backend/*` tree — flatter Gitea permissions
|-- AndroidCast - BE - vcs (UI) | @be-vcs | VCS frontend, former gitea, backend UI | git://f0xx.org/ac/ac-backend/ac-be-vcs
[AI]: Gitea **is** the UI — link from hub only; optional thin PHP wrapper if needed
|-- AndroidCast - MS - vcs | @ms-vcs | VCS microservice (gitea) | git://f0xx.org/ac/ac-microservice/ac-ms-vcs
|-- AndroidCast - BE - console (UI) | @be-console | git://f0xx.org/ac/ac-backend/ac-be-console | primary console, ./examples/build_console ? check if it is
[AI]: **FIX:** primary console = **ac-be-console** ← `examples/crash_reporter/backend`. Builder = **ac-be-builder** ← `examples/build_console`.
|-- AndroidCast - MS - common | @ms-common | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
|-- AndroidCast - BE - landing (UI) | @be-landing | git://f0xx.org/ac/ac-backend/ac-be-landing | landing page(s), now in ./examples/app_hub
|-- AndroidCast - MS - common | @ms-common | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
|-- AndroidCast - BE - issues (UI) | @be-issues | git://f0xx.org/ac/ac-backend/ac-be-issues | issues tracker UI, now is in ./examples somewhere
[AI]: **same PHP app** as tickets — `?view=` routes in crash_reporter views, not separate deployable
|-- AndroidCast - MS - tracker | @ms-tracker | git://f0xx.org/ac/ac-microservice/ac-ms-tracker | issues tracker microservice shared base, now is in ./examples somewhere
|-- AndroidCast - BE - tickets (UI) | @be-tickets | git://f0xx.org/ac/ac-backend/ac-be-tickets | tickets UI, now is in ./examples somewhere
|-- AndroidCast - MS - tracker | @ms-tracker | git://f0xx.org/ac/ac-microservice/ac-ms-tracker | issues tracker microservice shared base, now is in ./examples somewhere
|-- AndroidCast - BE - graphs (UI) | @be-graphs | git://f0xx.org/ac/ac-backend/ac-be-graphs | graphs UI, now somewhere in ./examples
|-- AndroidCast - MS - analytics | @ms-analytics | git://f0xx.org/ac/ac-microservice/ac-ms-analytics | analytics base microservice
|-- AndroidCast - BE - analytics (UI) | @be-analytics | git://f0xx.org/ac/ac-backend/ac-be-analytics | analytics and statistics web UI
|-- AndroidCast - MS - analytics | @ms-analytics | git://f0xx.org/ac/ac-microservice/ac-ms-analytics | analytics base microservice
[AI]: graphs + analytics UI = views inside **ac-be-console** today (`GraphRepository`, `AnalyticsHead`)
|-- AndroidCast - BE - remote access (UI) | @be-remote-access | git://f0xx.org/ac/ac-backend/ac-be-remote-access | remote access (now for vpn, somewhere in ./examples now) UI
|-- AndroidCast - MS - common | @ms-commmon | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
[AI]: typo **commmon** → common; RA UI = admin views in console + mobile app
|-- AndroidCast - MS - VPN | @ms-vpn | git://f0xx.org/ac/ac-microservice/ac-ms-vpn | VPN MS as a dependency
|-- AndroidCast - MS - RBAC | @ms-rbac | @ms-rbac | git://f0xx.org/ac/ac-microservice/ac-ms-rbac | RBAC MS as a dependency
|-- AndroidCast - BE - URL shortener (UI) | @be-url-shortener | git://f0xx.org/ac/ac-backend/ac-be-url-shortener | URL shortener UI, now is in separate submodule
[AI]: short-link **admin UI** is in crash console (`short_links.js`, hub tokens); **API** is submodule
|-- AndroidCast - MS - URL shortener | @ms-url-shortener | git://f0xx.org/ac/ac-microservice/ac-ms-url-shortener | URL shortener microservice, now is in separate submodule
|-- AndroidCast - BE - builder (UI) | @be-builder | git://f0xx.org/ac/ac-backend/ac-be-builder | orchestration builder UI, now in ./orchestration
[AI]: **FIX:** builder UI = `examples/build_console`; **orchestration/** = docker/cluster0 (**ac-deploy**)
|-- AndroidCast - MS - orchestration | @ms-orchestration | git://f0xx.org/ac/ac-microservice/ac-ms-orchestration | orchestration microservice, now in ./orchestration
|-- AndroidCast - BE - RBAC (UI) | @be-rbac | git://f0xx.org/ac/ac-backend/ac-be-rbac | RBAC backend, now mixed within ./examples somewhere
|-- AndroidCast - MS - common | @ms-common | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
|-- AndroidCast - MS - RBAC | @ms-rbac | git://f0xx.org/ac/ac-microservice/ac-ms-rbac | RBAC microservice
|-- AndroidCast - BE - OTA | @be-ota | git://f0xx.org/ac/ac-backend/ac-be-ota | OTA BE UI
|-- AndroidCast - MS - OTA | @ms-ota | git://f0xx.org/ac/ac-microservice/ac-ms-ota | OTA microservice, now intermixed in examples/ota and other commons
[AI]: OTA = static tree + builder publish + nginx location; extract API when `5.1 OTA` nginx stable
... TBD
ms-common (microservices common) needs a sharp eye, it may share project sensitive parts, like common stuff such as shared web sessions, DB access (or should be separated to @be-db-service/@ac-db-service sort of)
[AI]: **Agree.** Recommended split:
- **ac-platform-php** — session *code*, Auth, Rbac, PDO factory (reads DSN from env)
- **ac-platform-db** — SQL migrations + schema version table (shared by console, builder, shortener)
- **ac-deploy/secrets** — example only; prod DSN in vault / Alpine `config.php` outside git
- **No ac-db-service runtime** until you run DB proxy (PlanetScale, RDS, etc.) — app VMs use driver + secret
──────────────────────────────────────────────────────────────────────────────
G. FULL REPO CATALOG — microservice end-state (Gitea org **ac**)
──────────────────────────────────────────────────────────────────────────────
Canonical clone: `git://f0xx.org/ac/<repo>`. HTTPS UI: `https://apps.f0xx.org/app/androidcast_project/git/ac/<repo>.git` (path TBD with edge migration). Legacy `git://f0xx.org/android_cast` → read-only mirror until retired.
| ID | Repo | Git URL | Role | From (monolith) | VM / cloud slot |
|----|------|---------|------|-----------------|-----------------|
| **Workspace** |
| W0 | ac-workspace | `git://f0xx.org/ac/ac-workspace` | Optional submodule manifest; **no unified build** | `.gitmodules` | dev laptops |
| **Platform** |
| P0 | ac-platform-php | `git://f0xx.org/ac/ac-platform-php` | Composer: HTTP, auth client, PDO factory | Auth*, bootstrap | baked into PHP images |
| P1 | ac-platform-db | `git://f0xx.org/ac/ac-platform-db` | SQL migrations + schema version | sql/, migrations | be-data-1 job |
| P2 | ac-platform-web | `git://f0xx.org/ac/ac-platform-web` | Shared CSS/JS/theme | crashes/assets, hub links | CDN / edge static |
| P3 | ac-platform-edge | `git://f0xx.org/ac/ac-platform-edge` | nginx routes, cookie domain, proxy | fe/apps.conf | be-web-1 / FE |
| P4 | ac-scripts | `git://f0xx.org/ac/ac-scripts` | CI, OTA, native codec, pdf scripts | `scripts/` | CI runners |
| P5 | ac-docs | `git://f0xx.org/ac/ac-docs` | Documentation | `docs/` | — |
| **Deploy** |
| D0 | ac-deploy | `git://f0xx.org/ac/ac-deploy` | docker, cluster0, env templates; **uses ac-scripts** | `orchestration/` | all VMs |
| **Clients** |
| C0 | ac-mobile-android | `git://f0xx.org/ac/ac-mobile-android` | Android app + ndk + third-party | `app/`, `ndk/` | CI → OTA |
| C1 | ac-mobile-ios | `git://f0xx.org/ac/ac-mobile-ios` | Future iOS client | — | CI → OTA |
| C2 | ac-session-studio | `git://f0xx.org/ac/ac-session-studio` | Desktop analyzer | `desktop/session-studio/` | releases |
| **Microservices** |
| S0 | ac-ms-template | `git://f0xx.org/ac/ac-ms-template` | Cookiecutter for new PHP MS | url-shortener layout | — |
| S1 | ac-ms-identity | `git://f0xx.org/ac/ac-ms-identity` | Users, login, register, 2FA, sessions | Auth*, UserRepository | be-web-1 |
| S2 | ac-ms-rbac | `git://f0xx.org/ac/ac-ms-rbac` | Companies, memberships, privileges | Rbac*, RbacAdmin* | be-web-1 |
| S3 | ac-ms-devices | `git://f0xx.org/ac/ac-ms-devices` | Device registry | DeviceRepository | be-web-1 |
| S4 | ac-ms-issues | `git://f0xx.org/ac/ac-ms-issues` | Crash/report ingest + reports API | upload, Report*, Tag* | be-web-1 |
| S5 | ac-ms-tickets | `git://f0xx.org/ac/ac-ms-tickets` | Tickets workflow, comments, attachments | Ticket* | be-web-1 |
| S6 | ac-ms-graphs | `git://f0xx.org/ac/ac-ms-graphs` | Graph ingest + query | GraphRepository, graph_*.php | be-web-1 |
| S7 | ac-ms-remote-access | `git://f0xx.org/ac/ac-ms-remote-access` | WG + RSSH control API | RemoteAccess*, WireGuard*, Rssh* | be-ops-1 |
| S8 | ac-ms-url-shortener | `git://f0xx.org/ac/ac-ms-url-shortener` | Short link API | `backend/url-shortener/` | be-ops-1 |
| S9 | ac-ms-build | `git://f0xx.org/ac/ac-ms-build` | Build queue + worker API | BuildRunner, build_console APIs | be-build-1 |
| S10 | ac-ms-ota | `git://f0xx.org/ac/ac-ms-ota` | Channel manifests; multi-client SHAs | OTA scripts + v0 tree | be-web-1 / OBJ |
| S11 | ac-ms-notifications | `git://f0xx.org/ac/ac-ms-notifications` | SMTP / auth mail (post-alpha) | AuthMailer | be-ops-1 |
| **Backend UI** |
| B0 | ac-be-hub | `git://f0xx.org/ac/ac-be-hub` | Landing | `examples/app_hub/` | be-web-1 |
| B1 | ac-be-issues | `git://f0xx.org/ac/ac-be-issues` | Issues UI → S4 | views reports/* | be-web-1 |
| B2 | ac-be-tickets | `git://f0xx.org/ac/ac-be-tickets` | Tickets UI → S5 | views tickets/* | be-web-1 |
| B3 | ac-be-graphs | `git://f0xx.org/ac/ac-be-graphs` | Graphs dashboard UI | graphs views | be-web-1 |
| B4 | ac-be-remote-access | `git://f0xx.org/ac/ac-be-remote-access` | RA admin UI → S7 | view remote_access | be-web-1 |
| B5 | ac-be-access | `git://f0xx.org/ac/ac-be-access` | RBAC admin UI → S2 | view rbac | be-web-1 |
| B6 | ac-be-builder | `git://f0xx.org/ac/ac-be-builder` | Builder UI → S9 | `examples/build_console/` | be-build-1 |
| B7 | ac-be-auth | `git://f0xx.org/ac/ac-be-auth` | Login/register chrome → S1 | login.php shared | be-web-1 |
| **Third-party (submodules of C0 only)** |
| — | libvpx, opus, speex, wireguard-android | upstream URLs + Gitea mirrors under `ac/` | Native deps | `third-party/*` | mobile CI |
| **Future** |
| F1 | ac-ms-sfu-signaling | `git://f0xx.org/ac/ac-ms-sfu-signaling` | WebRTC signaling | examples/sfu | be-media-N |
| F2 | ac-ms-media-transcode | `git://f0xx.org/ac/ac-ms-media-transcode` | VOD/ffmpeg | V.2x draft | be-media-N |
**PO draft mapping fixes:**
- `ac-be-console` (monolith) → **dissolved** into B1–B5 + S4–S7
- `ac-ms-tracker` → **S4 + S5** (issues ≠ tickets — PO URL split confirms)
- `ac-ms-analytics` → **S6** (graphs) + later BI; GA config in **P2 platform-web**
- `ac-ms-orchestration` → **S9 + D0** (worker vs deploy)
- `ac-ms-vcs` → **not a repo**; document in **D0**
- `ac-be-url-shortener` UI → admin panel in **B5** or **B0** calling **S8** (tokens already in crashes console)
**ac-deploy ↔ ac-scripts:** D0 submodules P4 at pinned SHA; deploy-local scripts only for image entrypoints.
──────────────────────────────────────────────────────────────────────────────
H. ac-workspace sketch (optional — PO §5)
──────────────────────────────────────────────────────────────────────────────
**Purpose:** “I only work on mobile” or “I only work on tickets UI” — clone workspace, init selected submodules. **Not** for building everything in one Gradle/PHP tree.
```ini
# ac-workspace — example partial checkout
[submodule "mobile-android"]
path = clients/android
url = git://f0xx.org/ac/ac-mobile-android
[submodule "ms-issues"]
path = services/issues
url = git://f0xx.org/ac/ac-ms-issues
[submodule "be-issues"]
path = ui/issues
url = git://f0xx.org/ac/ac-be-issues
[submodule "deploy"]
path = deploy
url = git://f0xx.org/ac/ac-deploy
```
Developers without workspace clone individual repos directly.
──────────────────────────────────────────────────────────────────────────────
I. PO decisions (2026-06-18) — recorded
──────────────────────────────────────────────────────────────────────────────
| # | Question | Decision |
|---|----------|----------|
| 1 | Gitea org | **`ac`** |
| 2 | Architecture | **Full microservice** end-state (phased delivery) — §K |
| 3 | ac-scripts | **Separate repo**; **ac-deploy submodules/invokes it** |
| 4 | Public URLs | **Change desired** — path-per-service (§M); not frozen under `/crashes/?view=` |
| 5 | Meta repo | **`ac-workspace` optional** — selective clone only; **not** build-all-in-one; §N OTA |
| 6 | Cloud | **No K8s requirement** — start VM lift-and-shift (§O); K8s later optional |
──────────────────────────────────────────────────────────────────────────────
J. Microservice vs modular monolith — comparison (PO requested)
──────────────────────────────────────────────────────────────────────────────
| Criterion | Full microservice (PO choice) | Modular monolith (single ac-be-console) |
|-----------|------------------------------|----------------------------------------|
| **Independent deploy** | Deploy tickets without touching issues | One PHP deploy for any UI fix |
| **Team scaling** | Different owners per repo/service | Single codebase, simpler grep |
| **Failure isolation** | Ticket MS down ≠ crash ingest | Shared FPM pool — blast radius larger |
| **CI speed** | Small repos, faster per-service tests | One `./gradlew`-scale PHP test suite |
| **Auth/session** | Needs **ac-ms-identity** + edge cookie early | Already works (`ac_crash_sess`) |
| **DB** | Shared MariaDB OK initially; schema ownership per MS | One migration stream today |
| **Operational cost** | More nginx routes, more images, more logs | One BE tree to sync ([INFRA.md](../INFRA.md)) |
| **PO microservice learning curve** | Higher — mitigated by **ac-ms-template**, **D0** | Lower short-term |
| **Cloud path** | Maps 1:1 to VM roles then containers | Lift whole monolith first |
| **URL clarity** | Natural `/issues/`, `/tickets/` per service | Stuck on `?view=` without refactor |
| **Rollback** | Revert one service SHA | Revert one monolith commit |
| **Risk for alpha** | **Higher** if done before identity split | Lower — current prod pattern |
[AI]: PO choice is valid **if P3 (identity + platform) is non-negotiable first**. Without it, full MS split repeats the build-console/bootstrap.php class of failures across N repos.
──────────────────────────────────────────────────────────────────────────────
K. Service dependency graph (logical + structural + repo)
──────────────────────────────────────────────────────────────────────────────
```mermaid
flowchart TB
subgraph clients [Clients]
AND[ac-mobile-android]
IOS[ac-mobile-ios future]
HUB[ac-be-hub]
SS[ac-session-studio]
end
subgraph edge [Edge]
EDGE[ac-platform-edge]
end
subgraph core [Core platform MS]
ID[ac-ms-identity]
RB[ac-ms-rbac]
DV[ac-ms-devices]
end
subgraph domain [Domain MS]
IS[ac-ms-issues]
TK[ac-ms-tickets]
GR[ac-ms-graphs]
RA[ac-ms-remote-access]
UL[ac-ms-url-shortener]
BL[ac-ms-build]
OT[ac-ms-ota]
end
subgraph ui [Backend UI]
UI_I[ac-be-issues]
UI_T[ac-be-tickets]
UI_G[ac-be-graphs]
UI_R[ac-be-remote-access]
UI_A[ac-be-access]
UI_B[ac-be-builder]
end
AND -->|upload API| EDGE
AND -->|RA API| EDGE
AND -->|graph upload| EDGE
HUB --> EDGE
UI_I & UI_T & UI_G & UI_R & UI_A & UI_B --> EDGE
EDGE --> ID & IS & TK & GR & RA & UL & BL & OT & RB
ID --> RB
RB --> DV
IS --> DV
IS --> RB
TK --> RB
TK -.->|optional link| IS
GR --> ID
RA --> RB
RA --> DV
RA -.->|issue deep links| IS
BL --> ID
BL -->|git clone| GITEA[(Gitea infra)]
OT --> BL
OT --> AND
OT --> IOS
UL --> RB
```
**Dependency rules (enforce in DR/SPEC):**
| Service | Hard depends on | Soft / read depends on | Mobile / public API |
|---------|-----------------|------------------------|---------------------|
| ac-ms-identity | ac-platform-db (users schema) | ac-ms-notifications (mail) | register, login |
| ac-ms-rbac | identity (JWT/session validation) | — | — |
| ac-ms-devices | rbac (company scope) | identity | — |
| ac-ms-issues | devices, rbac | tags catalog | **upload.php** |
| ac-ms-tickets | rbac | issues (link ticket↔report) | — |
| ac-ms-graphs | identity | — | graph_upload |
| ac-ms-remote-access | rbac, devices | issues (device context URLs) | remote_access.php |
| ac-ms-url-shortener | rbac (API tokens) | — | s.f0xx.org |
| ac-ms-build | identity | gitea mirror | — |
| ac-ms-ota | build artifacts | mobile/ios SHAs | `/v0/ota/` |
| ac-be-hub | identity (session for nav) | **P2 platform-web** assets | — |
**Cross-cutting fixes vs PO draft:**
1. **Auth must be first MS** — every UI and API currently includes `bootstrap.php` → Auth.
2. **Devices before issues** — upload auto-registers device + company ([README](../examples/crash_reporter/backend/README.md) RBAC).
3. **Hub → crashes assets** — today `app_hub/index.php` loads `/crashes/assets/css/app.css`; extract **ac-platform-web** before splitting hub.
4. **Builder → same DB users** — build_console uses `androidcast_crashes.users`; cannot split S9 before S1.
5. **Short links admin** — UI in monolith, API in submodule; UI calls S8, not duplicate repo **ac-be-url-shortener**.
6. **Graphs** — already separate nginx location `/graphs/` ([apps.conf](../../examples/crash_reporter/backend/nginx/fe/apps.conf)) — good MS boundary.
**Suggested new intermediates (actors):**
| Service | Why |
|---------|-----|
| **ac-ms-identity** | Single login/register/2FA; avoids 8 copies of Auth.php |
| **ac-ms-devices** | Shared by issues, RA, graphs device keys |
| **ac-platform-edge** | One nginx config for PO’s URL map + SSO cookie |
| **ac-ms-ota** | PO multi-repo Android+iOS same version (§N) |
| **ac-ms-notifications** | Decouple SMTP from identity (task 2.x mail frozen) |
| **ac-ms-attachments** (later) | Ticket files → object store when > local disk |
──────────────────────────────────────────────────────────────────────────────
L. Public URL migration map (PO §4)
──────────────────────────────────────────────────────────────────────────────
Base prefix (discussion): keep **`/app/androidcast_project/`** for alpha compatibility, or shorten to **`/app/ac/`** later. Below uses PO’s intent under current prefix.
| Today | Target path | MS / UI | Notes |
|-------|-------------|---------|-------|
| `/crashes/?view=reports` | **`/issues/`** | B1 → S4 | “Issue tracker” / crash reports list |
| `/crashes/?view=report&id=N` | **`/issues/N`** or `/issues/report/N` | B1 | Detail |
| `/crashes/?view=tickets` | **`/tickets/`** | B2 → S5 | Tickets catalog |
| `/crashes/?view=ticket&id=N` | **`/tickets/N`** | B2 | Detail |
| `/crashes/?view=rbac` | **`/access/`** | B5 → S2 | RBAC admin |
| `/crashes/?view=remote_access` | **`/remote-access/`** | B4 → S7 | RA admin |
| `/crashes/?view=short_links` | **`/short-links/`** or hub panel | B0/B5 → S8 | API stays on s.f0xx.org |
| `/crashes/?view=home` | **`/`** hub or `/console/` | B0 | Retire crashes “home” |
| `/crashes/api/upload.php` | **`/issues/api/upload`** or `/api/v1/issues/upload` | S4 | Update `BackendEndpoints.java`, `CrashSettings.java` |
| `/crashes/api/*` (tickets) | **`/tickets/api/*`** | S5 | |
| `/graphs/` | **`/graphs/`** (keep) | B3 → S6 | Already separate vhost block |
| `/build/` | **`/build/`** (keep) | B6 → S9 | |
| `/` (hub) | **`/`** (keep) | B0 | |
| `/git/` | **`/git/`** (keep) | Gitea | Not a MS |
| `/v0/ota/` | **`/v0/ota/`** (keep) | S10 | |
**Migration mechanics:** ac-platform-edge 301 redirects old URLs ≥ 1 release; mobile app legacy URL rewrite already in `BackendEndpoints.normalizeUploadUrl()`.
**Naming note:** Today “reports” in code = **issues/crashes** in product language — align UI strings when moving to `/issues/`.
──────────────────────────────────────────────────────────────────────────────
M. ac-workspace + OTA multi-repo versioning (PO §5)
──────────────────────────────────────────────────────────────────────────────
**ac-workspace is NOT:**
- A single Gradle/Composer build root
- Required for CI (each repo has own pipeline)
**ac-workspace IS:**
- Optional manifest: “which repos at which SHA for a release train”
- Convenience for developers who want `clients/android` + `services/issues` side by side
**OTA same version, different client SHAs (Android + iOS):**
`ac-ms-ota` channel manifest example:
```json
{
"channel": "staging",
"version": "00.02.00.00",
"artifacts": {
"android": { "repo": "ac-mobile-android", "git_sha": "abc123", "apk_url": "..." },
"ios": { "repo": "ac-mobile-ios", "git_sha": "def456", "ipa_url": "..." }
}
}
```
Builder (S9) triggers per-repo builds; **S10** assembles manifest. Mobile apps read **`version`** for UX; **`git_sha`** per platform for support.
──────────────────────────────────────────────────────────────────────────────
N. Cloud & VM strategy (PO §6 — no K8s experience)
──────────────────────────────────────────────────────────────────────────────
**Recommendation: Phase 1 = VM lift-and-shift** (matches cluster0 lab, scaling SPEC §6.3, PO skill set).
| Stage | Where | Pattern |
|-------|-------|---------|
| **Now** | FE + BE Alpine | nginx + PHP-FPM + MariaDB ([INFRA.md](../INFRA.md)) |
| **Lab** | cast01–03 | GTID MariaDB, NFS config only, apps local ([cluster0 ARCHITECTURE](../../orchestration/sim/cluster0/ARCHITECTURE.md)) |
| **Cloud phase A** | 4–6 VMs (be-web, be-data, be-ops, be-build) | Same as scaling table; **docker only on build** |
| **Cloud phase B** | Managed DB (RDS-class) | Apps use DSN secret; **ac-platform-db** migrations as job |
| **Cloud phase C** (optional) | K8s / compose swarm | **Not required** — introduce when MS count > ~8 and PO wants auto-scale |
**ac-deploy deliverables for cloud:** Terraform optional; minimum **Ansible + docker-compose fragments per VM role** + env templates. Each MS = one PHP-FPM pool or one container behind edge nginx.
**Inter-service calls (PO poor on microservices):** Start with **HTTP on localhost/private VLAN** (same BE host), not service mesh. Edge nginx routes public paths; internal `127.0.0.1:900x` or docker network. OpenAPI spec in **ac-docs/openapi/** is the contract.
──────────────────────────────────────────────────────────────────────────────
O. DR readiness checklist
──────────────────────────────────────────────────────────────────────────────
- [x] Map monolith → full MS catalog (§G)
- [x] PO decisions recorded (§I)
- [x] Comparison table (§J)
- [x] Dependency graph + fixes (§K)
- [x] URL migration map (§L)
- [x] Workspace / OTA model (§M)
- [x] Cloud recommendation (§N)
- [ ] PO confirms URL prefix (`/app/androidcast_project/` vs `/app/ac/`) — **locked: full prefix**
- [ ] PO confirms **ac-ms-identity** as P3 gate — **approved**
- [x] Write `docs/DRs/20260618_repos_reorganizing.md` + PDF
- [ ] Update gitea migrate scripts for org **`ac`**
- [ ] Migration runbook P0–P7
[AI]: **Create Gitea repos from §G table in waves:** Wave 1 = P4,P5,D0,C2,S8; Wave 2 = S1,S2,S3; Wave 3 = S4,S5 + B1,B2; then edge + URL migration.

98
drafts/20260618_repos_reorganizing_r0000.txt Normal file
View File

@@ -0,0 +1,98 @@
Project repository and structure reorganization draft
milestone: R0-pre
severity: high
state: pre-implementation
author: Anton Afanasyeu
date: 2026-06-18
The problem:
The project looks like monolith. For better and more atomic support we need to reorganize the whole project repo structure
The what:
Current project structure
Project | Placement | Repo | State | Description
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
AndroidCast all in one | repo root | git://f0xx.org/android_cast | monolithic | all-in-one project, needs to be splitted
libvpx | third-party/libvpx | https://chromium.googlesource.com/webm/libvpx | 3rdparty exported submodule | VPx video codecs
opus | third-party/opus | https://github.com/xiph/opus | 3rdparty exported submodule | Opus voice codec
speex | third-party/speex | https://github.com/xiph/speex/ | 3rdparty exported submodule | Speex voice codec
wireguard | third-party/wireguard-android | https://git.zx2c4.com/wireguard-android | 3rdparty exported submodule | wireguard for android
url-shortener | backend/url-shortener | git://f0xx.org/androidcast_project/url-shortener | part of AndroidCast backend | url shortener standalone backend
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The how:
Next hope project structure
AndroidCast project - all in one | git://f0xx.org/ac
|-- AndroidCast - common | @common
|-- AndroidCast - mobile | @mobile
|-- AndroidCast - BE | @backend
... TBD
AndroidCast project | Placement (@ - submodule) | Description | Repo
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
AndroidCast - common | @common | project all common | git://f0xx.org/ac/ac-common
|-- AndroidCast - docs | @docs | project docs, BE, MS, mobile, web, everything | git://f0xx.org/ac/ac-docs
|-- AndroidCast - scripts | @scripts | common project scripts | git://f0xx.org/ac/ac-scripts
AndroidCast - mobile | @mobile | mobile app sources | git://f0xx.org/ac/ac-mobile
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
AndroidCast - Session Studio | @session-studio | offline analytic tools | git://f0xx.org/ac/ac-session-studio
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
AndroidCast - BE - MS | @backend | backend microservices common base (UI) | git://f0xx.org/ac/ac-be-microservice
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
AndroidCast - MS | @microservices | microservice commons | git://f0xx.org/ac/ac-microservice
|-- AndroidCast - common | @common | | git://f0xx.org/ac/ac-common
... TBD
AndroidCast BE microservices and their derivatives (OpenAPI providers, any REST API services for web and mobile apps)
AndroidCast - MS - template <- AndroidCast - MS (@ means suibmodule) |git://f0xx.org/ac/ac-microservice/ac-ms-template
| -- AndroidCast - common | @common | project all common | git://f0xx.org/ac/ac-common
| -- AndroidCast - MS - common | @ms-common | microservices common | git://f0xx.org/ac/ac-microservice/ac-ms-common
|-- AndroidCast - MS - builder | @ms-builder | builder microservice | git://f0xx.org/ac/ac-microservice/ac-ms-builder
|-- AndroidCast - MS - VPN | @ms-vpn | VPN microservice | git://f0xx.org/ac/ac-microservice/ac-ms-vpn
|-- AndroidCast - MS - VPN - RSSH | @ms-vpn-rssh | VPN RSSH microservice | git://f0xx.org/ac/ac-microservice/ac-ms-vpn-rssh
|-- AndroidCast - MS - VPN - WireGuard | @ms-vpn-wireguard | VPN WG microservice | git://f0xx.org/ac/ac-microservice/ac-ms-vpn-wireguard
|-- AndroidCast - MS - VCS | @ms-vcs | VCS microservice (gitea) | git://f0xx.org/ac/ac-microservice/ac-ms-vcs
|-- AndroidCast - MS - URL shortener | @ms-url-shortener | URL shortener microservice | git://f0xx.org/ac/ac-microservice/ac-ms-url-shortener
|-- AndroidCast - MS - analytics | @ms-analytics | analytics microservice | git://f0xx.org/ac/ac-microservice/ac-ms-analytics
|-- AndroidCast - MS - orchestration | @ms-orchestration | orchestration microservice | git://f0xx.org/ac/ac-microservice/ac-ms-orchestration
|-- AndroidCast - MS - tracker | @ms-tracker | issues & tickets tracker microservice | git://f0xx.org/ac/ac-microservice/ac-ms-tracker
|-- AndroidCast - MS - RBAC | @ms-rbac | RBAC microservice | git://f0xx.org/ac/ac-microservice/ac-ms-rbac
... TBD
Sub-project of AndroidCast BE/MS, mostly Web UI stuff, but also should be OpenAPI-compatible and provide services to the mobile app (@ means suibmodule)
AndroidCast - BE - common <- AndroidCast - BE - MS | git://f0xx.org/ac/ac-backend
|-- AndroidCast - BE - vcs (UI) | @be-vcs | VCS frontend, former gitea, backend UI | git://f0xx.org/ac/ac-backend/ac-be-vcs
|-- AndroidCast - MS - vcs | @ms-vcs | VCS microservice (gitea) | git://f0xx.org/ac/ac-microservice/ac-ms-vcs
|-- AndroidCast - BE - console (UI) | @be-console | git://f0xx.org/ac/ac-backend/ac-be-console | primary console, ./examples/build_console ? check if it is
|-- AndroidCast - MS - common | @ms-common | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
|-- AndroidCast - BE - landing (UI) | @be-landing | git://f0xx.org/ac/ac-backend/ac-be-landing | landing page(s), now in ./examples/app_hub
|-- AndroidCast - MS - common | @ms-common | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
|-- AndroidCast - BE - issues (UI) | @be-issues | git://f0xx.org/ac/ac-backend/ac-be-issues | issues tracker UI, now is in ./examples somewhere
|-- AndroidCast - MS - tracker | @ms-tracker | git://f0xx.org/ac/ac-microservice/ac-ms-tracker | issues tracker microservice shared base, now is in ./examples somewhere
|-- AndroidCast - BE - tickets (UI) | @be-tickets | git://f0xx.org/ac/ac-backend/ac-be-tickets | tickets UI, now is in ./examples somewhere
|-- AndroidCast - MS - tracker | @ms-tracker | git://f0xx.org/ac/ac-microservice/ac-ms-tracker | issues tracker microservice shared base, now is in ./examples somewhere
|-- AndroidCast - BE - graphs (UI) | @be-graphs | git://f0xx.org/ac/ac-backend/ac-be-graphs | graphs UI, now somewhere in ./examples
|-- AndroidCast - MS - analytics | @ms-analytics | git://f0xx.org/ac/ac-microservice/ac-ms-analytics | analytics base microservice
|-- AndroidCast - BE - analytics (UI) | @be-analytics | git://f0xx.org/ac/ac-backend/ac-be-analytics | analytics and statistics web UI
|-- AndroidCast - MS - analytics | @ms-analytics | git://f0xx.org/ac/ac-microservice/ac-ms-analytics | analytics base microservice
|-- AndroidCast - BE - remote access (UI) | @be-remote-access | git://f0xx.org/ac/ac-backend/ac-be-remote-access | remote access (now for vpn, somewhere in ./examples now) UI
|-- AndroidCast - MS - common | @ms-commmon | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
|-- AndroidCast - MS - VPN | @ms-vpn | git://f0xx.org/ac/ac-microservice/ac-ms-vpn | VPN MS as a dependency
|-- AndroidCast - MS - RBAC | @ms-rbac | @ms-rbac | git://f0xx.org/ac/ac-microservice/ac-ms-rbac | RBAC MS as a dependency
|-- AndroidCast - BE - URL shortener (UI) | @be-url-shortener | git://f0xx.org/ac/ac-backend/ac-be-url-shortener | URL shortener UI, now is in separate submodule
|-- AndroidCast - MS - URL shortener | @ms-url-shortener | git://f0xx.org/ac/ac-microservice/ac-ms-url-shortener | URL shortener microservice, now is in separate submodule
|-- AndroidCast - BE - builder (UI) | @be-builder | git://f0xx.org/ac/ac-backend/ac-be-builder | orchestration builder UI, now in ./orchestration
|-- AndroidCast - MS - orchestration | @ms-orchestration | git://f0xx.org/ac/ac-microservice/ac-ms-orchestration | orchestration microservice, now in ./orchestration
|-- AndroidCast - BE - RBAC (UI) | @be-rbac | git://f0xx.org/ac/ac-backend/ac-be-rbac | RBAC backend, now mixed within ./examples somewhere
|-- AndroidCast - MS - common | @ms-common | git://f0xx.org/ac/ac-microservice/ac-ms-common | microservice commons
|-- AndroidCast - MS - RBAC | @ms-rbac | git://f0xx.org/ac/ac-microservice/ac-ms-rbac | RBAC microservice
|-- AndroidCast - BE - OTA | @be-ota | git://f0xx.org/ac/ac-backend/ac-be-ota | OTA BE UI
|-- AndroidCast - MS - OTA | @ms-ota | git://f0xx.org/ac/ac-microservice/ac-ms-ota | OTA microservice, now intermixed in examples/ota and other commons
... TBD
ms-common (microservices common) needs a sharp eye, it may share project sensitive parts, like common stuff such as shared web sessions, DB access (or should be separated to @be-db-service/@ac-db-service sort of)