monitoring: fix false HttpEndpointDown alert for self-signed FE cert

- blackbox.yml: add http_2xx_insecure module (TLS skip) for FE probing
- prometheus.yml: split blackbox_http (internal HTTP) vs blackbox_https_external
  (external HTTPS targets using insecure module); remove notify.f0xx.org probe
  until FE nginx proxies it
- service_alerts.yml: HttpEndpointDown/SlowResponse cover both jobs via regex
- TODO: replace http_2xx_insecure with strict module once FE gets LE cert

Co-authored-by: Cursor <cursoragent@cursor.com>

monitoring/cast04/prometheus.yml

@@ -52,7 +52,7 @@ scrape_configs:
         target_label: instance
         replacement: 'artc0'
 
-  # ── blackbox_exporter — HTTP probes ─────────────────────────────────────────
+  # ── blackbox_exporter — internal HTTP probes (no TLS issues) ────────────────
   - job_name: 'blackbox_http'
     metrics_path: /probe
     params:
@@ -62,7 +62,27 @@ scrape_configs:
           - 'http://10.7.16.236/app/androidcast_project/'          # BE main app (cast01)
           - 'http://10.7.16.236/app/androidcast_project/issues/'   # crash reporter
           - 'http://10.7.16.239:3000/api/health'                   # Grafana health
-          - 'https://apps.f0xx.org/app/androidcast_project/'       # external FE probe
+        labels:
+          env: 'production'
+    relabel_configs:
+      - source_labels: [__address__]
+        target_label: __param_target
+      - source_labels: [__param_target]
+        target_label: instance
+      - target_label: __address__
+        replacement: 'localhost:9115'
+
+  # ── blackbox_exporter — external HTTPS probes (self-signed cert on FE) ──────
+  # Uses insecure_skip_verify until a valid LE cert is deployed on apps.f0xx.org
+  # TODO: switch back to http_2xx once cert is fixed (FE monstro: certbot --nginx)
+  - job_name: 'blackbox_https_external'
+    metrics_path: /probe
+    params:
+      module: [http_2xx_insecure]
+    static_configs:
+      - targets:
+          - 'https://apps.f0xx.org/app/androidcast_project/'       # external FE (self-signed cert)
+          # notify.f0xx.org: add here once FE nginx proxies notify.f0xx.org → cluster
         labels:
           env: 'production'
     relabel_configs: