# LLM Team UI — Product Requirements (for scrum-master audits)

This doc is the anchor the lakehouse scrum-master uses when auditing
this codebase. It is the "ground truth" — every scrum finding is
evaluated against what this doc says the system should be. Written
2026-04-24 to give the scrum pipeline a target surface; keep it tight
and aligned with what the code actually tries to do. Last reconciled
2026-06-20 against commit `ea0ba33` — the mode registry and unknown-mode
handling described below were verified against the live source on that date.

## What LLM Team UI is

Flask-based web UI that lets a user configure and run multi-model
"teams" — pipelines of LLMs (OpenRouter + Ollama Cloud + local Ollama)
that collaborate on a task with roles like executor, reviewer, critic,
sentinel. Lives at devop.live:5000 behind nginx. Main file
`llm_team_ui.py` handles HTTP routes, session auth, model orchestration,
SSE streaming. Backed by PostgreSQL for users + runs + audit.

## Core invariants (scrum should flag if violated)

1. **Every mode accepted by `/api/run` must be in the `_VALID_MODES`
   registry and have a matching `run_<mode>` handler.** As of 2026-06-20
   there are 24 registered modes (brainstorm, pipeline, debate, validator,
   roundrobin, redteam, consensus, codereview, ladder, tournament,
   evolution, blindassembly, staircase, drift, mesh, hallucination,
   timeloop, research, eval, extract, refine, adaptive, deep_analysis,
   distill), each with a `run_<mode>` function. `run_team()` validates
   `mode` against `_VALID_MODES` before opening the SSE stream and rejects
   anything else with HTTP 400 (see invariant below). The registry is kept
   in sync with the handlers manually — adding a `run_<mode>` without
   listing it in `_VALID_MODES` (or vice-versa) is the hard fail to flag.
2. **Unknown modes must fail fast with HTTP 400 + the valid-mode list.**
   `run_team()` returns `{"error": "Unknown mode: X", "valid_modes": [...]}`
   with status 400 *before* streaming. Returning 200-with-error-body (the
   old behavior) is a hard fail — it forces callers to parse SSE to learn
   the request was bad.
3. **All authenticated routes must call `@login_required` decorator.**
   The SESSION_COOKIE_HTTPONLY + SAMESITE=Lax + SECURE defaults must hold
   (SECURE added 2026-06-20; the app sits behind nginx TLS). Any route
   that mutates state without auth is a hard fail.
4. **Security logging goes to `/var/log/llm-team-security.log`** via
   the `sec_log` logger. Fail2ban watches this file. Any new auth path
   must emit there on failure.
5. **PostgreSQL connections come from `DB_URL` env.** Direct literal
   connection strings are a hard fail.
6. **SMTP alerts via `send_security_alert(subject, body)`** — never
   log secrets in the body.
7. **SSE streams must flush incrementally** — if an endpoint returns
   all-at-once, it should not claim SSE in the Content-Type.
8. **Password storage: bcrypt only.** Any plain-text / SHA256 password
   path is a hard fail.
9. **Rate limiting enforced** on /login, /register, and model-invocation
   endpoints. No unbounded loops that invoke paid APIs.

## Known gaps (scrum can work on these)

- **~~Only `extract` mode is registered.~~ RESOLVED (2026-06-20).** The
  mode set grew from one handler to 24 (`run_brainstorm` … `run_distill`),
  all listed in `_VALID_MODES` and dispatched by `run_team()`. Note the
  mode named in code is `codereview` (not `code_review`); callers using the
  underscored name still get a 400. Historical context: observer.ts in
  lakehouse was escalating to `/api/run?mode=code_review` and silently hit
  "Unknown mode" for weeks (see lakehouse memory
  `reference_llm_team_modes.md`) — the underscore-vs-no-underscore mismatch
  is the remaining footgun, not the missing handler.
- **File is 13K lines.** Cohesive modules (auth, teams, runs, alerts,
  SSE, templates) should be separated — but that's a refactor, out of
  scope for individual scrum-master reviews.
- **Audit log schema is thin.** The PostgreSQL `runs` table stores the
  fact of a run but not fine-grained per-step outcomes. Downstream
  correlation is manual.
- **Templates inline in Python as `render_template_string(...)`.**
  Makes HTML review impossible via the scrum pipeline — HTML is hidden
  inside Python string literals.

## Change proposal axis (what the scrum should optimize for)

1. **~~Every Unknown-mode endpoint should either be registered or return
   404/not-a-mode.~~ DONE (2026-06-20).** `run_team()` now returns HTTP 400
   (not 404 — chosen so it reads as "bad request / wrong mode" rather than
   "route missing") with a `valid_modes` list, before streaming. Callers
   can now distinguish a wrong call from a missing mode. Remaining: see the
   `codereview` vs `code_review` naming footgun in Known gaps.
2. **Auth decorators should be typed** — `@login_required` is a
   runtime check; adding an `Annotated[User, Depends(auth)]`-style
   type hint (possible via Flask's `g` object) would make review
   mechanical.
3. **Security logging should be structured** — current `sec_log.warning(
   f"...")` loses fields. Move to `logger.warning(msg, extra={...})`
   so fail2ban and external tools can parse.
4. **Secret handling** — `FLASK_SECRET` in env is correct, but the
   fallback to `~/.llm-team-secret` means dev mode silently generates
   a persistent secret. Not bad, but should log once.

## How scrum-master audits this repo

The lakehouse scrum pipeline is repo-agnostic via env vars. See
`/root/llm-team-ui/CLAUDE.md` for the exact runbook.