e346b54e0f
Implements PROMPT.md / docs/REVIEW_PIPELINE.md Phase 2:
- internal/llm/ollama.go — real Ollama provider:
- HealthCheck probes /api/tags + a 1-token completion + a JSON-mode
probe ({"ok": true} round-trip), populating the model-doctor.json
schema documented in docs/LOCAL_MODEL_SETUP.md
- Complete + CompleteJSON via /api/chat with stream=false
- think=false set for ALL completions (qwen3.5:latest is reasoning-
capable but the inner-loop hot path wants direct answers, not
reasoning traces consuming the token budget — same finding as
the Lakehouse-Go chatd 2026-04-30 wave)
- internal/llm/review.go — Reviewer wrapper:
- 2-attempt flow: prompt → parse → repair-prompt → parse
- Strict JSON shape enforced; markdown fences stripped before parse
- Severity normalized to enum; out-of-range confidence clamped
- Per-file chunking (file-level for v0; function-level Phase D+)
- Bounded by review-profile max_file_bytes + max_llm_chunk_chars
- pipeline.go — Phase 2 wired between static scan + report gen:
- --enable-llm flag opts in (off by default — static-only is
cheaper and faster)
- Raw output ALWAYS saved to llm-findings.raw.json (forensics)
- Normalized findings → llm-findings.normalized.json
- LLM findings merged into the report findings list (sourced
"llm" so consumers can filter)
- Receipts honestly mark phase status: "ok" | "degraded" | "skipped"
- cli model doctor — real probes replace the Phase A stub.
Verified:
- model doctor: status="ok" with qwen3.5:latest + qwen3:latest both
loaded, basic_prompt_ok=true, json_mode_ok=true
- insecure-repo with --enable-llm: 9 LLM findings; qwen3.5 correctly
flagged SQLi, RCE, hardcoded credentials as critical with verbatim
evidence; 27s wall for 3 chunks
- clean-repo with --enable-llm: 0 LLM findings, 4 parsed chunks, 2.8s
- self-review with --enable-llm: 77 LLM findings + 83 static; 3 of
~30 chunks needed retry (PROMPT.md, REPORT_SCHEMA.md,
SCRUM_TEST_TEMPLATE.md — all eventually parsed); 5min wall
go vet + go test -short clean. Fixture stray.go now `package fixture`
so go-tooling doesn't choke on the orphan.
Phase D (validator cross-check) + Phase E (memory + diff/rules
subcommands) remain.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
97 lines
5.0 KiB
Markdown
97 lines
5.0 KiB
Markdown
# Scrum Test — insecure-repo
|
|
|
|
**Generated:** 2026-04-30T06:07:13.931830669Z
|
|
**Branch:** main · **Commit:** 70d68757f78ea722bf24585c73120c09d82c4fea
|
|
|
|
## Verdict
|
|
|
|
**blocked** — critical-severity finding present. See Confirmed Risks; rotate any leaked credentials, then re-run.
|
|
|
|
## Evidence
|
|
|
|
- repo path: `/home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
|
|
- file count: 4
|
|
- languages: Go (2), JavaScript (1)
|
|
- dependency manifests: 0 ()
|
|
- test files/dirs: 0
|
|
- LLM review: **skipped** (Phase C not implemented OR provider unavailable; see model-doctor.json)
|
|
|
|
## Confirmed Risks
|
|
|
|
| Severity | File:Line | Title | Evidence |
|
|
|---|---|---|---|
|
|
| high | `.env` | Environment file in source tree | `filename=.env` |
|
|
| medium | `.` | No tests found | `No test files or test directories detected (looked for *_test.go, *.test.{js,ts}, test_*.py, tests/, spec/)` |
|
|
|
|
## Suspected Risks
|
|
|
|
Each entry is a static-scan regex hit awaiting validation (Phase D / LLM cross-check).
|
|
|
|
| Severity | File:Line | Title | Evidence |
|
|
|---|---|---|---|
|
|
| critical | `src/handler.go:23` | Possible secret committed to source | `const API_KEY = "sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV"` |
|
|
| critical | `src/handler.go:23` | Possible secret committed to source | `const API_KEY = "sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV"` |
|
|
| critical | `src/server.js:5` | Possible secret committed to source | `const AWS_KEY = "AKIAIOSFODNN7EXAMPLE";` |
|
|
| high | `src/handler.go:14` | Raw SQL interpolation | `q := fmt.Sprintf("SELECT * FROM users WHERE name = '%s'", name)` |
|
|
| high | `src/handler.go:19` | Shell command execution | `exec.Command("bash", "-c", cmd).Run()` |
|
|
| high | `src/server.js:2` | Wildcard CORS | `res.setHeader("Access-Control-Allow-Origin", "*");` |
|
|
| medium | `src/handler.go:10` | Hardcoded absolute path | `const HARDCODED_PATH = "/home/profit/secrets/key.pem"` |
|
|
| medium | `src/handler.go:11` | Hardcoded private-network IP | `const SERVER_IP = "192.168.1.176"` |
|
|
| medium | `src/huge.go:1-901` | Large file | `901 lines (limit: 800)` |
|
|
| medium | `src/server.js:7` | Mutation route in file with no visible auth | `app.post("/api/users", function(req, res) { /* no auth */ });` |
|
|
| medium | `src/server.js:8` | Mutation route in file with no visible auth | `app.delete("/api/admin", function(req, res) { /* no auth */ });` |
|
|
| low | `src/handler.go:22` | TODO/FIXME comment | `// FIXME: hardcoded creds` |
|
|
| low | `src/handler.go:9` | TODO/FIXME comment | `// TODO: rotate this and move to env` |
|
|
| low | `src/server.js:1` | TODO/FIXME comment | `// HACK: open CORS for now` |
|
|
|
|
## Blocked Checks
|
|
|
|
- LLM review (Phase 2 in REVIEW_PIPELINE.md). Reason: provider unavailable or stub. Next command: `review-harness model doctor`
|
|
|
|
## Sprint Backlog
|
|
|
|
**Sprint 0 — Reproducibility Gate**
|
|
|
|
- Wire `just verify` (or equivalent) to run the static checks before every commit/PR.
|
|
- Add a CI step that fails on `critical` findings.
|
|
- Triage the 16 findings emitted by this run; mark each as accepted / blocking / dismiss-with-reason.
|
|
|
|
**Sprint 1 — Trust Boundary Gate**
|
|
|
|
- Resolve every `critical` and `high` finding before non-loopback deploy.
|
|
- Confirm auth posture for any mutation endpoint flagged as exposed.
|
|
- Replace raw SQL interpolation with parameterized queries.
|
|
|
|
**Sprint 2 — Memory Correctness Gate**
|
|
|
|
- (Phase E) Wire append-only `.memory/` writes for known-risks + fixed-patterns.
|
|
- Add a regression test that re-runs the harness and asserts no regression in confirmed-finding count.
|
|
|
|
**Sprint 3 — Agent Loop Reality Gate**
|
|
|
|
- (Phase C) Wire local-Ollama LLM review.
|
|
- (Phase D) Validator pass cross-checks every LLM finding against repo evidence.
|
|
|
|
**Sprint 4 — Deployment Gate**
|
|
|
|
- Ship the harness as a single static binary (`go build -o review-harness`).
|
|
- Document operator runbook (model setup, profile editing, output retention).
|
|
|
|
## Acceptance Gates
|
|
|
|
Each gate must be testable. Format: command + verifiable post-condition.
|
|
|
|
1. **Reproducibility:** `review-harness repo .` exits 0; `reports/latest/repo-intake.json` exists with non-zero `file_count`.
|
|
2. **No false positives on a clean fixture:** `review-harness repo tests/fixtures/clean-repo` produces zero `confirmed` findings.
|
|
3. **Every documented static check fires on the insecure fixture:** `jq '[.findings[] | .check_id] | unique | length' reports/latest/static-findings.json` ≥ 8.
|
|
4. **Receipts are honest about degraded phases:** `jq '[.phases[] | select(.status == "degraded")]' reports/latest/receipts.json` lists every skipped/stubbed phase.
|
|
5. **Critical findings block production deploy:** at least one critical finding is currently present; resolve before deploy.
|
|
|
|
## Next Commands
|
|
|
|
1. Open the risk register: `cat reports/latest/risk-register.md`
|
|
2. Triage every `critical` finding; rotate any leaked credentials immediately.
|
|
- Probe the model provider: `review-harness model doctor`
|
|
- Re-run after fixes: `review-harness repo /home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
|
|
- Generate the full Scrum bundle: `review-harness scrum /home/profit/share/local-review-harness-full-md/tests/fixtures/insecure-repo`
|