profit/local-review-harness
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
local-review-harness/tests/fixtures/insecure-repo/reports/latest/static-findings.json
Claude (review-harness setup) e346b54e0f Phase C — local-Ollama LLM review wired end-to-end 
Implements PROMPT.md / docs/REVIEW_PIPELINE.md Phase 2:
- internal/llm/ollama.go — real Ollama provider:
  - HealthCheck probes /api/tags + a 1-token completion + a JSON-mode
    probe ({"ok": true} round-trip), populating the model-doctor.json
    schema documented in docs/LOCAL_MODEL_SETUP.md
  - Complete + CompleteJSON via /api/chat with stream=false
  - think=false set for ALL completions (qwen3.5:latest is reasoning-
    capable but the inner-loop hot path wants direct answers, not
    reasoning traces consuming the token budget — same finding as
    the Lakehouse-Go chatd 2026-04-30 wave)
- internal/llm/review.go — Reviewer wrapper:
  - 2-attempt flow: prompt → parse → repair-prompt → parse
  - Strict JSON shape enforced; markdown fences stripped before parse
  - Severity normalized to enum; out-of-range confidence clamped
  - Per-file chunking (file-level for v0; function-level Phase D+)
  - Bounded by review-profile max_file_bytes + max_llm_chunk_chars
- pipeline.go — Phase 2 wired between static scan + report gen:
  - --enable-llm flag opts in (off by default — static-only is
    cheaper and faster)
  - Raw output ALWAYS saved to llm-findings.raw.json (forensics)
  - Normalized findings → llm-findings.normalized.json
  - LLM findings merged into the report findings list (sourced
    "llm" so consumers can filter)
  - Receipts honestly mark phase status: "ok" | "degraded" | "skipped"
- cli model doctor — real probes replace the Phase A stub.

Verified:
- model doctor: status="ok" with qwen3.5:latest + qwen3:latest both
  loaded, basic_prompt_ok=true, json_mode_ok=true
- insecure-repo with --enable-llm: 9 LLM findings; qwen3.5 correctly
  flagged SQLi, RCE, hardcoded credentials as critical with verbatim
  evidence; 27s wall for 3 chunks
- clean-repo with --enable-llm: 0 LLM findings, 4 parsed chunks, 2.8s
- self-review with --enable-llm: 77 LLM findings + 83 static; 3 of
  ~30 chunks needed retry (PROMPT.md, REPORT_SCHEMA.md,
  SCRUM_TEST_TEMPLATE.md — all eventually parsed); 5min wall

go vet + go test -short clean. Fixture stray.go now `package fixture`
so go-tooling doesn't choke on the orphan.

Phase D (validator cross-check) + Phase E (memory + diff/rules
subcommands) remain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-30 01:13:39 -05:00

243 lines
9.0 KiB
JSON
Raw Blame History

{
"generated_at": "2026-04-30T06:07:13.951970576Z",
"findings": [
{
"id": "750676119e4a",
"title": "Environment file in source tree",
"severity": "high",
"status": "confirmed",
"file": ".env",
"evidence": "filename=.env",
"reason": ".env files commonly hold real secrets and should not be tracked. If this is a sample, rename to .env.example with placeholder values.",
"suggested_fix": "Rename to .env.example with placeholders; add .env to .gitignore; rotate any committed secrets.",
"source": "static",
"confidence": 0.9,
"check_id": "static.env_file_committed"
},
{
"id": "eb3c41b3a186",
"title": "Hardcoded absolute path",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "10",
"evidence": "const HARDCODED_PATH = \"/home/profit/secrets/key.pem\"",
"reason": "Absolute path encoded in source — couples the binary to one filesystem layout. Move to config or env var.",
"source": "static",
"confidence": 0.7,
"check_id": "static.hardcoded_paths"
},
{
"id": "5bf85ae888a0",
"title": "Shell command execution",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "19",
"evidence": "exec.Command(\"bash\", \"-c\", cmd).Run()",
"reason": "Direct subprocess/shell invocation. Confirm inputs are sanitized; prefer typed APIs over string-built commands.",
"source": "static",
"confidence": 0.6,
"check_id": "static.shell_execution"
},
{
"id": "3a198539c923",
"title": "Raw SQL interpolation",
"severity": "high",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "14",
"evidence": "q := fmt.Sprintf(\"SELECT * FROM users WHERE name = '%s'\", name)",
"reason": "SQL assembled via string formatting/concatenation rather than parameterized query. Verify inputs aren't user-controlled.",
"suggested_fix": "Use parameterized queries / prepared statements; pass values via driver placeholders, not string interpolation.",
"source": "static",
"confidence": 0.6,
"check_id": "static.raw_sql_interpolation"
},
{
"id": "9bc97c579efc",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "OpenAI/OpenRouter-shaped key detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "9bc97c579efc",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "23",
"evidence": "const API_KEY = \"sk-1234567890abcdefABCDEFGHIJKLMNOPQRSTUV\"",
"reason": "Hardcoded credential pattern detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "f3e510b70ec9",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "9",
"evidence": "// TODO: rotate this and move to env",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "f99cd5bb5f2c",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "22",
"evidence": "// FIXME: hardcoded creds",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "bb70e8e262d6",
"title": "Hardcoded private-network IP",
"severity": "medium",
"status": "suspected",
"file": "src/handler.go",
"line_hint": "11",
"evidence": "const SERVER_IP = \"192.168.1.176\"",
"reason": "RFC 1918 / link-local IP literal in source. Move to config so the binary isn't tied to one network.",
"source": "static",
"confidence": 0.7,
"check_id": "static.hardcoded_local_ip"
},
{
"id": "512b795dc551",
"title": "Large file",
"severity": "medium",
"status": "suspected",
"file": "src/huge.go",
"line_hint": "1-901",
"evidence": "901 lines (limit: 800)",
"reason": "File exceeds the configured size threshold. Long files are a refactor target — split by responsibility.",
"source": "static",
"confidence": 1,
"check_id": "static.large_files"
},
{
"id": "ef8bb39704d3",
"title": "Wildcard CORS",
"severity": "high",
"status": "suspected",
"file": "src/server.js",
"line_hint": "2",
"evidence": "res.setHeader(\"Access-Control-Allow-Origin\", \"*\");",
"reason": "Access-Control-Allow-Origin: * permits cross-origin reads from any domain. Narrow to an explicit allowlist unless this endpoint is intentionally public.",
"source": "static",
"confidence": 0.85,
"check_id": "static.broad_cors"
},
{
"id": "d3c2c5606e1d",
"title": "Possible secret committed to source",
"severity": "critical",
"status": "suspected",
"file": "src/server.js",
"line_hint": "5",
"evidence": "const AWS_KEY = \"AKIAIOSFODNN7EXAMPLE\";",
"reason": "AWS access key ID detected. If real, rotate immediately and move to a secret store.",
"suggested_fix": "Move secret to env var / secret manager; commit the .env.example with a placeholder; rotate the leaked credential.",
"source": "static",
"confidence": 0.75,
"check_id": "static.secret_patterns"
},
{
"id": "4a631055edd1",
"title": "TODO/FIXME comment",
"severity": "low",
"status": "suspected",
"file": "src/server.js",
"line_hint": "1",
"evidence": "// HACK: open CORS for now",
"reason": "Inline marker for deferred work. Audit whether the deferred concern is now blocking.",
"source": "static",
"confidence": 0.95,
"check_id": "static.todo_comments"
},
{
"id": "7ed1cab08825",
"title": "Mutation route in file with no visible auth",
"severity": "medium",
"status": "suspected",
"file": "src/server.js",
"line_hint": "7",
"evidence": "app.post(\"/api/users\", function(req, res) { /* no auth */ });",
"reason": "POST/PUT/DELETE/PATCH route registered in a file with no visible auth middleware. May still be auth'd at a higher layer — confirm.",
"source": "static",
"confidence": 0.4,
"check_id": "static.exposed_mutation_endpoint"
},
{
"id": "2b765c240c96",
"title": "Mutation route in file with no visible auth",
"severity": "medium",
"status": "suspected",
"file": "src/server.js",
"line_hint": "8",
"evidence": "app.delete(\"/api/admin\", function(req, res) { /* no auth */ });",
"reason": "POST/PUT/DELETE/PATCH route registered in a file with no visible auth middleware. May still be auth'd at a higher layer — confirm.",
"source": "static",
"confidence": 0.4,
"check_id": "static.exposed_mutation_endpoint"
},
{
"id": "4d59806aeb57",
"title": "No tests found",
"severity": "medium",
"status": "confirmed",
"file": ".",
"evidence": "No test files or test directories detected (looked for *_test.go, *.test.{js,ts}, test_*.py, tests/, spec/)",
"reason": "Repository has source code but no test surface. Refactoring or extending without test cover is high-risk.",
"source": "static",
"confidence": 0.95,
"check_id": "static.missing_tests"
}
],
"summary": {
"total": 16,
"confirmed": 2,
"suspected": 14,
"rejected": 0,
"critical": 3,
"high": 4,
"medium": 6,
"low": 3,
"by_source": {
"static": 16
},
"by_check": {
"static.broad_cors": 1,
"static.env_file_committed": 1,
"static.exposed_mutation_endpoint": 2,
"static.hardcoded_local_ip": 1,
"static.hardcoded_paths": 1,
"static.large_files": 1,
"static.missing_tests": 1,
"static.raw_sql_interpolation": 1,
"static.secret_patterns": 3,
"static.shell_execution": 1,
"static.todo_comments": 3
}
}
}
Reference in New Issue View Git Blame Copy Permalink