Claude (review-harness setup) f3ee4722a8 Phase A + B (MVP) — local review harness

Implements the MVP cutline from the planning artifact:
- Phase A: skeleton + CLI dispatch + provider interface + stub model doctor
- Phase B: scanner + git probe + 12 static analyzers + reporters + pipeline
- Phase B fixtures: clean-repo, insecure-repo, degraded-repo

12 static analyzers per PROMPT.md "Suggested Static Checks For MVP":
hardcoded_paths, shell_execution, raw_sql_interpolation, broad_cors,
secret_patterns, large_files, todo_comments, missing_tests,
env_file_committed, unsafe_file_io, exposed_mutation_endpoint,
hardcoded_local_ip.

Acceptance gates passing:
- B1 (intake produces accurate counts) ✓
- B2 (insecure fixture fires ≥8 distinct check_ids — actually 11/12) ✓
- B3 (clean fixture produces 0 confirmed findings — no false positives) ✓
- B4 (scrum mode produces all 6 required markdown + JSON reports) ✓
- B5 (receipts.json marks degraded phases honestly) ✓
- F  (self-review on this repo runs without crashing) ✓ — exit 66 (degraded
  because Phase C LLM review is hardcoded skipped)

Phases C (LLM review), D (validation cross-check), E (memory + diff +
rules subcommands) deferred per the cutline. The MVP delivers the
evidence-first path; LLM is purely additive.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-04-30 00:56:02 -05:00

1.6 KiB

Executable File

Raw Permalink Blame History

Local Model Setup

Purpose

The review harness should use local models first.

The first supported provider is Ollama.

The design must allow OpenAI-compatible local endpoints later.

Default Ollama Profile

provider: ollama
base_url: http://localhost:11434
model: qwen2.5-coder
fallback_model: llama3.1
timeout_seconds: 120
temperature: 0.1

Model Doctor Command

The harness must provide:

review-harness model doctor

Doctor Checks

The doctor command should test:

Ollama server availability
configured model availability
fallback model availability
basic prompt response
JSON response reliability
timeout behavior
degraded-mode behavior

Required Doctor Output

reports/latest/model-doctor.json

Required JSON Fields

{
  "provider": "ollama",
  "base_url": "http://localhost:11434",
  "primary_model": "",
  "fallback_model": "",
  "server_available": false,
  "primary_model_available": false,
  "fallback_model_available": false,
  "basic_prompt_ok": false,
  "json_mode_ok": false,
  "timeout_seconds": 120,
  "status": "ok|degraded|failed",
  "errors": []
}

Provider Interface

Do not hardcode Ollama into all logic.

Use a provider interface with these operations:

list_models()
complete(prompt, options)
complete_json(prompt, schema, options)
health_check()

Local Model Rules

temperature should default low for review tasks
prompts should request strict JSON where possible
raw model output must be saved for failed parse attempts
invalid model output must never be silently accepted
fallback model usage must be recorded

1.6 KiB Executable File Raw Permalink Blame History