agent-governance/docs/CREDENTIALS_SETUP.md

# Production Credentials Setup Guide

> How to configure external integration credentials for Slack, GitHub, and PagerDuty.

## Overview

The Agent Governance System uses HashiCorp Vault for secure credential storage. Each integration has a dedicated secret path with specific required keys. The system also supports environment variable overrides for development and CI environments.

## Prerequisites

- Vault server running at `https://127.0.0.1:8200` (or `VAULT_ADDR`)
- Vault root token or appropriate access policy
- Admin access to Slack, GitHub, and PagerDuty accounts

## Credential Priority

The secrets manager loads credentials in this order:

1. **Environment variables** (highest priority)
2. **Vault secrets** (production)
3. **Mock values** (testing only, when `INTEGRATION_DRY_RUN=true`)

---

## Slack Integration

### Vault Path
```
secret/data/integrations/slack
```

### Required Keys

| Key | Description | Format |
|-----|-------------|--------|
| `slack_bot_token` | Bot User OAuth Token | `xoxb-...` |
| `slack_webhook_url` | Incoming Webhook URL | `https://hooks.slack.com/services/...` |

### How to Obtain

#### Bot Token
1. Go to [api.slack.com/apps](https://api.slack.com/apps)
2. Create a new app or select existing
3. Navigate to **OAuth & Permissions**
4. Under **Bot Token Scopes**, add:
   - `chat:write` - Send messages
   - `chat:write.public` - Send to public channels
   - `files:write` - Upload files (optional)
5. Click **Install to Workspace**
6. Copy the **Bot User OAuth Token** (`xoxb-...`)

#### Webhook URL
1. In your Slack app, go to **Incoming Webhooks**
2. Enable incoming webhooks
3. Click **Add New Webhook to Workspace**
4. Select the target channel
5. Copy the **Webhook URL**

### Store in Vault

```bash
# Set Vault address and token
export VAULT_ADDR="https://127.0.0.1:8200"
export VAULT_TOKEN=$(cat /opt/vault/init-keys.json | python3 -c "import sys,json; print(json.load(sys.stdin)['root_token'])")

# Store Slack credentials
vault kv put secret/integrations/slack \
    slack_bot_token="xoxb-YOUR-BOT-TOKEN" \
    slack_webhook_url="https://hooks.slack.com/services/YOUR/WEBHOOK/URL" \
    configured=true

# Or using curl (if vault CLI not available)
curl -sk -X POST \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
        "data": {
            "slack_bot_token": "xoxb-YOUR-BOT-TOKEN",
            "slack_webhook_url": "https://hooks.slack.com/services/YOUR/WEBHOOK/URL",
            "configured": true
        }
    }' \
    "$VAULT_ADDR/v1/secret/data/integrations/slack"
```

### Environment Variable Override

```bash
export SLACK_BOT_TOKEN="xoxb-YOUR-BOT-TOKEN"
export SLACK_WEBHOOK_URL="https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
```

### Verify Configuration

```bash
# Test with Python
python3 -c "
from integrations.slack import SlackIntegration
slack = SlackIntegration()
print(f'Enabled: {slack.enabled}')
print(f'Connection test: {slack.test_connection()}')
"
```

---

## GitHub Integration

### Vault Path
```
secret/data/integrations/github
```

### Required Keys

| Key | Description | Format |
|-----|-------------|--------|
| `github_token` | Personal Access Token or App Token | `ghp_...` or `ghs_...` |
| `github_webhook_secret` | Webhook signature secret | Random string |

### How to Obtain

#### Personal Access Token (Classic)
1. Go to [github.com/settings/tokens](https://github.com/settings/tokens)
2. Click **Generate new token (classic)**
3. Select scopes:
   - `repo` - Full repository access
   - `write:org` - Organization webhooks (if needed)
   - `admin:repo_hook` - Manage webhooks
4. Click **Generate token**
5. Copy the token immediately (shown only once)

#### Fine-Grained Personal Access Token (Recommended)
1. Go to [github.com/settings/tokens?type=beta](https://github.com/settings/tokens?type=beta)
2. Click **Generate new token**
3. Set token name and expiration
4. Select **Repository access** (specific repos or all)
5. Under **Permissions**, enable:
   - Contents: Read and write
   - Pull requests: Read and write
   - Commit statuses: Read and write
   - Webhooks: Read and write
6. Click **Generate token**

#### Webhook Secret
1. Generate a random secret:
   ```bash
   openssl rand -hex 32
   ```
2. When creating GitHub webhooks, use this secret for signature verification

### Store in Vault

```bash
# Generate webhook secret
WEBHOOK_SECRET=$(openssl rand -hex 32)

# Store GitHub credentials
vault kv put secret/integrations/github \
    github_token="ghp_YOUR-PERSONAL-ACCESS-TOKEN" \
    github_webhook_secret="$WEBHOOK_SECRET" \
    configured=true

# Or using curl
curl -sk -X POST \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -d "{
        \"data\": {
            \"github_token\": \"ghp_YOUR-PERSONAL-ACCESS-TOKEN\",
            \"github_webhook_secret\": \"$WEBHOOK_SECRET\",
            \"configured\": true
        }
    }" \
    "$VAULT_ADDR/v1/secret/data/integrations/github"
```

### Environment Variable Override

```bash
export GITHUB_TOKEN="ghp_YOUR-PERSONAL-ACCESS-TOKEN"
export GITHUB_WEBHOOK_SECRET="your-webhook-secret"
```

### Verify Configuration

```bash
python3 -c "
from integrations.github import GitHubIntegration
gh = GitHubIntegration()
print(f'Enabled: {gh.enabled}')
print(f'Connection test: {gh.test_connection()}')
"
```

---

## PagerDuty Integration

### Vault Path
```
secret/data/integrations/pagerduty
```

### Required Keys

| Key | Description | Format |
|-----|-------------|--------|
| `pagerduty_routing_key` | Events API v2 Integration Key | 32-char hex string |
| `pagerduty_api_token` | REST API Token (optional) | 20-char string |
| `pagerduty_service_id` | Service ID (optional) | `P...` format |

### How to Obtain

#### Routing Key (Events API v2) - Required
1. Log into [app.pagerduty.com](https://app.pagerduty.com)
2. Go to **Services** > Select your service
3. Click **Integrations** tab
4. Click **Add Integration**
5. Select **Events API v2**
6. Copy the **Integration Key** (this is your routing key)

#### REST API Token (Optional)
1. Go to **Configuration** > **API Access**
2. Click **Create New API Key**
3. Enter a description (e.g., "Agent Governance")
4. Choose **Read/Write** access
5. Copy the generated token

#### Service ID (Optional)
1. Go to **Services** > Select your service
2. The Service ID is in the URL: `app.pagerduty.com/services/PXXXXXX`
3. Or find it under **Settings** > **Service ID**

### Store in Vault

```bash
# Store PagerDuty credentials
vault kv put secret/integrations/pagerduty \
    pagerduty_routing_key="YOUR-32-CHAR-ROUTING-KEY" \
    pagerduty_api_token="YOUR-API-TOKEN" \
    pagerduty_service_id="PXXXXXXX" \
    configured=true

# Or using curl
curl -sk -X POST \
    -H "X-Vault-Token: $VAULT_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
        "data": {
            "pagerduty_routing_key": "YOUR-32-CHAR-ROUTING-KEY",
            "pagerduty_api_token": "YOUR-API-TOKEN",
            "pagerduty_service_id": "PXXXXXXX",
            "configured": true
        }
    }' \
    "$VAULT_ADDR/v1/secret/data/integrations/pagerduty"
```

### Environment Variable Override

```bash
export PAGERDUTY_ROUTING_KEY="your-32-char-routing-key"
export PAGERDUTY_API_TOKEN="your-api-token"
export PAGERDUTY_SERVICE_ID="PXXXXXXX"
```

### Verify Configuration

```bash
python3 -c "
from integrations.pagerduty import PagerDutyIntegration
pd = PagerDutyIntegration()
print(f'Enabled: {pd.enabled}')
print(f'Connection test: {pd.test_connection()}')
"
```

---

## Runtime Environment Variables

The following environment variables control integration behavior:

| Variable | Description | Default |
|----------|-------------|---------|
| `VAULT_ADDR` | Vault server address | `https://127.0.0.1:8200` |
| `VAULT_TOKEN` | Vault authentication token | Read from `/opt/vault/init-keys.json` |
| `USE_VAULT` | Enable Vault secret loading | `true` |
| `INTEGRATION_DRY_RUN` | Prevent actual API calls (testing) | `false` |
| `TESTING` | Enable mock mode for all secrets | `false` |

### Production Configuration

```bash
# /etc/environment or systemd unit file
VAULT_ADDR=https://vault.internal:8200
USE_VAULT=true
INTEGRATION_DRY_RUN=false
```

### Development Configuration

```bash
# .env or shell profile
export SLACK_BOT_TOKEN="xoxb-dev-token"
export GITHUB_TOKEN="ghp_dev-token"
export PAGERDUTY_ROUTING_KEY="dev-routing-key"
export INTEGRATION_DRY_RUN=true
```

---

## Verification Script

Run this script to verify all integrations are properly configured:

```bash
#!/bin/bash
# verify-integrations.sh

cd /opt/agent-governance

echo "=== Integration Credential Verification ==="
echo ""

python3 << 'EOF'
import os
import sys
sys.path.insert(0, "/opt/agent-governance/integrations")

from common.secrets import SecretsManager

secrets = SecretsManager()

integrations = ["slack", "github", "pagerduty"]

for name in integrations:
    result = secrets.validate_integration(name)
    status = "READY" if result["ready"] else "MISSING KEYS"
    print(f"\n[{name.upper()}] {status}")
    if result["configured"]:
        print(f"  Configured: {', '.join(result['configured'])}")
    if result["missing"]:
        print(f"  Missing: {', '.join(result['missing'])}")
    if result["mock_mode"]:
        print(f"  (Running in mock mode)")

print("\n" + "=" * 45)
EOF
```

---

## Troubleshooting

### "No credentials found"
1. Check Vault is accessible: `curl -sk $VAULT_ADDR/v1/sys/health`
2. Verify token is valid: `vault token lookup`
3. Check secret path exists: `vault kv get secret/integrations/slack`

### "Connection test failed"
1. Verify credentials are correct (not placeholder values)
2. Check network connectivity to external services
3. Enable dry-run mode to test without making real API calls:
   ```bash
   INTEGRATION_DRY_RUN=true python3 -c "..."
   ```

### "Rate limit exceeded"
1. The integration base class handles rate limiting (60 req/min default)
2. Check `_request_times` in the integration instance
3. Wait 60 seconds and retry

---

## Security Best Practices

1. **Never commit credentials** to version control
2. **Rotate tokens regularly** (GitHub: 90 days, PagerDuty: annually)
3. **Use least privilege** - only request scopes you need
4. **Enable Vault audit logging** for credential access tracking
5. **Use environment variables** only in development; production should use Vault
6. **Monitor for credential leaks** with GitHub secret scanning

---

## Related Documentation

- [Architecture Overview](./ARCHITECTURE.md)
- [Integration Status](../integrations/STATUS.md)
- [Secrets Management Code](../integrations/common/secrets.py)

---
*Last updated: 2026-01-24*