security-check

Security Check Skill

Comprehensive security auditing for Clawdbot skills to detect malicious intent, prompt injection, secrets exposure, and misaligned behavior.

Quick Start

Pre-Installation Security Check

Before installing a new skill from ClawdHub or any source:

1. Download and inspect the skill files
2. Run the automated security scanner:

   python3 scripts/scan_skill.py /path/to/skill
   

3. Review the scanner output - Block any skill with HIGH severity issues
4. Manual review for MEDIUM severity issues
5. Verify behavior matches description before installation

Daily Security Audit

Run daily to ensure installed skills remain secure:

# Scan all skills in the skills directory
python3 scripts/scan_skill.py /path/to/skills/skill-1
python3 scripts/scan_skill.py /path/to/skills/skill-2
# ... repeat for each installed skill

Security Scanner

Running the Scanner

The scripts/scan_skill.py tool provides automated security analysis:

python3 scripts/scan_skill.py <skill-path>

Output includes:

• HIGH severity issues (immediate action required)
• MEDIUM severity warnings (review recommended)
• LOW severity notes (informational)
• Summary of checks performed

Example output:

{
  "skill_name": "example-skill",
  "issues": [
    {
      "severity": "HIGH",
      "file": "SKILL.md",
      "issue": "Potential prompt injection pattern",
      "recommendation": "Review and remove suspicious patterns"
    }
  ],
  "warnings": [
    {
      "severity": "MEDIUM",
      "file": "scripts/helper.py",
      "issue": "os.system() usage detected",
      "recommendation": "Review and ensure this is safe"
    }
  ],
  "passed": [
    {"file": "SKILL.md", "check": "Prompt injection scan", "status": "Completed"}
  ],
  "summary": "SECURITY ISSUES FOUND: 1 issue(s), 1 warning(s)"
}

What the Scanner Checks

1. SKILL.md Analysis

   • Prompt injection patterns
   • External network calls
   • Suspicious instructions

2. Scripts Directory Scan

   • Dangerous command patterns (rm -rf, eval, exec)
   • Hardcoded secrets and credentials
   • Unsafe subprocess usage
   • File system operations outside skill directory

3. References Directory Scan

   • Hardcoded secrets (passwords, API keys, tokens)
   • Suspicious URLs (pastebin, raw GitHub links)
   • Sensitive information exposure

Manual Security Checklist

Use the comprehensive checklist in references/security-checklist.md for manual reviews.

Critical Checks (Before Installation)

1. Documentation Integrity (SKILL.md)

• ✅ Description accurately reflects skill functionality
• ❌ No prompt injection patterns (see references/prompt-injection-patterns.md)
• ❌ No instructions to ignore/discard context
• ❌ No system override commands
• ✅ No hidden capabilities beyond description

2. Code Review (scripts/)

• ❌ No hardcoded credentials or secrets
• ❌ No dangerous file operations (rm -rf outside skill dir)
• ❌ No eval() or exec() with user input
• ❌ No unauthorized network requests
• ✅ All operations within skill directory
• ✅ Proper input validation

3. Reference Materials (references/)

• ❌ No hardcoded passwords, API keys, or tokens
• ❌ No production credentials in documentation
• ✅ Links only to legitimate, trusted sources
• ✅ No documentation of security bypasses

4. Behavior Alignment

• ✅ Every command matches stated purpose
• ✅ No hidden capabilities
• ✅ No unnecessary file system access
• ✅ Network access only when explicitly required

Daily Audit Checks

1. Scan all installed skills with the automated scanner
2. Review any new HIGH severity issues
3. Check for modified files in skill directories
4. Verify skill descriptions still match behavior
5. Audit new dependencies if added

Specific Security Concerns

Prompt Injection Detection

Read references/prompt-injection-patterns.md for comprehensive patterns.

Key indicators:

• Instructions to ignore/discard context
• System override or bypass commands
• Authority impersonation (act as administrator, etc.)
• Jailbreak attempts (unrestricted mode, etc.)
• Instruction replacement patterns

Detection:

# Automated pattern matching
import re
dangerous_patterns = [
    r'ignore\s+previous\s+instructions',
    r'override\s+security',
    r'act\s+as\s+administrator',
]

Secrets and Credentials Exposure

What to scan for:

• Hardcoded passwords, API keys, tokens
• AWS access keys and secret keys
• SSH private keys
• Database connection strings
• Other sensitive credentials

Patterns to detect:

password="..."
secret='...'
token="1234567890abcdef"
api_key="..."
aws_access_key_id="..."

Local Configuration Access

Block access to:

• ~/.clawdbot/credentials/
• ~/.aws/credentials
• ~/.ssh/ directory
• ~/.npmrc and other config files
• Shell history files
• System keychain

Allow only:

• Skill-specific configuration files
• User-provided file paths
• Designated workspace directories
• Approved environment variables

Command-Behavior Alignment

Verification process:

1. Extract all commands/operations from skill code
2. Compare against description in SKILL.md
3. Identify any operations not documented
4. Flag suspicious or hidden capabilities

Example misalignment:

❌ BLOCK:

• Description: "Format text documents"
• Actual: Scans filesystem, sends data to external server

✅ SAFE:

• Description: "Convert Markdown to PDF with templates"
• Actual: Reads Markdown, applies template, generates PDF

Security Severity Levels

HIGH (Immediate Block)

• Prompt injection patterns detected
• Hardcoded secrets or credentials
• Data exfiltration capabilities
• Unauthorized file system access
• Dangerous file operations (rm -rf, dd, etc.)
• eval() or exec() with untrusted input

Action: Do not install. Report to skill author.

MEDIUM (Review Required)

• Suspicious but not clearly malicious
• Requires user approval for specific operations
• Limited network access to unverified endpoints
• Unsafe subprocess usage (shell=True)
• Environment variable exposure risks

Action: Manual review. Install only if justified and understood.

LOW (Informational)

• Suspicious URLs (may be legitimate)
• Documentation of deprecated practices
• Minor code quality issues
• Potential improvements for security

Action: Note for future review. Generally safe to install.

Installation Decision Framework

When to BLOCK (Do Not Install)

• Any HIGH severity issues present
• Clear prompt injection attempts
• Hardcoded secrets
• Data exfiltration
• Unauthorized access patterns

When to WARN (Install with Caution)

• MEDIUM severity issues present
• Suspicious patterns requiring verification
• Needs specific user approvals
• Network access to unknown endpoints

Before installing with WARN:

1. Understand the risk
2. Verify the skill author's reputation
3. Test in isolated environment first
4. Monitor behavior closely
5. Be prepared to uninstall

When to APPROVE (Safe to Install)

• No security issues detected
• Well-documented and transparent
• Matches description perfectly
• From trusted source
• Regularly audited

Dependency Security

Check skill dependencies for vulnerabilities:

# For Node.js skills
npm audit
npm audit fix

# For Python skills
pip-audit
safety check

What to check:

• Known CVEs in dependencies
• Outdated packages with security updates
• Transitive dependency vulnerabilities
• Untrusted or unmaintained packages

Security Reporting

Report Template

# Security Audit Report
**Date:** [Date]
**Skill:** [Skill Name]
**Version:** [Version]

## Executive Summary
[Overall security posture: SAFE, WARNING, or BLOCK]

## Critical Issues (Immediate Action Required)
[List HIGH severity issues]

## Warnings (Review Recommended)
[List MEDIUM severity issues]

## Informational Notes
[List LOW severity issues]

## Recommendations
[Actionable items to address issues]

## Conclusion
[Final verdict: Install/Block/Requires Changes]

Escalation Process

1. Detect issue during scan or review
2. Document findings using report template
3. Assess severity (HIGH/MEDIUM/LOW)
4. Take action:
   • HIGH: Block skill, report to author
   • MEDIUM: Review, install with caution or wait for fix
   • LOW: Note, monitor
5. Follow up on resolved issues

Reference Materials

Essential Reading

1. Security Checklist (references/security-checklist.md)

   • Comprehensive security criteria
   • Command alignment verification
   • Secrets exposure checks
   • Installation guidelines
   • Daily audit procedures

2. Prompt Injection Patterns (references/prompt-injection-patterns.md)

   • Detection categories and patterns
   • Automated detection strategies
   • Red flag indicators
   • Mitigation techniques
   • Reporting templates

Internal Security Docs

Refer to workspace security documents:

• SECURITY_AUDIT_REPORT.md - Overall Clawdbot security posture
• Any additional security policies or guidelines

Workflow Examples

Example 1: New Skill from ClawdHub

User request: "Check if skill 'xyz' is safe to install"

Response:

1. Download skill to temporary location
2. Run scanner: python3 scripts/scan_skill.py /tmp/xyz-skill
3. Review output:
   • If HIGH issues: "❌ BLOCKED: [list issues] - Do not install"
   • If MEDIUM issues: "⚠️ WARNING: [list issues] - Requires manual review"
   • If clean: "✅ SAFE: No security issues detected - Can install"
4. If MEDIUM issues: Provide detailed manual review using checklist

Example 2: Daily Security Audit

Daily routine:

# Scan all installed skills
for skill in /Users/rlapuente/clawd/skills/*/; do
    python3 scripts/scan_skill.py "$skill"
done

# Review any HIGH issues immediately
# Monitor MEDIUM issues for trends

Example 3: Verification of Skill Update

---

Content truncated.