log-analyzer

MikoPBX Log Analyzing

Efficiently analyze logs inside MikoPBX Docker container to diagnose issues, monitor processes, and track system behavior.

What This Skill Does

• Intelligently searches logs based on problem context
• Identifies relevant log files for each issue type
• Filters noise to show only relevant entries
• Correlates logs across multiple files
• Tracks worker processes and their status
• Provides actionable insights from log analysis
• Works with both host-mounted logs and docker exec for flexibility

Host Log Access (Faster)

For local development, logs are mounted on the host at:

/Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/

Advantages of host access:

• ✅ Faster (no docker exec overhead)
• ✅ Can use Read tool directly
• ✅ Can use Grep tool with full power
• ✅ Better for large log files

When to use host path:

• Analyzing large log files
• Complex grep patterns
• Multiple log file analysis
• Performance-critical operations

When to Use This Skill

Use this skill when:

• User reports an error or issue
• Need to diagnose API problems
• Tracking specific worker processes
• Investigating system behavior
• User asks "check logs" or "what's in the logs?"
• After tests fail (to find root cause)
• Monitoring real-time system activity

Quick Start

Step 1: Choose Access Method

Option A: Host Path (Recommended for analysis)

# Direct access to logs (faster)
HOST_LOG_PATH="/Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log"

# Use Grep tool
# Pattern: errors in system log
# Path: $HOST_LOG_PATH/system/messages

# Or use Read tool for specific log files

Option B: Docker Exec (Needed for process checks)

# Find MikoPBX container (needed for worker checks)
CONTAINER_ID=$(docker ps | grep mikopbx | awk '{print $1}' | head -1)

Quick Decision Guide:

• Log file analysis → Use host path with Grep/Read tools
• Process/worker checks → Use docker exec with Bash
• Real-time monitoring → Use docker exec with tail -f

Step 2: Determine Log Context

Based on issue type, select appropriate logs:

API Issues:

• System messages (WorkerApiCommands)
• PHP error log
• Nginx logs

Call Issues:

• Asterisk messages/error/verbose
• Security log

Database Issues:

• System messages
• Debug query logs (if enabled)

Worker Issues:

• System messages (filter by worker name)
• Process status

Step 3: Execute Analysis

Use targeted search patterns (see search-patterns.md):

# Recent errors
docker exec $CONTAINER_ID tail -500 /storage/usbdisk1/mikopbx/log/system/messages | grep -i error

# Specific worker
docker exec $CONTAINER_ID tail -300 /storage/usbdisk1/mikopbx/log/system/messages | grep WorkerApiCommands

# Real-time monitoring
docker exec $CONTAINER_ID tail -f /storage/usbdisk1/mikopbx/log/system/messages

Top 5 Common Verification Patterns

Pattern 1: Recent System Errors

When to use: First step in diagnosing any issue

Method A: Host Path (Recommended)

# Use Grep tool
# Pattern: error
# Path: /Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/system/messages
# Options: -i (case insensitive), -C 2 (2 lines context)

Method B: Docker Exec

# Get last 500 lines, filter errors
docker exec $CONTAINER_ID tail -500 /storage/usbdisk1/mikopbx/log/system/messages | grep -i error

What to look for:

• Error timestamps (correlation)
• Error frequency (isolated vs recurring)
• Component name (which part failed)
• PID (which process)

Common findings:

• PHP fatal errors
• Database constraint violations
• Worker crashes
• Memory exhaustion

---

Pattern 2: API Request Debugging

When to use: API request fails or returns unexpected result

Method A: Host Path (Recommended)

# Use Grep tool for WorkerApiCommands activity
# Pattern: WorkerApiCommands
# Path: /Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/system/messages

# Use Read tool for PHP errors (recent)
# Path: /Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/php/error.log
# Offset: calculate from file size
# Limit: 50

# Use Grep for nginx access
# Pattern: POST /pbxcore/api
# Path: /Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/nginx/access.log

Method B: Docker Exec

# Check WorkerApiCommands activity
docker exec $CONTAINER_ID tail -200 /storage/usbdisk1/mikopbx/log/system/messages | grep WorkerApiCommands

# Check PHP errors
docker exec $CONTAINER_ID tail -50 /storage/usbdisk1/mikopbx/log/php/error.log

# Check nginx access log
docker exec $CONTAINER_ID tail -100 /storage/usbdisk1/mikopbx/log/nginx/access.log | grep "POST /pbxcore/api"

What to look for:

1. Request received by nginx
2. Request queued to Redis
3. WorkerApiCommands processing
4. PHP errors during execution
5. Response status code

Expected flow:

[06:52:10] nginx: POST /pbxcore/api/v3/extensions
[06:52:11] WorkerApiCommands: Processing request
[06:52:12] WorkerApiCommands: Executing SaveRecordAction
[06:52:13] WorkerApiCommands: Response sent (200 OK)

See analysis-scenarios.md for detailed example.

---

Pattern 3: Worker Process Monitoring

When to use: Worker not responding, system seems stuck

# Check if worker is running
docker exec $CONTAINER_ID ps aux | grep WorkerApiCommands

# Check for crashes
docker exec $CONTAINER_ID tail -500 /storage/usbdisk1/mikopbx/log/system/messages | grep -E "WorkerApiCommands|terminated|crash"

# Count instances
docker exec $CONTAINER_ID ps aux | grep WorkerApiCommands | grep -v grep | wc -l

Expected: WorkerApiCommands should have 3 instances

What to look for:

• Process terminated messages
• Orphaned process warnings
• Fatal PHP errors
• Last activity before crash

Common issues:

• Worker crashed (0 instances)
• Too many workers (restart loop)
• Zombie processes

See worker-processes.md for complete worker reference.

---

Pattern 4: Database Error Investigation

When to use: Constraint violations, lock errors, data inconsistency

# Check for database errors
docker exec $CONTAINER_ID tail -200 /storage/usbdisk1/mikopbx/log/system/messages | grep -iE "database|sqlite|constraint"

# Check PHP PDO errors
docker exec $CONTAINER_ID tail -100 /storage/usbdisk1/mikopbx/log/php/error.log | grep -iE "database|pdo|sql"

Common errors:

• UNIQUE constraint failed: Extensions.number - Duplicate data
• FOREIGN KEY constraint failed - Invalid reference
• Database is locked - Concurrent access
• Database disk image is malformed - Corruption

Investigation steps:

1. Identify which query failed
2. Check input data for duplicates/invalids
3. Verify foreign key references exist
4. Check for long-running transactions

See analysis-scenarios.md for detailed troubleshooting.

---

Pattern 5: Real-Time Activity Monitoring

When to use: Need to see what's happening right now

# Follow system log
docker exec $CONTAINER_ID tail -f /storage/usbdisk1/mikopbx/log/system/messages

# Follow with error filter
docker exec $CONTAINER_ID tail -f /storage/usbdisk1/mikopbx/log/system/messages | grep -i error

# Follow specific worker
docker exec $CONTAINER_ID tail -f /storage/usbdisk1/mikopbx/log/system/messages | grep WorkerApiCommands

Use cases:

• Monitor API request processing
• Watch worker activity
• See errors as they occur
• Debug timing issues

Pro tip: Press Ctrl+C to stop monitoring when done.

---

Critical Log Files

Host Paths (Fast Access)

# Log base directory on host
HOST_LOG_BASE="/Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log"

# PRIMARY: Main system log
${HOST_LOG_BASE}/system/messages

# PHP errors
${HOST_LOG_BASE}/php/error.log

# Asterisk errors
${HOST_LOG_BASE}/asterisk/error

# Web server errors
${HOST_LOG_BASE}/nginx/error.log

# Fail2ban logs
${HOST_LOG_BASE}/fail2ban/fail2ban.log

Container Paths (For docker exec)

# PRIMARY: Main system log
/storage/usbdisk1/mikopbx/log/system/messages

# PHP errors
/storage/usbdisk1/mikopbx/log/php/error.log

# Asterisk errors
/storage/usbdisk1/mikopbx/log/asterisk/error

# Web server errors
/storage/usbdisk1/mikopbx/log/nginx/error.log

Quick Access Commands

Using Host Path (Faster)

# Check recent errors in system log
# Use Grep tool:
#   Pattern: error
#   Path: /Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/system/messages
#   Options: -i -C 2

# Read PHP error log
# Use Read tool:
#   Path: /Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/php/error.log

# Search for specific pattern
# Use Grep tool:
#   Pattern: WorkerApiCommands
#   Path: /Users/nb/PhpstormProjects/mikopbx/dev_docker/tmp/projects/main/storage/usbdisk1/mikopbx/log/system/messages

Using Docker Exec (For processes)

# Find container
docker ps | grep mikopbx

# Tail system log
docker exec <id> tail -100 /storage/usbdisk1/mikopbx/log/system/messages

# Tail errors only
docker exec <id> tail -500 /storage/usbdisk1/mikopbx/log/system/messages | grep -i error

# Check PHP errors
docker exec <id> tail -50 /storage/usbdisk1/mikopbx/log/php/error.log

# Check workers (MUST use docker exec)
docker exec <id> ps aux | grep Worker

Output Format

Always structure log analysis output clearly:

📋 Log Analysis: <Component/Issue>
==================================

🔍 Search Criteria:
   • Log file: /path/to/log
   • Time ran

---

*Content truncated.*