Structure Validation - Verifies directory structure, file presence, and documentation standards
Script Testing - Tests Python scripts for syntax, functionality, and compliance
Quality Scoring - Provides comprehensive quality assessment across multiple dimensions

Quick Start

Validate a Skill

# Basic validation
python scripts/skill_validator.py engineering/my-skill

# Validate against specific tier
python scripts/skill_validator.py engineering/my-skill --tier POWERFUL --json

Test Scripts

# Test all scripts in a skill
python scripts/script_tester.py engineering/my-skill

# Test with custom timeout
python scripts/script_tester.py engineering/my-skill --timeout 60 --json

Score Quality

# Get quality assessment
python scripts/quality_scorer.py engineering/my-skill

# Detailed scoring with improvement suggestions
python scripts/quality_scorer.py engineering/my-skill --detailed --json

Components

Scripts

skill_validator.py (700+ LOC) - Validates skill structure and compliance
script_tester.py (800+ LOC) - Tests script functionality and quality
quality_scorer.py (1100+ LOC) - Multi-dimensional quality assessment

Reference Documentation

skill-structure-specification.md - Complete structural requirements
tier-requirements-matrix.md - Tier-specific quality standards
quality-scoring-rubric.md - Detailed scoring methodology

Sample Assets

sample-skill/ - Complete sample skill for testing the tester itself

Features

Validation Capabilities

SKILL.md format and content validation
Directory structure compliance checking
Python script syntax and import validation
Argparse implementation verification
Tier-specific requirement enforcement

Testing Framework

Syntax validation using AST parsing
Import analysis for external dependencies
Runtime execution testing with timeout protection
Help functionality verification
Sample data processing validation
Output format compliance checking

Quality Assessment

Documentation quality scoring (25%)
Code quality evaluation (25%)
Completeness assessment (25%)
Usability analysis (25%)
Letter grade assignment (A+ to F)
Tier recommendation generation
Improvement roadmap creation

CI/CD Integration

GitHub Actions Example

name: Skill Quality Gate
on:
  pull_request:
    paths: ['engineering/**']
    
jobs:
  validate-skills:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'
      - name: Validate Skills
        run: |
          for skill in $(git diff --name-only ${{ github.event.before }} | grep -E '^engineering/[^/]+/' | cut -d'/' -f1-2 | sort -u); do
            python engineering/skill-tester/scripts/skill_validator.py $skill --json
            python engineering/skill-tester/scripts/script_tester.py $skill
            python engineering/skill-tester/scripts/quality_scorer.py $skill --minimum-score 75
          done

Pre-commit Hook

#!/bin/bash
# .git/hooks/pre-commit
python engineering/skill-tester/scripts/skill_validator.py engineering/my-skill --tier STANDARD
if [ $? -ne 0 ]; then
    echo "Skill validation failed. Commit blocked."
    exit 1
fi

Quality Standards

All Scripts

Zero External Dependencies - Python standard library only
Comprehensive Error Handling - Meaningful error messages and recovery
Dual Output Support - Both JSON and human-readable formats
Proper Documentation - Comprehensive docstrings and comments
CLI Best Practices - Full argparse implementation with help text

Validation Accuracy

Structure Checks - 100% accurate directory and file validation
Content Analysis - Deep parsing of SKILL.md and documentation
Code Analysis - AST-based Python code validation
Compliance Scoring - Objective, repeatable quality assessment

Self-Testing

The skill-tester can validate itself:

# Validate the skill-tester structure
python scripts/skill_validator.py . --tier POWERFUL

# Test the skill-tester scripts
python scripts/script_tester.py .

# Score the skill-tester quality
python scripts/quality_scorer.py . --detailed

Advanced Usage

Batch Validation

# Validate all skills in repository
find engineering/ -maxdepth 1 -type d | while read skill; do
  echo "Validating $skill..."
  python engineering/skill-tester/scripts/skill_validator.py "$skill"
done

Quality Monitoring

# Generate quality report for all skills
python engineering/skill-tester/scripts/quality_scorer.py engineering/ \
  --batch --json > quality_report.json

Custom Scoring Thresholds

# Enforce minimum quality scores
python scripts/quality_scorer.py engineering/my-skill --minimum-score 80
# Exit code 0 = passed, 1 = failed, 2 = needs improvement

Error Handling

All scripts provide comprehensive error handling:

File System Errors - Missing files, permission issues, invalid paths
Content Errors - Malformed YAML, invalid JSON, encoding issues
Execution Errors - Script timeouts, runtime failures, import errors
Validation Errors - Standards violations, compliance failures

Output Formats

Human-Readable

=== SKILL VALIDATION REPORT ===
Skill: engineering/my-skill
Overall Score: 85.2/100 (B+)
Tier Recommendation: STANDARD

STRUCTURE VALIDATION:
  ✓ PASS: SKILL.md found
  ✓ PASS: README.md found
  ✓ PASS: scripts/ directory found

SUGGESTIONS:
  • Add references/ directory
  • Improve error handling in main.py

JSON Format

{
  "skill_path": "engineering/my-skill",
  "overall_score": 85.2,
  "letter_grade": "B+",
  "tier_recommendation": "STANDARD",
  "dimensions": {
    "Documentation": {"score": 88.5, "weight": 0.25},
    "Code Quality": {"score": 82.0, "weight": 0.25},
    "Completeness": {"score": 85.5, "weight": 0.25},
    "Usability": {"score": 84.8, "weight": 0.25}
  }
}

Requirements

Python 3.7+ - No external dependencies required
File System Access - Read access to skill directories
Execution Permissions - Ability to run Python scripts for testing

Contributing

See SKILL.md for comprehensive documentation and contribution guidelines.

The skill-tester itself serves as a reference implementation of POWERFUL-tier quality standards.