Decisions

  • Pending: Skill-based (/smoke-test) or script-based (pytest)?
  • Pending: Run automatically or on-demand?

User Tasks


FR-051: System Integration Testing

Summary

A /smoke-test skill and test suite that verifies skills, hooks, agents, rules, and templates work correctly together.

Problem / Motivation

FR-049 covers Python unit testing. But the Opus system is more than Python code — it’s a constellation of skills, hooks, agents, rules, templates, and vault conventions that must work together. Currently there’s no way to verify that:

  • Skills produce valid output and don’t error
  • Hooks fire correctly and don’t block valid operations
  • Templates contain all required sections
  • Rules load for the right paths
  • Frontmatter schemas are consistent across all FR files
  • Cross-references between files are valid (no broken links)

As the system grows, changes to one component can silently break others.

Proposed Solution

Two layers:

  1. /smoke-test skill — quick health check runnable in any session
  2. Automated test suite — deeper validation, runnable via script

Test Categories

CategoryWhat it checks
Template integrityAll templates have required sections, valid YAML frontmatter
FR consistencyAll FRs have valid frontmatter, status matches directory, no orphan references
Hook healthHook scripts exist, are executable, handle valid input without error
Skill syntaxSkill YAML/MD files parse correctly, required fields present
Rule loadingRules files exist at expected paths
Cross-referencesFR dependencies point to existing FRs, related links are valid

Open Questions

No open questions.


Phase Overview

PhaseDescriptionStatus
Phase 1/smoke-test skill with basic checks
Phase 2Automated test scripts (deeper validation)

Phase 1: Smoke Test Skill —

Goal: Quick health check as a slash command.

File / FeatureDetailsOwnerStatus
.claude/skills/smoke-test/SKILL.mdSkill definitionmv
Template validationCheck all templates parse correctlyopus
FR frontmatter scanValidate all FRs have required fieldsopus
Hook existence checkVerify registered hooks point to real scriptsmv

Phase 2: Deep Validation —

Goal: Thorough automated test suite.

File / FeatureDetailsOwnerStatus
src/tests/test_system_integrity.pyPytest-based system testsmv
Cross-reference validationCheck all FR links resolveopus
Directory-status consistencyFR in in-progress/ must have status: in-progressopus
Integration with CIRun on push if CI existsopus

Test

Manual tests

TestExpectedActualLast
pending-

AI-verified tests

ScenarioExpected behaviorVerification method

E2E tests

ScenarioAssertion

Integration tests

ComponentCoverage

Unit tests

ComponentTestsCoverage

History

DateEventDetails
2026-03-12CreatedIdentified as gap — no way to test system components work together

References

  • FR-049 (Testing Infrastructure) — Python unit tests; this FR covers system-level integration
  • FR-004 (Deterministic Validation Hooks) — hooks are one of the things tested here
  • FR-028 (Feature Overview Sync Check) — overlaps on FR consistency; this FR is broader