- YNX-1104a: Create Server-Client integration test framework - MockTransportPair for simulating network communication - createIntegrationTestContext() for test environment setup - First-time pairing flow integration tests - Reconnection flow tests - YNX-1105a: Create failure path test matrix documentation - MATRIX.md with PF/AF/RP/CF/HF/SR categories - Priority marking for critical security paths - YNX-1105b: Implement pairing failure path tests - PF-01: Invalid pairing code with retry - PF-02: Expired pairing code cleanup - PF-03: Unknown identifier rejection - PF-04: Admin notification failure handling - PF-05: Empty/whitespace code rejection - PF-06: Malformed payload handling - PF-07: Double pairing protection - Edge cases: concurrent pairing, state cleanup - Update TASKLIST.md with completion status
6.6 KiB
6.6 KiB
Yonexus Failure Path Test Matrix
This document defines the systematic test coverage for pairing and authentication failure scenarios.
Test Matrix Legend
- ✅ = Test implemented
- 🔄 = Test stub exists, needs implementation
- ⬜ = Not yet implemented
- 🔴 = Critical path, high priority
1. Pairing Failure Paths
| ID | Scenario | Trigger | Expected Behavior | Status |
|---|---|---|---|---|
| PF-01 | Invalid pairing code | Client submits wrong code | pair_failed(invalid_code), allow retry |
⬜ |
| PF-02 | Expired pairing code | Client submits after expiry | pair_failed(expired), reset to pair_required |
⬜ |
| PF-03 | Identifier not in allowlist | Unknown client tries to pair | pair_failed(identifier_not_allowed), close connection |
⬜ |
| PF-04 | Admin notification failed | Discord DM fails to send | pair_failed(admin_notification_failed), abort pairing |
⬜ |
| PF-05 | Empty pairing code | Client submits empty string | pair_failed(invalid_code) |
⬜ |
| PF-06 | Malformed pair_confirm payload | Missing required fields | Protocol error, no state change | ⬜ |
| PF-07 | Double pairing attempt | Client calls pair_confirm twice | Second attempt rejected if already paired | ⬜ |
| PF-08 | Pairing during active session | Paired client tries to pair again | Reject, maintain existing trust | ⬜ |
| PF-09 | Server restart during pairing | Server restarts before confirm | Pairing state preserved, code still valid | ⬜ |
| PF-10 | Client restart during pairing | Client restarts before submit | Client must restart pairing flow | ⬜ |
2. Authentication Failure Paths
| ID | Scenario | Trigger | Expected Behavior | Status |
|---|---|---|---|---|
| AF-01 | Unknown identifier | Auth from unpaired client | auth_failed(unknown_identifier) |
⬜ |
| AF-02 | Not paired | Auth before pairing complete | auth_failed(not_paired) |
⬜ |
| AF-03 | Invalid signature | Wrong private key used | auth_failed(invalid_signature) |
⬜ |
| AF-04 | Wrong secret | Client has outdated secret | auth_failed(invalid_secret) |
⬜ |
| AF-05 | Stale timestamp | Proof timestamp >10s old | auth_failed(stale_timestamp) |
⬜ |
| AF-06 | Future timestamp | Proof timestamp in future | auth_failed(future_timestamp) |
⬜ |
| AF-07 | Nonce collision | Reused nonce within window | auth_failed(nonce_collision) → re_pair_required 🔴 |
⬜ |
| AF-08 | Rate limited | >10 attempts in 10s | auth_failed(rate_limited) → re_pair_required 🔴 |
⬜ |
| AF-09 | Wrong public key | Key doesn't match stored | auth_failed(invalid_signature) |
⬜ |
| AF-10 | Malformed auth_request | Missing required fields | Protocol error | ⬜ |
| AF-11 | Tampered proof | Modified signature | auth_failed(invalid_signature) |
⬜ |
3. Re-pairing Triggers
| ID | Scenario | Cause | Server Action | Client Action | Status |
|---|---|---|---|---|---|
| RP-01 | Nonce collision | Replay attack detected | Clear secret, reset state | Enter pair_required |
⬜ |
| RP-02 | Rate limit exceeded | Brute force detected | Clear secret, reset state | Enter pair_required |
⬜ |
| RP-03 | Admin-initiated | Manual revocation | Mark revoked, notify | Enter pair_required |
⬜ |
| RP-04 | Key rotation | Client sends new public key | Update key, keep secret | Continue with new key | ⬜ |
4. Connection Failure Paths
| ID | Scenario | Trigger | Expected Behavior | Status |
|---|---|---|---|---|
| CF-01 | Network partition | Connection drops mid-auth | Client retries with backoff | ⬜ |
| CF-02 | Server unreachable | Initial connect fails | Exponential backoff retry | ⬜ |
| CF-03 | Duplicate connection | Same ID connects twice | Old connection closed, new accepted | ⬜ |
| CF-04 | Protocol version mismatch | Unsupported version | Connection rejected with error | ⬜ |
| CF-05 | Malformed hello | Invalid JSON | Error response, connection maintained | ⬜ |
| CF-06 | Unauthenticated rule message | Client sends before auth | Connection closed | ⬜ |
| CF-07 | Reserved rule registration | Plugin tries registerRule("builtin") |
Registration rejected | ⬜ |
5. Heartbeat Failure Paths
| ID | Scenario | Trigger | Expected Behavior | Status |
|---|---|---|---|---|
| HF-01 | 7-minute timeout | No heartbeat received | Status → unstable, notify |
⬜ |
| HF-02 | 11-minute timeout | Still no heartbeat | Status → offline, disconnect |
⬜ |
| HF-03 | Early heartbeat | Heartbeat before auth | Rejected/ignored | ⬜ |
| HF-04 | Heartbeat from unauthenticated | Wrong state | Error, possible disconnect | ⬜ |
6. State Recovery Scenarios
| ID | Scenario | Condition | Expected Recovery | Status |
|---|---|---|---|---|
| SR-01 | Server restart with pending pairing | Pairing in progress | Preserve pairing state, code valid | ⬜ |
| SR-02 | Server restart with active sessions | Online clients | All marked offline, reconnect required | ⬜ |
| SR-03 | Client restart with credentials | Has secret + keys | Resume with auth, no re-pairing | ⬜ |
| SR-04 | Client restart without credentials | First run | Full pairing flow required | ⬜ |
| SR-05 | Corrupted server store | File unreadable | Clear state, start fresh | ⬜ |
| SR-06 | Corrupted client state | File unreadable | Reset to initial state | ⬜ |
Implementation Priority
Phase 1: Critical Security Paths (🔴)
- AF-07 Nonce collision → re-pairing
- AF-08 Rate limiting → re-pairing
- PF-04 Admin notification failure
- CF-06 Unauthenticated message handling
Phase 2: Core Functionality
- PF-01/02 Invalid/expired pairing codes
- AF-03/04 Signature and secret validation
- AF-05/06 Timestamp validation
- HF-01/02 Heartbeat timeout handling
Phase 3: Edge Cases
- All connection failure paths
- State recovery scenarios
- Double-attempt scenarios
Test Implementation Notes
Running the Matrix
# Run specific failure path category
npm test -- pairing-failures
npm test -- auth-failures
npm test -- connection-failures
# Run all failure path tests
npm test -- failure-paths
Adding New Test Cases
- Add row to appropriate table above
- Assign unique ID (PF-, AF-, RP-, CF-, HF-, SR- prefix)
- Update status when implementing
- Link to test file location
Cross-References
- Protocol spec:
../PROTOCOL.md - Acceptance criteria:
../ACCEPTANCE.md - Server tests:
../Yonexus.Server/tests/ - Client tests:
../Yonexus.Client/tests/