168 lines
7.5 KiB
Markdown
168 lines
7.5 KiB
Markdown
# Yonexus Failure Path Test Matrix
|
||
|
||
This document defines the systematic test coverage for pairing and authentication failure scenarios.
|
||
|
||
## Test Matrix Legend
|
||
|
||
- ✅ = Test implemented
|
||
- 🔄 = Test stub exists, needs implementation
|
||
- ⬜ = Not yet implemented
|
||
- ⏸ = Deferred / intentionally out of v1 scope
|
||
- 🔴 = Critical path, high priority
|
||
|
||
---
|
||
|
||
## 1. Pairing Failure Paths
|
||
|
||
| ID | Scenario | Trigger | Expected Behavior | Status |
|
||
|----|----------|---------|-------------------|--------|
|
||
| PF-01 | Invalid pairing code | Client submits wrong code | `pair_failed(invalid_code)`, allow retry | ✅ |
|
||
| PF-02 | Expired pairing code | Client submits after expiry | `pair_failed(expired)`, reset to `pair_required` | ✅ |
|
||
| PF-03 | Identifier not in allowlist | Unknown client tries to pair | `pair_failed(identifier_not_allowed)`, close connection | ✅ |
|
||
| PF-04 | Admin notification failed | Discord DM fails to send | `pair_failed(admin_notification_failed)`, abort pairing | ✅ |
|
||
| PF-05 | Empty pairing code | Client submits empty string | `pair_failed(invalid_code)` | ✅ |
|
||
| PF-06 | Malformed pair_confirm payload | Missing required fields | Protocol error, no state change | ✅ |
|
||
| PF-07 | Double pairing attempt | Client calls pair_confirm twice | Second attempt rejected if already paired | ✅ |
|
||
| PF-08 | Pairing during active session | Paired client tries to pair again | Reject, maintain existing trust | ✅ |
|
||
| PF-09 | Server restart during pairing | Server restarts before confirm | Pairing state preserved, code still valid | ✅ |
|
||
| PF-10 | Client restart during pairing | Client restarts before submit | Client must restart pairing flow | ✅ |
|
||
|
||
---
|
||
|
||
## 2. Authentication Failure Paths
|
||
|
||
| ID | Scenario | Trigger | Expected Behavior | Status |
|
||
|----|----------|---------|-------------------|--------|
|
||
| AF-01 | Unknown identifier | Auth from unpaired client | `auth_failed(unknown_identifier)` | ✅ |
|
||
| AF-02 | Not paired | Auth before pairing complete | `auth_failed(not_paired)` | ✅ |
|
||
| AF-03 | Invalid signature | Wrong private key used | `auth_failed(invalid_signature)` | ✅ |
|
||
| AF-04 | Wrong secret | Client has outdated secret | 当前实现将其并入 `auth_failed(invalid_signature)`；`invalid_secret` 语义保留待后续确认 | ⏸ |
|
||
| AF-05 | Stale timestamp | Proof timestamp >10s old | `auth_failed(stale_timestamp)` | ✅ |
|
||
| AF-06 | Future timestamp | Proof timestamp in future | `auth_failed(future_timestamp)` | ✅ |
|
||
| AF-07 | Nonce collision | Reused nonce within window | `auth_failed(nonce_collision)` → `re_pair_required` 🔴 | ✅ |
|
||
| AF-08 | Rate limited | >10 attempts in 10s | `auth_failed(rate_limited)` → `re_pair_required` 🔴 | ✅ |
|
||
| AF-09 | Wrong public key | Key doesn't match stored | `auth_failed(invalid_signature)` | ✅ |
|
||
| AF-10 | Malformed auth_request | Missing required fields | Protocol error | ✅ |
|
||
| AF-11 | Tampered proof | Modified signature | `auth_failed(invalid_signature)` | ✅ |
|
||
|
||
---
|
||
|
||
## 3. Re-pairing Triggers
|
||
|
||
| ID | Scenario | Cause | Server Action | Client Action | Status |
|
||
|----|----------|-------|---------------|---------------|--------|
|
||
| RP-01 | Nonce collision | Replay attack detected | Clear secret, reset state | Enter `pair_required` | ✅ |
|
||
| RP-02 | Rate limit exceeded | Brute force detected | Clear secret, reset state | Enter `pair_required` | ✅ |
|
||
| RP-03 | Admin-initiated | Manual revocation | Mark revoked, notify | Enter `pair_required` | ⏸ |
|
||
| RP-04 | Key rotation | Client sends new public key | Update key, keep secret | Continue with new key | ⏸ |
|
||
|
||
---
|
||
|
||
## 4. Connection Failure Paths
|
||
|
||
| ID | Scenario | Trigger | Expected Behavior | Status |
|
||
|----|----------|---------|-------------------|--------|
|
||
| CF-01 | Network partition | Connection drops mid-auth | Client retries with backoff | ✅ |
|
||
| CF-02 | Server unreachable | Initial connect fails | Exponential backoff retry | ✅ |
|
||
| CF-03 | Duplicate connection | Same ID connects twice | Old connection closed, new accepted | ✅ |
|
||
| CF-04 | Protocol version mismatch | Unsupported version | Connection rejected with error | ✅ |
|
||
| CF-05 | Malformed hello | Invalid payload / missing required hello fields | Error response, connection maintained | ✅ |
|
||
| CF-06 | Unauthenticated rule message | Client sends before auth | Connection closed | ✅ |
|
||
| CF-07 | Reserved rule registration | Plugin tries `registerRule("builtin")` | Registration rejected | ✅ |
|
||
|
||
---
|
||
|
||
## 5. Heartbeat Failure Paths
|
||
|
||
| ID | Scenario | Trigger | Expected Behavior | Status |
|
||
|----|----------|---------|-------------------|--------|
|
||
| HF-01 | 7-minute timeout | No heartbeat received | Status → `unstable`, notify | ✅ |
|
||
| HF-02 | 11-minute timeout | Still no heartbeat | Status → `offline`, disconnect | ✅ |
|
||
| HF-03 | Early heartbeat | Heartbeat before auth | Rejected/ignored | ✅ |
|
||
| HF-04 | Heartbeat from unauthenticated | Wrong state | Error, possible disconnect | ✅ |
|
||
|
||
---
|
||
|
||
## 6. State Recovery Scenarios
|
||
|
||
| ID | Scenario | Condition | Expected Recovery | Status |
|
||
|----|----------|-----------|-------------------|--------|
|
||
| SR-01 | Server restart with pending pairing | Pairing in progress | Preserve pairing state, code valid | ✅ |
|
||
| SR-02 | Server restart with active sessions | Online clients | All marked offline, reconnect required | ✅ |
|
||
| SR-03 | Client restart with credentials | Has secret + keys | Resume with auth, no re-pairing | ✅ |
|
||
| SR-04 | Client restart without credentials | First run | Full pairing flow required | ✅ |
|
||
| SR-05 | Corrupted server store | File unreadable | Surface corruption error clearly for operator handling | ✅ |
|
||
| SR-06 | Corrupted client state | File unreadable | Surface corruption error clearly for operator handling | ✅ |
|
||
|
||
---
|
||
|
||
## Implementation Priority
|
||
|
||
### Phase 1: Critical Security Paths (🔴)
|
||
1. AF-07 Nonce collision → re-pairing
|
||
2. AF-08 Rate limiting → re-pairing
|
||
3. PF-04 Admin notification failure
|
||
4. CF-06 Unauthenticated message handling
|
||
|
||
### Phase 2: Core Functionality
|
||
5. PF-01/02 Invalid/expired pairing codes
|
||
6. AF-03/04 Signature and secret validation
|
||
7. AF-05/06 Timestamp validation
|
||
8. HF-01/02 Heartbeat timeout handling
|
||
|
||
### Phase 3: Edge Cases
|
||
9. All connection failure paths
|
||
10. State recovery scenarios
|
||
11. Double-attempt scenarios
|
||
|
||
---
|
||
|
||
## Test Implementation Notes
|
||
|
||
### Running the Matrix
|
||
|
||
```bash
|
||
# Run specific failure path category
|
||
npm test -- pairing-failures
|
||
npm test -- auth-failures
|
||
npm test -- connection-failures
|
||
|
||
# Run all failure path tests
|
||
npm test -- failure-paths
|
||
```
|
||
|
||
### Current Notes
|
||
|
||
- AF-04 (`invalid_secret`) 目前明确按 v1 语义并入 `invalid_signature`，不再单独视为未完成缺口；若后续要保留独立错误码，需要先同步更新协议与实现。
|
||
- RP-03（管理员主动撤销）与 RP-04（key rotation）继续作为 v2+ 议题保留，不阻塞当前 v1 交付判断。
|
||
- 本轮已补齐 AF-01/02/03/05/06/09/10/11、RP-01/02、CF-01/02/03/04/05/07、HF-01/02、PF-08/09/10、SR-01/02/03/04/05/06。
|
||
|
||
### Umbrella Validation Entry Point
|
||
|
||
在 umbrella 仓库根目录可运行：
|
||
|
||
```bash
|
||
./scripts/validate-v1.sh
|
||
```
|
||
|
||
它会顺序执行：
|
||
- `Yonexus.Protocol` 测试
|
||
- `Yonexus.Server` 类型检查 + 测试
|
||
- `Yonexus.Client` 类型检查 + 测试
|
||
|
||
### Adding New Test Cases
|
||
|
||
1. Add row to appropriate table above
|
||
2. Assign unique ID (PF-, AF-, RP-, CF-, HF-, SR- prefix)
|
||
3. Update status when implementing
|
||
4. Link to test file location
|
||
|
||
---
|
||
|
||
## Cross-References
|
||
|
||
- Protocol spec: `../PROTOCOL.md`
|
||
- Acceptance criteria: `../ACCEPTANCE.md`
|
||
- Server tests: `../Yonexus.Server/tests/`
|
||
- Client tests: `../Yonexus.Client/tests/`
|