QA Audit Log System

Added: 2026-02-08 Status: Active PR: #499

---

Overview

The QA audit log tracks manual end-to-end test coverage across 14 functional scopes. It provides visibility into what has been tested, what is stale (code changed since last test), and what has never been tested.

Key Components:

1. Smoke test definitions (docs/smoke-test.md) - Scope definitions and file-to-scope mapping
2. QA log (docs/qa-log.md) - Coverage matrix and test history
3. /qa-update skill - Automated skill that updates the log based on test results

---

How It Works

Scopes

The system tracks 14 functional scopes:

Scope	Description
Auth	Firebase authentication flows
Equipment	10 equipment types CRUD and bow setup
RoundCreation	Round configuration and setup
ActiveScoring	Real-time score entry during a live round
RoundHistory	Historical rounds list and statistics
Tournaments	Tournament creation, discovery, join codes
TournamentScoring	Tournament live scoring, sync, leaderboards
Leaderboards	Global leaderboards, publishing, verification
CameraScoring	ML arrow detection and camera scoring
Analytics	Equipment analytics and shot grouping
Admin	Admin panel, moderation, audit logs
Settings	User preferences, theme
Navigation	Tab navigation, deep links, app launch
Database	Schema integrity, migrations

Status Values

• PASS - All acceptance criteria met
• FAIL - One or more criteria failed
• PARTIAL - Some criteria passed, some failed or untested
• STALE - Production code changed since last test
• — - Never tested

Staleness Detection

The /qa-update skill automatically detects stale scopes by:

1. For each previously-tested scope, running git diff --name-only <scope-SHA> HEAD
2. Filtering out non-production paths (tests, docs, config)
3. Matching changed files against the file-to-scope mapping table
4. Marking the scope as STALE if any mapped production file changed

This means a scope tested at commit abc1234 becomes stale as soon as any of its mapped production files change in a later commit.

File-to-Scope Mapping

Each scope has a defined set of glob patterns mapping production source files to it. For example, the Auth scope maps to:

• app/**/ui/authentication/** (Android)
• shared/**/data/repositories/FirebaseAuthRepository.kt (KMP)
• iosApp/**/Auth/** (iOS)

The complete mapping is in docs/smoke-test.md.

---

Usage

Recording Test Results

Run the /qa-update skill with a description of what was tested:

/qa-update tested auth pass, equipment partial - sight form crashes on save

The skill will:

1. Parse scope names and results from the input
2. Run staleness analysis on all previously-tested scopes
3. Update docs/qa-log.md with new results and stale markers
4. Report a summary

Coverage Matrix

The coverage matrix in docs/qa-log.md shows the current state of all scopes:

Scope	Status	Last Tested	SHA	Version	Tester	Notes
Auth	PASS	2026-02-08	abc1234	1.1	BlameChris
Equipment	STALE	2026-01-15	def5678	1.0	BlameChris	Changed: SightFormViewModel.swift

Test History

Each test run creates a history entry:

### 2026-02-08 — BlameChris
- **Auth**: PASS @ `abc1234` (v1.1)
- **Equipment**: PARTIAL @ `abc1234` (v1.1) — sight form crashes on save

---

Best Practices

Prioritizing Test Scopes

1. STALE scopes - Production code changed, retest needed
2. Untested scopes (--) - No baseline coverage
3. Core user flows - Auth, ActiveScoring, RoundCreation

When to Run /qa-update

• After manual testing sessions
• Before App Store submissions
• After major feature merges
• When preparing release notes

---

Related Documentation

• Test Coverage Strategy - Automated test coverage approach
• Test Quality Standards - Test writing guidelines
• Tournament Testing Checklist - Tournament-specific manual tests

---

Tags: testing qa manual-testing coverage Status: Active