# Dashboard methodology (SIMULATED)

> All data is synthetic and the company is fictional. Nothing here is client work or an assurance opinion.

As-of date: **2026-09-30**. Generated from `data/processed/dashboard.json` by `scripts/build_dashboard_reports.py`; the JSON is produced by `scripts/build_dashboard_data.py`.

## Principles

- No metric is typed by hand; each is computed from a source CSV or JSON file listed beside it.
- Projects are not blended into one false total. Audit findings (P1), vendor issues (P3), framework gaps (P2) and AI gates (P4) are counted separately, and the only combined figure (K03, overdue items) shows its parts.
- Status is validated state, not reported state: a remediation action reported complete but failing retest counts as not complete.
- Enterprise risks keep their recorded pre-audit residual scores. The dashboard does not re-rate them; it shows open findings against each risk for the risk owner to act on.
- Every chart has a table with the same figures, and a test checks that each chart total equals its source row count.

## Indicators

### K01 Open audit findings

- **Value:** 12 findings
- **Definition:** Findings from the Project 1 ITGC audit that are not closed by retest and not formally risk-accepted.
- **Calculation:** `count(findings where status not in {'Closed (retested)','Risk accepted'})`
- **Source:** data/analysis/findings.csv
- **Inclusion / exclusion:** Includes inconclusive F-13 and partially remediated findings. Excludes 2 closed or accepted. Project 1 only; vendor issues and framework gaps are counted separately.

### K02H Open High findings

- **Value:** 5 findings
- **Definition:** Open Project 1 findings rated High.
- **Calculation:** `count(open findings where severity = 'High')`
- **Source:** data/analysis/findings.csv
- **Inclusion / exclusion:** Same inclusion as K01.

### K02M Open Medium findings

- **Value:** 7 findings
- **Definition:** Open Project 1 findings rated Medium.
- **Calculation:** `count(open findings where severity = 'Medium')`
- **Source:** data/analysis/findings.csv
- **Inclusion / exclusion:** Same inclusion as K01.

### K02L Open Low findings

- **Value:** 0 findings
- **Definition:** Open Project 1 findings rated Low.
- **Calculation:** `count(open findings where severity = 'Low')`
- **Source:** data/analysis/findings.csv
- **Inclusion / exclusion:** Same inclusion as K01.

### K03 Overdue remediation items

- **Value:** 6 items
- **Definition:** Remediation actions (Project 1) plus vendor remediation issues (Project 3) past their due date and not validated complete.
- **Calculation:** `P1: count(actions where target_date < as_of and not (Completed and retest Pass)); P3: count(issues where overdue = Yes and status != Closed)`
- **Source:** data/analysis/remediation_actions.csv; projects/03-third-party-risk/remediation-and-monitoring-tracker.csv
- **Inclusion / exclusion:** P1 contributes 1, P3 contributes 5. A management-reported completion that failed retest still counts as overdue.

### K04 Controls tested not operating effectively

- **Value:** 12 tests
- **Definition:** Project 1 control tests concluded 'Not operating effectively' out of 16 tests.
- **Calculation:** `count(control_tests where conclusion starts with 'Not operating effectively')`
- **Source:** data/analysis/control_tests.csv
- **Inclusion / exclusion:** Counts tests, not controls. 18 of 33 controls were not tested (see audit program). Conclusions are limited to the stated populations.
- **Note:** Tested controls are a risk-based selection, not a random sample, so this is not a rate for the whole control environment.

### K05 Vendors with High residual risk

- **Value:** 2 vendors
- **Definition:** Assessed vendors whose residual risk after considering the evidence is High.
- **Calculation:** `count(vendor_risk_register where residual_risk = 'High')`
- **Source:** projects/03-third-party-risk/vendor-risk-register.csv
- **Inclusion / exclusion:** Only the 3 vendors assessed in Project 3; the other vendors in the inventory were not assessed.

### K06 AI use cases approved to proceed (unconditionally)

- **Value:** 0 use cases
- **Definition:** AI use cases with an unconditional approval decision.
- **Calculation:** `count(ai_use_case_inventory where decision starts with 'Approved')`
- **Source:** projects/04-ai-governance/ai-use-case-inventory.csv
- **Inclusion / exclusion:** Of 2 use cases: 1 Conditionally approved (limited pilot); 1 Not approved: must not proceed

### K07 AI approval gates evidenced

- **Value:** 0 of 16 gates
- **Definition:** Approval gates for AI use cases with closing evidence on file.
- **Calculation:** `count(gates where status = Done) over count(gates)`
- **Source:** projects/04-ai-governance/ai-approval-gates.csv
- **Inclusion / exclusion:** Vendor-linked gates read their status from the Project 3 tracker.

### K08 Audit-readiness gaps identified

- **Value:** 54 gaps
- **Definition:** Gaps found when 32 unified controls were assessed against the paraphrased framework requirements and Project 1 results.
- **Calculation:** `count(audit-readiness-gap-assessment rows)`
- **Source:** projects/02-unified-controls/audit-readiness-gap-assessment.csv
- **Inclusion / exclusion:** Gap types include missing evidence, operating failures, design weaknesses, mapping uncertainty and 'Not applicable' items.

### K09 Actionable gaps (excluding Not applicable)

- **Value:** 50 gaps
- **Definition:** Gaps that need action, excluding rows recorded as not applicable.
- **Calculation:** `count(gaps where gap_type != 'Not applicable')`
- **Source:** projects/02-unified-controls/audit-readiness-gap-assessment.csv
- **Inclusion / exclusion:** 4 'Not applicable' rows excluded.

### K10 AI evidence review: exception recall (held-out, assisted mode)

- **Value:** 0.93 ratio
- **Definition:** Share of the 43 labelled exceptions in the held-out set that the rules-plus-mock-model pipeline flagged as exceptions.
- **Calculation:** `TP / (TP + FN) from evaluation.json, held-out set of 108 cases`
- **Source:** projects/05-ai-evidence-review/evaluation.json
- **Inclusion / exclusion:** 95% interval 0.814-0.976. The 'model' is a heuristic mock, labels are author judgement, and no human review has been performed.
- **Note:** Illustrative only; not evidence that AI improves audit quality.

### K11 Enterprise risks needing executive acceptance (recorded residual >= 15)

- **Value:** 0 risks
- **Definition:** Enterprise risks whose recorded residual score meets the fictional executive-acceptance threshold.
- **Calculation:** `count(risks where residual_score >= 15)`
- **Source:** data/raw-synthetic/risks.csv
- **Inclusion / exclusion:** Residual scores are the pre-audit values recorded in the risk register. They were NOT re-rated after the audit results; see the heatmap note.

## Charts

- **C1 Findings by severity and status** (data/analysis/findings.csv): Project 1 findings by severity, split into open and closed/accepted.
- **C2 Open findings by time to target date** (data/analysis/findings.csv): Open findings grouped by days from the as-of date to the target remediation date. All findings share one final-report date (2026-09-18), so finding age is the same for each; time to target date is shown instead.
- **C3 Open findings by control domain** (data/analysis/findings.csv; data/raw-synthetic/control_library.csv): Domain taken from the control cited in each finding's criteria reference.
- **C4 Control test outcomes** (data/analysis/control_tests.csv): Project 1 test conclusions. No test concluded fully effective with zero exceptions. 4 tests also noted a design deficiency.
- **C5 Project 1 remediation actions by validated state** (data/analysis/remediation_actions.csv): Management-reported status combined with the audit retest result. 'Reported complete, retest failed' is not counted as done.
- **C6 Open vendor issues by age and severity** (projects/03-third-party-risk/remediation-and-monitoring-tracker.csv): Open or in-progress Project 3 vendor issues; age = as-of date minus opened date.
- **C7 Vendor inherent vs residual risk** (projects/03-third-party-risk/vendor-risk-register.csv): Residual risk applies the documented step rule to the evidence status. Only 3 vendors were assessed. Vendor V-02 moves up because of unanswered questionnaire items.
- **C8 AI use-case decisions and risk** (projects/04-ai-governance/ai-use-case-inventory.csv): Committee decision, risk ratings and gate progress for the two proposed AI use cases.
- **C9 Enterprise risk heatmap (recorded residual, pre-audit)** (data/raw-synthetic/risks.csv): Count of the 14 enterprise risks at each recorded residual likelihood and impact (5 x 5). These are the scores recorded before the audit. They have not been re-rated; the table on the page shows open audit findings against each risk as a signal for management to re-rate.
- **C10 Enterprise risks and open audit findings** (data/raw-synthetic/risks.csv; data/analysis/findings.csv): Open findings linked to each risk through the control cited.
- **C11 Audit-readiness gaps by type and roadmap phase** (projects/02-unified-controls/audit-readiness-gap-assessment.csv): Phase = the 30/60/90-day roadmap slot each gap is scheduled in.
- **C12 AI evidence review variants (held-out 108 cases)** (projects/05-ai-evidence-review/evaluation.json): Held-out evaluation of three review variants. Mock model, author-labelled synthetic cases, no human review. Recall confidence intervals for rules-only and assisted overlap.

## Known limitations

- Small synthetic populations: one record can move a percentage by several points.
- All Project 1 findings share one final-report date, so finding age is uniform; time to target date is charted instead.
- Only 3 of 9 vendors and 2 AI use cases were assessed.
- The AI evidence-review indicator comes from a mock model on author-labelled cases and has not been reviewed by a human.