Cognitive Domains
CAAPES Key Performance Indicators (KPI)
KPI formulas for scoring the subcomponents in CAAPES (provided in the table below) are reported as accuracy-based scores. These scores provide the most meaningful interpretation for performance in CAAPES and have been validated for reliability. It is HIGHLY suggested that these KPIs be used as the primary dependent variables associated with CAAPES performance.
For end-user convenience, IHMC has provided executable data analysis Python scripts for CAAPES KPIs. KPIs are automatically computed through running the CAAPES Score Processor file, which is attached to the CAAPES software package when downloaded.
KPI Scoring Formulas
| KPI Name | Formula |
| Tracking Weighted Accuracy Score | % on Target x Difficulty Level |
| Math Weighted Accuracy Score | % Math Accuracy x Difficulty Level |
| Manikin Weighted Accuracy Score | % Manikin Accuracy x Difficulty Level |
| Memory Weighted Accuracy Score | % Memory Accuracy x Difficulty Level |
| Math Accuracy | Math Correct Responses / Math Total Count |
| Manikin Accuracy | Manikin Correct Responses / Manikin Total Count |
| Memory Accuracy | Memory Correct Responses / Memory Total Count |
| Pensacola Composite (PCOLAC) Score | Tracking Weighted Accuracy (WA) + Math WA + Manikin WA + Memory WA |
Point Based Scoring
Point based scores (provided in the CAAPES Studies logs) are metrics that were designed to provide momentary performance results. These scores are not the primary performance indicator for CAAPES, however they are still available the end-user and can be found in the MOMENTARY log files.
The way point based scores work is each task assigns points based on the participant’s response (correct, incorrect, or no response) and are scaled based on the level of difficulty. For more detailed information on point based scores, refer to the CAAPES user documentation which can be accessed for CAAPES users.
Additional Metrics
Beyond KPIs and scores, CAAPES includes response time-related metrics and difficulty levels for each task, as well as data segmented by specific time windows within a session or iteration. All variables used to compute accuracy and response scores are included in the output logs and script generated results tables. A full list of all available variables and their definitions can be found in the accompanying documentation and data dictionary.