Sat., 10/6/12 General Interest, PAPER #107, 2:36 pm OTA-2012
The PROMIS Physical Function Computerized Adaptive Test Is as Reliable and Valid as the Short Musculoskeletal Function Assessment in the Orthopaedic Trauma Population With Less Ceiling Effect
Man Hung, PhD; Thomas F. Higgins, MD; Charles L. Saltzman, MD; Ami R. Stuart, PhD;
Shirley Hon; Stefan Rhodewalt; Ashley M. Woodbury, BS; Gregory M. Daub, BS; Erik N. Kubiak, MD;
University of Utah Department of Orthopaedic Surgery, Salt Lake City, Utah, USA
Background/Purpose: Understanding the outcomes of treatment is a fundamental step toward improving care of patients. Patient-reported outcomes have been recognized as critically important to guiding treatment decisions and assessing the effectiveness of clinical interventions. For orthopaedic trauma patients, the Short Musculoskeletal Function Assessment (SMFA) is a commonly used questionnaire to determine effectiveness of treatment and has been recommended by the American Academy of Orthopaedic Surgeons. It has been shown to demonstrate excellent internal consistency, content and convergent validity, reliability, and responsiveness. More recently developed is the Patient-Reported Outcomes Measurement Information System Physical Function Computerized Adaptive Test (PROMIS PF CAT) v.1, which dynamically administers questions from an item response theory (IRT)–calibrated bank of 124 questions. Based on the previous response to a question, the CAT selects the most relevant next question to administer. The PROMIS CATs have been shown to be highly reliable while drastically shortening the number of questions required to complete the questionnaires. This study seeks to compare the usability of the PROMIS PF CAT and SMFA in terms of floor and ceiling effects, reliability, and administration time.
Methods: Orthopaedic trauma patients who were seen for operative follow-up or nonoperative fracture care and consultation were invited to participate in this IRB-approved study. When the patient arrived for his or her clinic visit, he or she was asked to consent to participation in the study by a research assistant. Upon receipt of consent, participants’ basic demographic information was recorded. The patient was then given an iPad to complete the SMFA and the PROMIS PF CAT. Standard descriptive statistics such as means, standard deviations, frequencies, and patterns were used to describe patient demographic characteristics. Advanced psychometric techniques were used to evaluate the PROMIS PF CAT and the SMFA. In particular, we used a one-parameter Rasch IRT model to examine the psychometric properties of the instruments. We also calculated the average time required for each patient to complete the instruments.
Results: 153 patients participated in the study. Of those 55.7% were male, 81% self-identified as White, and 11.1% Hispanic or Latino; 53.2% had private insurance, 8.5% had Medicaid, and 22.7% had Medicare. On average, each patient had responded to 4 questions from the PROMIS PF CAT, and responded to all 46 questions from the SMFA. The mean administration time was significantly less for the PROMIS PF CAT than for the SMFA (P <0.05), 44 seconds and 599 seconds respectively. Both instruments showed an extremely high item reliability (Cronbach alpha of 0.98). In terms of instrument coverage, neither instrument showed any floor effect; however, the SMFA revealed 14.4% ceiling effect while the PROMIS PF CAT had none.
Conclusion: Administered by electronic means with one question automatically advanced per page, the PROMIS PF CAT required less than one-tenth the amount of time than the SMFA for patients to complete while achieving equally high reliability as the SMFA. The PROMIS PF CAT also showed desirable psychometric properties while the SMFA shows some restrictions in the upper end of the physical function continuum, which can be problematic for assessing changes in outcomes. This improved instrument coverage and diminished time for patient completion may make the PROMIS PF CAT potentially more effective and useful than the SMFA for both clinical and research purposes. Further study may be needed to examine responsiveness to change of the PROMIS PF CAT over time.
Alphabetical Disclosure Listing (808K PDF)
• The FDA has not cleared this drug and/or medical device for the use described in this presentation (i.e., the drug or medical device is being discussed for an “off label” use). ◆FDA information not available at time of printing. Δ OTA Grant.