Physical Therapist and Physical Therapist Assistant Clinical

2023 No. 007

Physical Therapist and Physical Therapist Assistant

Clinical Performance Instruments: Validation Study

Technical Brief

Prepared

for:

Steven B. Chesbro, PT, DPT, EdD

American Physical Therapy Association

3030 Potomac Avenue, Suite 100

Alexandria, VA 22305-3085

Authors:

Brittany F. Crawford

Andrea L. Sinclair

Date:

January 31, 2023

 
PT and PTA Clinical Performance Instruments: Validation Study | ©2023 American Physical Therapy Association  i 
Physical Therapist and Physical Therapist Assistant Clinical  
Performance Instruments: Validation Study  
Table of Contents 
Introduction ................................................................................................................................... 2 
Clarifying the Intended Score Uses of the CPIs ............................................................................ 2 
Formative Uses (Low-Stakes) ................................................................................................. 3 
Summative Use (High-Stakes) ................................................................................................ 3 
Inappropriate Uses .................................................................................................................. 3 
Revisiting Alignment with Essential Elements of Clinical Practice ................................................ 4 
Revising the PT and PTA CPI Performance Criteria and Rating Scales ...................................... 5 
Revising the PT and PTA CPI Scoring Model ............................................................................... 6 
Summary ....................................................................................................................................... 7 
References .................................................................................................................................... 9 
 
List of Tables 
 
Table 1. Preliminary Passing Standard and Scoring Model for the PT and PTA CPI ................... 7 
 
Developed and produced by HumRRO for the APTA Clinical Performance Instruments for PTs 
and PTAs. ©2023 American Physical Therapy Association. All rights reserved.

Physical Therapist and Physical Therapist Assistant Clinical

Performance Instruments: Validation Study

Introduction

The Physical Therapist (PT) and Physical Therapist Assistant (PTA) Clinical Performance

Instruments (CPIs) are performance assessments designed to evaluate student performance during

their clinical experiences. The CPI is completed by the Clinical Instructor (CI) at midterm and at the

end of the clinical experience. Students also complete a self-evaluation using the same tool. During

the student’s clinical experience, the CI provides opportunities that allow the student to practice

specific skills and behaviors that correspond with the student’s current level of academic preparation

(American Physical Therapy Association, 2018).

The PT CPI was developed in 1997 and revised in 2006. The PTA CPI was developed in 1998 and

revised in 2009. Both instruments underwent content and format changes and were adapted from

paper-and-pencil administration to online administration. The PT WebCPI was launched in 2008 and

the PTA WebCPI was launched in 2010. Since the CPIs were last revised, the American Physical

Therapy Association (APTA) has gathered anecdotal evidence from users suggesting that some of

the CPI’s performance criteria are redundant and lack clarity, leading to inconsistent ratings and

results (Sinclair, 2020; Wetherbee et al., 2018).

In 2020, the Human Resources Research Organization (HumRRO) partnered with APTA to conduct

the first phase of the CPI evaluation research, which included an in-depth review of CPI

documentation and research, including materials from the Liaison International CPI Help Center

website pertaining to the CPIs (e.g., rater training materials), published research on the CPIs,

and

APTA’s internal documentation gathered from CPI users. Additionally, HumRRO reviewed the PT and

PTA CPIs through an account provided by APTA to better understand the functionality of the CPIs.

This review resulted in a series of recommendations by Sinclair (2020) to help increase the reliability

and validity of the CPIs. The recommendations were classified by priority level: urgent priority, high

priority, lower priority, and easier-to-implement, or longer-term recommendations.

This technical brief provides an overview of HumRRO’s work to address one urgent-priority

recommendation and three high-priority recommendations from Sinclair (2020), which thereby

resulted in revised PT and PTA CPIs. Throughout each of the data collection activities described

below, APTA and HumRRO worked collaboratively to include a diverse composition of stakeholders

who provided insight and expertise to inform revisions to the CPIs. The stakeholders represented

individuals in different roles (e.g., Directors of Clinical Education, Clinical Instructors), of varying

tenures (i.e., early versus late career), and in a multitude of work settings (e.g., hospital-based

outpatient facility, private outpatient office, academic institution), thereby ensuring the revised PT and

PTA CPIs are representative of the PTs and PTAs in the field.

Clarifying the Intended Score Uses of the CPIs

During the second phase of this research, HumRRO addressed the first urgent priority

recommendation identified by Sinclair (2020): Clarify the intended score uses of the CPIs (Crawford &

Sinclair, 2022a). According to Standard 1.0 in the Standards for Educational and Psychological

Testing (hereafter Standards), “Clear articulation of each intended test score interpretation for a

See Sinclair, 2020 for the full list of published and unpublished research.

While there was a subset of stakeholders who participated in multiple stakeholder workshops, the majority of

the stakeholders participated in a single workshop.

specified use should be set forth, and appropriate validity evidence in support of each interpretation

should be provided.” (AERA, APA, NCME, 2014, p. 11).

HumRRO conducted the second phase of research to determine the intended score uses of the PT and

PTA CPIs. In February 2022, we conducted six virtual interviews, each with one stakeholder of the PT

and PTA CPIs (interviews were split evenly between users of the PT CPI and users of the PTA CPI).

Specifically, we conducted interviews with two Site Coordinators of Clinical Education (SCCEs), three

Directors of Clinical Education (DCEs), and one Academic Coordinator of Clinical Education (ACCE).

HumRRO facilitators followed an interview protocol that focused on stakeholders’ experiences using CIs’

evaluations along with students’ self-evaluations to inform critical decisions related to students’ clinical

education programs.

Following the virtual interviews, HumRRO facilitated a series of focus group workshops, also in

February 2022, with 13 expert users of the PT and PTA CPIs. Stakeholder composition of the focus

groups included seven DCEs, three ACCEs, two CIs, and one SCCE. As a group, stakeholders were

asked to review each intended CPI use identified from the virtual interviews, offer their thoughts and

opinions regarding the frequency and appropriateness of each use, and engage in discussion with

other participants to resolve disagreements. We also asked the key stakeholders to consider any

additional CPI uses that we might not have identified during the key stakeholder interviews.

Through this process, HumRRO identified four intended uses of the PT and PTA CPIs (Crawford &

Sinclair, 2022a). According to the Standards, assessments often serve more than one purpose

(AERA, APA, NCME, 2014). These purposes can be classified into lower-stakes, formative purposes

and higher-stakes, summative purposes. The stakes of an assessment refer to the importance of the

outcomes. The importance of gathering evidence to support an assessment’s high-stakes purpose(s)

is greater compared to supporting an assessment’s low-stakes purpose(s) because the high-stakes

purposes are typically tied to critical student outcomes (AERA, APA, NCME, 2014). Of the four

intended uses, there was a single summative use that is considered higher stakes than the other

formative uses given that it is tied to a critical student outcome. The four intended uses include:

Formative Uses (Low-Stakes)

• Providing a checkpoint for the student’s progress during their clinical experience while helping to

identify deficits and areas for growth in the student’s performance and/or skills.

• Facilitating the student’s self-assessment of their clinical performance.

• Identifying areas of discordance in evaluation and/or expectations between the CI and the

student.

Summative Use (High-Stakes)

• Guiding the DCE’s decision on the student’s pass-fail status for their terminal clinical

experience.

Standards 4.1 and 6.10 of the Standards establish the need for test developers to identify potential

limitations and inappropriate uses of test results to avoid misinterpretation and misuse of test scores

by test users (AERA, APA, NCME, 2014). Throughout the stakeholder interviews and focus group

workshops, we identified three inappropriate uses of the CPIs (Crawford & Sinclair, 2022a):

Inappropriate Uses

• Determining if a student is ready to sit for the Board exam.

• Making comparisons about the relative effectiveness of education programs.

• Using the CPI as the single deciding factor for whether a student should be recommended for

removal from the PT/PTA program.

Revisiting Alignment with Essential Elements of Clinical Practice

During the third phase, HumRRO addressed the first high-priority recommendation identified by

Sinclair (2020): Revisit Alignment with Essential Elements of Clinical Practice (Crawford, et al.,

2022a). When the PT CPI was developed in 1997, the APTA Board of Directors recommended that

CPIs be reviewed every three years to maintain alignment with current APTA documentation and

terminology (Roach et al., 2012, p.417). HumRRO found no documentation of such reviews. Thus,

HumRRO compiled the most recent professional documentation and guidelines on PT and PTA

practice and then re-examined the alignment of the CPIs’ content to the most recent practice

standards. We also identified any important changes to the profession since the CPIs were last

revised over 15 years ago. This was an important step in supporting the validity of the CPIs, as

accurate interpretation of indicators of performance on a set of standards relies heavily on the

alignment of the assessment to the standards (AERA, APA, & NCME, 2014).

With support from APTA, HumRRO compiled the most recent professional documentation and

guidelines against which the content on the existing CPIs was evaluated, including (a) Commission on

Accreditation in Physical Therapy Education’s (CAPTE’s) Standards and Required Elements for

Accreditation of Physical Therapist/Physical Therapist Assistant Education Programs (CAPTE, 2020a,

2020b), (b) APTA’s Core Competencies of a Physical Therapist Resident (2020), (c) APTA’s Core

Values for the Physical Therapist and Physical Therapist Assistant (2021), and (d) the findings from a

recent practice analysis conducted by HumRRO and the Federation of State Boards of Physical

Therapy (FSBPT; Harris et al., 2021) that included an evaluation of ongoing and emerging trends in

entry-level PT requirements and changes in the profession that necessitated adjustments to PT

licensure examinations. HumRRO developed separate PT and PTA workbooks that included

descriptions of the relevant practice standards for stakeholders to use as a reference in the subsequent

focus group workshops.

In March 2022, HumRRO facilitated a series of focus group workshops to gain an initial understanding

of the alignment of the current CPI content to current practice. Seventeen stakeholders participated,

including nine DCEs, four ACCEs, two SCCE/CIs, and two CIs. Stakeholders reflected and provided

feedback on a variety of topics, including (a) their experiences evaluating PT or PTA students in their

clinical experience, (b) the most important skills and behaviors that should be evaluated during a

student’s clinical experience, (c) performance criteria or essential skills that are not captured by the CPI

content, (d) performance criteria that could be considered outdated or redundant, (d)

language/terminology that could be considered more current or intuitive, and (e) examples of how the

CPI content could be reorganized. Then, HumRRO consolidated the feedback from the stakeholder

workshops to develop a “Content Re-examination Survey.”

The Content Re-examination Survey was administered to gather input from a larger, representative

sample of experts on the relevancy of the existing CPI content to current practice and updated

content recommendations that were provided during the focus group workshops. The survey link was

distributed by APTA via email to 58,814 CPI users and shared by APTA via the Clinical Education

Hub (approximately 344 subscribers), the National Consortium of Clinical Educators email newsletter

(425 members), and the Clinical Education Special Interest Group Discussion Forum (approximate

number of subscribers/users unknown). Responses were collected between April 19

- May 3

, 2022,

and the final analytic sample consisted of 2,253 PT and/or PTA CPI users.

The findings from the Phase 3 data collection activities indicated that while most stakeholders agree

that the previous PT and PTA CPI content was comprehensive and aligned with current practice

standards, there was a critical need to restructure, clarify, and consolidate the PT and PTA CPI

content (Crawford et al., 2022a). Specifically, stakeholders vocalized their desires for more user-

friendly CPIs—namely, simplified language, elimination of redundancies, and a less time-consuming

format.

Revising the PT and PTA CPI Performance Criteria and Rating Scales

To address stakeholders’ concerns from the third phase, the fourth phase of research included two

tasks: (a) revising the CPI performance criteria and (b) revising the CPI rating scale.

The first task (revising the performance criteria) included three steps. In Step 1, HumRRO further

analyzed the open-ended comments in the Content Re-examination Survey. In total, 464 and 122

open-ended responses were analyzed from the PT and PTA CPI surveys, respectively. Most of the

responses pertained to collapsing specific performance criteria, for example, collapsing Examination

and Evaluation on both CPIs and collapsing interventions on the PTA CPI. In Step 2, the most

common suggestions from Step 1 were presented to 11 stakeholders (ten CIs and one DCE) as part

of a content revision activity, which took place in July 2022. In this content revision activity, the

stakeholders independently drafted (a) revised performance criteria and descriptions and (b) sample

behaviors for each level of performance on the rating scale. Following Step 2, HumRRO compiled the

stakeholder input and drafted revised PT and PTA CPI performance criteria and descriptions. In Step 3,

the revised CPIs were reviewed by four additional stakeholders (all current CIs) in August 2022. Most of

the revisions focused on clarifying the descriptions for performance criteria and splitting previously

consolidated performance criteria that stakeholders felt covered distinct concepts (e.g., splitting ‘Ethical

Practice’ into ’Ethical Practice’ and ‘Legal Practice’). At the conclusion of these three steps, the revised

PT and PTA CPIs included 12 and 11 performance criteria, respectively (Crawford et al., 2022b).

For the second task, we revised the PT and PTA CPI rating scales. This was the second high-priority

recommendation identified by Sinclair (2020). As Sinclair (2020) noted, a 2003 study found that raters

were only able to discriminate between six levels of performance (Straub & Campbell, 2003), yet the

rating scales for the CPIs contained 21 performance levels (i.e., intervals). Thus, HumRRO revised

the rating scales to have six levels. Then, in late August 2022, HumRRO facilitated focus group

workshops to brainstorm sample behaviors for each performance level and for each newly drafted

performance criteria. Nine stakeholders (three CIs, three DCEs, two ACCEs, and one SCCE) were

provided the CPIs’ current list of sample behaviors for each performance criterion as a reference

while drafting their own sample behaviors. HumRRO consolidated all the sample behaviors recorded

in the stakeholders’ individual workbooks and used that information to develop sample behaviors for

each performance criterion.

Given the overlap in many of the stakeholders’ sample behaviors for

neighboring performance levels (e.g., Beginning Performance and Advanced Beginner Performance),

we created “performance levels spans” such that each set of sample behaviors spans across two

neighboring performance levels. We also included percentage ranges for level of supervision and

caseload for each performance level span to help guide raters in their evaluations. These

percentages were adapted from the previous versions of PT and PTA CPIs. Finally, the revised PT

and PTA rating instructions and rating scales were reviewed by four stakeholders (three CIs and one

DCE) who were familiar with both CPIs (i.e., PT and PTA). Each revised rating scale includes four

important pieces of information to help ensure that CIs are accurately evaluating students during their

clinical placements:

• A description of the performance criterion,

This resulted in what is called a Behaviorally Anchored Rating Scale or “BARS.”

• Statements or “anchors” that describe the six performance levels (Beginning Performance,

Advanced Beginner Performance, Intermediate Performance, Advanced Intermediate

Performance, Entry-Level Performance, and Beyond Entry-Level Performance),

• Percentage ranges for the student’s level of required clinical supervision and caseload (except

for the ‘Professionalism’ domain where these benchmarks are less applicable), and

• Sample behaviors that further clarify the performance level spans.

Revising the PT and PTA CPI Scoring Model

The fifth and final phase addressed the third high-priority recommendation identified by Sinclair

(2020): Revisit the Scoring Model (Crawford & Sinclair, 2022b). In October 2022, HumRRO facilitated

focus group workshops to gather feedback on the preliminary passing standard and scoring model.

Stakeholders included eight DCEs, six ACCEs, five CIs, and one Assistant DCE. The large majority of

the PTA stakeholders had 15 years or more experience and the majority of the PT stakeholders had

five or more years of experience, with no one in either stakeholder group having fewer than two years

of experience in their current role. Per the Standards:

The process must be such that well-qualified participants can apply their knowledge and

experience to reach meaningful and relevant judgments that accurately reflect their

understandings and intentions. A sufficiently large and representative group of participants

should be involved to provide reasonable assurance that the expert ratings across judges are

sufficiently reliable and that the results of the judgements would not vary greatly if the process

were replicated. (AERA, APA, & NCME, 2014; p. 101).

Notable revisions resulting from their feedback included revising the PT and PTA rating scales to

better delineate the sample behaviors for the Advanced Intermediate Performance and Entry-Level

Performance levels and reframing the percentage caseload and supervision descriptions for the

Entry-Level and Beyond Entry-Level Performance span in terms of what students are capable of

doing. Given these changes, a second group of PT and PTA stakeholders (three DCEs, two ACCEs,

and one CI) reviewed and provided feedback on the revisions. The only remaining suggestion was to

include a checkbox at the end of each CPI where CIs can indicate if there are any safety concerns

during the student’s terminal clinical experience (i.e., for the sake of documentation and potential

legal purposes). This checkbox would be followed by a narrative feedback box where CIs can

describe their safety concerns.

Stakeholders agreed that the CPIs should continue to use a non-compensatory scoring model such

that a student is required to earn a score of Entry-Level Performance (i.e., a ‘5’) or greater on each

performance criterion to pass their terminal clinical experience. As one stakeholder mentioned, these

performance criteria should be viewed as core competencies. Stakeholders also preferred the non-

compensatory scoring model, as opposed to a compensatory scoring or hybrid model where students

can “make-up” for poor performance on one criterion by having high performance on another

criterion, because they believe it will help provide consistency in CIs’ ratings. Specifically, they felt a

hybrid scoring model could introduce too much flexibility and inconsistency into the evaluation

We refer to this as a “preliminary passing standard” given that the revised CPIs have not been released for

operational use. We recommend that the preliminary passing standard be revisited after the instruments have

been used operationally and data (i.e., ratings on the CPIs) is available to investigate the impact of the

preliminary passing standard.

process because the specific performance criteria requiring a rating of Entry-Level Performance or

greater could vary across clinical settings.

The PTA stakeholders also stated that the three intervention-related performance criteria under the

‘Technical/Procedural’ domain should include a ‘Not Applicable’ indicator on the PTA CPI because

PTA students may or may not have the opportunity to perform certain types of interventions

depending on the clinical setting in which they practiced (e.g., pediatrics). As a result, CIs may

indicate ‘Not Applicable’ on one or more of the PTA interventions performance criteria; however, ‘Not

Applicable’ ratings must be accompanied by a brief written rationale. An overview of the final scoring

model and the passing standard is provided in Table 1.

Table 1. Preliminary Passing Standard and Scoring Model for the PT and PTA CPI

Performance

Criteria

Description

PT CPI

Entry-Level Performance or greater on all performance criteria

PTA CPI

Entry-Level Performance or greater on all applicable performance criteria; Not

Applicable ratings must be accompanied by a written rational

Potential non-applicable performance criteria are limited to:

• Interventions: Therapeutic Exercise and Techniques

• Interventions: Mechanical and Electrotherapeutic Modalities

• Interventions: Functional Training and Application of Devices and

Equipment

Finally, we asked stakeholders to describe the pros and cons of requiring raters to provide narrative

feedback on performance criteria only when the student’s performance on the criterion falls below a

specific, agreed-upon level. Nearly all stakeholders agreed that narrative feedback should remain a

requirement for each performance criterion regardless of the student’s performance on that specific

criterion. Stakeholders felt that the students benefit greatly from the written descriptions of their

strengths and weaknesses. Thus, removing or reducing this requirement could lead to less useful

feedback for students.

The final revised CPIs can be found in Appendices A and B for PT and PTA, respectively.

Summary

Throughout the previous phases of research, HumRRO and APTA partnered to streamline the PT

and PTA CPI content and rating scales and, as a result, continued to build support for the validity and

reliability of each CPI. During Phase 1, we gained a deeper understanding of the areas in which to

improve the CPIs, resulting in the recommendations that drove each subsequent phase of research

(Sinclair, 2020). During Phase 2, we clarified the intended score uses of the CPIs and identified the

single summative use of the CPIs: Guiding the Directors of Clinical Education’s (DCE’s) decision on

the student’s pass-fail status for their terminal clinical experience (Crawford & Sinclair, 2022a). Phase

3 provided an opportunity to re-examine the alignment of the current CPI content to the most recent

practice standards and identify any important changes to the profession since the CPIs were last

revised over 15 years ago (Crawford, et al., 2022a). Phase 4 resulted in two significant updates to the

CPIs: (a) restructured, clarified, and consolidated performance criteria for the PT and PTA CPIs,

reducing the PT CPI from 18 to 12 performance criteria and the PTA CPI from 14 to 11 performance

criteria and (b) a new rating scale that includes six anchor points and sample behaviors for each

performance span of each performance criterion (Crawford, et al., 2022b). Finally, during the fifth

phase, we worked with PT and PTA stakeholders to set a preliminary passing standard and scoring

model for each CPI (Crawford & Sinclair, 2022b).

Each of the previous phases of research and any additional research conducted on the revised

instruments helps build support for the validity and reliability of the CPIs for their high-stakes,

summative use (i.e., Guiding the DCE’s decision on the student’s pass-fail status for their terminal

clinical experience). After the instruments have been used operationally and data is available,

HumRRO and APTA will be able to investigate the impact of the preliminary passing standard and,

over time, collect additional evidence to support the reliability of the revised PT and PTA CPIs.

 
PT and PTA Clinical Performance Instruments: Validation Study | ©2023 American Physical Therapy Association  9 
References 
American Educational Research Association, American Psychological Association, & National 
Council on Measurement in Education (Eds.). (2014). Standards for educational and 
psychological testing. American Educational Research Association. Retrieved from: 
https://www.testingstandards.net/uploads/7/6/6/4/76643089/9780935302356.pdf   
American Physical Therapy Association (2018). Using the APTA physical therapist clinical 
performance instrument for students: A self-guided training course. 
American Physical Therapy Association (2020). Core competencies of a physical therapist resident. 
Retrieved from: https://abptrfe.apta.org/globalassets/abptrfe/for-programs/clinical-
programs/abptrfe-core-competencies-physical-therapist-resident.pdf  
 
American Physical Therapy Association (2021). Core values for the physical therapist and physical 
therapist assistant. Retrieved from: https://www.apta.org/apta-and-you/leadership-and-
governance/policies/core-values-for-the-physical-therapist-and-physical-therapist-assistant    
 
Commission on Accreditation in Physical Therapy Education (CAPTE) (2020a). Standards and 
required elements for accreditation of physical therapist education programs. Retrieved from: 
https://www.capteonline.org/globalassets/capte-docs/capte-pt-standards-required-
elements.pdf  
 
Commission on Accreditation in Physical Therapy Education (CAPTE) (2020b). Standards and 
required elements for accreditation of physical therapist assistant education programs. 
Retrieved from: https://www.capteonline.org/globalassets/capte-docs/capte-pta-standards-
required-elements.pdf  
 
Crawford, B. F., Borawski, E. A., & Sinclair, A. L. (2022a). Alignment of the Physical Therapist and 
Physical Therapist Assistant Clinical Performance Instrument Content to Current Practice 
Standards (2022 No. 052). Human Resources Research Organization.  
 
Crawford, B. F., May, M., Sinclair, A. L., & Borawski, E. A. (2022b). Revising the Physical Therapist 
and Physical Therapist Assistant Clinical Performance Instrument Performance Criteria and 
Rating Scales (2022 No. 104). Human Resources Research Organization.  
 
Crawford, B. F. & Sinclair, A. L. (2022a). Intended Uses of the Physical Therapist and Physical 
Therapist Assistant Clinical Performance Instruments (2022 No. 037). Human Resources 
Research Organization.  
 
Crawford, B. F. & Sinclair, A. L. (2022b). Developing a Preliminary Passing Standard and Scoring 
Model for the Revised Physical Therapist and Physical Therapist Assistant Clinical 
Performance Instruments (2022 No. 116). Human Resources Research Organization.  
 
Harris, J. L., Rogers, A. P., & Caramagno, J. P (2021). Analysis of practice for the physical therapy 
profession: Report memo 2021 (2021 No. 100). Human Resources Research Organization.  
 
Roach, K. E., Frost, J. S., Francis, N. J., Giles, S., Nordrum, J. T., & Delitto, A. (2012). Validation of 
the revised physical therapist clinical performance instrument (PT CPI): Version 2006. 
Physical Therapy, 92 (3), 416-428. https://academic.oup.com/ptj/article/92/3/416/2735258  
 

Sinclair, A. L. (2020). Research studies to support reliability and validity of the PT CPI and PTA CPI

(2020 No. 072). Human Resources Research Organization.

Straube, D. & Campbell, S.K. (2003). Rater discrimination using the visual analog scale of the

physical therapist clinical performance instrument. Journal of Physical Therapy Education, 17

(1), 33-38.

Wetherbee, E., Dupre, A. M., Feinn, R. S., & Roush, S. (2018). Relationship between narrative

comments and ratings for entry-level performance on the clinical performance instrument A

call to rethink the clinical performance instrument. Journal of Physical Therapy Education,

32(4), 333-343.