Location: Institute for Simulation and Training, Partnership
II, Room 235.
Dissertation Title: The Development and Validation of a Measurement System to Assess Intensive Care Unit Team Performance
Explanation: Teamwork is essential for ensuring the quality and safety of healthcare delivery in the intensive care unit (ICU). Complex procedures must be executed with a diverse team of clinicians with unique roles and responsibilities. Information about care plans and goals must also be developed, communicated, and coordinated across multiple disciplines and transferred effectively between shifts and personnel. The intricacies of routine care are compounded during emergency events, which require ICU teams to adapt to rapidly changing patient conditions while facing intense time pressure and conditional stress. Realities such as these emphasize the need for teamwork skills in the ICU.
The scientific community has responded to these challenges by developing systems to optimize and quantify team performance in the ICU. Despite the expansion of ICU teamwork and team training literature, there is no unifying theoretical framework to conceptualize team performance or to operationalize team constructs. Further complicating the scientific study of teamwork is the absence of a standardized lexicon to describe team constructs, which has resulted in inconsistent term use across this care setting. Without a strong theoretical foundation, the link between teamwork and safety and performance outcomes may be misrepresented or misleading. The purpose of this study is to develop and empirically validate the first behavioral marker system able to measure ICU team performance across multiple task types. As measurement should be grounded in theory, this study first developed a framework of ICU team performance to guide measurement. The significance of developing a behavioral marker system is to provide ICUs across the nation with a standardized, theoretically-based mechanism to (1) discuss central components of teamwork, (2) assess team performance, (3) structure feedback around core team competencies, and (4) evaluate the impact of future team improvement initiatives. Equally important to developing good theory is ensuring that measurement instruments accurately elicit operationalizations of constructs; if observed scores are the product of bias rather than manifestations of target constructs, the researcher is unable to draw valid conclusions about research findings. Collecting multiple forms of reliability and validity evidence is the psychometric paragon for demonstrating a measurement tool is actually measuring what it intends to measure. In addition to calculating traditional forms of reliability evidence (intraclass correlations and percent agreement), this study also modeled the systematic variance associated with raters by applying generalizability (G) theory. G theory was also employed to provide evidence that the marker system adequately discriminates among teamwork competencies targeted for measurement. The confluence of evidence supported the prediction that the marker differentiates among teamwork subdmensions. That said, different reliability indices suggested varying levels of confidence in rater consistency depending on the teamwork competency that was measured. Implications of these findings for future research and development are discussed.
Read More