Objective Assessment of Point-of-Care Ultrasound (POCUS) Competency Using Arm Motion Data and Machine Learning Classifiers

10 pages•Published: July 12, 2024

Ryan Harari, Nicole Duggan, Madeline Schwid, Andrew Eyre, Lauren Selame, Munna Dashti, Andrew Goldsmith and Roger Dias

Abstract

Point-of-care ultrasound (POCUS) is becoming an increasingly important tool for diagnostic evaluation, clinical decision-making, and procedural guidance in the emergency department (ED). POCUS image acquisition is cognitively demanding and operator-dependent, making rigorous competency assessment critically valuable. Traditional methods for assessing POCUS competency are limited by subjectivity, requiring a more objective approach. In this study, we aimed to investigate an objective method for assessing POCUS competency using arm motion data and employing machine learning (ML) classification methods. We utilized a motion-capturing system to extract motion data while ED clinicians performed POCUS tasks. We used logistic regression (LR) and random forest (RF) classifiers to predict the expertise level (expert versus novice) based on flexion, abduction, and pronation motion metrics from the right and left wrists and elbows. The mean accuracy of the LR model was 0.80 (95% CI [0.76, 0.84]) with an area under the curve (AUC) of 0.84. The mean accuracy of the RF model was 0.91 (95% CI [0.89, 0.93]) with an AUC of 0.95. These results suggest that both methods show promise for predicting the level of clinical expertise based on arm motion data during POCUS, with the RF model outperforming LR in terms of accuracy. Our finding highlights the potential of using motion capture data and ML approaches to objectively evaluate POCUS competency with high accuracy for distinguishing between novice and expert ED clinicians. Future studies should include larger sample sizes to further improve the accuracy of the models, as well as to investigate other ML techniques.

Keyphrases: competency, decision making, machine learning, point of care ultrasound (pocus)

In: Kenneth Baclawski, Michael Kozak, Kirstie Bellman, Giuseppe D'Aniello, Alicia Ruvinsky and Candida Da Silva Ferreira Barreto (editors). Proceedings of Conference on Cognitive and Computational Aspects of Situation Management 2023, vol 102, pages 90-99.

Links:	https://easychair.org/publications/paper/wmBVV
	https://doi.org/10.29007/6xlf

BibTeX entry

@inproceedings{CogSIMA2023:Objective_Assessment_Point_Care,
  author    = {Ryan Harari and Nicole Duggan and Madeline Schwid and Andrew Eyre and Lauren Selame and Munna Dashti and Andrew Goldsmith and Roger Dias},
  title     = {Objective Assessment of Point-of-Care Ultrasound (POCUS) Competency Using Arm Motion Data and Machine Learning Classifiers},
  booktitle = {Proceedings of Conference on Cognitive and Computational Aspects of Situation Management 2023},
  editor    = {Kenneth Baclawski and Michael Kozak and Kirstie Bellman and Giuseppe D'Aniello and Alicia Ruvinsky and Candida Da Silva Ferreira Barreto},
  series    = {EPiC Series in Computing},
  volume    = {102},
  publisher = {EasyChair},
  bibsource = {EasyChair, https://easychair.org},
  issn      = {2398-7340},
  url       = {/publications/paper/wmBVV},
  doi       = {10.29007/6xlf},
  pages     = {90-99},
  year      = {2024}}

Download PDF Open PDF in browser