Research Fellow University of Pittsburgh School of Medicine Nutley, NJ, US
Introduction: In the rapidly evolving healthcare field, the integration of artificial intelligence (AI) is set to transform diagnostic processes, including the interpretation of clinical documentation. Microsoft Copilot, an enterprise-level AI tool, offers a promising solution for enhancing the efficiency and accuracy of medical report summarization. The enterprise version of Copilot uses Microsoft’s 365 permissions model to ensure that the AI model is not trained on inputted information and that data does not leak between users. This makes it an ideal tool for handling protected health information compared to other AI tools. This study aims to evaluate the effectiveness of Microsoft Copilot in summarizing radiology reports.
Methods: The study involved 29 neurosurgical clinic patients. For each patient, all radiology reports from the past three months were inputted into the Copilot notebook prompt for summarization. Providers who had independently reviewed these radiology reports were surveyed about the quality of Copilot’s summaries. These providers were neurosurgery residents, fellows, and advanced practice providers. Providers assessed accuracy, content, and organization using a five-point Likert scale relative to their own review. Additionally, the readability of the summaries was evaluated with the Flesch Reading Ease (FRE) using Readability Studio Professional Edition Version 13.2.1 (Oleander Software, Ltd, Vandalia, OH).
Results: All surveys were completed. Copilot's summaries achieved an average accuracy score of 4.69 ± 0.46, content score of 4.45 ± 0.81, and organization score of 4.55 ± 0.72. The mean FRE for the summaries was 23.25 ± 8.70, indicating a high level of complexity.
Conclusion : This pilot study supports the utility of Microsoft Copilot in summarizing radiology reports for neurosurgical clinic patients, with strong provider satisfaction regarding accuracy, content, and organization. Despite its complex readability, Copilot’s potential to reduce the time spent reviewing charts suggests that AI integration into clinical practice can significantly enhance operational efficiency.
.jpg)
