Fine Tuning a Large Language Model to Extract Thoracolumbar Injury Classification Score

Introduction: Thoracolumbar spinal injuries are a common consequence of trauma, requiring accurate classification for guiding clinical management. The Thoracolumbar Injury Classification and Severity Score (TLICS) is widely used for evaluating the severity of these injuries, assisting in determining the need for surgical intervention. However, manual extraction and classification of injury data from many different parts of the patient chart is time consuming. This study aims to fine-tune a large language model (LLM) to automatically extract TLICS parameters from clinical texts, thereby streamlining the classification process.

Methods: The large language model Llama 3 was fine-tuned on a curated dataset of 1085 spine radiology reports. The dataset was annotated by two raters for injury morphology and integrity of the posterior ligamentous complex. 100 random patients from this set were selected and their physical exam notes were annotated by two raters for neurological status. 19 exams were used for prompt engineering, and the rest were reserved for testing. The model’s performance was evaluated on the final test set of 81 radiology reports and physical exams.

Results: The fine-tuned LLM demonstrated an overall accuracy of 93%, 91%, and 90% at detecting injury morphology, PLC status, and neurological status respectively.

Conclusion : Fine-tuning a large language model to extract TLICS parameters from clinical notes presents a promising method for automating the classification of thoracolumbar injuries. The model’s high accuracy and efficiency suggest it could support clinical decision-making by providing consistent and rapid classifications, reducing clinician workload and improving the timeliness of patient care. Further refinement of the model, especially for more nuanced cases, may enhance its utility in broader clinical settings.