Ignite Creation Date:
2024-05-06 @ 8:22 PM
Last Modification Date:
2024-10-26 @ 3:26 PM
Study NCT ID:
NCT06351943
Status:
ACTIVE_NOT_RECRUITING
Last Update Posted:
2024-04-08
First Post:
2024-03-26
Brief Title:
Proximal Femur Image Database Validation
Sponsor:
AO Innovation Translation Center
Organization:
AO Innovation Translation Center
Study Overview
Official Title:
Validation of the Fracture Classification Accuracy Ground Truth of Anteroposterior X-ray of the Proximal Femur According to the Arbeitsgemeinschaft für OsteosynthesefragenOrthopedic Trauma Association Classification Done by a Single Center A Pilot Validation Study
Status:
ACTIVE_NOT_RECRUITING
Status Verified Date:
2024-03
Last Known Status:
None
Delayed Posting:
No
If Stopped, Why?:
Not Stopped
Has Expanded Access:
False
If Expanded Access, NCT#:
N/A
Has Expanded Access, NCT# Status:
N/A
Acronym:
None
Brief Summary:
The AOAI Turin project is a collaborative project with a Turin group and the AO Arbeitsgemeinschaft für Osteosynthesefragen or in English Association for the Study of Internal Fixation foundation An Image database DB has been built to host AP pelvic radiographs ready for artificial intelligence AI development
The goal of this project is to determine the agreement between the Turin annotation of fracture status and the annotation from an external group of AO expert surgeons for a random subset of the Turin images
The goal of this project is to determine the agreement between the Turin annotation of fracture status and the annotation from an external group of AO expert surgeons for a random subset of the Turin images
Detailed Description:
The AOAI Turin project is a collaborative project with a Turin group who has collected 2932 anteroposterior AP pelvic radiographs of which 1811 are fracture images and 1121 are non-fracture images The Turin group has developed an artificial intelligence AI algorithm for fracture classification using these images These anonymized images with all metadata or personal identifiers removed have been uploaded to a cloud-based image database DB hosted and managed by the AO Foundation
The Turin group has established the ground truth using the methods of consensus by experts Two radiologists from their medical team have reviewed and classified the fracture status fracture vs non-fracture and if fracture the AOOrthopedic Trauma Association OTA classification
The next steps goal is the ground truth validation plan to test the accuracy of the Turin annotation of fracture classification of the already uploaded AP pelvic images This is to ensure that the image DB offers accurate quality annotations to allow AI development
For the pilot phase a random subset of the Turin images 300 of images will be drawn from the image DB These images will be reviewed by an external group of AO expert surgeons who will annotate the images per their fracture status ie fracture vs non-fracture and if fracture the AOOTA classification
The group of AO expert surgeons consists of four surgeons who will independently review the 300 images and a fifth surgeon who serves as an adjudicator if necessary The expert surgeons will be given access to the 300 images via the cloud-based image DB and annotate the images The expert surgeons will be blinded to the Turin annotations The expert surgeons annotations will be entered into a DB built for the purpose for the pilot study
To determine the ground truth the annotations of the four surgeons will be compared and discrepancies will be identified A meeting will then be arranged among the surgeons to resolve by consensus the discrepancies with the potential involvement of the fifth surgeon as the adjudicator After the resolution meeting there will be a single set of annotations for the 300 images from the exert surgeon group
The Turin annotations will also be entered into the study DB to allow comparisons with the expert surgeon groups annotation
In case of disagreement between the Turin annotation and the AO expert surgeon annotations a consensus will be sought to establish a new ground truth If this process results in significant revisions to the annotations the entire dataset will be reviewed to set this new standard Following such a comprehensive dataset revision the algorithm for automated fracture classification of the proximal femur which has already been developed by the Turin group will be re-trained After re-training the algorithms performance will be evaluated through metrics such as precision recall and F1-score to ensure its accuracy and effectiveness in classifying proximal femur fractures
The Turin group has established the ground truth using the methods of consensus by experts Two radiologists from their medical team have reviewed and classified the fracture status fracture vs non-fracture and if fracture the AOOrthopedic Trauma Association OTA classification
The next steps goal is the ground truth validation plan to test the accuracy of the Turin annotation of fracture classification of the already uploaded AP pelvic images This is to ensure that the image DB offers accurate quality annotations to allow AI development
For the pilot phase a random subset of the Turin images 300 of images will be drawn from the image DB These images will be reviewed by an external group of AO expert surgeons who will annotate the images per their fracture status ie fracture vs non-fracture and if fracture the AOOTA classification
The group of AO expert surgeons consists of four surgeons who will independently review the 300 images and a fifth surgeon who serves as an adjudicator if necessary The expert surgeons will be given access to the 300 images via the cloud-based image DB and annotate the images The expert surgeons will be blinded to the Turin annotations The expert surgeons annotations will be entered into a DB built for the purpose for the pilot study
To determine the ground truth the annotations of the four surgeons will be compared and discrepancies will be identified A meeting will then be arranged among the surgeons to resolve by consensus the discrepancies with the potential involvement of the fifth surgeon as the adjudicator After the resolution meeting there will be a single set of annotations for the 300 images from the exert surgeon group
The Turin annotations will also be entered into the study DB to allow comparisons with the expert surgeon groups annotation
In case of disagreement between the Turin annotation and the AO expert surgeon annotations a consensus will be sought to establish a new ground truth If this process results in significant revisions to the annotations the entire dataset will be reviewed to set this new standard Following such a comprehensive dataset revision the algorithm for automated fracture classification of the proximal femur which has already been developed by the Turin group will be re-trained After re-training the algorithms performance will be evaluated through metrics such as precision recall and F1-score to ensure its accuracy and effectiveness in classifying proximal femur fractures
Study Oversight
Has Oversight DMC:
None
Is a FDA Regulated Drug?:
False
Is a FDA Regulated Device?:
False
Is an Unapproved Device?:
None
Is a PPSD?:
None
Is a US Export?:
None
Is an FDA AA801 Violation?:
None