Ignite Creation Date:
2024-10-26 @ 3:41 PM
Last Modification Date:
2024-10-26 @ 3:41 PM
Study NCT ID:
NCT06608758
Status:
NOT_YET_RECRUITING
Last Update Posted:
None
First Post:
2024-09-18
Brief Title:
End Diagnostic OvershadowingAddressing Ableism in Diagnoses
Sponsor:
None
Organization:
None
Study Overview
Official Title:
End Diagnostic Overshadowing Addressing Ableism in the Healthcare Context
Status:
NOT_YET_RECRUITING
Status Verified Date:
2024-09
Last Known Status:
None
Delayed Posting:
No
If Stopped, Why?:
Not Stopped
Has Expanded Access:
No
If Expanded Access, NCT#:
N/A
Has Expanded Access, NCT# Status:
N/A
Acronym:
EDO
Brief Summary:
As a result of ableism people with disabilities PWD experience diagnostic overshadowing-attributing symptoms to disability rather than a potentially new or co-morbid condition the goal of this researh is to identify and create understanding of what underlies and contributes to diagnostic overshadowing The investigators plan to develop ways to reduce it specifically ways to identify people with disabilities at risk of diagnostic overshadowingdiagnostic error The investigators will also develop education programs and decision supports targeted to healthcare professionals If it is effective ways to reduce diagnostic overshadowingdiagnostic error will have been developed among people with disabilities
Detailed Description:
Aim 1 Identify and create understanding of mechanisms underlying diagnostic overshadowing
A baseline audit of CPT code usage between patients aged 3-89 years old with and without specified disabilities
The Hypothesis is that there will be statistical difference in use of CPT EM codes between people with the specified disabilities and people without the specified disabilities
2 The investigators also will conduct manual chart reviews on PWDs with lower and higher CPT EM codes addressing demographic characteristics age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site and the following 10 issues suggested by stakeholders caregiver involved transfer referral from other institution verbal co-morbidity numbers and types any refused tests previous history visitsadmissions for same issue ongoing use of medical equipment use of chemical or physical restraints during visits and admissions history of trauma and negative descriptors 15 terms for which Black vs White patients had higher odds ratio for use in medical records
The hypothesis is that information will be developed that can be used in development of mock tracers and in the Participatory Planning and Decision Making process to identify themes that may underly diagnostic overshadowing
3 Based on found differences from baseline audits of use of CPT EM codes and follow-up chart reviews the investigators will conduct targeted interviews with staff then follow-up with stakeholders to develop Joint Commission-style individual mock tracers to follow the care of specific populations of PWD listed above and develop system of care tracers focused on evaluating the system surrounding diagnostic overshadowing including trauma-informed care
The hypothesis is that information will be developed sufficient to develop mock tracers following the care of the specific populations of PWD and systems of care tracers
Aim 2 Co-produce a frame of themes underlying diagnostic overshadowing DO to develop algorithms to identify patients with the specified disabilities at risk for DO Then co-produce case studies for education on DO and EHR prompts alerts and decision supports related to the algorithms along with education on the EHR materials
The hypothesis is that information will be developed that can be used to develop algorithms for identifying PWD from the specific populations at risk of DO as evidenced by CPT EM code usage and DE as evidence by time to diagnostic evaluation
Aim 3 Evaluate for change after implementation of algorithms to identify patients with the specified disabilities at risk for DO
The Hypotheses are that there will be statistical change in CPT EM code usage for PWD from the specified populations
Data evaluation CPT codes were developed by the American Medical Association and undergo periodic revisions and ongoing maintenance CPT codes are the universal way that providers document their services providing standardized reporting needed for billing and reimbursement of healthcare providers including physicians nurse practitioners physician assistants other professionals The system provides numeric codes for issues such as 1 The site of service eg Emergency Department inpatient outpatient preventive services 2 The service provided 3 The complexity of clinical information-gathering and decision-making and 4 Time spent Each setting has a specific group of CPT EM codes ranging from lowest time and complexity of decision making to highest time and complexity of decision making
Incidence among the groups of patients with specific disabilities will be calculated for diagnoses identified as having high incidence of diagnostic error in the general population and for diagnoses identified as problematic for diagnostic error among children Incidence will be calculated on the following using primary or secondary ICD 10 diagnosis codes in the database Aortic aneurysm and dissection Arterial thromboembolism Venous thromboembolism Congestive heart failure Stroke Myocardial infarction Spinal abscess Meningitis and encephalitis Endocarditis Sepsis Pneumonia Lung cancer Melanoma Colorectal cancer Breast cancer Prostate cancer
With focus on pediatric Arterial ischemic stroke Appendicitis Asthma Retinal blastoma Brain tumor Polyateritis Congenital heart disease Duchense muscular dystrophy Inflammatory bowel disease scleroderma
Demographic information on diagnoses will provide background for following processes
Quality assurance The Joint Commission developed mock tracers to use in preparing for accreditation visits and for ongoing quality assurance and professional development efforts Mock tracers provide information on patient experiences quality of care healthcare processes and products and areas needing improvement Tracers involve one-on-one and small group interviews in addition to review of patient charts and forms For this project questions will center around processes and considerations in evaluation and management of patients Deeper inquiry is expected based on answers
EHR prompts and alerts The stage of the diagnostic process requires different clinical decision supports Furthermore conditions that are not common require specific supports Through standard order sets alerts and reminders and other means of diagnostic support eg website clinician can access guidelines more easily
Algorithms Algorithms are in use to address diagnostic error such as identifying patients at risk of delayed test results delays in follow up of chest imaging results tests for hypothyroidism and delayedmissed diagnoses related to abdominal pain In a randomized clinical trial algorithms were used to prospectively identify patients at high risk of delayedmissed diagnoses of lung colorectal or prostate cancer Time to diagnostic evaluation was significantly reduced in the intervention group vs control group for colorectal and prostate cancers but not lung cancers
Co-production of healthcare programs Involving impacted persons in co-production of services impacting them is considered an ethical issue in healthcare transcending the traditional dichotomy between knowledge and program developers and users Co-production involves building collaboration of people from impacted groups in the production and use of knowledge and programs from the start of the process Figure 1 Participation of PWD impacted by the results of research and program planning is often limited to providing input after key decisions have already been made rather than throughout the process However beginning work to involve people with IDD in the co-production of programs for behavioral health indicated improvements in social networks and confidence for participation In addition co-production has been used in developing a framework in healthcare quality improvement In co-production the investigators will work with involved stakeholders in reducing bias in development of algorithms and other materials
Participatory Planning and Decision Making PPDM PPDM is a model-building process based on principles of participatory action research It is designed for groups to make judgments about multiple alternatives and weigh the importance that each of these play with respect to the outcome of interest The process first requires participants to identify or confirm broad domainsthemes and subdomainssubthemes contributing to the phenomenon under study Following the identification of themes and subthemes importance weights are assigned to each and proportional importance weights are then derived There are eight basic phases to the PPDM process beginning with pre-surveys regarding initial themes to presenting participants meetings on the themes and presenting participants with a final model of themes Based on broad input from members of different groups and communities staff at the University of Minnesota Institute on Community Integration have successfully used this process to develop conceptual frames for self-determination and healthcare coordination and to refine the National Quality Forums Home and Community-based Services Outcome Measurement Framework These efforts have resulted in frameworks that possess high content and concurrent and predictive validity The use of the PPDM model and processes enables a diverse group of voices to be heard in the development of targeted education programs and EHR decision supports ensuring that materials fit the needs of the multiple groups In co-production of algorithms as a means of retrospective review for quality assurance and prospective use to identify patients with disabilities at risk of diagnostic overshadowingdiagnostic error the PPDM process is expected to be useful in addressing bias in materials
STUDY ENDPOINTS
Primary At Year 5 compared to Year 1 CPT EM code usage with the five identified groups of people with disabilities quantitative will be evaluated for changes following implementation of algorithms to identify people with disabilities at risk of DO along with EHR decision supports and promptsalerts on specific issues Statistical changes are expected
Education programs will be evaluated through 1 pre- and post- knowledge checks of usage and 2 descriptive data on use of specific EHR decision supports Statistical change is expected in knowledge
Pre-post time to diagnostic evaluation is planned for 2-3 issues still yet to be determined to address delayedmissed diagnoses Time to diagnostic evaluation isexpected to decrease
Secondary For quantitative measures CPT EM code usage analysis knowledge checks descriptive data on use of EHR decision supports and promptsalert ANCOVA will be used to probe for interaction effects using pre-test measures age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site and the 10 issues suggested by stakeholders singly or in composites as covariates and evaluate for variation in post-test results Interaction effects are exxpected
Final mock tracers will be conducted at the end of Year 4 and in Year 5 Tracer notes will be compared to notes before the intervention using qualitative analysis Changes will provide context for any changes in quantitative measures
Framework The Collective Impact Model for social change is the organizing framework for this project Multiple impacted groups are brought to bear on the problem of diagnostic overshadowing Previous efforts to develop algorithms to identify patients at high risk of diagnostic error have not specifically addressed diagnostic overshadowing affecting patients with disabilities and have not previously addressed intersectionality The Collective Impact Model has not previously been used to address diagnostic overshadowing or the overall problem of diagnostic errors The following five tenets must be met to facilitate organization and planning with multiple impacted groups for a Collective Impact project 1 achieving a common agenda 2 ensuring continuous Communication 3 identifying shared measurement strategies 4 employing mutually reinforcing activities to deliver programs and services that will achieve the intended outcome of Collective Impact efforts and 5 employing a dedicated staff as backbone support Partnering requires attention to bringing in the experiences and voices of all impacted groups
Building Organizational Structures using the Collective Impact Model In the first six months members of research team will meet at least once a month to solidify the team hire new staff and create structures based on the Collective Impact Model A Cross-Sector Partnership Steering Committee the Cross-Disability Advocate Advisory Committee and three Consortium Action Networks Communication Measurement Education will be organized The Steering Committee and Cross-Disability Advocate Committee will take overall accountability for developing a shared agenda Collective Impact Tenet 1 Practices that improve understanding of diagnostic overshadowing and the identification of underlying mechanisms will be developed through continuous communication The Communication Action Network will take accountability Collective Impact Tenet 2 The Measurement Action Network will take accountability for ongoing evaluation and final evaluation in Year 5Collective Impact Tenet 3 The Education Action Network will take accountability for facilitating development of targeted education programs and EHR decision supports to mitigate diagnostic overshadowing Collective Impact Tenet 4 The developed infrastructure will facilitate mutually reinforcing activities that encourage the sharing of perspectives and best practices of the projects interdisciplinary partners Processes leading to the achievement of project goals and outcomes will be facilitated by dedicated backbone staff who will assist with the management planning and logistics required by the project RUSH University is the lead institution and each consortium partner has specific responsibilities Collective Impact Tenet 5
Design Aim 1 Identify and create understanding of mechanisms underlying diagnostic overshadowing
Introduction Partnership will be built between three not for profit medical center systems that place prominence on improving health equity RUSH University System for Health and affiliated RUSH University Medical Center RUSH Oak Park Hospital and RUSH Copley Medical Center Rochester Regional Health and affiliated Rochester General Hospital and Erie County Medical Center ECMC These will be sites for pre-post analysis of diagnostic overshadowing via CPT code data implementation of mock tracers and then implementation of targeted education programs and EHR decision supports Data use agreements between the three institutions will be obtained
Data Collection will be in three steps First the investigators will retrieve and auditanalyze for CPT EM code use differences in complexity by the presence of specific disabilities PWD with major mobility impairments mental health concerns severe visual impairments blindness severe hearing lossdeafness and IDD versus no presence of the specified disabilities Patient CPT EM codes data will be from RUSH University Medical Center RUSH Oak Park Hospital RUSH Copley Hospital Rochester General Hospital Erie County Medical Center and associated outpatient practices The investigators will retrieve and analyze CPT EM codes from the EDs codes 99281-99285 CPT EM codes from inpatient services 99221-99223 99231-99233 99238-99239 CPT EM codes from outpatient visits with new patients 99202-99205 with established patients 99211-99215 and for preventive care 99384-99387 to identify potential differences in usage Changes were made to EM codes in 2023 in a way expected to reduce burden Data will be retrieved for the period January 1 2023 through June 30 2024 CPT EM codes will be used as a proxy for diagnostic overshadowing in clinical information-gathering and decision-making complexity EM is a stage in the diagnostic process where errors can occur Quantitative methods for evaluation of CPT EM codes data will be used Data will be programmed with variable range checks and skip rules and will be exported in an automated manner into SPSS Based on experience patients with the specific disabilities will be identified through a comprehensive list of secondary diagnosis codes for the specific disabilities for patients aged 3-89 years old Data will be age-disaggregated in groupings of five years except the group aged 3-5 years old All variables will be checked for errant values Descriptive statistics will be computed for all items CPT EM codes and distributions examined for non-normality and outliers Descriptive statistics for all measures will be reported For each type of visit the investigators will first compare the overall proportion of each CPT EM codes by disability status using pairwise Fisherampampamp39s exact tests with a descriptive analysis of case frequency age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site Evaluation will be conducted on whether data can be collapsed into values of higher level and lower level complexity of evaluation and management codes If so binary logistic regression analysis will be conducted using the outcome of higher level and lower level of complexity of CPT evaluation and management codes and addressing the influence of race ethnicity gender age ranges disability type insurance status severity of illness and 10 chart review questions previously described Otherwise the investigators will use ordinal regression analysis on the outcomes of the CPT EM codes using all levels of complexity Site-specific analyses will be conducted ie ED inpatient outpatient preventive care and a combined model that accounts for site using cluster-robust standard errors
Binary or ordinal regression analysis will be conducted for each setting Emergency Department inpatient outpatient preventive care with the dependent variable being EM codes separately or split into lower and higher complexity codes Independent variables will be demographics of age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site and the 10 chart review issues listed above Variable loadings will be assessed for use in developing algorithms for PWD from the specified groups at risk of diagnostic overshadowing
Second based on found differences from the CPT EM code analyses targeted retrospective manual chart reviews will be conducted to improve identification of underlying mechanisms of diagnostic overshadowing At each partnering institution 5 associated with the three medical centers at least five charts will be reviewed of patients from each of the five specific populations of people with disabilities See previous paragraph from which data are being collected 25 at each of the five institutions and associated outpatient practices - 125 in total
The charts will be reviewed for issues which may provide insight into diagnostic overshadowing In discussions with staff suggestions to explore include the same issues listed above in Previous work Notes on the above ten issues will be taken Notes will be evaluated for themes using inductive thematic analysis The team will meet weekly to discuss themes If during the chart review other issues are determined an IRB amendment to evaluate additional issues will be submitted
Following the chart reviews interviews with staff will be conducted in identified areas to explore perceptions of why there may be differences expecting at least five in-depth interviews and multiple short interviews at each of the five sites that can be conducted during shift huddles and staff meetings
Third with these baseline data the Measurement Action Network and the Advocate Advisory Committee will be consulted on development of separate mock tracers following the care of each specific population of PWD prone to diagnostic overshadowing 25 retrospective tracers will be conducted of patients 5 each from the specific populations listed above with 10 tracers to be among children and at least 10 to be among patients from marginalized racialethnic groups at each of the five partnering institutions with associated outpatient practices 25 each at RUSH University Medical Center RUSH Oak Park Hospital RUSH Copley Hospital Rochester General Hospital and Erie County Medical Center For tracers patients with issues such as high acuity complexity of care eg multiple tests surgeries transfers between units and history of trauma will be chosen recognizing that issues affecting care differ by population Also system of care tracers focused on understanding system facilitators and barriers to reducing diagnostic overshadowing including trauma-informed care will be conducted
The development and first baseline implementation of mock tracers and any edits will be completed at Rush by end of Year one At other systems mock tracers will be conducted in year 2 This will be a total of 125 tracers at baseline With guides developmental formative evaluation methods will be used for the mock tracers with the formative evaluation communicated to the respective units and practices During years 3 and 4 mock tracers will be conducted in EDs nationally 70 of inpatients at hospitals are processed through EDs selective inpatient units including pediatrics and selective outpatient practices including pediatrics across the five institutions - 15 tracers at each for a total of 75
Aim 2 Co-produce a frame of themes underlying diagnostic overshadowing to develop algorithms to identify patients with specific disabilities at risk of DODE along with EHR decision supports and promptsalerts on specific issues Educational materials on the algorithms and the EHR decision supports and promptsalerts along with case studies to educate providers on DODE and EHR materials will be developed
Participatory Planning and Decision-Making PPDM Process Introduction By the end of Year 2 the investigators will have applied a mixed method analysis of CPT EM audits and analyses staff interviews and chart review quantitative data and qualitative notes An inductive thematic analysis will then be conducted of chart review staff interview and mock tracer notes to identify an initial set of themes of underlying mechanisms of diagnostic overshadowing for use in PPDM processes See Table 1
Table 1 Participatory Planning and Decision-Making PPDM Process Phase 1 Two weeks prior to meeting participants complete a pre-meeting questionnaire or interview to solicit their thoughts about the initially identified themes related to diagnostic overshadowing
Information solicited via a series of open-ended queries through an online survey or interview to elicit responses regarding themes in question
Phase 2 Stakeholder input analyzed using Constant Comparative Analysis154155 The degree of consensus will be determined as to the themes identified as most important for targeted education programs and EHR decision supports Preliminary models will be built to reflect the themes and subthemes developed for each PPDM group
Phase 3 Each PPDM group 5-7 persons will meet for 2 hours each group will be facilitated by ICI staff
Participants respond to questions regarding their groups framework at both theme and subtheme levels
Nominal group procedures will be used to facilitate discussion156
Provisional frame of themes based on Phases 1 ampampampampampampampampampampampampampamp 2 will be presented
The facilitator will guide group discussion of participants moving toward consensus
Nominal group process procedures will be used to facilitate discussion Phase 4 Using tablets supplied by facilitators and ICI-developed software stakeholders will provide importance weightings on a 0-100 or 0-10 depending on group of equal interval scale for each theme and subtheme
All measurement elements identified
The highest rated theme and subtheme within a theme will receive a score of 100 or 10 depending on group
All others are 0-99 Or 0-9 depending on the group
Phase 5 Following ratings de-identified importance weights of entire groups displayed Each participant can view
Their own importance weights
De-identified importance weights of all other stakeholders
Range of importance weights
Mean importance weights of their PPDM group for all themes and subthemes Phase 6
PPDM groups will meet a second time for one hour
Similarities and differences in importance weights will be discussed among participants
Participants will be given an opportunity to share with others the reasoning behind their importance weights
Movement toward consensus using nominal group processes will be achieved Phase 7 Using tablets stakeholders will provide a second set of importance weights for all themes and all subthemes
Phase 8 Proportion weights will be automatically computed through the support system
Participants will be presented with final model of themes
Methods The PPDM process will be as part of the co-production of a frame of important themes related to underlying mechanisms of DODE Participants in the PPDM process will be people from impacted groups including PWD family members health professionals health professional students quality improvement professionals people with expertise in child health and representatives from disability-related organizations health systems payor organizations and community-based services
An initial set of themes will be formulated related to underlying mechanisms of DODE The themes will be inclusive are inclusive of children Approximately 100 participants across impacted groups are expected There will be initial survey a meeting to prioritize themes The initial prioritization will be sent to participants and there will be a second meeting to further confirm and prioritize themes At each PPDM meeting a trained facilitator will use nominal group process procedures to guide the group ensuring stakeholders stay on task respond to questions and move toward consensus To facilitate the virtual PPDM process ICI staff developed software that works on laptops tablet computers and cellphones and that provides a versatile application enabling participants to easily view all themes and subthemes and relevant definitions identified as underlying diagnostic overshadowing and to directly enter priorityimportance weights of these themes during PPDM meetings A second staff person referred to as a chauffeur will operate this software system monitoring participant data entry offering support when necessary providing a visual display of the de-identified weightings once the rating process has concluded and recording the proceedings The themes will be reviewed by the Cross-Disability Advocate Committee the Education Action Network and the Steering Committee prior to implementation
Data Analysis As part of the PPDM process data are continuously analyzed by stakeholders at each stage When the weightings of group members are significantly different discussion is then facilitated to achieve an enhanced degree of consensus Data analysis plans for this study will be both qualitative and quantitative in nature Qualitative analysis based on the recorded statements of stakeholders during the PPDM process will be conducted with NVivo software to discern common themes using constant comparative analysis Each theme produced by the PPDM groups will be summarized Similarities and differences between the themes and the weighting of themes developed by different PPDM stakeholder groups will be identified Specific elements identified for each theme and subtheme will then be analyzed again identifying similarities and differences among groups Finally the importance weights that PPDM groups assign to the themes and subthemes will be examined for each stage of the PPDM process Table 1 Findings will represent the themes underlying DODE The Cross-Disability Advocate Committee and the Consortium Action Networks will review the themes and each will discuss implications of the themes for their work The Steering Committee will review the process and make recommendations on any adjustments of plans for the next steps
Program Development Following Phase 8 of the PPDM process with a final model of identified themes algorithms will be developed to identify patients with specific disabilities at risk diagnostic overshadowingdiagnostic errors The investigators will simultaneously collaborate with Digital and Information services at Rush specifically EPIC to develop the EHR prompts alerts and decision supports that reduce ambiguity and provide clear guidelines in the diagnostic process along with EHR decision supports and promptsalerts on specific issues Educational materials on the algorithms and the EHR decision supports and promptsalerts will be developed along with case studies to educate providers on diagnostic overshadowingdiagnostic errors In these efforts principles of instructional design in which learning environments and materials are developed in a way that motivates gaining knowledge and skills will be used Information services will be involved Staff at Rush and GIDDN the three medical centers RUSH University System for Health Rochester General Hospital an Erie County Medical Center and members of the Steering Committee Cross-Disability Advocate Advisory Committee and Action Networks will be integrated into these efforts Notably the three institutions have structures to implement instructional design and information technology The Steering Committee Cross- Disability Advocate Advisory Committee and the Education Action Network will conduct ongoing review of program development and implementation
Data analysis plan
At Year 5 compared to Year 1 CPT EM code usage with the five identified groups of people with disabilities quantitative will be conducted to evaluate whether coding usage changed for PWD after implementation of algorithms to identify people with disabilities at risk of diagnostic overshadowingdiagnostic error along with EHR decision supports and promptsalerts on specific issues Binary or ordinal pre-post ANCOVA regression analysis will be conducted for each setting ED inpatient outpatient preventive care either separately or as clusters with the pretreatment outcome and post-treatment outcomes as binary or ordinal percentages Independent variables will be patient demographics of age increments gender raceethnicity urbanrural co-morbidities disability type type of insurance and applicable severity index and the 10 chart review issues listed above either separately or as composites Developed education programs will be evaluated through 1 pre- and post- knowledge checks of usage and 2 descriptive data on use of specific EHR decision supports as independent t tests and as regression analysis with independent variables being setting gender raceethnicity age range type of provider and setting of the provider ED inpatient outpatient preventive care91 Pre-post time to diagnostic evaluation using ANCOVA to evaluate for differences will be conducted for 2-3 issues still yet to be determined with demographic characteristics of patients with disabilities and provider characteristics as independent variables Interaction effects will be evaluated For quantitative measures CPT EM code usage analysis knowledge checks descriptive data on use of EHR decision supports and promptsalert ANCOVA will be used to probe for interaction effects using pre-test measures and independent variables from Year one
Final mock tracers and analysis will be conducted at the end of Year 4 and in Year 5 Qualitative analysis of review notes compared to notes before intervention will be conducted
A baseline audit of CPT code usage between patients aged 3-89 years old with and without specified disabilities
The Hypothesis is that there will be statistical difference in use of CPT EM codes between people with the specified disabilities and people without the specified disabilities
2 The investigators also will conduct manual chart reviews on PWDs with lower and higher CPT EM codes addressing demographic characteristics age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site and the following 10 issues suggested by stakeholders caregiver involved transfer referral from other institution verbal co-morbidity numbers and types any refused tests previous history visitsadmissions for same issue ongoing use of medical equipment use of chemical or physical restraints during visits and admissions history of trauma and negative descriptors 15 terms for which Black vs White patients had higher odds ratio for use in medical records
The hypothesis is that information will be developed that can be used in development of mock tracers and in the Participatory Planning and Decision Making process to identify themes that may underly diagnostic overshadowing
3 Based on found differences from baseline audits of use of CPT EM codes and follow-up chart reviews the investigators will conduct targeted interviews with staff then follow-up with stakeholders to develop Joint Commission-style individual mock tracers to follow the care of specific populations of PWD listed above and develop system of care tracers focused on evaluating the system surrounding diagnostic overshadowing including trauma-informed care
The hypothesis is that information will be developed sufficient to develop mock tracers following the care of the specific populations of PWD and systems of care tracers
Aim 2 Co-produce a frame of themes underlying diagnostic overshadowing DO to develop algorithms to identify patients with the specified disabilities at risk for DO Then co-produce case studies for education on DO and EHR prompts alerts and decision supports related to the algorithms along with education on the EHR materials
The hypothesis is that information will be developed that can be used to develop algorithms for identifying PWD from the specific populations at risk of DO as evidenced by CPT EM code usage and DE as evidence by time to diagnostic evaluation
Aim 3 Evaluate for change after implementation of algorithms to identify patients with the specified disabilities at risk for DO
The Hypotheses are that there will be statistical change in CPT EM code usage for PWD from the specified populations
Data evaluation CPT codes were developed by the American Medical Association and undergo periodic revisions and ongoing maintenance CPT codes are the universal way that providers document their services providing standardized reporting needed for billing and reimbursement of healthcare providers including physicians nurse practitioners physician assistants other professionals The system provides numeric codes for issues such as 1 The site of service eg Emergency Department inpatient outpatient preventive services 2 The service provided 3 The complexity of clinical information-gathering and decision-making and 4 Time spent Each setting has a specific group of CPT EM codes ranging from lowest time and complexity of decision making to highest time and complexity of decision making
Incidence among the groups of patients with specific disabilities will be calculated for diagnoses identified as having high incidence of diagnostic error in the general population and for diagnoses identified as problematic for diagnostic error among children Incidence will be calculated on the following using primary or secondary ICD 10 diagnosis codes in the database Aortic aneurysm and dissection Arterial thromboembolism Venous thromboembolism Congestive heart failure Stroke Myocardial infarction Spinal abscess Meningitis and encephalitis Endocarditis Sepsis Pneumonia Lung cancer Melanoma Colorectal cancer Breast cancer Prostate cancer
With focus on pediatric Arterial ischemic stroke Appendicitis Asthma Retinal blastoma Brain tumor Polyateritis Congenital heart disease Duchense muscular dystrophy Inflammatory bowel disease scleroderma
Demographic information on diagnoses will provide background for following processes
Quality assurance The Joint Commission developed mock tracers to use in preparing for accreditation visits and for ongoing quality assurance and professional development efforts Mock tracers provide information on patient experiences quality of care healthcare processes and products and areas needing improvement Tracers involve one-on-one and small group interviews in addition to review of patient charts and forms For this project questions will center around processes and considerations in evaluation and management of patients Deeper inquiry is expected based on answers
EHR prompts and alerts The stage of the diagnostic process requires different clinical decision supports Furthermore conditions that are not common require specific supports Through standard order sets alerts and reminders and other means of diagnostic support eg website clinician can access guidelines more easily
Algorithms Algorithms are in use to address diagnostic error such as identifying patients at risk of delayed test results delays in follow up of chest imaging results tests for hypothyroidism and delayedmissed diagnoses related to abdominal pain In a randomized clinical trial algorithms were used to prospectively identify patients at high risk of delayedmissed diagnoses of lung colorectal or prostate cancer Time to diagnostic evaluation was significantly reduced in the intervention group vs control group for colorectal and prostate cancers but not lung cancers
Co-production of healthcare programs Involving impacted persons in co-production of services impacting them is considered an ethical issue in healthcare transcending the traditional dichotomy between knowledge and program developers and users Co-production involves building collaboration of people from impacted groups in the production and use of knowledge and programs from the start of the process Figure 1 Participation of PWD impacted by the results of research and program planning is often limited to providing input after key decisions have already been made rather than throughout the process However beginning work to involve people with IDD in the co-production of programs for behavioral health indicated improvements in social networks and confidence for participation In addition co-production has been used in developing a framework in healthcare quality improvement In co-production the investigators will work with involved stakeholders in reducing bias in development of algorithms and other materials
Participatory Planning and Decision Making PPDM PPDM is a model-building process based on principles of participatory action research It is designed for groups to make judgments about multiple alternatives and weigh the importance that each of these play with respect to the outcome of interest The process first requires participants to identify or confirm broad domainsthemes and subdomainssubthemes contributing to the phenomenon under study Following the identification of themes and subthemes importance weights are assigned to each and proportional importance weights are then derived There are eight basic phases to the PPDM process beginning with pre-surveys regarding initial themes to presenting participants meetings on the themes and presenting participants with a final model of themes Based on broad input from members of different groups and communities staff at the University of Minnesota Institute on Community Integration have successfully used this process to develop conceptual frames for self-determination and healthcare coordination and to refine the National Quality Forums Home and Community-based Services Outcome Measurement Framework These efforts have resulted in frameworks that possess high content and concurrent and predictive validity The use of the PPDM model and processes enables a diverse group of voices to be heard in the development of targeted education programs and EHR decision supports ensuring that materials fit the needs of the multiple groups In co-production of algorithms as a means of retrospective review for quality assurance and prospective use to identify patients with disabilities at risk of diagnostic overshadowingdiagnostic error the PPDM process is expected to be useful in addressing bias in materials
STUDY ENDPOINTS
Primary At Year 5 compared to Year 1 CPT EM code usage with the five identified groups of people with disabilities quantitative will be evaluated for changes following implementation of algorithms to identify people with disabilities at risk of DO along with EHR decision supports and promptsalerts on specific issues Statistical changes are expected
Education programs will be evaluated through 1 pre- and post- knowledge checks of usage and 2 descriptive data on use of specific EHR decision supports Statistical change is expected in knowledge
Pre-post time to diagnostic evaluation is planned for 2-3 issues still yet to be determined to address delayedmissed diagnoses Time to diagnostic evaluation isexpected to decrease
Secondary For quantitative measures CPT EM code usage analysis knowledge checks descriptive data on use of EHR decision supports and promptsalert ANCOVA will be used to probe for interaction effects using pre-test measures age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site and the 10 issues suggested by stakeholders singly or in composites as covariates and evaluate for variation in post-test results Interaction effects are exxpected
Final mock tracers will be conducted at the end of Year 4 and in Year 5 Tracer notes will be compared to notes before the intervention using qualitative analysis Changes will provide context for any changes in quantitative measures
Framework The Collective Impact Model for social change is the organizing framework for this project Multiple impacted groups are brought to bear on the problem of diagnostic overshadowing Previous efforts to develop algorithms to identify patients at high risk of diagnostic error have not specifically addressed diagnostic overshadowing affecting patients with disabilities and have not previously addressed intersectionality The Collective Impact Model has not previously been used to address diagnostic overshadowing or the overall problem of diagnostic errors The following five tenets must be met to facilitate organization and planning with multiple impacted groups for a Collective Impact project 1 achieving a common agenda 2 ensuring continuous Communication 3 identifying shared measurement strategies 4 employing mutually reinforcing activities to deliver programs and services that will achieve the intended outcome of Collective Impact efforts and 5 employing a dedicated staff as backbone support Partnering requires attention to bringing in the experiences and voices of all impacted groups
Building Organizational Structures using the Collective Impact Model In the first six months members of research team will meet at least once a month to solidify the team hire new staff and create structures based on the Collective Impact Model A Cross-Sector Partnership Steering Committee the Cross-Disability Advocate Advisory Committee and three Consortium Action Networks Communication Measurement Education will be organized The Steering Committee and Cross-Disability Advocate Committee will take overall accountability for developing a shared agenda Collective Impact Tenet 1 Practices that improve understanding of diagnostic overshadowing and the identification of underlying mechanisms will be developed through continuous communication The Communication Action Network will take accountability Collective Impact Tenet 2 The Measurement Action Network will take accountability for ongoing evaluation and final evaluation in Year 5Collective Impact Tenet 3 The Education Action Network will take accountability for facilitating development of targeted education programs and EHR decision supports to mitigate diagnostic overshadowing Collective Impact Tenet 4 The developed infrastructure will facilitate mutually reinforcing activities that encourage the sharing of perspectives and best practices of the projects interdisciplinary partners Processes leading to the achievement of project goals and outcomes will be facilitated by dedicated backbone staff who will assist with the management planning and logistics required by the project RUSH University is the lead institution and each consortium partner has specific responsibilities Collective Impact Tenet 5
Design Aim 1 Identify and create understanding of mechanisms underlying diagnostic overshadowing
Introduction Partnership will be built between three not for profit medical center systems that place prominence on improving health equity RUSH University System for Health and affiliated RUSH University Medical Center RUSH Oak Park Hospital and RUSH Copley Medical Center Rochester Regional Health and affiliated Rochester General Hospital and Erie County Medical Center ECMC These will be sites for pre-post analysis of diagnostic overshadowing via CPT code data implementation of mock tracers and then implementation of targeted education programs and EHR decision supports Data use agreements between the three institutions will be obtained
Data Collection will be in three steps First the investigators will retrieve and auditanalyze for CPT EM code use differences in complexity by the presence of specific disabilities PWD with major mobility impairments mental health concerns severe visual impairments blindness severe hearing lossdeafness and IDD versus no presence of the specified disabilities Patient CPT EM codes data will be from RUSH University Medical Center RUSH Oak Park Hospital RUSH Copley Hospital Rochester General Hospital Erie County Medical Center and associated outpatient practices The investigators will retrieve and analyze CPT EM codes from the EDs codes 99281-99285 CPT EM codes from inpatient services 99221-99223 99231-99233 99238-99239 CPT EM codes from outpatient visits with new patients 99202-99205 with established patients 99211-99215 and for preventive care 99384-99387 to identify potential differences in usage Changes were made to EM codes in 2023 in a way expected to reduce burden Data will be retrieved for the period January 1 2023 through June 30 2024 CPT EM codes will be used as a proxy for diagnostic overshadowing in clinical information-gathering and decision-making complexity EM is a stage in the diagnostic process where errors can occur Quantitative methods for evaluation of CPT EM codes data will be used Data will be programmed with variable range checks and skip rules and will be exported in an automated manner into SPSS Based on experience patients with the specific disabilities will be identified through a comprehensive list of secondary diagnosis codes for the specific disabilities for patients aged 3-89 years old Data will be age-disaggregated in groupings of five years except the group aged 3-5 years old All variables will be checked for errant values Descriptive statistics will be computed for all items CPT EM codes and distributions examined for non-normality and outliers Descriptive statistics for all measures will be reported For each type of visit the investigators will first compare the overall proportion of each CPT EM codes by disability status using pairwise Fisherampampamp39s exact tests with a descriptive analysis of case frequency age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site Evaluation will be conducted on whether data can be collapsed into values of higher level and lower level complexity of evaluation and management codes If so binary logistic regression analysis will be conducted using the outcome of higher level and lower level of complexity of CPT evaluation and management codes and addressing the influence of race ethnicity gender age ranges disability type insurance status severity of illness and 10 chart review questions previously described Otherwise the investigators will use ordinal regression analysis on the outcomes of the CPT EM codes using all levels of complexity Site-specific analyses will be conducted ie ED inpatient outpatient preventive care and a combined model that accounts for site using cluster-robust standard errors
Binary or ordinal regression analysis will be conducted for each setting Emergency Department inpatient outpatient preventive care with the dependent variable being EM codes separately or split into lower and higher complexity codes Independent variables will be demographics of age increments gender raceethnicity urbanrural co-morbidities disability type insurance type and severity of illness relevant to site and the 10 chart review issues listed above Variable loadings will be assessed for use in developing algorithms for PWD from the specified groups at risk of diagnostic overshadowing
Second based on found differences from the CPT EM code analyses targeted retrospective manual chart reviews will be conducted to improve identification of underlying mechanisms of diagnostic overshadowing At each partnering institution 5 associated with the three medical centers at least five charts will be reviewed of patients from each of the five specific populations of people with disabilities See previous paragraph from which data are being collected 25 at each of the five institutions and associated outpatient practices - 125 in total
The charts will be reviewed for issues which may provide insight into diagnostic overshadowing In discussions with staff suggestions to explore include the same issues listed above in Previous work Notes on the above ten issues will be taken Notes will be evaluated for themes using inductive thematic analysis The team will meet weekly to discuss themes If during the chart review other issues are determined an IRB amendment to evaluate additional issues will be submitted
Following the chart reviews interviews with staff will be conducted in identified areas to explore perceptions of why there may be differences expecting at least five in-depth interviews and multiple short interviews at each of the five sites that can be conducted during shift huddles and staff meetings
Third with these baseline data the Measurement Action Network and the Advocate Advisory Committee will be consulted on development of separate mock tracers following the care of each specific population of PWD prone to diagnostic overshadowing 25 retrospective tracers will be conducted of patients 5 each from the specific populations listed above with 10 tracers to be among children and at least 10 to be among patients from marginalized racialethnic groups at each of the five partnering institutions with associated outpatient practices 25 each at RUSH University Medical Center RUSH Oak Park Hospital RUSH Copley Hospital Rochester General Hospital and Erie County Medical Center For tracers patients with issues such as high acuity complexity of care eg multiple tests surgeries transfers between units and history of trauma will be chosen recognizing that issues affecting care differ by population Also system of care tracers focused on understanding system facilitators and barriers to reducing diagnostic overshadowing including trauma-informed care will be conducted
The development and first baseline implementation of mock tracers and any edits will be completed at Rush by end of Year one At other systems mock tracers will be conducted in year 2 This will be a total of 125 tracers at baseline With guides developmental formative evaluation methods will be used for the mock tracers with the formative evaluation communicated to the respective units and practices During years 3 and 4 mock tracers will be conducted in EDs nationally 70 of inpatients at hospitals are processed through EDs selective inpatient units including pediatrics and selective outpatient practices including pediatrics across the five institutions - 15 tracers at each for a total of 75
Aim 2 Co-produce a frame of themes underlying diagnostic overshadowing to develop algorithms to identify patients with specific disabilities at risk of DODE along with EHR decision supports and promptsalerts on specific issues Educational materials on the algorithms and the EHR decision supports and promptsalerts along with case studies to educate providers on DODE and EHR materials will be developed
Participatory Planning and Decision-Making PPDM Process Introduction By the end of Year 2 the investigators will have applied a mixed method analysis of CPT EM audits and analyses staff interviews and chart review quantitative data and qualitative notes An inductive thematic analysis will then be conducted of chart review staff interview and mock tracer notes to identify an initial set of themes of underlying mechanisms of diagnostic overshadowing for use in PPDM processes See Table 1
Table 1 Participatory Planning and Decision-Making PPDM Process Phase 1 Two weeks prior to meeting participants complete a pre-meeting questionnaire or interview to solicit their thoughts about the initially identified themes related to diagnostic overshadowing
Information solicited via a series of open-ended queries through an online survey or interview to elicit responses regarding themes in question
Phase 2 Stakeholder input analyzed using Constant Comparative Analysis154155 The degree of consensus will be determined as to the themes identified as most important for targeted education programs and EHR decision supports Preliminary models will be built to reflect the themes and subthemes developed for each PPDM group
Phase 3 Each PPDM group 5-7 persons will meet for 2 hours each group will be facilitated by ICI staff
Participants respond to questions regarding their groups framework at both theme and subtheme levels
Nominal group procedures will be used to facilitate discussion156
Provisional frame of themes based on Phases 1 ampampampampampampampampampampampampampamp 2 will be presented
The facilitator will guide group discussion of participants moving toward consensus
Nominal group process procedures will be used to facilitate discussion Phase 4 Using tablets supplied by facilitators and ICI-developed software stakeholders will provide importance weightings on a 0-100 or 0-10 depending on group of equal interval scale for each theme and subtheme
All measurement elements identified
The highest rated theme and subtheme within a theme will receive a score of 100 or 10 depending on group
All others are 0-99 Or 0-9 depending on the group
Phase 5 Following ratings de-identified importance weights of entire groups displayed Each participant can view
Their own importance weights
De-identified importance weights of all other stakeholders
Range of importance weights
Mean importance weights of their PPDM group for all themes and subthemes Phase 6
PPDM groups will meet a second time for one hour
Similarities and differences in importance weights will be discussed among participants
Participants will be given an opportunity to share with others the reasoning behind their importance weights
Movement toward consensus using nominal group processes will be achieved Phase 7 Using tablets stakeholders will provide a second set of importance weights for all themes and all subthemes
Phase 8 Proportion weights will be automatically computed through the support system
Participants will be presented with final model of themes
Methods The PPDM process will be as part of the co-production of a frame of important themes related to underlying mechanisms of DODE Participants in the PPDM process will be people from impacted groups including PWD family members health professionals health professional students quality improvement professionals people with expertise in child health and representatives from disability-related organizations health systems payor organizations and community-based services
An initial set of themes will be formulated related to underlying mechanisms of DODE The themes will be inclusive are inclusive of children Approximately 100 participants across impacted groups are expected There will be initial survey a meeting to prioritize themes The initial prioritization will be sent to participants and there will be a second meeting to further confirm and prioritize themes At each PPDM meeting a trained facilitator will use nominal group process procedures to guide the group ensuring stakeholders stay on task respond to questions and move toward consensus To facilitate the virtual PPDM process ICI staff developed software that works on laptops tablet computers and cellphones and that provides a versatile application enabling participants to easily view all themes and subthemes and relevant definitions identified as underlying diagnostic overshadowing and to directly enter priorityimportance weights of these themes during PPDM meetings A second staff person referred to as a chauffeur will operate this software system monitoring participant data entry offering support when necessary providing a visual display of the de-identified weightings once the rating process has concluded and recording the proceedings The themes will be reviewed by the Cross-Disability Advocate Committee the Education Action Network and the Steering Committee prior to implementation
Data Analysis As part of the PPDM process data are continuously analyzed by stakeholders at each stage When the weightings of group members are significantly different discussion is then facilitated to achieve an enhanced degree of consensus Data analysis plans for this study will be both qualitative and quantitative in nature Qualitative analysis based on the recorded statements of stakeholders during the PPDM process will be conducted with NVivo software to discern common themes using constant comparative analysis Each theme produced by the PPDM groups will be summarized Similarities and differences between the themes and the weighting of themes developed by different PPDM stakeholder groups will be identified Specific elements identified for each theme and subtheme will then be analyzed again identifying similarities and differences among groups Finally the importance weights that PPDM groups assign to the themes and subthemes will be examined for each stage of the PPDM process Table 1 Findings will represent the themes underlying DODE The Cross-Disability Advocate Committee and the Consortium Action Networks will review the themes and each will discuss implications of the themes for their work The Steering Committee will review the process and make recommendations on any adjustments of plans for the next steps
Program Development Following Phase 8 of the PPDM process with a final model of identified themes algorithms will be developed to identify patients with specific disabilities at risk diagnostic overshadowingdiagnostic errors The investigators will simultaneously collaborate with Digital and Information services at Rush specifically EPIC to develop the EHR prompts alerts and decision supports that reduce ambiguity and provide clear guidelines in the diagnostic process along with EHR decision supports and promptsalerts on specific issues Educational materials on the algorithms and the EHR decision supports and promptsalerts will be developed along with case studies to educate providers on diagnostic overshadowingdiagnostic errors In these efforts principles of instructional design in which learning environments and materials are developed in a way that motivates gaining knowledge and skills will be used Information services will be involved Staff at Rush and GIDDN the three medical centers RUSH University System for Health Rochester General Hospital an Erie County Medical Center and members of the Steering Committee Cross-Disability Advocate Advisory Committee and Action Networks will be integrated into these efforts Notably the three institutions have structures to implement instructional design and information technology The Steering Committee Cross- Disability Advocate Advisory Committee and the Education Action Network will conduct ongoing review of program development and implementation
Data analysis plan
At Year 5 compared to Year 1 CPT EM code usage with the five identified groups of people with disabilities quantitative will be conducted to evaluate whether coding usage changed for PWD after implementation of algorithms to identify people with disabilities at risk of diagnostic overshadowingdiagnostic error along with EHR decision supports and promptsalerts on specific issues Binary or ordinal pre-post ANCOVA regression analysis will be conducted for each setting ED inpatient outpatient preventive care either separately or as clusters with the pretreatment outcome and post-treatment outcomes as binary or ordinal percentages Independent variables will be patient demographics of age increments gender raceethnicity urbanrural co-morbidities disability type type of insurance and applicable severity index and the 10 chart review issues listed above either separately or as composites Developed education programs will be evaluated through 1 pre- and post- knowledge checks of usage and 2 descriptive data on use of specific EHR decision supports as independent t tests and as regression analysis with independent variables being setting gender raceethnicity age range type of provider and setting of the provider ED inpatient outpatient preventive care91 Pre-post time to diagnostic evaluation using ANCOVA to evaluate for differences will be conducted for 2-3 issues still yet to be determined with demographic characteristics of patients with disabilities and provider characteristics as independent variables Interaction effects will be evaluated For quantitative measures CPT EM code usage analysis knowledge checks descriptive data on use of EHR decision supports and promptsalert ANCOVA will be used to probe for interaction effects using pre-test measures and independent variables from Year one
Final mock tracers and analysis will be conducted at the end of Year 4 and in Year 5 Qualitative analysis of review notes compared to notes before intervention will be conducted
Study Oversight
Has Oversight DMC:
None
Is a FDA Regulated Drug?:
None
Is a FDA Regulated Device?:
None
Is an Unapproved Device?:
None
Is a PPSD?:
None
Is a US Export?:
None
Is an FDA AA801 Violation?:
None