The development of electronic health record—extractable quality indicators for osteoporosis in primary care: A rand-modified Delphi method

Abstract

Background

Osteoporosis is a common chronic condition and a cause of morbidity and mortality worldwide. Currently, osteoporosis is under-diagnosed and under-treated. There is an important role for the general practitioner (GP) in the prevention and management of this condition. The electronic health record (EHR) can be used to effectively pinpoint at-risk patients and enhance the quality of care provided to those suffering from osteoporosis. This study aims to develop evidence-based and EHR-extractable quality indicators (QIs).

Methods

The RAND/UCLA- modified Delphi method was used. After an extensive literature search, recommendations were retrieved from the selected evidence-based guidelines and included in a questionnaire if they met the ‘SMART’-criteria (specific, measurable, acceptable, realistic and time-related). Next, an expert panel (8 general practitioners, 2 patients, a geriatrician, a rheumatologist and an EHR specialist) was formed and asked to grade the selected recommendations individually. A consensus meeting was hosted to discuss the results. After their final appraisal, a set of quality indicators was developed out of the included recommendations.

Results

Out of 11 evidence-based guidelines 478 recommendations were retrieved. After applying the ‘SMART’ criteria, 38 recommendations were presented to the panel of experts. After the written questionnaire round and the consensus meeting 25 recommendations were included. Twenty recommendations remained after final appraisal and were converted into 34 quality indicators of which 13 are currently extractable out of the electronic health record in Belgium.

Conclusion

This study generated 34 evidence-based quality indicators for osteoporosis in primary care. This set enables general practitioners to measure and enhance the quality of care for osteoporosis patients through automated audit and feedback.

Introduction

Osteoporosis is a common chronic condition that represents a growing health problem with major impact on morbidity and mortality [1]. It is characterized by the loss of bone mass and a deterioration of bone quality, predisposing to an increased risk of fracture. Those fractures are associated with a decreased quality of life and an increased mortality [2].

In total, 22% of women and 7% of men over the age of 50 are affected worldwide. In 2019 approximately 681 372 people suffered from osteoporosis in Belgium and the cost of osteoporotic fractures is estimated to be 2.4% of the national total healthcare spending. Due to aging, this number will continue to rise [3].

Unfortunately, osteoporosis remains under-diagnosed and therefore under-treated [4]. The survey conducted by the international osteoporosis foundation revealed that postmenopausal women tend to downscale their personal risk and often avoid discussing osteoporosis with their doctors. Additionally, there seems to be limited access to diagnosis and treatment prior to the first fracture [5].

Given the impact on morbidity, mortality, and economic cost, it is primordial to recognize osteoporosis in an early stage and take preventive measures. Primary care plays a significant role in the early identification of patients at risk, the initial approach and the integrated management between primary and secondary care [6, 7].

The electronic health record (EHR) can be used to overcome the challenge of screening and managing patients at risk for osteoporosis by an automated quality assessment and the generation of a feedback report [8]. In this way, patients at risk can be identified and become more involved in managing their disease by initiating adequate prevention and proper treatment. However, this requires evidence-based and EHR-extractable quality indicators (QIs) that ensure a holistic approach to the patient with osteoporosis in primary care.

Quality indicators have been previously developed for the care of osteoporosis in frail elderly and post-menopausal women [9, 10]. Van Der Ploeg et al. even transferred QIs for the general practice care management of falls in the frail elderly population from the United States to the Netherlands [11]. However, none of these previously defined QIs are EHR extractable and they do not allow automated audit and feedback.

The purpose of this study was to develop EHR-extractable and evidence-based QIs by using the RAND/UCLA-modified Delphi method [12]. Through this, the study provides a framework for evaluating and improving quality of care for people diagnosed with osteoporosis.

Methods

The study was conducted between January 2022 and August 2023.

Study design

The RAND/UCLA—modified Delphi method was used to define QIs, including 5 steps [8, 13]. (a) Recommendations were retrieved from evidence-based guidelines and evaluated for inclusion. (b) An expert panel was formed and asked to grade the selected recommendations individually. The results were analyzed, and a feedback report was sent to them (written questionnaire round). (c) A consensus meeting with panel members was organized to discuss the results of the written questionnaire and recommendations were reviewed for in- or exclusion (consensus round). (d) Adapted recommendations were sent to the participants for final appraisal (e) and a final set of QIs was formulated.

Study population

A panel was composed to select the QIs, consisting of 13 members: 8 general practitioners (GPs), 1 geriatrician, 1 rheumatologist, 1 EHR expert and 2 patients of whom one is also a nurse. All professionals worked in Belgian hospitals or practices.

Data collection

Selection of recommendations

The search was conducted in the following databases: Pubmed [14], Gin (Guidelines international network) [15], and Trip (Turning Research Into Practice) database [16] until 1 August 2022. The search strategy was the same for all databases except for some adaptions necessary to suit each database search style. The following Medical Subject Headings (MeSH) terms and their synonyms (appendix 1) were used for Pubmed and GIN database: ‘osteoporosis’ and ‘recommendations’. For Trip only’ osteoporosis’ was used. An additional search in EB Practicenet [17] was also performed using ‘osteoporosis’ as a search term. Participants were allowed to suggest other guidelines for inclusion.

Additional filters of language (English and Dutch) and year of publication (after 2018) were applied. Only evidence-based guidelines from reputable institutions and guidelines from western demographics were selected for inclusion. Reputable institutions define institutions known for their scientific research, that make use of peer review, that are frequently cited by other institutions and exceed the score of 90 by the AGREE II tool [18].

In cases where multiple recommendations on the same topic were published by the same research group, only the most recent one was considered for inclusion. Afterwards, each included guideline was assessed by two different authors using the AGREE II tool [18]. Only guidelines with an overall score of ≥ 90 were included.

Out of these guidelines, all recommendations relevant to osteoporosis care were listed. All the recommendations had to meet the ‘SMART-criteria’ (specific, measurable, acceptable, realistic and time-related, Table 1) [19] and had to be extractable from the EHR to ensure the possibility of automated feedback. Any disagreement among both authors was resolved by discussion.

Table 1 Examples and explanations of exclusion based on the ‘SMART-principle

The selected recommendations were collected in a questionnaire with 6 categories: definition, screening, diagnosis, prevention and treatment, follow-up and referral (appendix 2). Since our panel consisted of two patients, an adapted questionnaire was made for them. The professional-specific recommendations such as definitions, specific laboratory requests or technical investigations and pharmacological therapy (appendix 3) were left out.

Written questionnaire round

Completion

The selected recommendations were presented to our expert panel in a questionnaire. The participants were asked to rate the appropriateness of each recommendation to measure the quality of osteoporosis care, considering their relevance and EHR-extractability. They did this by assigning a score on a 9-point Likert scale with 1 being the lowest score and 9 the highest score.

Further, they were requested to evaluate each recommendation for EHR-extractability. If not extractable, they had the possibility to label the recommendations as “not assessable”.

In addition, panel members were asked to rank the recommendations per category in a top-5 (prioritization) based on their importance for measuring quality of care.

Finally, space was provided to write down additional comments or additions to the recommendations.

Analysis

Recommendations were classified into three categories: demonstrating high, uncertain or low potential as a quality indicator for osteoporosis. These categories were established using three different criteria: median Likert scale score, prioritization and level of agreement among panel members [8].

• A. Median Likert scale score

For each recommendation, the median of the Likert scale score given by each panel member was calculated.

• B. Prioritization

Prioritization is expressed as a percentage based on the top 5 ranking made by the participants. This percentage was calculated as follows:

• 1. Points were assigned to the recommendations based on the top 5 ranking. The recommendation mentioned first got 5 points; the second got 4 points, the third 3 points, and so on.

• 2. For every recommendation, the total score was calculated as the sum of all the points awarded by the panel members.

• 3. The maximum score of a recommendation was calculated (e.g., in case of 13 panel members and a top-5 ranking, this maximum score is 13 × 5 = 65)

• 4. The prioritization was calculated as the ratio of the total received score and the maximum score (e.g., if 6 out of 13 panel members ranked a recommendation first in their top-5 and 7 panel members did not mention it, the prioritization was 30/65 or 46,2%).

• C. Agreement

Agreement was defined as ≥ 70% of the panel members assigning a score of ≥ 7. If ≥ 30% of the panel members scored ≥ 7, and ≥ 30% scored ≤ 3, there was disagreement. Scores falling outside these ranges were categorized as no agreement.

• D. Classification of recommendations

Considering these criteria, each recommendation was categorized as having either high, uncertain or low potential as a quality indicator to measure osteoporosis care. The division was done as follows (see Fig. 1):

• 1. High potential recommendations were defined as having a median score on the Likert scale of ≥ 7 AND a prioritization percentage of ≥ 20% AND panel agreement.

• 2. Uncertain potential recommendations:

  • a. Median score of ≥ 7 AND a top percentage between 1 and 20% AND panel agreement or,

  • b. Median score of < 7 AND a top percentage of ≥ 20% AND panel agreement or disagreement

• 3. Other recommendations were categorized as having low potential.

Fig. 1

Division of recommendations as having high, uncertain or low potential based on Median Likert scale score, prioritization and panel agreement

• E Feedback report

Each recommendation was color coded to represent its appropriateness as quality indicator for osteoporosis care. High potential recommendations were coded green, uncertain potential recommendations yellow and low potential recommendations red (Fig. 1). All panel members received a feedback report. This contained all color-coded recommendations, the Likert scale score assigned by the participant and the median Likert scale score for each individual recommendation.

Consensus meeting round

During the consensus meeting all recommendations were reviewed, focusing most attention on the low and uncertain recommendations. All high potential recommendations were considered included unless a panel member asked for more elaborative discussion. Finally, all accepted recommendations were reviewed and adjusted considering the ‘SMART-principle’, EHR-extractability and the panel members' remarks.

Final appraisal

All recommendations included were collected in a report sent to the panel members for final appraisal.

Translation into quality indicators

A final set of quality indicators was formed by expressing the included recommendations as a percentage. For example, “Weight, height and BMI must be measured annually in patients with osteoporosis” was transformed into “The percentage of patients with osteoporosis whose weight, height and BMI are measured annually”. The final list of quality indicators was reviewed and approved by all authors.

Results

Literature search

The initial database search yielded 802 records. Two additional records [20, 21] were manually added as suggested by our expert panel. Each article's title and abstract were reviewed by two authors. Out of these, 753 articles were excluded for not meeting the predetermined criteria regarding patient population, outcomes, or study design. Additionally, 13 duplicates were manually removed. Out of 36 remaining guidelines 16 were selected and were screened for eligibility by the two authors using the AGREE II instrument (Appraisal of Guidelines for Research & Evaluation) [18]. Eventually, 11 guidelines were included (Table 2).

Table 2 Overview of included guidelines

The study selection process is documented in a flowchart that can be found in appendix 4.

Selection of recommendations

Out of those guidelines, 478 recommendations were extracted and categorized by subject. Both authors independently screened these recommendations for inclusion using the ‘SMART-principle’ [19]. They excluded 253 recommendations, merged 216 recommendations addressing the same topic and directly included 9 recommendations without alterations (see Fig. 2). The main reasons for exclusion were non-specific phrasing, non-measurable outcome or no EHR-extractability (for example patient input, compliance control, importance of multidisciplinary management). Recommendations not attainable in primary care were also excluded. Ultimately, 38 recommendations remained and were presented to the panel of experts.

Fig. 2

Selection of recommendations

Written questionnaire round

All panel members completed the questionnaire. The calculation of the median Likert score, prioritization and agreement resulted in 16 recommendations having low potential, 4 having uncertain potential and 18 having high potential (appendix 5). All comments were included and discussed with our panel.

An adapted questionnaire with 21 recommendations was presented to the two participating patients.

Consensus meeting round

Eventually, 7 experts joined the consensus meeting round (1 geriatrician, 1 rheumatologist and 5 general practitioners). Two patients also provided input on the recommendations separately. Out of the initial set of recommendations, eleven were excluded because they were either not relevant to primary care or not extractable from electronic health records (EHRs). Additionally, two recommendations were merged (Fig. 2, appendix 5). The panel agreed to exclude all recommendations related to osteopenia.

Certain complex recommendations were simplified. For example, the risk factors for developing osteoporosis and the blood tests to rule out secondary causes of osteoporosis at diagnosis.

Discussions among specialists and general practitioners occurred on several topics, including vitamin substitution and treatment of osteoporosis. Through discussion consensus was reached. Furthermore, in some cases extra information was added to recommendations, others were modified (appendix 5).

The specialists missed a recommendation about new anabolic osteoporotic treatments. Although this therapy is not used in primary care, they felt that high-risk patients who qualify for it should be identified and referred to specialized care. An adapted recommendation was suggested, based on recent available evidence. Ultimately, this was discussed and voted on in our panel and finally accepted. The other recommendations regarding treatment were simplified.

During the consensus meeting, emphasis was placed on medical correctness of the recommendations. Further EHR-extractability was taken into account, which resulted in minor adjustments to the recommendations. Additionally, four recommendations were merged to facilitate EHR-extractability. Finally, one additional recommendation about vertebral fracture diagnosis was excluded due to lack of specificity and inability to be extracted from the EHR. Eventually a set of 20 recommendations remained.

Final appraisal

The final set of 20 recommendations was approved by the panel members.

Translation into quality indicators

To improve EHR-extractability, some of the recommendations were split into several QIs. Out of the 20 recommendations, 34 Quality Indicators (QIs) were developed of which 13 are fully EHR-extractable at present in Belgium (Table 3). Provided some modifications to the EHR, it will be possible to extract all the QIs from the EHR. This set consisted of 1 QI about diagnosis, 3 QIs about screening, 2 QIs about technical investigations, 4 QIs about lifestyle management, 14 QIs about treatment, 5 QIs about follow-up and finally 5 QIs about referral (Table 3).

Table 3 List of QI’S

Discussion

Principal findings

This study used a RAND-modified Delphi method to generate a set of 34 QIs to measure and improve the quality of primary care for osteoporosis. Of these QIs, 13 QIs can currently be extracted from the Belgian EHR systems. Panel members agreed that screening and early detection of high-risk patients for osteoporosis is primordial in primary care and that GPs are essential for this. The GPs in the panel declared that with tools integrated in their EHR, they would think about screening more regularly. The earlier the diagnosis is made the more fractures and long-term consequences can be avoided.

The QIs on screening indicate the necessity of calculating the FRAX-score every two years in patients > 65 years or in patients with risk factors for developing osteoporosis. Screening before age 50 is only appropriate in the presence of severe clinical risk factors. These risk factors and the FRAX-score determine whether a DXA-scan is warranted. Osteoporosis is diagnosed if there is either a fragility fracture or a T-score of -2.5 or lower on 'dual energy X-ray absorptiometry' (DXA) in the lumbar spine (antero-posterior), femoral neck, hip or 1/3 radius.

Moreover, the panel, and especially the included patients, indicated that GPs need to inform their patients about lifestyle management such as smoking cessation and alcohol restriction. The importance of compliance control was also stressed and added to the recommendations. The need for a multidisciplinary approach to reduce sedentary lifestyle was also mentioned. However, since this is not EHR-extractable only referral to a physical therapist could be included in the QIs.

Several guidelines mentioned the utility of bone turnover markers (BTM) to diagnose osteoporosis and follow up treatment [22,23,24]. However, this was not included in our final set of QIs due to the high cost and difficulty of use in primary care. Furthermore, two scoring systems were retained from our literature search, namely FRAX [32] and Q fracture [33]. Both tools differ in risk factors that are being considered and Q Fracture is only validated in the UK. In Belgium, FRAX is widely used and known because of its requirement in the request of a DXA-scan. Our panel agreed to only include FRAX in the QIs. Sanchez-Rodriguez et al. mentioned the Garvan fracture risk calculator as well. They indicate this risk score could be a good alternative for FRAX, as they include quantitative risk factors [21].

Ten out of 15 recommendations regarding pharmacological treatment were categorized as low potential after the written questionnaire round. After the consensus meeting, 5 of these recommendations were included and modified based on the suggestions made by our panel members. First, there was disagreement about the measurement of vitamin D levels in the blood. It was pointed out that standardization of vitamin D measurements is still a problem in Belgian laboratories [34] and that osteoporosis patients should always start vitamin D supplementation without checking blood values first. On the other hand, it was discussed that for patients with an elevated risk but no diagnosis of osteoporosis, testing vitamin D blood levels can be useful. A level of 30 ng 25-OH vitamin D was agreed upon as a cut-off level for starting supplementation. Although this cut-off is arbitrary, it ensures EHR-extractability.

In addition, the specialists in our panel pointed out that anabolic therapy wasn’t mentioned in the recommendations even though it is considered a first-choice drug in a certain subgroup of osteoporosis patients. However, referral to a specialist is necessary to start this medication. Two recommendations (about treatment and referral) were adapted (appendix 5) and the indications for anabolic therapy were included (fragility fracture in the last 2 years, multiple fragility fractures, T-score < -3.5 or FRAX risk of major osteoporotic fracture > 30%, hip fracture > 4.5%) [21, 22].

Finally, the recommendations stated that referral to a specialist is also needed if the patient takes corticosteroids (≥ 5 mg/day prednisolone or equivalent for more than 3 months) or if the renal function is below 30 ml/min/1.73 m2. The panel decided that corticosteroid intake could be excluded from the referral criteria because treatment in primary care with bisphosphonates is efficient as supported by recent data [35]. The threshold of renal function was also modified. Only a GFR below 15 ml/min requires advice from a specialist.

Different recommendations regarding the treatment of osteoporosis for postmenopausal women, secondary-induced osteoporosis [20, 22, 30] and osteoporosis in men [27, 31] were found in recent literature. The panel indicated that treatment may be equalized for these groups. Therefore, the recommendations about secondary and steroid-induced osteoporosis were excluded.

Strengths and limitations

To our knowledge, this study is among the first to develop evidence-based and EHR-extractable QIs for osteoporosis. This set of QI enables automated quality assessments to monitor and improve the quality of primary care for patients with osteoporosis. Moreover, this set will also be used in a national quality project based on automatic audit and feedback intervention in Belgium, called the barometer project. One strength of this study is the exhaustive (inter)national guideline review and usage of the AGREE II instrument, enabling this set of QI to be used in other countries once their local EHR extractability is verified. Additionally, these QIs can also be applied to the whole osteoporosis population rather than a specific subgroup, facilitating their use in primary care.

However, some limitations also apply. The EHR extractability is directly linked to the correct and complete coding of risk factors, medical history and clinical findings in the EHR. The data completeness of the EHR systems must therefore be taken into account. Implementation of a parameter set in the EHR, which compiles all information important for osteoporosis (risk factors, Frax score, treatment,….) has been shown to have a positive effect on coding [8]. Furthermore, several elements in osteoporosis care cannot yet be coded in the Belgian EHR systems, in which ICD-10 codes are used [36], and these will have to be added in the future (e.g. a parameter set). However, steps have already been taken to improve this. Finally, the expert panel did contain a nurse, but she was consulted as a 'patient expert' and not because of her nursing expertise.

Conclusion

In this study, 34 QIs were identified to measure the quality of primary care for osteoporosis patients of which 13 QIs are already EHR extractable in Belgium. This set of QI is evidence-based and covers various aspects of osteoporosis care including diagnosis, screening, technical investigations, lifestyle management, treatment, follow-up and referral. Prevention and early diagnosis are seen as the most important task of the GP in osteoporosis care. This set of QIs will allow GPs to measure and improve the quality of care for osteoporosis patients by automated audit and feedback.