Particularly in the case of accelerated drug approvals and drugs for rare diseases (orphan drugs), the evidence available at the time of market access is often insufficient for the early benefit assessment of drugs. Often, the studies are too short or no data on patient-relevant outcomes were collected. Comparisons with the German standard of care are also often lacking. In order to close such evidence gaps, in future, routine practice data are also to be included in early benefit assessments of drugs.
But how must the data be collected and processed so that they can be used by the Federal Joint Committee (G-BA) for benefit assessments in Germany? In order to answer this question, the G-BA commissioned the Institute for Quality and Efficiency in Health Care (IQWiG) to develop scientific concepts for the generation of routine practice data and their analysis for benefit assessments of drugs – especially with regard to the option of quantifying the added benefit of a new drug. According to the “Gesetz für mehr Sicherheit in der Arzneimittelversorgung” (GSAV, Law for More Safety in the Supply of Medicines), the G-BA may in future commission the collection of routine practice data on selected drugs to support the quantification of added benefit.
Summarizing the most important result of the IQWiG analysis, Jürgen Windeler, IQWiG’s Director, notes: “Extensive analyses of the methodological literature and intensive discussions with registry operators and external statisticians have led us to the conclusion that, in the case of high-quality patient registries, it is possible to base studies on these registries and use the routine practice data collected for extended benefit assessments of drugs.”
Such registry studies can be conducted either with or without randomization, but the high quality of the data is the decisive factor in both cases.
In order to support the individual registries in particular and the registry landscape in Germany in general in the collection of routine practice data, on the basis of current national and international recommendations, IQWiG compiled criteria for data quality and for ensuring data quality for routine practice data collections for benefit assessments of drugs, condensed them to the essentials, and organized them in a clear and concise manner. In addition, the rapid report provides registry operators, sponsors of registry studies as well as health policy decision-makers with specific recommendations for action on how the collection of routine practice data in registries can be made usable for benefit assessments of drugs.
Focus on collection of routine practice data in registries
Routine practice data are data collected within the context of usual health care in patient populations that can receive the drug under assessment in the approved therapeutic indication. The data can be collected in studies with or without randomization.
In their rapid report, the IQWiG authors describe that the use of routine practice data for benefit assessments of drugs mandatorily requires a comparison between the new drug and the comparator therapy specified by the G-BA, which makes it necessary to conduct comparative studies. In general, four data collection tools are available for comparative studies: study-specific data collection as well as data collection from registries, electronic patient records, and claims data of health insurance funds.
The IQWiG authors are convinced that the collection and processing of routine practice data from electronic patient records and claims data from health insurance funds is currently not possible with regard to benefit assessments of drugs and will not be possible in the near future. This is mainly because the data quality in these sources is insufficient and important data are not collected. These problems cannot be solved in the short or medium term. In contrast, the assessment of disease-related patient registries yielded positive results.
Data quality of registries has improved
As the IQWiG authors note, of the data collection tools not primarily geared towards comparative studies, registries are most likely to offer the option of adapting the data collection requirements for these studies. This concerns both the specification of the necessary data and the data quality.
The authors also note that the question as to whether existing patient registries are currently suitable for the collection of routine practice data according to §35a Social Code Book (SGB V) cannot be answered in a general way. This depends on the respective registry and, above all, on the specific research questions posed. In the discussions with selected registry operators, however, it also became apparent that from a technical and organizational point of view, the registries are generally prepared to implement any necessary extensions of the data set.
Thomas Kaiser, Head of IQWiG’s Drug Assessment Department explains: “In recent years, the objectives and scope of documentation of registries have been extended. In particular, the increasing documentation of clinical information in registries that can be used to describe patient populations, interventions and outcomes for benefit assessments is an important step forward. For certain research questions, data on patient-reported outcomes should also be included in registries. This is already the case in some registries.”
Benefit assessments always require fair comparisons
As emphasized by the IQWiG authors, if routine practice data are to be used in benefit assessments, it must be taken into account that the basis of any conclusion on the effects of interventions is a comparison. This is because only on the basis of a comparison is it possible to distinguish between “after intervention A” and “due to intervention A”; this distinction is necessary for a causal conclusion. A comparison is only meaningful if the starting conditions are fair (similarity of the groups in terms of prognostic factors). Ideally, this is achieved through randomization, i.e. the random allocation of study participants to the two study arms.
When studies are conducted without randomization, the adjustment of interfering factors (confounders) is an essential part of the assessment. For this purpose, the relevant confounders – such as the severity of a concomitant disease or a genetic mutation – must be determined and documented in the data collection. The completeness and accuracy of the data on confounders is just as important as that of the other data. Depending on the research question and the data already available, it may therefore be less resource-intensive to conduct a study with randomization.
As the IQWiG authors note, in order to be able to use routine practice comparative studies for benefit assessments, it should already be ensured in the study planning phase that the study process and the data collected are of the necessary quality to produce interpretable results.
They therefore compiled a clear list of criteria to ensure that only data of sufficient quality are used. This list is divided into four categories: mandatory criteria for ensuring data quality; general criteria that are always relevant for registry studies used in benefit assessments of drugs; general criteria that, depending on the research question, are relevant for registry studies used in benefit assessments of drugs; and criteria whose degree of fulfilment is to be assessed in relation to the research question.
Thomas Kaiser notes: “In the context of the suitability testing of a specific registry, this list should be used to evaluate for the respective research question whether all necessary data have been collected or whether possible deficits can be corrected with reasonable effort in a registry-based study.”
Without randomization, no more than a hint of an effect is conceivable
The smaller the expected differences in treatment effects in a comparison, the more important is a fair comparison in terms of the similarity of the groups in terms of prognostic factors described above. From this, the IQWiG authors conclude that from comparative studies without randomization, a conclusion drawn from the observed effects with regard to the benefit or harm of an intervention is only meaningful if a certain effect size is exceeded. Otherwise, it cannot be excluded that the observed effect was not caused by the intervention, but by confounders. Since without randomization it cannot be excluded, even in a good study, that unknown confounders may influence the results, it is therefore generally not possible to derive more than a hint of an effect from comparative studies without randomization.
According to IQWiG’s analysis, whether it is possible to consider retrospective study designs depends on whether the available data sources contain the necessary data in the required quality. Thus, comparisons of patient populations receiving a new drug with patient populations comprising historical controls only appear realistic if the same data source is used for both (e.g. a disease-specific clinical registry).
Registry-based randomized trials as an option
In general, comparative studies with randomization always have a higher informative value than those without randomization. They remain the gold standard because quantification of the added benefit is more reliable. The IQWiG authors emphasize that, particularly after drug approval, routine practice comparative trials with randomization can – depending on the existing research question – also be conducted with a limited collection of data in “large simple trials”. Conducting studies in registries has an additional potential to accelerate the studies and make them less complex and resource-intensive (registry-based comparative studies with randomization).
Jürgen Windeler, IQWiG’s Director, concludes: “The generation of routine practice data and their analysis is potentially feasible in the near future – but for the time being, in addition to study-specific data collection, only via data collection from registries. We have documented which data must be available in the registries and in what quality. The registry operators were very open-minded in their discussions with us, so I expect that the first data from high-quality registries will soon be available for use in benefit assessments of drugs.” In this context, Windeler also calls on politicians to act: “The conditions for high quality registries could be better. This concerns both funding and the fact that there are different requirements for data protection in different German federal states.”
###
Process of report production
On 2 May 2019, the Federal Joint Committee (G-BA) commissioned IQWiG to develop scientific concepts for the generation of routine practice data and their analysis for the benefit assessment of drugs. According to the commission, the report was to be prepared in an accelerated procedure as a so-called rapid report. Interim products were therefore not published or made available for a hearing. This rapid report was sent to the contracting agency, the G-BA, on 10 January 2020.
An English-language extract is available and the English-language full report will be available soon. If you would like to be informed when it is published, please send a request to info@iqwig.de.