Critical appraisal of hybrid designs studies.

The unreal mixture of different parts of animals has been an obsession of so-called human beings since immemorial time. The most emblematic case is that of Chimera (which gives its name to the whole family of mixtures of different animals). This mythological being, daughter of Typhon and the viper Echidna, had a lion’s head, a goat’s body and a dragon’s tail, which allowed him to breathe in flames and scare everyone up who passed by.

Of course, it did not help him when Belloforontes, mounted on Pegasus (another weirdo, a horse with wings), insisted on crossing it with his lead spear. You see, in his strength was his downfall: the fire melted the tip of the spear into this rare creature, which resulted in its death.

Besides Chimera, there are many more of these beings, all of them fruit of human imagination. To name a few, we can remember the unicorns (these had worse luck than Pegasus, instead of wings they had horns, one each animal), the basilisks (a kind of snake rooster of quite bad character), the gryphon (lion’s body and eagle for the rest) and all those in which part of the mixture is human, such as manticores (head of man and body of lion), centaurs, Minotauro, Medusa (with their snakes instead of hair), mermaids…

In any case, among all the beings of this imaginary zoo, I am left with the chickenphant (gallifante in Spanish). This was a mixture of chicken and elephant that was used on TV to reward the wit of children who attended a popular contest. Millenials will have no idea what I’m talking about, but surely those who grew up in the 80s do know what I mean.

And all this came to my mind when I was reflecting on the number of chimeras that also exist among the possible types of scientific study designs, especially among observational studies. Let’s get to know a little three of these chickenphants of epidemiology: the case-control studies nested in a cohort and the case and cohort studies , to end with another particular specimen, the case-crossover or self-controlled studies.

Clasic observational studies

Within observational studies, we all know the classic cohorts and the cases and controls studies, the most frequently used.

In a cohort study, a group or cohort is subjected to an exposure and followed over time to compare the frequency of appearance of the effect compared to an unexposed cohort, which acts as a control. These studies tend to be of an antegrade direction, so they allow us to measure the incidence of the disease and calculate the risk ratio between the two groups. On the other hand , a case-control study starts from two population groups, one of which presents the effect or disease under study and compares its exposure to a specific factor with respect to the group that does not have the disease and that acts as a control.

Being of retrograde direction and directly selecting cases of disease, it is not possible to directly calculate the incidence density and, therefore, the risks ratios between the two groups, making the odds ratio the measure of association typical of case-control studies.

The cohort study is the most solid of the two from a methodological point of view. The problem is that they usually require long follow-up periods and large cohorts, especially when the frequency of the disease studied is low, which leads to the need to manage all the covariates of this large cohort, which increases the costs of the study.

Hybrid designs

Well, for these cases in which neither the cases and controls nor the cohorts are well suited to the needs of the researcher, epidemiologists have invented a series of designs that are halfway between the two and can mitigate their shortcomings. These hybrid designs are the case-control studies nested in a cohort and the case and cohort studies to which we have already referred.

In another order of things, in classical observational studies the key point is in the selection of controls, which have to be representative of the level of exposure to the risk factor evaluated in the population from which the cases originate. An adequate selection of controls becomes even more difficult when the effect occurs abruptly. For example, if we want to know if a copious meal increases the risk of heart attack, we would have great difficulty in collecting representative controls of the population, since the risk factors can act instants before the event.

To avoid these difficulties, the principle of “you make your bed, you lie in it” was applied and the third type of chimera we have mentioned was designed, in which each participant acts, at the same time, as his own control. They are case-crossover studies, also known as self-monitoring cases studies.

Cases and controls nested in a cohort

Let’s see these weirdos, beginning with cases and controls nested in a cohort.

Suppose we have done a study in which we have used a cohort with many participants. Well, we can reuse it in a nested case-control study. We took the cohort and followed it over time, selecting as cases those subjects who are developing the disease and assigning them as controls individuals from the same cohort who have not yet presented it (although they can do it later).

Thus, cases and controls come from the same cohort. It is convenient to match them taking into account confusing and time-dependent variables, such as the years they have been included in the cohort. In this way, the same subject can act as a control on several occasions and end as a case in another, which will have to be taken into account at the time of the statistical analysis of the studies. As this seems a bit confusing, I show you a scheme of this type of studies in the first attached figure.

As we are seeing how cases arise, we are doing a sampling by density of incidence, which will allow us to estimate risks ratios. This is an important difference with conventional case-control studies, in which an odds ratio is usually calculated, which can only be assimilated to the relative risk when the frequency of the effect is very low.

Another difference is that all the information about the cohort is collected at the beginning of the study, so there is less risk of producing the classic information biases of the case-control studies, usually of a retrospective nature.

Case and cohort studies

The other type of hybrid observational design that we are going to discuss is that of the case and cohort studies. Here we also start from a large initial cohort, from which we select a more manageable sub-cohort that will be used as a comparison group. Thus, we see which individuals of the initial cohort develop the disease and compare them with the sub-cohort (regardless of whether or not they belong to the sub-cohort). You can see the outline of a case study and cohort in the second figure attached.

As in the previous example, when choosing cases over time we can estimate the density of incidence in cases and not cases, calculating the risk ratio from them. As we can imagine, this design is cheaper than conventional studies because it greatly reduces the volume of information of healthy subjects that must be handled, without losing efficiency when studying rare diseases.

The problem that arises is that the sub-cohort has an overrepresentation of cases, so that the analysis of the results cannot be done as in traditional cohorts, but has its own methodology, much more complicated.

To summarize what has been said so far, we will say that the nested case-control study is more like the classic case-control study, while the case and cohort study is more like the conventional cohort study. The fundamental difference between the two is that in the nested study the sampling of the controls is done by incidence density and by pairing, so we must wait until all cases have been produced to select the entire reference population.

This is not the case in the case and cohort study, which is much simpler, in which the reference population is selected at the beginning of the study.

When the time of event matters

To put an end to these hybrid studies, we will say some things about case-crossover studies. These focus on the moment in which the event occurs and try to see if there has been something unusual that has favored it, comparing the expositions of moments immediate to the event with previous ones that serve as control. Therefore, we compare case moments with control moments, each individual acting as their own control.

For the study to be valid from the methodological point of view, the authors have to clearly describe a series of characteristic periods of time. The first is the induction period, which is the delay time that occurs from the beginning of the exposure until the production of the effect.

The second is the period of effect, which is the interval during which exposure can trigger the effect. Finally, the period of risk would be the sum of the two previous periods, from the moment of exposure to the beginning of the event.

The induction period is usually very brief most of the times, so the period of risk and effect are usually equivalent. In the attached figure I show you the relationship between the three periods so that you understand it better.

It is essential that these three periods be clearly specified, since a poor estimate of the period of effect, both by excess and by defect, produces a dilution of the effect of the exposure and makes its detection more difficult.

Some of you will tell me that these studies are similar to other self-controlled studies, such as paired cases and controls studies. The difference is that in the latter one or more similar controls are chosen for each case, while in the self-controlled each one is its own control.

They also look a little like cross-over clinical trials, in which all participants are subjected to intervention and control, but these are experimental studies in which the researcher intervenes in the production of the exposure, while self-controlled studies are observational studies.

In what it resembles paired cases and controls is in the statistical analysis, only here case moments and control moments are analyzed. In this way, it is usual to use conditional logistic regression models, being the most common measure of association the odds ratio.

Critical appraisal of hybrid designs

As you can see, hybrid studies are a whole new family that threatens to grow in number and complexity. As far as I know, there are no checklists to critically aprraise these types of designs, so we will have to apply judiciously the principles we apply when analyzing classical observational studies, taking into account, in addition, the particularities of each type of study.

For this, we will follow our three pillars: validity, relevance and applicability.

In the VALIDITY section we will assess the methodological quality with which the study was made. We will check that there is a clear definition of the study population, the exposure and the effect. If we use a reference cohort, it should be representative of the population and should be followed completely. On the other hand, the cases will be representative of the population of cases from which they come and the controls have to come from a population with an exposure level representative of the case population.

The measurement of the exposure and the effect must be done blindly, being independent the measurement of the effect and the knowledge of the level of exposure. In addition, we will analyze if attention has been paid to the temporal relationship of events between exposure and effect and if there was a relationship between the level of exposure and the degree of effect.

Finally, the statistical analysis should be correct, taking into account the control of possible confounding factors. This part can be complicated by the complexity of the statistical studies that usually require this type of designs.

In addition, as we have already mentioned, if we are facing a case-crossover study, we must ensure that there has been a correct definition of the three periods, especially the period of effect, whose inaccuracy may affect the conclusion of the study to a greater degree.

Next, we will evaluate the RELEVANCE of the results and their accuracy as measured by their confidence intervals. We will look for the impact measurements calculated by the authors of the study and, if they do not provide them, we will try to calculate them ourselves. Finally, we will compare the results with other previously published in the literature to see if they are concordant with the existing knowledge and what new knowledge are provided.

We will finish the critical appraising assessing the APPLICABILITY of the results. We will think if the participants can be assimilated to our patients and if the conclusions are applicable to our environment.

We’re leaving…

And here we are going to finish this post. We have seen a whole new range of hybrid studies that combine the advantages of two observational studies to better adapt to situations in which classical studies are more difficult to apply.

The drawback of these studies, as we have said, is that the analysis is a bit more complicated than that of the conventional studys, since it is not enough to get a crude analysis of the results, but must be adjusted by the possibility of that a participant can act as control and case (in the nested studies) and by the overrepresentation of the cases in the sub-cohort (in the cohort and cases).

I just finish commenting that all I have said about the case-crossover studies refers to the so-called unidirectional case-crossover ones, studies in which there is a very precise temporal relationship between exposure and effect. For the cases in which the exposure is more maintained, other types of case-crossover studies called bidirectional case-crossover studies can be used, in which control periods are selected before and after the effect. But that is another story…