PRISMA statement.
PRISMA statement establishes the methodological requirements that a systematic review with meta-analysis must comply.
A long, long time ago a squirrel could cross the Iberian Peninsula without getting off the trees. Such was the lushness of our land. But don’t you be so sure that this is true, because some people think that it’s nothing more than a myth. Anyway, I wonder if the squirrel in question would realize it was in a great forest. I guess yes, but you never know: sometimes you can’t see the forest for the trees or, rather, you can’t see the entire forest.
In any case, a modern squirrel would not have this problem. There’s no doubt that today it could not cross the Peninsula without getting out of the trees but, instead, it could cross the entire country without getting off the head of a fool. As I read one day on a blog, there’re more stupid people than bottles of beer, and they are also strategically placed so you run into, at least, a couple of then each day.
Meta-analysis is also sort of a forest where its primary studies would be the trees. How poetic it is!. But, in this case, trees not only don’t prevent you of seeing anything, they help you not only to see the forest, but to see the entire forest as a whole. Of course, for that, the meta-analysis results must be presented in a proper way.
PRISMA statement
Until recently we could follow the advices of the QUOROM statement, but this statement was updated to become PRISMA, which devotes seven of its 27 items to give us advice on how to present the results of a meta-analysis.
First, we have to detail the study selection process: how many of them we have found and evaluated, how many selected and how many rejected and why. For this purpose it can come very handy the flow chart that the systematic review should include if the authors have followed the PRISMA statement scheme.
Second, you must specify the characteristics of the primary studies, detailing what data we have extracted from each one of them. Also, we must provide their corresponding citations in order to facilitate the work to any reader who want to check the data if he or she doesn’t trust us. On the same direction is the third recommendation, which refers to the assessment of studies for its risk of bias and internal validity.
Fourth, we have to present the results of each individual study given a summary measure of each intervention group analyzed as well as their calculated estimates with their confidence intervals. These data will provide the information we’ll need in the next step, the fifth PRISMA’s point recommendation concerning the presentation of results, which is non-other that the global synthesis of meta-analysis studies, confidence intervals, homogeneity study result and so on.
Forest plot
This is usually done graphically using a popular tool known by the name of forest plot. This is a kind of forest where trees are the meta-analysis’ primary studies and which summarizes all relevant results of quantitative synthesis.
The Cochrane Collaboration recommends structuring the forest plot in five different columns. Column 1 shows the primary studies or the groups or subgroups of patients included in the meta-analysis. They are usually represented by an ID composed by the author’s name and the date of publication.
Column 2 gives you the effect measure as recorded or calculated by the authors of each included study.
Column 3 is the actual forest plot. It shows the effect measures represented on both sides of the vertical line of no effect. We already know the null value is equal to zero for mean differences and to one for odds ratios, relative risks, hazard ratios, etc. Each study is represented by a blob whose area is proportional to the weigh with which each study contributes to the pooled effect. Also, each blob is represented in a horizontal line which represents its confidence interval.
These confidence intervals inform about the precision of studies and tell us which ones are statistically significant: those whose interval doesn’t cross the line of no effect. Anyway, you should not forget that, even though they cross the line of no effect and are not statistically significant, the position of the confidence intervals limits give us a lot of information about clinical relevance of results obtained from each study.
Finally, at the bottom there’s a diamond that represents the pooled effect of the meta-analysis. Its position with respect to the line of no effect will tell us about the statistical significance of the effect, while its width will give us information about its precision (its confidence interval). Also, at the top you can find the type of measure that is represented in the forest plot, the statistical method used to pool it (fixed effect model or the random effects model) and the measure of the confidence interval given (conventionally taken as 95%).
The graph is usually completed with a fourth column with the percent weight of the estimates of each study respect the pooled result and a fifth column with the effect estimates in numbers. And in a little corner of that forest you will find the heterogeneity measure that the authors have calculated, along with its statistical significance when appropriate.
At the end of results presentation, PRISMA recommend a sixth section dedicated to detail any risk of bias that have been assessed and a seventh with all additional analysis that have been performed, as needed: stratification, sensitivity analysis, meta-regression, etc.
As you can see, nothing is simple about meta-analysis. Therefore, Cochrane’s recommend us to follow a series of steps to interpret results properly. These are the following:
- Check the variable that is compared and how. You can usually find it out at the top of the forest plot.
- Find out the effect measure used. You can understand this is needed to interpret results accurately. A hazard ratio is different from a mean difference or whatever it has been used.
- Look at the diamond, its location and width. It’s also very convenient to exam the numerical value of the pooled estimate and its confidence interval.
- Check that heterogeneity has been determined. You can find it out at a glance looking whether segments representing primary studies are widely scattered or not, and checking if they overlap with each other. In any case, there must always be a statistical parameter to quantify heterogeneity. If we find that there’s heterogeneity, next thing we have to do is to look for the explanation provided by the authors about its existence.
- Draw your conclusions. We will look at which side of the line of no effect the pooled estimate is and its confidence interval. You know that, even though statistically significant, its lower limit should be as far away from that line as possible: clinical relevance and statistically significant is not always synonym. Finally, go back again to homogeneity study. If heterogeneity exists, reliability of results could be compromised.
We’re leaving…
And here we end up with results presentation and forest plot. As a matter of fact, the forest plot is not exclusive to meta-analysis as it can be used whenever we want to compare studies to assess their clinical or statistical significance, or in others cases such us in equivalence studies, in which the null line is flanked by the equivalence thresholds. But it still has further utility. A variation of the forest plot is also used to assess the publication bias in systematics reviews, although in those cases it is often called funnel plot instead of forest plot. But that’s another story…