The recent presentation of the French government’s anti-pollution plan was an opportunity for many in the media to remind us of a significant figure: air pollution from fine particles is estimated to cause 48,000 deaths per year in France. Despite its surprising nature (this would mean that about 9% of all deaths were caused by pollution), this figure was treated with few reservations in the press, and even less in political circles (Anne Hidalgo had already used this argument in 2017 for the City of Paris’ anti-pollution plan) It is true that it comes from an irreproachable source, as it is a report by the Agence Santé Publique France (SPF)[i] [Public Health Agency France], which estimated the number of premature deaths attributable in France to PM 2.5 particles (particles with a radius less than or equal to 2.5 micrometres), using a new mathematical model, based on a map of spatialised air pollution over the entire French territory, at the municipal level. But can this figure of 48,000 deaths be taken as a basis for consideration without reservations? And what exactly does it measure? A careful reading of the SPF report reveals some surprises…
A range of uncertainty between 11 and 48,000 deaths!
The first note of caution (the only one mentioned in the SPF press release, and therefore the only one sometimes taken into account by the more vigilant media) is to state that 48,000 is only the top of a particularly gaping range of uncertainty. In fact, the number of premature deaths calculated by the SPF model varies considerably according to the basic level of pollution considered as “normal”: from 48,000 annual deaths if we take as a reference the least polluted parts of French territory… to 11 deaths if we take as a reference the threshold of fine particles recommended by the WHO ! It is clear, however, that SPF clearly favours a maximising interpretation of 48,000 deaths, and this is the only figure cited by most of the newspapers that reported on this publication. Even if the SPF does not express it very explicitly, it therefore amounts to severely criticising the current standards on air pollution. The INVS [Institut de veille sanitaire, Public Health Surveillance] (now part of SPF) had already questioned these standards in a previous study, which focused on short-term deaths caused by pollution[ii] peaks… forgetting in passing its own work on the effects of the heat wave (the short-term effects of pollution tend to occur only during summer pollution peaks, and not in winter[iii]).
A purely theoretical result, presented as an established truth
On such a serious subject, it therefore seems necessary to examine the arguments and the method employed by SPF in meticulous detail. From this point of view, the report presents some surprising gaps:
- It contains NO results validating the model used, i.e. no comparison between the mortalities calculated according to their model in each commune, and the actual recorded mortalities. When the authors tell us that their model is an accurate representation of reality, we are therefore obliged to take their word for it.
- This calculation is based on the application of a relative risk (RR), which defines the relationship between mortality and fine particle concentration in the air. In the usual publications on this subject, this RR is statistically calculated by combining a mortality data set with pollution data covering the same geographical area. This is not the method followed by SPF, which in this report has calculated mortalities directly from its spatialised pollution map, with an RR chosen by the authors. This choice of methodology is all the more audacious as the value used is very high compared to existing bibliographical references: RR=1.15 for an increase of 10 μg/m3, i.e. more than double the values used by the previous meta-analyses that the authors cite in their bibliography (0.6 and 0.7). The only justification for this surprising choice is that this value has already been observed in France (by the same SPF authors, there being no better reference than oneself…). Why not, but we would still like to know the RR value that would have been obtained directly from the actual mortality data at the commune level. Once again, an essential element for validation of the SPF model is missing from the report.
- To be scientifically sound, the authors show results from a sensitivity analysis, which theoretically allows the robustness of the model’s predictions to be assessed, based on the uncertainty in its input data. Unfortunately, this analysis only covers relatively secondary parameters, and dodges the essential question, linked to the previous point: what is the effect of uncertainty on Relative Risk, in estimating the number of premature deaths? For example, we would like to know what would have happened to the results if SPF had used a more conventional RR of 0.6 or 0.7.
Air pollution versus olive oil
So, the discussions of the results completely obscure the real scientific questions posed by the method used. The statistical model used, like the previous studies on which it is based, calculates geographical correlations between mortality (all non-accidental causes combined) and the level of air pollution. This type of correlation is likely to be affected by obvious but difficult to correct confounding factors:
- Life expectancy is strongly correlated to socio-professional category. In large cities, however, the wealthiest people rarely live in the most polluted neighbourhoods. There is therefore an obvious potential for bias here, especially when working at the commune level in large conurbations.
- On the national scale, the least polluted areas are of course rural areas, with demographics, lifestyles, and diet, which are clearly distinct from those of urban populations. It is therefore risky to compare their mortality levels with those of polluted cities. Moreover, in the case of France, this type of large-scale geographical comparison is complicated by regional inequalities in life expectancy, long observed by demographers.[iv] In particular, there is a North-South gradient in life expectancy, which is generally explained by the benefits of the Mediterranean diet. This North-South gradient is also observed for the PM 2.5 concentration. Consequently, the effects of diet, if they exist, could be a confounding factor with the effect of pollution, but this question is not at all addressed by SPF. However, it would be a very necessary question: the effect on life expectancy that they attribute to pollution (3 to 4.5 years difference between the most and least polluted areas) would single-handedly explain the difference between the departments where life expectancy is the lowest and those where it is the highest. Since the supposed effect of diet would go in the same direction overall, and should therefore be added to that of PM 2.5, one of the two effects must be imaginary… or both exist but are both overestimated!
- 1 : On the left, map of life expectancy at birth by department (INED data), on the right the increase in life expectancy in the absence of pollution (SPF – French Public Health Model). The two maps show similarities when viewed on a very large scale (Comparison between the North and East, heavily polluted, and with low life expectancy, and the mountainous areas of southern France). Since the increase in life expectancy calculated by SPF is of the same order of magnitude as the differences in life expectancy actually observed between departments, this would therefore mean that pollution alone explains most of the regional disparities in mortality observed in France. This is of course possible, but still quite surprising, especially since other causes are often proposed for these disparities, such as the favourable effect of the Mediterranean diet. To verify the hypothesis of the dominant effect of PM 2.5 on mortality, it would therefore be very interesting to verify the correlation between mortality and pollution at a more detailed geographical scale. For example, if the “excess mortality corridor” of the Rhône valley calculated by the SPF model is observed in mortality statistics by commune, this would be a decisive argument to remove all doubt on the effect of PM on mortality. Unfortunately, the authors do not provide any validation results for their model.
In addition to these fundamental problems, there are problems of form that are awkward for a scientific publication:
- The authors repeatedly congratulate themselves on the fact that their results are consistent with previous studies, as if this confirmed the accuracy of their work. This is not surprising, however, since the results presented are just simulations of a model based precisely on these previous surveys, and not validated from actual field data. Their claim that their study confirms previous assessments of the health effects of pollution, is therefore perfectly tautological: their model in no way confirms the previous results, it simply applies them on a different geographical scale.
- They use the term preventable deaths several times, which is alarmist language to use, where it would be better to refer to premature deaths. The term preventable death would be acceptable for a disease which is otherwise very rare without a contributory environmental factor (such as pleural cancer for asbestos), but absolutely not in this case.
38,000 dead globally, including 48,000 in France?
In its rationale, this SPF study consequently inclines dangerously towards pseudoscience. The confusing presentation suggests that the mortalities mentioned were calculated from actual mortality data, when in fact they are theoretical simulations run from an unvalidated model, and whose potential confounding factors, though obvious, have not been studied at all. It should be noted that the methodological anomalies we mentioned are not all the responsibility of the authors: they broadly followed recommendations from WHO, which considers the effect of fine particles on health as demonstrated. While the studies on which this consensus is based provide consistent results, this is not surprising since they all use more or less the same method. Before this publication, several other works, not spatialised to such a degree, but based on the same methods, had already obtained results in the order of 40,000 premature deaths per year in France. This estimate has therefore come to the fore, despite its surprising nature (the attribution of 9% of deaths to pollution on national average, up to 13% in large cities). It implies that pollution must be the major determinant of geographical inequalities in life expectancy in France, which contradicts the other explanations considered as well as those clearly demonstrated, the effect of the socio-professional category, and that of the Mediterranean diet. With its detailed spatialisation of the distribution of PM 2.5 at the commune level, the new SPF model could be an excellent tool to check the consistency of the “pollution hypothesis” at any geographical scale. In particular, while the North/South gradient of particles coincides fairly well with the life expectancy gradient, this is not the case for their East-West distribution. A validation of the model for areas of discrepancy between fine particle density and life expectancy would therefore be exciting. A missed opportunity (for now?), since the authors were content to use their model to spatialise mortalities to a finer level, calculated with parameters validated in different models.
The question here is not whether this figure of 48,000 deaths per year due to PM 2.5 is correct or not, since the report does not provide the necessary elements to judge that. No effort is made to distinguish the effect of PM 2.5 from an effect considered to be just as scientifically proven, that of diet. It is as if SPF were to do epidemiological studies on lung cancer, without adjusting the results according to tobacco consumption. So it is all proceeding as if chronic mortality due to PM 2.5 has become a dogma, which pollution specialists no longer even try to verify. Yet even the proponents of this hypothesis show that much remains to be clarified. At the same time, an article published in Nature estimated the deaths due to PM 2.5 and NOx [i] at 38,000 per annum worldwide, including 28,000 in the European Union… compared to the 48,000 for France alone, and PM 2.5 alone, calculated by SPF! Spot the mistake… or the mistakes?
Give a dog a bad name…
In summary, this figure of 48,000 deaths:
- Is the upper range of an interval whose lower range is… 11
- Comes from a purely theoretical statistical model:
- with an unusual methodological choice (relative risk chosen a priori by the authors, and not calculated from actual mortality data)
- whose authors do not present any element of comparison with the reality on the ground
- amounts to attributing to air pollution the entire difference in life expectancy which has long been recorded between the North and the South of France, and which is usually explained by diet: this is of course possible, but it deserves to be reasoned through (whereas this “slight detail” is not even mentioned in the SPF report)
None of this is really scientifically beyond the pale, but it does show that these famous 48,000 deaths are at this stage only a combination of bold hypotheses, which very much need to be proven. This report remains in the field of theoretical speculation, and not in the quantification of a phenomenon solidly demonstrated on the ground: a legitimate stance for researchers (provided they then move on to validating their hypotheses…), but rather more surprising for a health assessment agency, supposed to rely on methodologies which are proven… and validated at the international level! And it is precisely if it were to be applied internationally that the SPF model would demonstrate its full impact. We have seen that at the national level, this model estimates that fine particles are responsible for 9% of deaths throughout France, including rural areas. Out of curiosity, we would like to know what percentage of mortality this model would reach for a city like Beijing, incomparably more polluted than even the worst French cities: 80, 150%? Similarly, SPF states without turning a hair that the number of excess deaths caused by pollution would be 48,000 if you take as a reference the French regions without anthropogenic pollution, and 11 if we take as a reference the European threshold of 25 μg of PM2,5/m3of air. This means that the European standard underestimates the number of victims of pollution by a factor of more than 4000. Why not? But this would also deserve international validation, unless we assume that the French are a people particularly sensitive to pollution.
The most elementary scientific caution would therefore have suggested not publishing such a bold hypothesis, without providing comparisons with actual mortality data by commune, and without having tested its plausibility for countries other than France. But when a hypothesis suits the political concerns of the moment so well, it is sometimes difficult to wait…
Fig. 2: The French Public Health report, which published the figure of 48,000 victims of air pollution in France per annum, was published one month after an article in the prestigious journal Nature, which attempted the same assessment on a worldwide scale. The militant environmental press largely parroted both these two publications, without ever noting their obvious contradictions:
- On the one hand, an estimate of 38,000 victims each year from PM 2.5 fine particulate matter and nitrogen oxides, on a global scale, thus including the hundreds of millions of inhabitants of the metropolises of emerging countries
- On the other hand, an estimate of 48 000 victims for MP 2.5 alone, on the scale of the 66 million French population, with considerably lower levels of air pollution.
[i] NOx : Nitrogen oxides