A colleague sent us a paper by Rana et al. called “Global magnitude-frequency statistics of the failures and impacts of large water-retention dams and mine tailings impoundments.” We reference it herein as Rana et al. (2022). It is available online from Earth-Science Reviews (2022).
At first, I was quite intimidated. Almost a decade earlier, we wrote “Factual and Foreseeable Reliability of Tailings Dams and Nuclear Reactors - a Societal Acceptability Perspective,” for Tailings and Mine Waste (TMW) 2013. We will reference our paper herein as Oboni & Oboni (2013). We had produced a first estimate of the rate of occurrence of major catastrophic tailings dam failures in the world. Thus, it was with a mix of anxiety and trepidation that I undertook the reading of the paper by such a group of experts and academics.
Rana et al. (2022)’s Scope
Rana et al. (2022) analyzes worldwide datasets of large reservoir facilities (LRFs) and tailings storage facilities (TSFs). It covers the period of 1965−2020. The paper handles unavoidable data gaps by adopting multiple estimation/extrapolation approaches. It discusses the assumptions and illustrates the resulting uncertainties.
In our discussion, we will focus on TSFs only, due to our specific experience in this matter. Let’s note right away that the uncertainties are very significant, even with basic data such as the number of tailings dams in the world. Indeed, Rana et al. (2022) indicates a low estimate of constructed TSFs at 6,810, and an upper estimate of 20,230.
Let’s Start with a Couple of Definitions
Rana et al. (2022) clearly defines the glossary it uses. We note the following definitions:
- “Tailings storage facility (TSF): an impoundment of tailings, supernatant including a dam, tailings, and water.” We note that in our experience, many TSFs include more than one dam to impound tailings in a specific pond. This generates uncertainties on the census. We also note that the failure definition in Rana et al. (2022) does not necessarily include a breach/catastrophic failure of impoundments.
- “Annual failure rate: the number of failures per the number of constructed facilities worldwide in a given year.” Rana et al. (2022) expresses this as a percentage.
- “Cumulative failure rate: the cumulative number of failures per the cumulative number of constructed facilities worldwide at the end of a given period.” Rana et al. (2022) expresses this as a percentage as well.
In Riskope’s practice, we prefer the scientific notation to the percentage as it better covers the hopefully small numbers depicting the probability of failure.
Oboni & Oboni (2013) Estimated the Following
Assumptions
Following Davies & Martin (2000) and Lemphers (2010), there were more than 3,500 tailings dams of various kinds and construction type, with at least 50% of the upstream design around the world. In the absence of more accurate data, and more particularly, on the evolution of this number, Oboni & Oboni (2013) considered a portfolio of 3,500 dams globally and 1,000 dams in the US alone. Portfolios were assumed constant over the years. However, we noted that even if that assumption was wrong by one order of magnitude, there would be no significant alterations in the conclusions of the paper.
From the United Nations Environmental Programme (1998) global data, we read that there were 44 major TSF failures reported in 1974−1984, and seven in 1994−2004.
Results
Using the numbers above in 2013, we evaluated for the worldwide portfolio in 1974−1984 a rate of 44/(3,500*10)= 10-3 per annum, and a rate of 7/35,000= 2*10-4 per annum for the 1994−2004. Oboni & Oboni (2013) also gave values for the US portfolio alone.
Oboni & Oboni (2013) vs. Rana et al. (2022) Worldwide Tailings Benchmarks Ten Years Later
First of all, let’s note that Rana et al. (2022) state that the global rate of reported TSF failures has remained constant since the mid-1960s. Indeed, the annual number of TSF failures worldwide mostly ranges between 2 and 6, closely matching the Oboni & Oboni (2013) estimates.
Rana’s 2022 Low TSF Estimate
So, Rana et al. (2022) and Oboni & Oboni (2013) agree that the annual number of TSF failures stayed relatively constant, despite a reported increase in the annual construction rate of TSFs. Rana et al. (2022) states that the cumulative failure rate of TSFs declined over time. When assuming Rana et al.’s lower estimate of the number of constructed TSFs (6,810), they estimate a cumulative failure rate of ~4.4% by the end of 2020.
Rana et al. (2022)’s Upper Estimate Generates Some Questions
Are These Results Counterintuitive?
However, when adopting their upper estimate (20,230 TSFs), Rana et al. (2022) obtained a rate of ~1.5%. This value is in the same order as the corresponding rate of LRFs. We perceive this result as strange. Indeed, the general perception is that hydro reservoirs are safer than tailings dams. At Riskope, we believe this strange value is the result of an exaggeration of the TSF count and “mixing oranges and apples” in the TSF worldwide basket.
“Advocates” Bias?
Let’s also note that some recent “advocate” papers seem to have inflated the number of dams in order to demonstrate the hazard. This approach does not help their discourse because inflating the number decreases the rates; an inflated number shows exactly the opposite of the desirable “advocate” result. Lastly, if the failure rates of tailings and hydro dams were indeed similar, public opinion should be as concerned by hydro as it is by tailings. This is not the case. However, things may change as LFRs age and begin to fail.
Using the same Oboni & Oboni (2013) data and converting them into the Rana et al. (2022) glossary definition, we would have:
Annual Failure Rate:
The estimation follows a few simple steps: 1974−1984 44/10= 4.4 major failures per annum, per 3,500 assumed TSF= 10-3 or 0.1%, respectively 7/10=0.7 per 3,500 leading to 2*10-4 or 0.02%. As we will see in the next section, these estimates are in good agreement with those of other researchers.
For benchmarking of dams, a standard procedure in the ORE2_Tailings™ quantitative risk assessment, which is compliant with the Global Industry Standard on Tailings Management, the annual failure rate is paramount. Indeed, it allows for the anchoring of results to factual reality. Furthermore, we can tell clients if their dams are better or worse with respect to the worldwide portfolio performance.
Cumulative Failure Rate:
By using the 1965−2020 timeframe (56 years) and a long-term average yearly number of failures of 3.5, again with 3,500 TSFs, and apply Rana et al. (2022)’s definition, we get 3.5*56/3,500=0.06 or 6%. If we use the average values of our two decades of 1974−1984 and 1994−2004, we find (4.4+0.7)/2=2.55. Thus, by applying Rana et al. (2022)’s definition, we get a cumulative failure rate of 2.55*56/3,500=0.04 or 4%. Again, the match between Oboni & Oboni (2013) and Rana et al. (2022) is excellent.
The cumulative failure rate is not useful for benchmarking single dams as it is the victim of “excessive averaging,” a point we discuss below.
Other “Statistical Studies” and Estimates
Azam & Li (2010)
In our book Tailings Dam Management for the Twenty-First Century, we quoted, like Rana et al. (2022) does, Azam & Li (2010). These researchers suggested a historical global failure rate of 1.2% for TSFs. One gets to that value by dividing 220 recorded failures by 18,400 mines, assuming 1 TSF per mine. It is important to note that this value covers over a hundred years of records. Thus, the actual annualized rate of failure is 1.2%/100= 1.2*10-4 in full agreement with the value by other researchers, such as Halabi et al. (2022). Indeed, they assumed the global estimate of 18,400 existing TSFs and calculated the annual probability of TSF failures to have been in the order of 10-4 on average in the period of 1950−2019.
Other Statistical Approaches
Other statistical approaches bear on the rate of failure, potential consequences and potential failure modes. Their discussions in 2018−2019 are present on this blog as follows:
- Larrauri & Lall (2018) and Azam & Li (2010) on consequences of dam failures
- Azam & Li (2010) on failure modes showing that causality is more important than failure modes
- Lyu et al. (2019)’s paper “A Comprehensive Review on Reasons for Tailings Dam Failures Based on Case History.” Our related discussion on statistical studies taxonomies and their impact on the conclusions is in consequences of failure and geoethics.
Thus, the benchmark values we use in ORE2_Tailings since 2014 are confirmed by Rana et al. (2022) paper as well as the other researchers referenced above. Worldwide tailings dam benchmarks ten years later are a success!
Using Rates and Evaluating “Returns”
At Riskope, we are reticent to use the term “returns,” because laymen tend to understand a return of n as “the next failure will occur in n years.” This is not the only reason we think the return is a general disservice to the public.
However, for the sake of comparison, we have picked a selected number of dam portfolios with varying volumes in various parts of the world. We analyzed these 57 dams for various clients using ORE2_Tailings. They are a subset of the large number of structures we have evaluated to date using the same methodology.
Three Sample Portfolios
- ·Portfolio A: A portfolio of four upstream dams in Central Asia with an average yearly probability of failure of 7.80E-03. These are poorly documented, monitored and maintained dams that give a return of ~32 years.
- Portfolio B: A selection of 14 dams in the US with excellent documentation, analyses, mitigation and good monitoring. Their yearly average probability of failure is 7.69E-04, giving a return of ~93 years.
- Portfolio C: 39 Canadian mixed-quality dams with a yearly average probability of failure of 6.79E-04, giving a return of ~38 years.
Three Sample Conclusions
- For the entire portfolio A+B+C, 57 dams total, the evaluated return is one failure on average every 21 years because a few bad apples spoil the lot in terms of expected returns.
- If we assumed that the entire portfolio of 57 dams is representative of the worldwide portfolio, and use the values above, we would get 2.9 failures per year. This value falls well within the observational range of the last hundred years.
- However, the fact that the long-term average worldwide is around 3.5 dam failures per year shows that the worldwide portfolio is “worse” than our sample of 57 dams.
Important Notes
Thus, not only are the benchmarks correct, but ORE2_Tailings results are proven to be realistic and well-calibrated at the worldwide scale. Furthermore, the returns yielded by the ORE2_Tailings analyses in our day-to-day practice also match the returns evaluated by Rana et al. (2022). This constitutes more proof of the correct calibration of the methodology based, for example, on more than a hundred dams evaluated in our forthcoming paper titled “Optimizing mitigation of tailings dam portfolios” at TMW 2022.
Averages Need to be Taken with a Grain of Sand (Pun Intended)
As we mentioned earlier, one should consider averages with care because they tend to smooth out data. Thus, they depict an optimistic image of the whole portfolio. This hinders good mitigative decisions and overall management.
To illustrate this, we have prepared a graph of the annual probability of failure (vertical axis) for the three portfolios A, B and C, and the global (A+B+C) results. The graph depicts Oboni & Oboni (2013) min-max benchmarks and pre-catastrophic threshold as horizontal lines. For each portfolio, we show the min-max annualized estimated probabilities of catastrophic failure, the average and the standard deviation range (blue bar).
We can Observe the Following:
- Portfolio A has a low variability because of the small sample size.
- Portfolio B has a larger sample size. Despite having an average within the worldwide benchmark, it tends to overlap the worldwide benchmark with some dams.
- Portfolio C has some dams with their max estimates being pre-catastrophic, despite having an average within the worldwide benchmark.
- The global portfolio (A+B+C) carries the min-max estimates of the best and worse estimates of the overall portfolio. It has an average and standard deviation influenced by each dam. However, note that the influence of the Portfolio A dams gets smoothed down by averaging.
- This leads to a statement we have often made. When looking at tailings dams, the devil is in the details; oversimplification or “bulk” judgements can be extremely misleading.
