FFIs prediksjonsturnering – idé- og metodebeskrivelse

In all defence planning, it is necessary to make some assumptions about the future strategic environment. However, given that it is impossible to be certain about what the future may entail, defence and security analyses, expert opinions, threat assessments and scenarios will always involve some degree of forecasting. To forecast, or to predict, is to make estimates about future events, e.g. about the scope of threats, changes in defence spending or the conduct of war. Yet, the forecasting accuracy of the Norwegian defence and security community has never been measured. Furthermore, all researchers, experts and officers cannot be equally right about what may happen in the future. Given the potentially catastrophic consequences of preparing for the wrong threats, the degree to which the predictions we base our planning on are correct, will be decisive for how well prepared the Norwegian Armed Forces will be in face of future challenges. Over the next three years (2017–2020), FFI will therefore be conducting a forecasting tournament for everyone who is interested in or works on defence and security policy-related topics in Norway. The main objective is to measure the Norwegian defence and security community’s ability to forecast developments and events of relevance to the Norwegian Armed Forces. The tournament will be conducted through a monthly survey with five questions on topics ranging from international politics and the economy to armed conflicts, technology and future warfare. The tournament will be open to everyone, both from within and outside the Norwegian defence sector, regardless of formal background or previous experience. Thus, it will be possible to compare the forecasting ability of amateurs with that of professionals who work on the questions posed on a daily basis, including military planners and intelligence analysts. Forecasting accuracy will be measured using Brier score, which is a commonly used proper score function that measures the accuracy of probabilistic predictions. Thus, all participants will be asked to estimate how likely (in percentages) they believe each different outcome is. The goal is to assign high probabilities to outcomes that do occur, whilst assigning low probabilities to outcomes that do not. All participants will receive an individual score and rank on every question. The five best participants will be named and winners will be announced once a year. All scores will also be measured against various algorithms and groups of other participants. The tournament’s results will provide the first quantitative measure of the Norwegian defence and security policy community’s forecasting accuracy. It will also provide insights on whether there are some topics that are harder to predict than others, whether some groups of people are better than others and whether there are some individuals in Norway who are particularly good. Hopefully, the knowledge gained through this forecasting tournament will help increase the accuracy of analyses in support of decisions on the future of the Norwegian Armed Forces.