2020/31 LEM Working Paper Series

Automated and Distributed Statistical Analysis of Economic Agent-Based Models

Andrea Vandin, Daniele Giachini, Francesco Lamperti and Francesca Chiaromonte
ABM; Automated and Distributed Simulation-based Analysis; Statistical Model Checking; Steady-state and Transient analysis; Warmup estimation; T-test and power; Prediction markets; Macro ABM.

  JEL Classifications
C15, C18, C63, D53, E30
This paper proposes a novel approach to the statistical analysis of simulation models and, especially, agent-based models (ABMs). Our main goal is to provide a fully automated and model-independent framework to inspect simulations and perform model-based counter-factual analysis that (i) is easy-to-use by the modeller, (ii) improves reproducibility of results, (iii) is as much fast as possible given the modeller’s machine by exploiting multi-core architectures, (iv) automatically chooses the number of required simulations and simulation steps to reach a user-specified statistical confidence, and (v) automatically runs a variety of statistical tests that are often overlooked. In particular, the proposed approach allows distinguishing the transient dynamics of the model from its steady state behaviour (if any), to estimate properties of the model in both “phases” and to equip the results with statistical guarantees, allowing also for robust comparison of model behaviours across computational experiments. The approach instantiates a family of analysis techniques from the computer science community known as statistical model checking, by redesigning and extending the statistical model checker MultiVeStA. The authors showcase the usefulness of the approach within two models from the literature: a large scale macro financial ABM and a small scale prediction market model obtaining new insights on the studied models and identifying and fixing erroneous analysis from previous publications.
download pdf