Sprecher
Beschreibung
The race for the most efficient, accurate, and universal algorithm in scientific computing drives innovation. However, this healthy competition is only beneficial if research outputs from different projects are actually comparable to one another. Fairly comparing algorithms can be a complex endeavor, as the implementation, configuration, compute environment, and test problems need to be well defined. Due to the increase in computer-based experiments, new infrastructure for facilitating the exchange and comparison of new algorithms is also needed. To this end, we propose a benchmark framework, which is a generic toolkit for comparing implementations of algorithms using test problems native to a community. Its value lies in its ability to fairly compare and validate existing methods for new applications, as well as compare newly developed methods with existing ones.
As a prototype for a more general framework, we have begun building a benchmark tool for the Model Order Reduction Wiki (MORWiki). The wiki features three main categories: benchmarks, algorithms, and software. An editorial board curates submissions and edits entries. Data sets for linear and parametric-linear models are already well represented in the existing collection. Data sets for non-linear or procedural models are being added and extended. Searchable attributes for all categories are actively being aggregated in metadata databases.
The MORWiki collection will serve as the primary basis for our model reduction benchmark tool. To this end, experiences from related projects serve as prototypes and will be extended to encompass diverse model types and performance measures. The MORWiki will serve as a proof-of-concept for a living document and progress-tracker of a field, while also facilitating fair comparisons of new findings and methods. Its core information will be mirrored in the MaRDI-Portal, which is concurrently under development.