Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks. © 2012 ACM.

Mcdermott, J., White, D., Luke, S., Manzoni, L., Castelli, M., Vanneschi, L., et al. (2012). Genetic programming needs better benchmarks. In GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation (pp.791-798) [10.1145/2330163.2330273].

Genetic programming needs better benchmarks

MANZONI, LUCA;CASTELLI, MAURO;VANNESCHI, LEONARDO;
2012

Abstract

Genetic programming (GP) is not a field noted for the rigor of its benchmarking. Some of its benchmark problems are popular purely through historical contingency, and they can be criticized as too easy or as providing misleading information concerning real-world performance, but they persist largely because of inertia and the lack of good alternatives. Even where the problems themselves are impeccable, comparisons between studies are made more difficult by the lack of standardization. We argue that the definition of standard benchmarks is an essential step in the maturation of the field. We make several contributions towards this goal. We motivate the development of a benchmark suite and define its goals; we survey existing practice; we enumerate many candidate benchmarks; we report progress on reference implementations; and we set out a concrete plan for gathering feedback from the GP community that would, if adopted, lead to a standard set of benchmarks. © 2012 ACM.
Si
paper
benchmarks; genetic programming; Computational Theory and Mathematics; Applied Mathematics
English
14th International Conference on Genetic and Evolutionary Computation, GECCO'12
9781450311779
Mcdermott, J., White, D., Luke, S., Manzoni, L., Castelli, M., Vanneschi, L., et al. (2012). Genetic programming needs better benchmarks. In GECCO'12 - Proceedings of the 14th International Conference on Genetic and Evolutionary Computation (pp.791-798) [10.1145/2330163.2330273].
Mcdermott, J; White, D; Luke, S; Manzoni, L; Castelli, M; Vanneschi, L; Jaśkowski, W; Krawiec, K; Harper, R; De Jong, K; O'Reilly, U
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/60799
Citazioni
  • Scopus 203
  • ???jsp.display-item.citation.isi??? 157
Social impact