Updates on Benchmarking Network Activities

[benchmarking-network]

Dear all,

This mail is to update you about the activities of the Benchmarking Network https://sites.google.com/view/benchmarking-network/

1. The GECCO workshop was a nice opportunity to discuss the goals of benchmarking, good benchmarking practices, open issues, as well as ongoing activities.
   More than 70 colleagues attended the workshop.
   In case you have missed it, you can find the slides here: https://sites.google.com/view/benchmarking-network/home/activities/GECCO20
   
   
2. Survey paper: the first draft of the survey Benchmarking in Optimization: Best Practice and Open Issues is available at https://arxiv.org/abs/2007.03488
   We are planning to extend this first version significantly, so feedback, additional pointers to good sources for good benchmarking practices and all other suggestions are very welcome.
   If you want to contribute to this survey, please get in touch with Thomas Bartz-Beielstein and Carola Doerr
   
   
3. The real-world optimization survey is available at https://docs.google.com/forms/d/e/1FAIpQLSf27nQcgJ4X690gcL6CBDcLEMUdESUYarz5_7dFFOj89U8rZQ/viewform
   Please consider to contribute, to help us design/select better benchmark problems
   
   
4. Competitions:
   
1. The Game-Benchmark for Evolutionary Algorithms (GBEA) competition invites entries until August 7: http://www.gm.fh-koeln.de/~naujoks/gbea/gamesbench_ppsn20comp.html
   
2. The Open Optimization Competition 2020, organized by the Nevergrad and the IOHprofiler teams, has been extended and accepts contributions until September 30: https://github.com/facebookresearch/nevergrad/blob/master/docs/opencompetition2020.md
   
   
5. IEEE Task Force: In collaboration with the Benchmarking Network, there is now also an IEEE Task Force, which you can find here: https://cmte.ieee.org/cis-benchmarking/
   
   
6. We are looking forward to our Benchmarking workshop at PPSN. Please mark it in your calendar!
   Saturday, September 5, 16:00-17:30 (Dutch time zone).
   The workshop will be hybrid, onsite and online.
   Online participation will be possible for everyone, not only for people registered at PPSN (login details will be made available before the workshop)
   We have an exciting schedule, with lots of room for discussion
   

   1. Reproducibility and Replicability
      Manuel López-Ibáñez
      
      
   2. Performance Measures
      Jakob Bossek
      
      
   3. Fairly Assessing Algorithms' Contribution to the State of the Art
      Holger Hoos 
      
      
   4. Ongoing benchmarking initiatives and how to get involved
      Carola
      

   
   
7. Travel grants for research visits and conference participation: if you are based in Europe and are planning a short research visit before September 30 that is related to benchmarking, please consider to apply for a "short-term scientific mission" within COST action CA15140: Improving Applicability of Nature-Inspired Optimisation by Joining Theory and Practice (ImAppNIO): https://imappnio.dcs.aber.ac.uk/stsms
   
   For colleagues affiliated with an institution in the following countries, ITC conference grants are available (covering either full travel cost or registration for online participation in a conference): https://imappnio.dcs.aber.ac.uk/itc-conference-grants  (eligible countries are Albania, Bosnia-Herzegovina, Bulgaria, Cyprus, Czech Republic, Estonia, Croatia, Hungary, Lithuania, Latvia, Luxembourg, Malta, Moldova, Montenegro, Poland, Portugal, Romania, Slovenia, Slovakia, Republic of North Macedonia, Republic of Serbia and Turkey)
   
   
8. Active Network Members: If you want to be listed in the list of active members at https://sites.google.com/view/benchmarking-network/home/members, please let us know. We may have missed a few mails, sorry for this. No intention or anything, just overwhelmed/-worked during the crisis. We should now be able to process all incoming requests more efficiently, so please just (re-)send your request by e-mail to mailto: benchmarking network
   

Best wishes!

Carola Doerr, Boris Naujoks, Pascal Kerschke, Vanessa Volz, Mike Preuss

-- 
Carola Doerr, CNRS researcher at LIP6, Sorbonne University, http://www-ia.lip6.fr/~doerr/