[robotics-worldwide] [Meetings] CfP to the Open forum on evaluation of results, replication of experiments and benchmarking in robotics research, Hamburg, Germany, September 28, 2015 (full day)

Fabio Bonsignorio fabio.bonsignorio at gmail.com
Wed Aug 5 04:55:09 PDT 2015

Apologies for multiple postings

Dear All,
you are kindly invited to submit to the IROS 2015  Open forum on evaluation
of results, replication of experiments and benchmarking in robotics
research,  which will be held in Hamburg, Germany on September 28th, 2015.

In Robotics Research the replicability of results and their objective
evaluation and comparison is very difficult to put into practice.
Furthermore, there is no solid theoretical foundation for experimental
replicability in robotics.

This situation impairs at the same time research progress and technology
Significant progress has been made in this respect in recent years. A RAM
special issue composed of replicable experiments and guest edited by some
of the proposers will be published at the end of 2015, while next September
- just before IROS in Spain -  there will be the 2015 IEEE RAS Summer
School dedicated to same topics and organized by the proposers. There are
still available places and special rates for students coming from
developing countries. Check here: http://ieee-raspebras2015.org if you are
interested to apply. Selection will be on a First Come First Served basis
until we will have filled all the available places.

Submission info

Key Dates

August 20st, 2014 (23:59 PST): Abstract Submission Deadline
August, 23rd, 2014: Notification of Acceptance

Abstract Format:

Abstract should be submitted as PDF files (from minimum 1  to  maximum
 3 pages, plus references) in  IROS regular paper format.

Depending on the number of submissions, content and quality accepted
presentations will be in poster or short oral presentation format.


Pedro Lima, Instituto Superior Técnico, Portugal

Gurvinder Virk, Univ. of Gävle & KTH, Sweden

Daniele Nardi, Sapienza University of Rome, Italy

Matteo Matteucci, Politecnico di Milano, Italy

Maria Gini, University of Minnesota, USA

Olivier Michel,Cyberbotics Ltd., Switzerland

Adriana Tapus, ENSTA-ParisTech, France (pending confirmation)

Vincent C. Mueller, Amerikaniko Kollegio Anatolia, Greece (pending

Amit Kumar Pandit, Aldebaran Robotics, France

Fabio Bonsignorio, The BioRobotics Institute, Scuola Superiore Sant’Anna
and Heron Robots, Italy

Elena Messina, NIST, USA

Angel P. Del Pobil, Universidad Jaume I, Spain

Up to date info will be posted here:

There will be post-proceedings.

We look forward your active participation

All the best
Fabio, Angel and Elena


This workshop aims to gather researchers active in the academia and the
industry to share the ideas so far developed and discuss the challenges
still ahead.

The best contributions will be invited to submit to a refereed edited book
or special issue on an high impact robotics journal.
Robotics is a broad science and though we will try to cover different
aspects of the discipline, the emphasis of the workshop will be on
principles, methods, and applications in terms of cognitive capabilities
and autonomy. We will address the issue of how to define and measure system
level characteristics like autonomy, cognition or intelligence.

Another key topic will be a capability-led understanding of cognitive
robots: how to define shared ontologies or dictionaries to discuss robotic
cognitive systems in terms of their performance, relationships between
different cognitive robotics capabilities, requirements, theories,
architectures, models and methods that can be applied across multiple
engineering and application domains, detailing and understanding better the
requirements for robots in terms of performance, the approaches to meeting
these requirements and the trade-offs in terms of performance.

Finally, epistemological issues in robotics research and its evaluation
will be presented and discussed, related to performance measurement,
methods for the objective comparison of different algorithms and systems,
and the replication of published results.

The workshop will provide a forum for the discussion of these issues and
will show a number ofexamples of replicable experiments in Robotics

Topics of interest

- Metrics of Dexterity, adaptivity, flexibility, robustness
- Replication of experiments in robotics
- Metrics for visual servoing effectiveness and efficiency
- Metrics for shared control effectiveness and efficiency
- Benchmarking autonomy and robustness to changes in the environment/task
- Scalable autonomy measurements
- Reporting experiments in Robotics
- Epistemological issues
- Examples of Good Practice
- Evaluation of Experimental Robotics Work
- Proposals for Promotion of Good Experimental Work

Fabio P. Bonsignorio

IEEE Senior Member

Scuola Superiore Sant'Anna, Pisa, Italy
The BioRobotics Institute
viale Rinaldo Piaggio 34, 56025 Pontedera (Pisa)

Heron Robots s.r.l.
Via Malta, 3/7
I-16121 Genova

Research Board of Directors Member
euRobotics aisbl

Founding Past Director
euRobotics aisbl

More information about the robotics-worldwide mailing list