[robotics-worldwide] Reminder: CFP: RSS'08 WS on Experimental Methodology and Benchmarking in Robotics Research

Fabio Bonsignorio fabio.bonsignorio at gmail.com
Wed May 21 09:46:10 PDT 2008

(Apologies if you receive this twice)

Dear All,
this a reminder that there are a few days left before the deadline of the
RSS workshop on GEM and benchmarking, which will be a wonderful opportunity
to exchange ideas on a somehow 'hot' topic
All the best


2008 Robotics: Science and Systems Workshop on

Half-day workshop to be held in conjuntion with the 2008 Robotics: Science
and Systems Conference at Zurich on June 28.

Important dates:

- *May 25 - Submission of a 1-2 page extended abstract*
- June 2 - Notification of acceptance
- June 16 - Full papers due

Rationale and goals
As the complexity of current robotic and embodied intelligent systems grows,
it is more and more necessary to define proper experimental approaches and
benchmarking procedures. On the one hand, reliable benchmarks are called for
in order to allow the comparison of the many research results in robotics
research, so that their industrial application is eventually possible. On
the other hand, if robotics aims to be  regarded as serious science,
replication of experiments deserves consciencious attention; it is necessary
to be able to verify if and by which measure new procedures and algorithms
proposed in research papers constitute a real advancement and can be used in
new applications.  New more successful implementations of concepts already
presented in the literature, but not implemented with exhaustive
experimental methodology, risk to be ignored, if appropriate benchmarking
procedures are not in place, allowing to compare the actual practical
results with reference to standard accepted procedures. Both replication and
benchmarking are needed to foster a cumulative advancement of our knowledge
of intelligent physical agents and even to correctly appreciate disruptive
innovation in the science and technology of robots. Should we take
inspiration from experimental practice in disciplines such as biology or
medicine? In order to address these needs the European Robotics Network of
Excellence  (EURON) has funded a Special Interest Group on Good Experimental
Methodology and Benchmarking. This workshop aims to provide a discussion
forum on these topics and to identify guidelines for the future. The
workshop will be organized in such a way as to generate fruitful
discussions, it will consist of invited presentations and regular
presentations with a significant amount of additional time for discussions.
The primary audience of the workshop is intended to be researchers and
practitioners both from academia and industry with an interest in
experimental robotics, benchmarking and objectively evaluating performance
of robots. Accordingly, it is envisioned to be useful for anyone who has an
interest in quantitative performance evaluation of robots and/or robot
algorithms. Some controversial issues will be discussed such as: measuring
autonomy or information metrics of intelligent systems, or the concept
itself  of replicability or benchmarking of research results in robotics.

List of topics
o       Good Experimental Methodology in Robotics Research
o     Replication in embodied cognitive systems
o      Methodological/experimental best practices
o     Benchmark/comparison examples
o  Benchmarking autonomy
o  Information metrics of cognitive natural and artificial systems
o        Benchmarking autonomy and robustness to changes in the
o        Requirements, theories, architectures, models and methods that can
be applied across multiple engineering and application domains
o      Detailing and understanding better the requirements for robots in
terms of performance, the approaches to meeting these requirements, the
trade-offs in terms of performance
o       The development of experimental scenarios to evaluate performance,
demonstrate generality, and measure robustness

Workshop Organizers
Angel P. del Pobil, Universitat  Jaume I, Spain, <pobil at icc.uji.es>
John Hallam, University of Southern Denmark, <john at mmmi.sdu.dk>
Fabio Bonsignorio, Heron Robots, Italy, <fabio.bonsignorio at heronrobots.com>

Related event
Please be aware that there is another related RSS workshop on:

Quantitative Performance Evaluation of Navigation Solutions for Mobile
co-organized by  Raj Madhavan, Chris Scrapper and Alex Kleiner

Both are half-day events and will share the same room.

Fabio P. Bonsignorio
Heron Robots s.r.l.
Via R.Ceccardi 1/18
I-16121 Genova
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://duerer.usc.edu/pipermail/robotics-worldwide/attachments/20080521/0d362fd8/attachment.html

More information about the robotics-worldwide mailing list