[robotics-worldwide] CfP: IROS 2008 Sept 26 W on Performance Evaluation and Benchmarking for Intelligent Robots and Systems

Fabio Bonsignorio fabio.bonsignorio at gmail.com
Sat Jul 12 01:20:03 PDT 2008

Dear all,

you will find below the call for participation for the IROS workshop
on performance evaluation and benchmarking for intelligent robots to
be held in NICE on September 26th during IROS 2008.

This is an hot topic and high quality and even provocative
contributions on measuring somehow elusive capabilities of artificial
agents, such as autonomy and cognition, as well as more established
like SLAM and Visual Servoing, are welcome.

If you are interested just drop an email to one of the organizers

All the best

ps: previous mail had formatting commands not understandable from some
email client

Important Dates

Papers/Ext. Abstracts: July 28th

Acceptance: August 11th

Full Papers: September 12th

Workshop: September 26th


It is a well-known fact that current robotics research makes it
difficult not only to compare results of different approaches, but
also to assess the quality of individual work. Some steps have been
taken to address this problem by studying the ways in which research
results in robotics can be assessed and compared. In this context, the
European Robotics Research Network EURON has as one of its major goal
the definition and promotion of benchmarks for robotics research. The
Performance Metrics for Intelligent Systems (PerMIS) Workshop series
has been dealing with similar issues in the context of intelligent
systems. A related issue, if not a prerequisite, is the capability of
replicating research results. The EURON Good Experimental Methodology
and Benchmarking Special Interest Group aims to gather efforts in
replication of results and benchmarking in order to increase the
overall experimentation benefit in robotics research. The main purpose
of this workshop is to contribute to the progress of performance
evaluation and benchmarking, focusing in intelligent robots and
systems, by providing a forum for participants to exchange their
on-going work and ideas in this regard.

The emphasis of the workshop will be on cognitive solutions to
practical problems. These cognitive approaches should enable an
"intelligent" system to behave appropriately in real-world scenarios
in various application domains. We also propose to discuss the
distinction between autonomy and intelligence (if any) and how one
influences the other. We welcome any topic relevant to benchmarking
and performance evaluation in the context of cognitive solutions to
practical problems, such as:

- Knowledge representation, perception (sensing), and learning

- Uncertainty management in robot navigation, path-planning and control

- Cognitive manipulation

- Benchmarking autonomy and robustness to changes in the environment/task

- Capability-led understanding of cognitive robots

- Shared ontologies to discuss robotic cognitive systems in terms of
their performance capabilities

- Relationships between different cognitive robotics capabilities

- Requirements, theories, architectures, models and methods that can
be applied across multiple engineering and application domains

- Detailing and understanding better the requirements for robots in
terms of performance, the approaches to meeting these requirements,
the trade-offs in terms of performance

- The development of experimental scenarios to evaluate performance,
demonstrate generality, and measure robustness

Workshop Format & Date

The workshop will consist of invited presentations and regular
presentations. These will be included in the IROS 2008 Tutorials and
Workshops DVD. The full-day workshop is scheduled to be held on
September 26th 2008. You can register for the workshop via the IROS'08
website at http://www.iros2008.org/.

Intended Audience

The primary audience of the proposed workshop is intended to be
researchers and practitioners both from academia and industry with an
interest in cognitive robotics and how these approaches can be
utilized in generating intelligent behaviors amidst uncertainty for
robots in the service and commercial sectors. The workshop is also
aimed at benchmarking and objectively evaluating performance of such
robots. Accordingly, it is envisioned to be useful for anyone who has
an interest in quantitative performance evaluation of robots.

Contact Details

Angel P. del Pobil,
Univ. Jaume I
pobil at icc.uji.es

Raj Madhavan
raj.madhavan at nist.gov

Fabio Bonsignorio
Heron Robots
fabio.bonsignorio at heronrobots.com

About the Organizers

The organizers of the workshop have been actively involved in
performance evaluation and benchmarking for robotics. The organizers
are currently involved in several projects to define methods for
measuring cognitive abilities for robots, both at the component and
the systems level.

Fabio P. Bonsignorio
Heron Robots s.r.l.
Via R.Ceccardi 1/18
I-16121 Genova

More information about the robotics-worldwide mailing list